AGC system reliability early warning system based on video decoding and model fusion

Abstract

This invention relates to the field of wind farm AGC system technology, and discloses an AGC system reliability early warning system based on video decoding and model fusion. The system bypasses the monitoring equipment via a video acquisition security access module to capture the interface video stream in real time; a video parsing feature extraction module synchronously decodes and extracts visual state features and multi-source data features; an intelligent diagnostic reasoning module independently diagnoses the system's dead state based on visual features, and verifies information consistency and lockout status based on multi-source data through a logical knowledge base. Ultimately, this achieves the dual effect of reliable external detection of a complete failure of internal monitoring and deep intelligent diagnosis of logical conflicts in multi-source information, solving the problems of blind spots and shallow alarms in traditional monitoring.

Description

Technical Field

[0001] This invention relates to the field of wind farm AGC system technology, and in particular to an AGC system reliability early warning system based on video decoding and model fusion. Background Technology

[0002] The global energy structure is accelerating its transition to clean and low-carbon energy, and wind power, as a mature renewable energy source, is seeing its share in the power system continue to increase. As the scale of wind farms connected to the grid expands, their impact on power system stability is becoming increasingly significant. Grid dispatching agencies are placing higher demands on the operation and control of wind farms, requiring them not only to respond precisely to dispatch instructions and ensure stable output of active and reactive power, but also to possess ancillary service capabilities such as primary frequency regulation to cope with grid frequency fluctuations.

[0003] Against this backdrop, wind farms have added a series of new equipment and systems, the power grid has increased its control and assessment of wind farm grid-connected equipment, and the electricity trading market has increasingly higher requirements for the actual power generation and forecast accuracy of the stations. If these systems malfunction and cause the wind farm to operate in violation of requirements, it will simultaneously trigger grid assessment and electricity trading losses.

[0004] The aforementioned and existing related technologies often suffer from the following shortcomings: First, for wind farm AGC systems experiencing a server kernel-level failure or a complete freeze of the core process, resulting in a completely unresponsive system crash where all internal monitoring processes, heartbeat messages, and software watchdogs simultaneously fail, existing monitoring methods relying on the system's internal self-reporting have blind spots and cannot achieve effective detection and rapid identification. Second, regarding the dynamic logical inconsistencies between multiple sources of information during AGC system operation, such as scheduling commands, internal AGC target values, actual wind turbine output, and measured values ​​at the grid connection point, as well as cross-system command conflicts caused by control link anomalies, existing alarm mechanisms based on single-point thresholds or single data sources lack the ability to fuse and analyze multi-source information and perform logical reasoning, making it difficult to detect such deep-seated, coupled system defects. Summary of the Invention

[0005] The technical problem to be solved by this invention is that the existing technology has the disadvantage of being unable to effectively detect and quickly identify such deep and coupled system defects. To this end, we propose an AGC system reliability early warning system based on video decoding and model fusion.

[0006] To achieve the above objectives, this application adopts the following technical solution: an AGC system reliability early warning system based on video decoding and model fusion, comprising: a video acquisition security access module, used to bypass access to the video output port of AGC-related monitoring equipment in the wind farm control room, to capture the video stream of the human-machine interface in real time and transmit it through a secure network; and a video parsing feature extraction module, used to decode and extract frames from the video stream, and simultaneously perform visual state feature extraction and multi-source data and state feature extraction, wherein the visual state features are used to characterize the overall visual activity of the interface, and the multi-source data and state features include scheduling instructions extracted from different display areas in the same frame. The system includes: visual status values, AGC target values, real-time active power values, measured values ​​at the grid connection point, equipment status indicators, and alarm text; an intelligent diagnostic reasoning module, used to run the visual perception diagnostic submodule based on the aforementioned visual status characteristics, to diagnose whether the system is in a completely unresponsive, dead state in a way independent of the internal signals of the AGC, and to run the fusion reasoning diagnostic submodule based on the aforementioned multi-source data and status characteristics, verifying the consistency between multi-source information, the correctness of logical direction, and the degree of consistency of dynamic locking status through a logical relationship knowledge base; and an early warning and handling closed-loop module, used to execute a graded response and closed-loop process tracking from on-site alarms and remote notifications to automated work order dispatch based on the risk level of the diagnostic results.

[0007] Furthermore, the video parsing feature extraction module includes: in the decoded video image frames, synchronously locating and cropping multiple corresponding region-of-interest (ROI) image blocks based on pre-configured ROI coordinates; performing parallel recognition processing on the ROI image blocks, including optical character recognition for numerical display area image blocks to obtain digital values, image classification for status indicator and icon area image blocks to obtain status labels, and text detection and recognition for alarm text area image blocks to obtain alarm statements.

[0008] Furthermore, the visual perception diagnosis submodule is configured to, through time-series analysis of the stability of the image pixel change rate and key data area identification values ​​in the visual state features, determine the system crash when the time exceeds a first preset duration threshold and the AGC system is in a remote control state; the logical relationship knowledge base built into the fusion reasoning diagnosis submodule contains the adjustment dead zone setpoint and safety interlocking strategy of the AGC system. This submodule is configured to, in real time calculate and compare the deviation between any two of the scheduling instruction value, AGC target value, real-time active power value and grid connection point measured value. If the deviation continuously exceeds the adjustment dead zone setpoint, an inconsistency alarm is triggered and the identified alarm statement is matched with the safety interlocking strategy.

[0009] Furthermore, when performing parallel recognition processing, the video parsing feature extraction module dynamically schedules computing resources based on the Parallel Extraction Performance Factor (PEEF). The formula for calculating PEEF is as follows: ,in, This represents the total number of regions of interest. The key weight for the k-th ROI; This represents the estimated accuracy of the current identification task in this area. This represents the actual total processing time. The absolute deviation between the timestamp of the k-th ROI identification result and the frame reference timestamp; The maximum allowed synchronization deviation threshold.

[0010] Furthermore, the visual perception diagnostic submodule quantifies the coupling relationship between visual activity decay and data stagnation by calculating the System Stagnation Index (SFI). The formula for calculating the SFI is as follows: ,in, The system death index at time t; Time window at time t The entropy of visual information within the image. This is the baseline value of visual information entropy during normal operation. The sliding variance of key data identification values ​​within the same window. To identify the inherent noise variance of the system, As a remote control mode indicator factor; when exceeding the threshold continuously The duration is greater than the dead time. Furthermore, if the system is in remote mode, it is considered to be frozen.

[0011] Furthermore, the logical relationship knowledge base in the fusion reasoning diagnosis submodule is constructed based on the core logical flow of AGC power adjustment. It is configured to perform consistency verification between the video recognition results and the expected state of the business process for each link in the process, including AGC function input verification, instruction source determination, deviation dead zone judgment, lockout adjustment verification, single adjustment amount limit, controlled equipment identifier generation, allocation strategy execution and closed-loop control determination.

[0012] Furthermore, the system adopts a cloud-edge collaborative architecture. The video acquisition security access module includes an industrial-grade high-definition encoder deployed at the wind farm site, used to encode the video stream and push it to the cloud platform via the management information regional network. The video parsing feature extraction module and the intelligent diagnostic reasoning module are deployed on the cloud platform. The cloud platform provides a visual configuration interface to define the coordinates of the region of interest and calls a cloud-based parallel AI service cluster to identify and process the image blocks of the region of interest. Specifically, the visual perception diagnostic submodule calculates the global inter-frame difference pixel ratio of consecutive video frames, and determines that the system interface is frozen and crashed when the ratio is continuously lower than the activity threshold and the key data identification value remains unchanged for more than a second preset duration threshold.

[0013] Furthermore, the early warning and handling closed-loop module includes an early warning engine and a message gateway deployed on a cloud platform; the early warning engine is configured to simultaneously trigger the following actions for the crash alarm determined by the visual perception diagnosis submodule: push high-priority alarm information to a designated mobile terminal group through the message gateway, initiate telephone voice broadcast through the voice gateway, and automatically create an emergency defect work order in the enterprise asset management system through the application programming interface.

[0014] Furthermore, the system adopts an edge autonomous architecture, with the video acquisition security access module, the video parsing feature extraction module, and the intelligent diagnostic reasoning module integrated into an edge AI computing all-in-one machine deployed at the wind farm site. The video acquisition security access module is connected in series with the AGC monitoring host display through the hardware video interface of the all-in-one machine. The region of interest coordinates and recognition model used by the video parsing feature extraction module are stored in the local storage of the all-in-one machine. The visual perception diagnostic submodule is configured to immediately drive the local audible and visual alarm of the all-in-one machine to sound an alarm after detecting that the image is still for more than a third preset duration threshold.

[0015] Furthermore, the early warning and handling closed-loop module is implemented using an edge-cloud collaborative approach: the edge AI computing all-in-one machine is configured to autonomously execute on-site audible and visual alarms based on local configuration for diagnosed abnormal events, and simultaneously upload the structured record of the diagnosed event to the cloud platform through a secure link; the cloud platform is configured to receive and aggregate event records from each edge AI computing all-in-one machine, and perform centralized display and statistical analysis, and generate a defect elimination work order in the enterprise asset management system based on the event records.

[0016] The technical effects and advantages of this invention are as follows: This invention directly captures the human-machine interface video stream of the wind farm AGC monitoring equipment via a bypass access (non-intrusive method) through a secure video acquisition access module, ensuring the independence and non-dependency of the data source. The video parsing feature extraction module decodes the video stream and simultaneously extracts visual state features characterizing the overall visual activity of the interface. Based on these visual state features, the visual perception diagnosis submodule in the intelligent diagnosis reasoning module, independent of the internal software signals and heartbeat mechanism of the AGC system, analyzes the temporal characteristics of the pixel change rate and data stability to achieve autonomous analysis of the external visual evidence chain. Ultimately, this achieves real-time, independent, and reliable detection of completely unresponsive crash states caused by server kernel crashes, core process freezes, etc., which lead to complete failure of internal monitoring, overcoming the monitoring blind spot problem of traditional internal self-reporting monitoring methods. In this invention, a video analysis feature extraction module synchronously extracts multi-source data and status features from the same decoded frame, including scheduling instruction values, AGC target values, real-time active power values, measured values ​​at grid connection points, equipment status indicators, and alarm texts, distributed across different display areas. This ensures the spatiotemporal consistency of information acquisition. The intelligent diagnostic reasoning module's fusion reasoning diagnostic submodule utilizes a built-in logical relationship knowledge base to perform real-time calculations, comparisons, and logical correlation verifications on the aforementioned multi-source information, covering multiple dimensions such as consistency, instruction direction correctness, and dynamic interlocking state matching. This achieves intelligent fusion diagnosis and precise localization of deep, coupled system defects such as dynamic logical inconsistencies, instruction conflicts, and interlocking logic failures between the scheduling instruction flow, internal control flow, and equipment feedback flow. It represents a leap from traditional single-point threshold alarms to deep logical verification based on business flows. Attached Figure Description

[0017] The disclosure of this invention is illustrated with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. In the drawings, the same reference numerals are used to refer to the same parts:

[0018] Figure 1 This is an overall flowchart of the present invention; Figure 2 This is a schematic diagram illustrating the wireless communication method of transmitting video to a cloud platform according to the present invention. Figure 3 This is the AGC control logic diagram of the present invention. Detailed Implementation

[0019] It is readily understood that, based on the technical solution of this invention, those skilled in the art can propose various interchangeable structural methods and implementations without altering the essential spirit of the invention. Therefore, the following detailed embodiments and accompanying drawings are merely illustrative examples of the technical solution of this invention and should not be considered as the entirety of the invention or as limitations or restrictions on the technical solution of this invention.

[0020] Reference Figure 1 As shown, this invention provides a technical solution: an AGC system reliability early warning system based on video decoding and model fusion. This system includes: a video acquisition secure access module, used to access the video output of key monitoring equipment in the wind farm control room in a non-intrusive manner, capturing the human-machine interface video stream reflecting the AGC system's operating status in real time and transmitting it through an independent secure network channel; a video parsing feature extraction module, connected to the video acquisition secure access module, used to decode and extract frames from the video stream, and simultaneously perform visual state feature extraction and multi-source data and state feature extraction, obtaining the system's global activity index and structured data and status labels scattered across different display areas from the same frame; and an intelligent diagnostic reasoning module, connected to the video parsing feature extraction module, used to run a visual perception diagnostic submodule based on visual state features, diagnosing whether the system is in a completely unresponsive, dead state in a way independent of the AGC's internal signals; and used to run a fusion reasoning diagnostic submodule based on multi-source data and state features, diagnosing the consistency between multi-source information, the correctness of logical direction, and the degree of dynamic locking state matching through a built-in logical relationship knowledge base and rule engine. The early warning and response closed-loop module is connected to the intelligent diagnostic reasoning module. It is used to execute a graded response from on-site alarm to remote platform notification and automated work order dispatch based on the fault type and risk level output by the diagnostic module, and to track the entire response process to form a closed loop of operation and maintenance management.

[0021] The aforementioned system establishes an observation channel independent of the internal reliability of the AGC system by introducing an external video monitoring perspective. This fundamentally avoids the risk of monitoring failure caused by the collapse of the internal monitoring process itself and solves the problem of detecting completely unresponsive crashes. At the same time, by synchronously extracting and fusing heterogeneous information from a single video source, it achieves real-time verification of the dynamic logical relationship between the scheduling command flow, internal control flow, and equipment execution feedback flow. This enables the detection of deep-seated faults that traditional point-based monitoring cannot reach, such as data link inconsistencies and control logic conflicts.

[0022] Example 1: This example details the implementation of the aforementioned system under a cloud-edge collaborative architecture with cloud-based intelligent analysis at its core and lightweight on-site data collection at its edge.

[0023] Please see Figure 2The specific implementation of the video acquisition security access module is as follows: In the central control room of the wind farm's booster station, an industrial-grade high-definition HDMI encoder is connected in series to the display output ports of key equipment such as the AGC monitoring host and remote control devices. This encoder achieves one loop-out of the video signal to ensure normal original display, while the other channel internally performs H.264 / H.265 encoding compression. To ensure the safety of the production control area, the encoder is connected to an independent management information area switch or dedicated wireless AP within the station via a dedicated network port, forming a physically isolated data channel. The encoder pushes the compressed video stream in real time via the RTMP protocol at a constant bitrate to the designated streaming media service address of the booster station intelligent inspection cloud platform deployed in the group's data center.

[0024] The video parsing feature extraction module is implemented in the cloud as follows: An NRV streaming media server cluster deployed on the cloud platform receives and parses video streams from various sites. The platform provides a visual configuration interface, allowing administrators to precisely select and define the pixel coordinates of multiple regions of interest (ROIs) for each monitoring software interface template. These include, but are not limited to: a scheduling instruction value display box, an AGC target value display box, a real-time total active power display box for the entire site, an active power display box for the grid connection point, AGC / remote status indicator lights, an active power adjustment direction icon area, and an alarm information scrolling window area. For each received video stream, the platform extracts image frames at a preset frame rate (e.g., 1 frame / second) based on its bound ROI template and calls a parallel AI service cluster in the cloud to identify and process image blocks in the ROIs. To optimize processing efficiency and ensure the temporal consistency of multi-source data, a parallel extraction performance factor (PEEF) is used to dynamically schedule computing resources.

[0025] The calculation of this factor takes into account the critical weight of each region, the accuracy of identification and prediction, processing time, and result synchronization deviation. .in, This represents the total number of regions of interest. The critical weight for the k-th ROI is dynamically allocated based on the importance of the information (e.g., the scheduling instruction value has the highest weight). The estimated accuracy of the current recognition task in this area is negatively correlated with image contrast and sharpness. This represents the actual total processing time. The absolute deviation between the timestamp of the k-th ROI identification result and the frame reference timestamp; This is the maximum allowable synchronization deviation threshold. The system scheduler aims to maximize PEEF, thereby completing the synchronization retrieval of all critical information with high accuracy in the shortest possible time. It is a very small positive number, and its core function is to prevent division by zero errors. In the specific implementation of this invention, This refers to the actual time required to extract features from all regions of interest in a single frame. The set value is much smaller than the time taken for a normal processing cycle (e.g., nanoseconds versus milliseconds), so it is only used as a safety baseline and will not affect the performance evaluation in normal or high-latency processing scenarios.

[0026] The intelligent diagnostic reasoning module is implemented in the cloud as follows: Visual perception diagnostic submodule: This module receives a sequence of time-series video frames and key data OCR result sequences from the video parsing module. It diagnoses system crashes by analyzing the visual activity and data stability of the images over time. Its core is the calculation of a System Deadness Index (SFI), which comprehensively quantifies the coupling relationship between visual activity decay and data stagnation. .in, The system death index at time t; Time window at time t The specific formula for calculating the visual information entropy of the image within the scene is as follows: .in, The grayscale level (e.g., 256). In the time window The probability of the i-th gray level appearing in the normalized gray-level histogram of all video frames. A lower value indicates that the image content tends to be static and the amount of information is reduced; This is the baseline value of visual information entropy when the system is operating normally; The sliding variance of OCR recognition values ​​for key data (such as total active power) within the same window; This refers to the inherent noise variance of the OCR system; This is the remote control mode indicator factor (remote = 1, local = 0). When exceeding the threshold continuously The duration is greater than the dead time. Furthermore, if the system is in remote mode, it is considered to be frozen.

[0027] The integrated reasoning and diagnostic submodule uses a cloud-based rule engine to subscribe in real-time to multi-source feature streams in a time-series database and performs concurrent diagnostics. Its core function is to perform multi-step correlation verification.

[0028] Please see Figure 3 , Figure 3The original core logic flow of the wind farm AGC system is demonstrated, which is also the core basis for the construction of the logical relationship knowledge base in the integrated reasoning and diagnosis submodule of this invention. Its complete execution flow and the diagnostic verification rules of this invention for each link are as follows: 1. Process start-up and data reading: After the AGC system enters the adjustment cycle, it first reads in all the running data. The video parsing feature extraction module of this invention simultaneously identifies the core data of this link from the video stream by OCR (including actual power value, target power value, AGC function input status identifier, operation mode identifier, station / controlled equipment lockout identifier, etc.), and ensures the consistency of data timing through the pipeline optimized by PEEF factor.

[0029] 2. AGC Function Activation Verification: The system first determines whether the AGC function is activated. If it is not activated, it enters a delay waiting phase and then re-reads the data. In this phase, the diagnostic module of this invention verifies the consistency between the "AGC function activation status indicator" and the AGC / remote status indicator in the video screen. If a contradiction occurs where "the function is not activated but the indicator shows that it is activated", a "status indicator logic conflict" alarm is triggered.

[0030] 3. Command Source Determination: If the AGC function is enabled, further determine whether the operating mode is "remote": In remote mode, obtain the scheduling command value (corresponding to the "Scheduling Command Gradient Limit" verification dimension in Table 1), and in local mode, obtain the local command value; In this step, the diagnostic module of this invention verifies the matching between the command source (scheduling / local) and the "remote / local" indicator light in the video screen, and at the same time verifies whether the scheduling command value exceeds the "Command Gradient Limit" (2%PF / min) in Table 1.

[0031] 4. Deviation Dead Zone Judgment: The system calculates the deviation between the real-time active power value and the target value. If the deviation does not exceed the "Active Power Adjustment Dead Zone" (1% of rated capacity) in Table 1, the system enters the delay stage and re-reads the data. The diagnostic module of this invention continuously monitors the timing changes of the deviation in this stage. If the deviation continues to exceed the dead zone but the system does not enter the next adjustment stage, the "Adjustment Dead Zone Verification Abnormality" alarm is triggered.

[0032] 5. Interlocking Adjustment Verification: If the deviation exceeds the dead zone, the system reads the station interlocking identifier and the controlled equipment interlocking identifier to determine whether interlocking adjustment is required. If interlocked, the system returns to the delay stage; if not interlocked, the system enters the adjustment calculation stage. In this stage, the diagnostic module of this invention associates the safety policy knowledge base in Table 2 to verify the matching of the "interlocking identifier" with the alarm text in the video (such as "master station communication interruption" or "grid connection point data jump"). For example, if the alarm text "grid connection point data jump" is identified but the interlocking identifier is not triggered, it is immediately diagnosed as "interlocking logic failure".

[0033] 6. Single adjustment limit: The system determines whether "actual power value - target power value" is less than or equal to the maximum adjustment amount in a single operation. If it exceeds the limit, the target power value is adjusted cyclically until the limit is met. The diagnostic module of this invention verifies whether the adjustment amount conforms to the "maximum adjustment amount in a single operation" setting customized by the station at this stage. If an over-adjustment occurs but there is no alarm, the "adjustment amount exceeding the limit" diagnosis is triggered.

[0034] 7. Controlled device identifier generation: The system generates an "adjustable / downward adjustable identifier set" for the controlled device. The diagnostic module of this invention verifies the consistency between the identifier set and the "active power adjustment direction icon" in the video image (if the identifier set is "downward adjustable" but the icon displays "↑", an "adjustment direction logic conflict" alarm is triggered).

[0035] 8. Allocation Strategy Execution: The system allocates calculation parameters according to the preset allocation method and the optimal strategy generation principle; the diagnostic module of this invention verifies the dynamic matching between the allocated parameters and the "total active power in real time across the entire field" and "active power at the grid connection point" in the video screen at this stage.

[0036] 9. Closed-loop control determination: The system determines whether closed-loop control is enabled: In non-closed-loop mode, an open-loop alarm prompt is generated, and in closed-loop mode, control commands are output; The diagnostic module of this invention verifies the correlation between the "closed-loop control status" and the alarm scrolling window in the video screen at this stage - for example, in closed-loop mode, the alarm text "control denied" appears but the command is still output, triggering the "closed-loop control logic abnormal" alarm.

[0037] 10. Process Cycle: After completing one round of adjustment, the system enters the next round of adjustment cycle. The diagnostic module of this invention synchronously writes all the verification results of this round into the time series database for subsequent trend analysis and fault tracing.

[0038] For example, given the following characteristics: dispatch instruction Ps=50MW, AGC target Pt=48MW, actual active power Pa=52MW, and the direction icon displays "↓", the engine executes the following diagnostic chain: 1) Calculate the deviation between Ps and Pa, determining that the theoretical adjustment direction should be power reduction; 2) Compare the icon direction, and the consistency check passes; 3) Calculate the deviation between Ps and Pt Δ=2MW, query the station's setpoint table, and if Δ continuously exceeds the active power regulation dead zone (e.g., 1% of rated capacity) for more than 5 calculation cycles, a "Dispatch instruction and AGC internal target value tracking deviation exceeds limit" alarm is triggered, indicating a possible anomaly in the internal control loop. Simultaneously, the engine matches the alarm text keywords identified by OCR with a pre-set safety policy knowledge base to infer the system's likely locked or abnormal state.

[0039] Table 1: Reference Table for Key Diagnostic Values

[0040]

[0041] Table 2: Security Policy Knowledge Base

[0042]

[0043] The specific implementation of the early warning and response closed-loop module is as follows: The cloud-based early warning engine, based on the risk level of the diagnostic results, calls the integrated message gateway to execute a tiered response. For "critical" level crash alarms, the following actions are performed simultaneously: a high-priority alarm card containing the site, fault type, and screenshot is sent to the relevant operations and maintenance group via the WeChat API; a synthesized voice broadcast is delivered by calling the on-duty supervisor via the voice gateway; and an "emergency defect" work order is automatically created through a predefined EAM system integration interface, with the work order title, description, handling level, and responsible team automatically filled in according to the diagnostic template. Operations and maintenance personnel receive work orders via mobile devices, handle them on-site, and provide feedback in the system. The platform monitors the entire lifecycle status of the work order and automatically links it to the alarm closed loop after the fault is cleared and AGC is restored, forming a complete digital operations and maintenance management closed loop.

[0044] Example 2: This example details the implementation of the aforementioned system under an integrated autonomous architecture that fully sinks core analytical capabilities to the local edge devices of the site.

[0045] The specific implementation of the video capture security access module is as follows: A customized edge AI computing all-in-one machine is deployed on-site. This highly integrated device includes multiple HDMI input / output interfaces, a high-performance embedded AI chip (such as the NVIDIA Jetson AGX), audio codec and amplifier units, multi-color LED indicators, and a dual-network port module. During installation, the display output cable of the AGC monitoring host is connected to the HDMI input port of the all-in-one machine, and then one of the HDMI output ports of the all-in-one machine is connected to the original monitor, achieving physical layer serial access and lossless video signal transmission. The entire process is uninterrupted by power outages and service interruptions. The video stream is directly sent to the AI ​​chip's memory processing area via the video capture chip within the device. The raw video data never leaves the device's hardware boundaries, constituting a built-in secure access method. The device sends lightweight diagnostic result messages to the upper-level platform only through another network interface.

[0046] The video parsing feature extraction module is implemented at the edge as follows: Inside the edge all-in-one machine, the embedded software stack directly acquires raw video frames from the video capture driver. Before leaving the factory or during on-site commissioning, a human-machine interface analysis template (including ROI coordinates and feature classification rules) for the specific version of the monitoring software at that site is embedded into the device's flash memory using a dedicated tool. After the device's built-in lightweight AI inference engine starts, it executes a parallel processing pipeline for each frame of image on the pipeline. Similarly, to ensure processing efficiency and timeliness at resource-constrained edge devices, the system adopts a simplified parallel extraction strategy. Its scheduling logic also follows the optimization objective of the Parallel Extraction Performance Factor (PEEF), but the parameters (such as...) are adjusted. , The edge computing power was adapted and tailored accordingly. For example, priority was given to ensuring the extraction speed and synchronization of core data such as scheduling command values ​​and real-time active power values. The value is small. Focus on a few core ROIs.

[0047] The intelligent diagnostic reasoning module is specifically implemented at the edge as follows: Visual perception diagnostic submodule: This module directly receives raw video frames within the edge device and performs real-time analysis. It employs the same System Dead Index (SFI) calculation logic as in Example 1, including using the same visual information entropy. The calculation formula quantifies screen activity, but uses a shorter detection window. and stricter thresholds To meet the requirements of rapid response at the edge. Once calculated If the limit is exceeded, a local hardware interrupt is immediately triggered, driving an audible and visual alarm.

[0048] The integrated reasoning and diagnostic submodule embeds a streamlined real-time rule reasoning engine. At the edge, this streamlined rule engine executes low-complexity association rules based on the AGC core adjustment process. It maintains a multi-source feature sliding window in memory and executes low-complexity association rules. For example, when it synchronously identifies a scheduling command value greater than the actual active power, but the direction icon shows reduced power, the engine immediately records a "command direction display logic conflict" event in local non-volatile memory. Simultaneously, according to pre-configured rules, it can drive the indicator light to switch to slow yellow flashing, providing on-site auxiliary prompts. The engine can also perform keyword matching in the local knowledge base based on identified alarm text fragments to infer basic system status (such as "communication interruption"), enriching the contextual information of local alarms.

[0049] The specific implementation of the early warning and response closed-loop module under edge-cloud collaboration is as follows: Edge-side-led on-site alarm: For any diagnosed anomaly, the edge all-in-one machine autonomously drives its own audible and visual alarm unit according to the locally configured pre-set plan. The alarm mode (sound mode, light color and frequency) strictly corresponds to the diagnosed fault level and type. This is the fastest response and the guarantee link that does not rely on any external network or system.

[0050] Cloud-side collaborative management: Edge devices send a "heartbeat" message containing device status and resource utilization to the cloud platform every 5 minutes via a secure link, and upload structured log records of all diagnostic events in real time (including event codes, timestamps, risk levels, and snapshots of key characteristic values). The cloud platform is responsible for aggregating records from all sites, providing macro-level situational awareness, generating performance and health reports, and automatically or manually generating corresponding troubleshooting work orders in EAM based on the reported event records. During temporary network outages, the edge independently completes all monitoring, diagnostics, on-site alarms, and local event caching; after the network is restored, the devices automatically re-upload the cached event records to the cloud, and the cloud continues to generate work orders and other management processes, realizing a complete closed loop combining "real-time autonomous response at the edge" and "asynchronous collaborative management at the cloud."

[0051] Specifically, the system of this invention acquires visual observation signals independent of the internal state of the AGC system through an external video acquisition unit. The core of its working principle lies in two parallel analysis paths: Path one analyzes the spatiotemporal activity characteristics of the image using machine vision and, based on the comprehensive quantitative indicator of the system's "zombie index," directly diagnoses whether the system's human-computer interaction layer has entered a zombie state, overcoming the "black box" detection challenge when internal monitoring completely fails; Path two extracts multi-source information aligned with spatiotemporal coordinates from a single frame simultaneously, and utilizes a parallel extraction efficiency factor-optimized processing pipeline to ensure the efficiency and consistency of multi-source data extraction. Then, based on a built-in domain knowledge base, it performs real-time correlation comparison and logical reasoning to diagnose coupled, deep-seated logical faults such as inconsistent data links and contradictory control command directions. Based on the risk level of the diagnostic results, the system drives a tiered response mechanism from rapid autonomous alarms at the edge to collaborative handling in the cloud, achieving an intelligent closed loop for operation and maintenance management. Example 1 relies on the powerful computing power of the cloud, excelling in complex model analysis and centralized control; Example 2 relies on real-time edge processing capabilities, focusing on extremely low latency determinism and high availability under network interruptions. Both embodiments can effectively achieve the inventive objective of this invention.

[0052] The technical scope of this invention is not limited to the content described above. Those skilled in the art can make various modifications and variations to the above embodiments without departing from the technical concept of this invention, and all such modifications and variations should fall within the protection scope of this invention.

Claims

1. A reliability early warning system for AGC systems based on video decoding and model fusion, characterized in that, include: The video acquisition security access module is used to bypass the video output port of the AGC-related monitoring equipment in the wind farm control room, capture the video stream of the human-machine interface in real time, and transmit it through a secure network. The video parsing feature extraction module is used to decode and extract frames from the video stream, and simultaneously perform visual state feature extraction and multi-source data and state feature extraction. The visual state features are used to characterize the overall visual activity of the interface. The multi-source data and state features include scheduling instruction values, AGC target values, real-time active power values, grid connection point measured values, equipment status indicators, and alarm texts extracted from different display areas in the same frame. The intelligent diagnosis and reasoning module is used to run the visual perception diagnosis submodule based on the visual state features, diagnose whether the system is in a completely unresponsive dead state in a way independent of the internal signals of the AGC, and run the fusion reasoning diagnosis submodule based on the multi-source data and state features, and verify the consistency between multi-source information, the correctness of logical direction, and the consistency of dynamic locking state through a logical relationship knowledge base. The early warning and handling closed-loop module is used to execute a graded response and closed-loop process tracking from on-site alarm and remote notification to automated work order dispatch based on the risk level of the diagnostic results. The visual perception diagnosis submodule is configured to determine that the system crashes when the pixel change rate and key data area identification value in the visual state features are analyzed in a time series and the value continuously exceeds a first preset time threshold while the AGC system is in a remote control state. The logical relationship knowledge base built into the fusion reasoning diagnosis submodule contains the adjustment dead zone setpoint and safety interlocking strategy of the AGC system. This submodule is configured to calculate and compare the deviation between any two of the scheduling instruction value, AGC target value, real-time active power value, and grid connection point measured value in real time. If the deviation continuously exceeds the adjustment dead zone setpoint, an inconsistency alarm is triggered and the identified alarm statement is matched with the safety interlocking strategy. When performing parallel recognition processing, the video parsing feature extraction module dynamically schedules computing resources based on the parallel extraction performance factor (PEEF). The calculation formula for PEEF is: ,in, This represents the total number of regions of interest. The key weight for the k-th ROI; This represents the estimated accuracy of the current identification task in this area. This represents the actual total processing time. The absolute deviation between the timestamp of the k-th ROI identification result and the frame reference timestamp; The maximum allowable synchronization deviation threshold, It is a very small positive number; the visual perception diagnostic submodule quantifies the coupling relationship between visual activity decay and data stagnation by calculating the System Stagnation Index (SFI), and the formula for calculating the SFI is: ,in, The system death index at time t; Time window at time t The entropy of visual information within the image. This is the baseline value of visual information entropy during normal operation. The sliding variance of key data identification values ​​within the same window. To identify the inherent noise variance of the system, As a remote control mode indicator factor; when exceeding the threshold continuously The duration is greater than the dead time. Furthermore, if the system is in remote mode, it is considered to be frozen.

2. The AGC system reliability early warning system based on video decoding and model fusion according to claim 1, characterized in that: The video parsing feature extraction module includes: in the decoded video image frames, based on the pre-configured region of interest coordinates, synchronously locating and cropping multiple corresponding region of interest image blocks; performing parallel recognition processing on the region of interest image blocks, including performing optical character recognition on numerical display region image blocks to obtain digital values, performing image classification on status indicator and icon region image blocks to obtain status labels, and performing text detection and recognition on alarm text region image blocks to obtain alarm statements.

3. The AGC system reliability early warning system based on video decoding and model fusion according to claim 1, characterized in that: The logical relationship knowledge base in the fusion reasoning diagnosis submodule is built based on the core logical process of AGC power adjustment. It is configured to perform consistency verification between the video recognition results and the expected state of the business process for each link in the process, including AGC function input verification, instruction source determination, deviation dead zone judgment, lockout adjustment verification, single adjustment amount limit, controlled equipment identifier generation, allocation strategy execution and closed-loop control determination.

4. The AGC system reliability early warning system based on video decoding and model fusion according to claim 2, characterized in that: The system adopts a cloud-edge collaborative architecture. The video acquisition and secure access module includes an industrial-grade high-definition encoder deployed at the wind farm site, used to encode the video stream and push it to the cloud platform via the management information regional network. The video parsing feature extraction module and the intelligent diagnostic reasoning module are deployed on the cloud platform. The cloud platform provides a visual configuration interface to define the coordinates of the region of interest and calls a cloud-based parallel AI service cluster to identify and process the image blocks of the region of interest. Specifically, the visual perception diagnostic submodule calculates the global inter-frame difference pixel ratio of consecutive video frames, and determines that the system interface is frozen and crashed when the ratio is continuously lower than the activity threshold and the key data identification value remains unchanged for a duration exceeding a second preset duration threshold.

5. The AGC system reliability early warning system based on video decoding and model fusion according to claim 4, characterized in that: The early warning and handling closed-loop module includes an early warning engine and a message gateway deployed on a cloud platform. The early warning engine is configured to simultaneously trigger the following actions for a crash alarm determined by the visual perception diagnosis submodule: push high-priority alarm information to a designated mobile terminal group through the message gateway, initiate telephone voice broadcast through the voice gateway, and automatically create an emergency defect work order in the enterprise asset management system through the application programming interface.

6. The AGC system reliability early warning system based on video decoding and model fusion according to claim 1, characterized in that: The system adopts an edge autonomous architecture. The video acquisition security access module, the video parsing feature extraction module, and the intelligent diagnostic reasoning module are integrated into an edge AI computing all-in-one machine deployed at the wind farm site. The video acquisition security access module is connected in series with the AGC monitoring host display through the hardware video interface of the all-in-one machine. The region of interest coordinates and recognition model used by the video parsing feature extraction module are stored in the local storage of the all-in-one machine. The visual perception diagnostic submodule is configured to immediately activate the local sound and light alarm of the all-in-one machine after detecting that the screen remains still for more than a third preset time threshold.

7. The AGC system reliability early warning system based on video decoding and model fusion according to claim 6, characterized in that: The early warning and handling closed-loop module is implemented using an edge-cloud collaborative approach: the edge AI computing all-in-one machine is configured to autonomously execute on-site audible and visual alarms based on local configuration for diagnosed abnormal events, and simultaneously upload the structured record of the diagnosed event to the cloud platform through a secure link; the cloud platform is configured to receive and aggregate event records from each edge AI computing all-in-one machine, and perform centralized display and statistical analysis, and generate a defect elimination work order in the enterprise asset management system based on the event records.

Images