A non-invasive AI-based new energy production automation monitoring system

Abstract

The application discloses a new energy production automatic monitoring system based on non-invasive AI, relates to the technical field of industrial automation and intelligent monitoring, and comprises a screen acquisition module, a motion execution module, a multi-modal information extraction module, a state fusion and confidence evaluation module, an SC3 closed-loop monitoring control module, a process arrangement module and a log auditing module. The system locks the interface snapshot in each monitoring round, generates an interface anchor point set and a reading set, carries out structured extraction on the readings through view positioning and area constraint to form a state candidate set, calculates the state confidence and generates a state vector in combination with recognition confidence, cross-frame stability and cross-view consistency, constructs a conflict set and carries out gate decision, carries out rollback or locking processing according to the afterimage checking result, and the log auditing module records an evidence chain in the whole process, so that the credible closed-loop control of the monitoring process is realized.

Description

Technical Field

[0001] This invention relates to the field of industrial automation and intelligent monitoring technology, and more specifically, to a new energy production automation monitoring system based on non-intrusive AI. Background Technology

[0002] As the scale of new energy power plants expands and the production system becomes more multi-source and heterogeneous, on-duty personnel need to frequently switch between multiple monitoring interfaces to verify alarms and key parameters. Under conditions of complex screens and dense information, problems such as high monitoring load, delayed anomaly detection, and difficulty in ensuring consistency in handling are likely to occur.

[0003] Existing public literature 1 (Design and Implementation of a Dispatch Automation Master Station Operation Monitoring System, 2014) monitors the operation of the master station hardware and application systems by building an operation monitoring system, and analyzes and judges abnormal situations to meet the needs of real-time and unified management of system operation status. However, such solutions usually rely on existing monitoring links and platform capabilities. When the system interfaces are not unified or the modification of existing systems is limited, it is difficult to cover multiple types of terminals and complex interface information; moreover, the lack of unified constraints on "sufficiency and verifiability of evidence" in the anomaly identification and handling stages can easily lead to insufficient reliability of monitoring results.

[0004] Existing public literature 2 (Research on Deep Semi-Supervised Learning Algorithm Based on Adaptive Thresholding and Soft Self-Stepping Learning, 2025) proposes, as Figure 1 As shown, this paper utilizes an adaptive threshold and soft-step learning weight mechanism to dynamically allocate sample weights based on prediction confidence and participate in loss calculation, thereby improving the utilization rate and learning stability of low-confidence information. However, this paper focuses on sample weighting and threshold adjustment during model training and does not address interface state forensics in production monitoring scenarios, or cross-temporal / cross-view scenarios. Figure 1 Engineering-based closed-loop requirements include consistency verification, conflict gating, and post-action verification and audit traceability.

[0005] Therefore, there is an urgent need in this field for an automated monitoring solution that can reliably acquire key statuses under complex monitoring interfaces and reduce the risk of misjudgment and misoperation. Summary of the Invention

[0006] To overcome the aforementioned deficiencies of existing technologies, this invention provides a non-intrusive AI-based automated monitoring system for new energy production. This system generates structured readings through non-intrusive screen acquisition and employs a trusted monitoring closed-loop decision-making algorithm for confidence gating and conflict determination. It combines verification / handling, afterimage verification, rollback locking, and log audit evidence chain to achieve closed-loop monitoring, thereby solving the problems mentioned in the background technology.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A new energy production automation monitoring system based on non-intrusive AI. Non-intrusive means that the monitored system's hardware and software are not modified, and the backend interface is not called. The closed loop is completed only through screen evidence collection and external input simulation. The "monitoring" in this invention includes monitoring, alarm, generation of handling suggestions, and automatic handling closed loop under safety gating. The action execution of this invention is a human-machine interaction layer operation of the existing HMI / monitoring interface, without modifying the internal logic of PLC / SCADA, without accessing the control protocol, and without modifying the field equipment.

[0009] The system of this invention includes a screen capture module, an action execution module, a multimodal information extraction module, a state fusion and confidence assessment module, an SC3 closed-loop monitoring control module, a process orchestration module, and a log auditing module. Its key feature is that the SC3 closed-loop monitoring control module executes a trusted closed-loop monitoring decision algorithm with triple consistency constraints. In each monitoring round, it locks an interface snapshot and generates an interface anchor point set and a reading set. After performing view positioning and region constraints on the anchor point set, it performs structured extraction on the reading set to obtain a state candidate set. The state fusion and confidence assessment module calculates the confidence score of the state candidate set and generates a state vector, wherein the confidence score is composed of recognition confidence, cross-frame stability, and cross-view... Figure 1 Consistency constraints and cross-frame stability require that the deviation of the same state variable does not exceed the threshold band within N consecutive frames, and cross-view stability. Figure 1 Consistency requires that the deviation of the same state variable between at least two views does not exceed the cross threshold; the SC3 closed-loop monitoring control module constructs a conflict set and makes gating decisions accordingly. When the confidence level is insufficient or the conflict set is not empty, only the generation of a review action sequence is allowed to improve the quality of evidence. When the confidence level is sufficient and the conflict set is empty, a disposal action package is generated from the verifiable action template set and bound to the expected afterimage verification rule; after the action is executed, evidence is collected again, and the consistency of the action is judged according to the expected afterimage verification rule. If it fails, a rollback is triggered first; if it is still inconsistent after the rollback or it is determined that there is an irreversible risk, it is locked and enters the next round (or transferred to manual review); the log audit module associates the recorded evidence, conflict set, threshold, action and verification result to form an evidence chain.

[0010] As a further aspect of the present invention, the interface anchor point set includes static anchor points and dynamic anchor points. Static anchor points include at least any one of the following: page title area, menu area, table area, and alarm list area. Dynamic anchor points include at least any one of the following: pop-up window, color state change, icon flashing, and limit violation mark. The view positioning and area constraint use the anchor point combination matching result as the boundary condition of the effective reading area.

[0011] As a further aspect of the present invention, the structured extraction includes at least text-numeric reading extraction, table key-value pair extraction, alarm item extraction, and trend feature extraction, and generates a corresponding reading source identifier and view identifier for each reading, for cross-view processing of the same state quantity. Figure 1 Consistency test.

[0012] As a further aspect of the present invention, the threshold band and the cross threshold are configured to correspond to different state quantity categories, wherein a discrete consistency criterion is used for alarm state quantities, a relative deviation criterion is used for continuous numerical state quantities, and a morphological similarity criterion is used for trend state quantities, thereby generating the conflict set uniformly under the same algorithm framework.

[0013] As a further aspect of the present invention, the review action sequence is generated by a review action template set provided by the process orchestration module. The review action template set includes at least two of the following: refreshing the page, switching to the details view, zooming in on the key area, switching to the backup view, and repositioning the target area. The review action sequence terminates when the conflict set is empty or the confidence level reaches a threshold.

[0014] As a further aspect of the present invention, each template of the verifiable action template set includes a precondition, an operation sequence, a postcondition, and a prohibition condition; the prohibition condition includes at least one of the following: the conflict set is not empty, the confidence level is lower than the threshold, and the view positioning fails, so as to ensure that the action package is generated only under the premise of satisfying the triple consistency constraint.

[0015] As a further aspect of the present invention, the action package is divided into low-risk actions, medium-risk actions, and high-risk actions according to the risk level of the actions. High-risk actions include at least one of alarm confirmation or silencing, action conclusion confirmation, and work order submission. Moreover, the generation and execution of high-risk actions must meet a confidence threshold higher than that of low-risk actions and satisfy cross-view requirements. Figure 1 Consistency test.

[0016] As a further aspect of the present invention, the expected afterimage verification rules include at least one or more of the following: alarm status bit change, key parameters falling into the expected range, work order number generation, event record addition, and page status identifier change. The expected afterimage verification rules are bound one-to-one with verifiable action templates for action consistency determination.

[0017] As a further aspect of the present invention, the rollback or locking includes any one or more of the following: undoing the operation, returning to the security page, restoring the default filtering conditions, stopping subsequent actions, and locking the process. The threshold band, cross threshold, and number of consecutive frames N are adaptively updated based on the error distribution of the failed verification samples, and the log audit module records the parameters before and after the update and the corresponding evidence chain to support playback reproduction.

[0018] The technical effects and advantages of the new energy production automation monitoring system based on non-invasive AI of the present invention are as follows:

[0019] This invention achieves automated monitoring closed-loop through screen evidence collection and external input without modifying the software and hardware of the monitored system or calling the backend interface. It reduces the manual monitoring load and frequent switching costs under complex interfaces, shortens the response time for anomaly detection and handling, and is suitable for multi-source heterogeneous site operation environments.

[0020] The trusted monitoring closed-loop decision algorithm of this invention transforms the evidence collection results into computable evidence sufficiency constraints, and generates a path with conflict interception and gating decision control actions. When the evidence is insufficient, priority is given to review and supplementary evidence, and only when the evidence is sufficient and consistent is action allowed. This mechanism reduces the risk of misreading, errors, and erroneous execution, improves the stability and credibility of monitoring conclusions, and ensures the safety of high-risk operations.

[0021] This invention binds verifiable postconditions and post-image verification rules when generating the action. After the action is executed, evidence is collected again to confirm the action's effectiveness. If it fails, it automatically triggers rollback or locks and enters the next round. Log auditing forms a replayable evidence chain, enabling the action results to have objective evidence support and full traceability, solving the problems of traditional automated actions being difficult to prove and difficult to review.

[0022] This invention uses evidence collection, decision-making, action, and verification to form a closed-loop coupled link. The trusted monitoring closed-loop decision-making algorithm runs through gating, review, handling, and rollback locking. Each module restricts and triggers each other to form a whole, realizing the unity of reliable evidence collection, credible decision-making, verifiable handling, and traceable auditing. Attached Figure Description

[0023] Figure 1 A schematic diagram of a semi-supervised learning framework for adaptive thresholding and soft autostepping learning in existing technologies.

[0024] Figure 2 This is a structural diagram of a new energy production automation monitoring system based on non-intrusive AI, according to the present invention.

[0025] Figure 3 This is a flowchart of the SC3 closed-loop monitoring control process of the present invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] Example 1

[0028] This embodiment deploys a non-intrusive AI-based automated monitoring system for new energy production in the actual duty environment of a new energy centralized control center. The monitored objects are the production monitoring main station interface and the work order system interface running simultaneously on the duty computer. On-site constraints are: no plugins or scripts are installed on the main station, no main station backend interfaces are called, no existing main station configuration is changed, and only screen capture via external devices is allowed, along with simulation of keyboard and mouse input via external means. The system structure is as follows... Figure 2 As shown, the system includes a screen capture module, an action execution module, a multimodal information extraction module, a state fusion and confidence assessment module, an SC3 closed-loop monitoring control module, a process orchestration module, and a log auditing module. The system uses a "monitoring round" as its basic operating unit: each round begins by locking an interface snapshot and establishing the evidence baseline for that round. This process involves anchor point and reading generation, view positioning and area constraints, structured extraction, confidence calculation and state fusion, conflict set construction and gating decision-making, review or handling, action post-image verification, rollback or locking, and ending after the evidence chain is closed. The next round begins with re-entry from a secure page, thus preventing the accumulation of risks caused by superimposing actions on uncertain interface states.

[0029] In this embodiment, the screen capture module connects to the video output link of the duty computer via an external capture card. The capture resolution is fixed at 1920×1080, and the capture frame rate is adaptive within the range of 12 to 60 fps. Each frame is appended with a capture timestamp and frame sequence number for cross-frame stability verification and time alignment of the before / after image windows. To reduce random errors such as single-frame tearing, mouse cursor occlusion, and pop-up animation jitter, three frames are continuously captured at the beginning of each monitoring cycle, and the median frame is locked as the interface baseline snapshot S0. S0 is used to determine the anchor point positioning and ROI clipping parameters for that cycle and is written to the log audit as a "baseline evidence frame" before the action is executed. Subsequently, without changing the anchor point and ROI of that round, the system continues to collect subsequent consecutive frames to form a cross-frame evidence sequence {S1, S2, ..., SN} (preferably N=5), and performs cross-frame consistency verification on the readings / state variables within the same ROI in the sequence to obtain the structured extraction fusion results (e.g., median / mean and confidence interval) and output the corresponding confidence level. If the sequence does not meet the cross-frame stability threshold, the reading / state variable is marked as unstable and added to the conflict set, triggering review or delayed processing. To ensure the repeatability of evidence collection, the acquisition link maintains a constant display scaling factor, and environmental parameters such as resolution, scaling factor, and refresh rate are written into the evidence chain.

[0030] In this embodiment, the action execution module is implemented via USB-simulated keyboard and mouse injection, externally displaying clicks, scrolling, dragging, and text input consistent with human actions. To ensure verifiable and reproducible actions, actions are not directly clicked using absolute screen coordinates, but rather represented using "anchor reference + relative offset," with each action recorded as (anchor...).id ,Δx,Δy,type,param), where anchor id The anchor point identifier is obtained by this wheel recognition. Δx and Δy are the pixel offsets relative to the anchor reference point. type can be one of single click / double click / right click / scroll / drag / input. param is a parameter such as scroll wheel steps, input content, and drag endpoint offset. During execution, the coordinates of the anchor reference point are first located, and then Δx and Δy are superimposed to obtain the action landing point. This transforms the uncertainties such as interface scaling, column width fine-tuning, and window position changes into a calculable condition of "whether the anchor point matches", significantly reducing the risk of misclicks caused by script-based coordinate clicks. This embodiment uses a fixed action timing sequence: single click press-release interval 120ms, double click interval 180ms, drag duration 500ms, text input rate 8 characters / second, fixed scroll step and recording the scroll direction and number of steps each time. All action parameters are entered into the evidence chain to support playback reproduction.

[0031] Within each monitoring round, the SC3 closed-loop monitoring control module executes a trusted closed-loop monitoring decision algorithm with triple consistency constraints. Its core link is as follows: Figure 3 As shown: First, S0 is locked, and a set of interface anchor points and a set of readings are generated. Then, the anchor point set is positioned and constrained by the view, and the reading set is extracted in a structured manner to obtain a set of state candidates. Next, the state fusion and confidence evaluation module calculates the confidence of the state candidate set and generates a state vector. The SC3 closed-loop monitoring control module constructs a conflict set based on the state vector and makes gating decisions: when the evidence is insufficient or the conflict set is not empty, only a review action sequence is allowed to be generated to improve the quality of evidence. After the review action is executed, evidence is collected again, and the state vector and conflict set are updated before entering the next gating decision. When the evidence is sufficient and the conflict set is empty, a disposal action package is generated from the verifiable action template set, and the expected afterimage verification rule corresponding to the disposal action package is bound. After the disposal action is executed, evidence is collected again, and the consistency of the action is determined according to the bound expected afterimage verification rule: if consistent, it indicates that the disposal has achieved the expectation, the system restores the monitoring state and enters the next monitoring round; if inconsistent, a rollback or locking strategy is triggered to correct the current state before entering the next monitoring round. The log auditing module runs through the entire process, linking evidence, conflict sets, thresholds, actions, and verification results to form a traceable chain of evidence.

[0032] To adapt to different page formats and dynamic changes, the system constructs a set of interface anchor points A in the t-th round. tThe anchor point set includes static and dynamic anchor points: static anchor points must come from at least one of the following categories: page title area, menu area, table area, and alarm list area, such as the overall operation title text, menu tree root node icon, device table header, alarm list header, etc.; dynamic anchor points must come from at least one of the following categories: pop-ups, color status changes, icon flashing, and limit violation markers, such as confirmation / mute pop-up buttons, device row red, yellow, and green status blocks, bell flashing icons, trend limit violation triangle markers, and selected row highlight boxes, etc. Each anchor point is recorded as a 'a'. i =(class i bbox i ,s i ,flag i ), where class i For anchor point category, bbox i For bounding box, s i To match the score, flag i The stability marker is determined by three computable conditions: the anchor point is visible at least once in continuous evidence collection, the bounding box scale changes no more than once, and the relative topological relationship with other key anchor points does not change abruptly. This provides verifiable evidence for "view stability".

[0033] The set of readings generated in parallel with the set of anchor points is R. t The generation of the reading set is strictly limited by the boundary conditions of "view positioning and region constraints": the system first determines the current view V by anchor point combination matching. t Then, based on the geometric relationship of the anchor points, the set of effective reading regions Ω(V) is derived. t ), only in Ω(V t Reading candidates are generated within the range. View positioning uses a combination of multiple anchor points and topological constraints instead of a single anchor point, for example, if and only if the anchor point combination {a title ,a filter ,a table Only when the topological relationship of "title above, filter in the middle, table below" is met simultaneously will the alarm table row area be included in Ω(V). t It allows the generation of alarm entry readings; when the combination is not met, no alarm table readings are generated, and the view positioning failure is recorded as a fatal conflict. The same applies to the device details view: reading candidates are allowed to be generated in the parameter panel and trend area only if the device name anchor, parameter panel anchor, and trend label anchor combination matches and their relative positions satisfy the constraints. Through this mechanism of "anchor combination matching results as boundary conditions for the valid reading area," the reading source is spatially constrained to a reliable area, reducing the probability of misreading background elements and irrelevant button text.

[0034] The multimodal information extraction module performs structured extraction on the reading set to obtain a state candidate set C. tStructured data extraction includes at least four steps: text and numeric reading extraction, table key-value pair extraction, alarm entry extraction, and trend feature extraction. Text and numeric reading extraction is used for areas such as parameter cards and detail panels, outputting numerical values, units, and out-of-limit markers. Table key-value pair extraction uses header anchors and column separators to create a grid, performs OCR on cells, and maps them to key-value pairs semantically by column. Alarm entry extraction maintains in-row associations and outputs at least the alarm level, alarm time, device name, alarm content, and confirmation status. Trend feature extraction prioritizes obtaining numerical labels next to curves or readings from floating prompts; if no readable numerical labels are available, it extracts trend pattern feature vectors. For each reading candidate, a reading source identifier (src) is generated. i With view identifier v i And write the candidate entry (key) i ,value i unit i ,src i ,v i To avoid source identifier abstraction leading to inoperability, src i In this embodiment, the value is taken from one of the following enumeration sets: src i ∈{OCR cell OCR card ICON status COLOR patch TREND tooltip TREND feature}, which respectively correspond to table cell OCR, card / panel OCR, icon recognition, color block recognition, trend floating prompt reading, and trend feature vector extraction. i As a view identifier, used for subsequent cross-view... Figure 1 Consistency testing and conflict interpretation.

[0035] This embodiment sets clear dimensions and reasonable engineering ranges for key state variables, and incorporates range verification as part of the candidate validity constraint: active power P (MW, 0~300), reactive power Q (MVar, -100~100), energy storage state of charge (SOC) (%, 0~100), wind speed V w (m / s, 0~40), Photovoltaic branch current I s (A, 0~15), Alarm Count N a (Non-negative integers), alarm time is "YYYY-MM-DD HH:MM", and work order number IDw is a 10-24 alphanumeric string. When a candidate value exceeds the limit but there is an over-limit marker on the interface or a corresponding alarm to support it, it is marked as "abnormal but interpretable"; when it exceeds the limit and lacks support, it is marked as "high-risk candidate", enters the conflict set and triggers gating interception to avoid directly converting misidentification into handling actions.

[0036] The state fusion and confidence evaluation module calculates the confidence score of the candidate state set and generates the state vector S. t The confidence level is determined by the identification confidence C. rec Cross-frame stability C frame Cross-view Figure 1 C view Common constraints. Cross-frame stability requires that the deviation of the same state variable does not exceed the threshold band within N consecutive frames, and the system forms a continuous evidence collection sequence in round t:

[0037]

[0038] in This represents the set of valid results obtained from the k-th evidence collection in this round, where N is the number of consecutive evidence collections; in this embodiment, N=5. For continuous numerical state variables x∈{P,Q,SOC,V} w ,I s Cross-frame stability is determined using the relative deviation criterion:

[0039]

[0040] Where x t,k This refers to the value of the same state variable obtained during the k-th verification in this round. This is the average of N evidence collections within this round. ≥x min When the relative deviation criterion is used, when <x min The absolute deviation criterion maxx is used at this time. t,k -minx t,k ≤δ x The threshold band is configured as δ according to the state variable category. P =0.6MW, δ Q =0.3MVar, δ SOC =0.4%, δ Vw =0.5m / s, δ Is =0.2A.

[0041] Cross-view Figure 1 The consistency requirement is that the deviation of the same state variable between at least two views does not exceed the cross threshold ε, and the cross threshold for continuous values ​​is set to ε. P =1.2MW, ε SOC =0.8%, ε Vw =1.0m / s, ε Is =0.4A, the text field uses a normalized edit distance threshold ε text =0.15. The confidence level is obtained by combining the three types of consistency. C rec C frame C viewAll are linearly attenuated to 0 according to the proportion exceeding the threshold. The exceeding proportion is defined as the ratio of the excess amount to the corresponding threshold band or cross threshold, and is truncated to the interval [0, 1]. In this embodiment, the weight w rec = 0.45, w frame = 0.30, w view = 0.25.

[0042] The SC3 closed-loop monitoring control module obtains S t and then constructs the conflict set E t and performs gating decision-making. The conflict set at least includes three types: same-key multi-value conflict, physical constraint conflict and logical constraint conflict, and is classified into general, important and fatal. View positioning failure, missing key fields, and failure to uniquely determine the action object are fatal conflicts.

[0043] Gating determination takes the key fields of the object to be disposed as the objectification threshold. For alarm-type disposal, the key field set K={DeviceName,AlarmTime,AlarmLevel,AlarmContent} is defined, and its minimum key confidence is calculated C(k), where C(k) is the comprehensive confidence of the field k. When the action template to be executed in this round involves other key fields (such as work order number, disposal object identifier, etc.), they are incorporated into the template key field set K tpl and participate in the C min calculation.

[0044] The gating rule is: when E t is not an empty set or C min,K < T, it is regarded as insufficient evidence, and only the generation of a review action sequence is allowed; when E t is an empty set and C min,K ≥ T, the generation of a disposal action package is allowed. For low-risk actions and review actions, the threshold T low = 0.78, and for high-risk actions, the threshold T high = 0.90, and it is required that key fields are consistent across views Figure 1 and the static anchor topology remains stable in consecutive N times of forensics.

[0045] The review action sequence is generated by the process orchestration module based on the review action template set. Each time a review action is executed, the interface snapshot is re-locked and C t , S t and E t are recalculated. The successful termination condition of review is that E t is an empty set and C min,K ≥ T are satisfied at the same time, and then the process enters the disposal gating; if the maximum number of review steps is reached or a fatal conflict occurs, locking is taken as the failure termination condition and the process enters the next round, so as to avoid repeated oscillation when the conflict is not resolved.

[0046] When the gating allows the action, the system generates an action package from the verifiable action template set. Each template includes preconditions, operation sequence, postconditions, and prohibition conditions. Postconditions define the set of observable afterimages that should appear, and afterimage verification rules determine whether this set is satisfied within the verification window. After the action is executed, the system re-examines the evidence and performs afterimage verification. For the verification of "key parameters falling within the expected range," the expected range is derived solely from verifiable information on the same screen: if upper and lower limit fields exist, they are used as the range; if only a set value field exists, the range [set value − 2δ] is used. x , set value + 2δ x If the interface does not provide any upper or lower limits or set values, this parameter will not be used as the minimum afterimage condition. If any minimum afterimage condition is met within the verification window, the action consistency is determined to be passed; if no minimum afterimage condition is met within the verification window, the action consistency is determined to be failed. Action consistency failure triggers rollback or locking and enters the next round. Rollback is used for reversible failures, and locking is used for irreversible risks or environmental anomalies. For high-risk actions, the minimum afterimage condition is preferably set as a strong afterimage condition, which at least includes state change evidence that can uniquely confirm the handling result (such as number generation, state field disking, log addition, etc.).

[0047] The threshold band, cross threshold, and consecutive frame number N are adaptively updated based on the error distribution of the failed samples. When the cumulative number of similar failed samples reaches 50, an update is triggered. The new values ​​of δ and ε are taken as the 95th percentile of the corresponding error samples and truncated to the safe upper limit. N is taken as the smallest integer that makes the stability pass rate reach the target value. All updates record the version number, previous and subsequent parameters, trigger sample index, and rollback pointer to support parameter replayability and reproducibility.

[0048] The log auditing module runs throughout the entire process, forming a chain of evidence corresponding to the closure of each round. The chain of evidence must at least link the following records: lock snapshot S0 and keyframe sequence, and anchor point set A. t and matching score, view location result V t With the effective reading range Ω (V t ), Reading set R t With candidate set C t Confidence decomposition term C rec C frame C view Compared with overall confidence level and minimum confidence level C of key fields min,K Conflict set E tIn addition to the template identifiers and action parameters of the classification, gating conclusions, review action sequences or handling action packages, the set of afterimages defined by postconditions, the afterimage sequence and consistency judgment results in the afterimage verification window, the fallback or locking path and reasons when failure occurs, and the parameter adaptive update versions and the sample index, this embodiment calculates a summary of key evidence fragments and forms a chain association, so that any round of evidence collection-decision-action-verification process can be replayed and reproduced.

[0049] To verify that this embodiment can solve the problems of stable object positioning, cross-view reliable verification, and self-verifiable action effectiveness that are difficult to simultaneously satisfy in traditional automation under non-intrusive conditions, this embodiment was compared and tested in a semi-realistic joint debugging environment. The environment consisted of a master station image and a work order system image, with a resolution of 1920×1080, running continuously for 8 hours, including 4 hours each for the wind power scenario and the photovoltaic / energy storage combined scenario; the average alarm arrival rate was 18 alarms / hour, with a peak of 45 alarms / hour. Three scenarios were compared: Scenario A was a single identification trigger (without cross-frame stability and cross-view verification). Figure 1 Consistency gating), Case B is recognition + cross-frame stabilization (only cross-frame stabilization is performed, not cross-view stabilization). Figure 1 (Consistency and conflict gating), Case C is the triple consistency constraint closed loop of this embodiment. Statistical indicators include alarm object location accuracy rate, high-risk false execution rate, action post-image verification pass rate, and average closed loop delay for forming a replayable evidence chain. The results are shown in Table 1.

[0050] Table 1 Comparison of test results for three scenarios (total over 8 hours)

[0051]

[0052] As shown in Table 1, scenario A is prone to object misalignment in scenarios such as list scrolling bounce, table rearrangement, and short-term loading delays, leading to high-risk actions such as confirming or submitting incorrect objects. The root cause lies in its reliance on single-shot recognition confidence, lacking cross-frame stability and cross-view stability. Figure 1 The evidence constraints are inconsistent, and there is a lack of conflict set gating to intercept inconsistent evidence for review and supplementary evidence. Scenario B reduces the false execution rate by suppressing digit skipping and short-term jitter through cross-frame stability, but the risk of "single-view stability but cross-view inconsistency" may still occur when the list and details are temporarily out of sync or when fields in different views are inconsistent. Scenario C addresses this by identifying confidence, cross-frame stability, and cross-view... Figure 1 Consistency triple consistency is used to generate confidence scores, and the minimum confidence score C of the key field is used. min,K With Conflict Set E tBy implementing gating, inconsistent evidence is directed to review and supplementary evidence or locked for exit, reducing the high-risk erroneous execution rate to zero. At the same time, the effectiveness of actions is determined by post-image verification, and all actions and non-actions form a replayable evidence chain, solving the problem of traditional automation where "it is difficult to prove whether an action is truly effective and difficult to trace back afterward." The average closed-loop latency in case C is slightly higher than in case A, with the increase coming from review and supplementary evidence and post-image waiting windows. However, this results in zero erroneous execution of high-risk actions and full-process traceability, enabling calculable, verifiable, and reproducible reliable closed-loop monitoring under strict non-intrusive constraints.

[0053] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0054] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A non-intrusive AI-based automated monitoring system for new energy production, comprising a screen capture module, an action execution module, a multimodal information extraction module, a state fusion and confidence assessment module, an SC3 closed-loop monitoring control module, a process orchestration module, and a log auditing module; characterized in that, The SC3 closed-loop monitoring control module executes a reliable closed-loop monitoring decision algorithm with triple consistency constraints. In each monitoring round, it locks the interface snapshot and generates an interface anchor point set and a reading set. After performing view positioning and region constraints on the anchor point set, it performs structured extraction on the reading set to obtain a state candidate set. The state fusion and confidence evaluation module calculates the confidence of the state candidate set and generates a state vector. The confidence is jointly constrained by identification confidence, cross-frame stability, and cross-view consistency. Cross-frame stability requires that the deviation of the same state quantity does not exceed the threshold band within N consecutive frames. Cross-view consistency requires that the deviation of the same state quantity does not exceed the cross threshold between at least two views. The SC3 closed-loop monitoring and control module constructs a conflict set and makes gating decisions accordingly. When the confidence level is insufficient or the conflict set is not empty, only the generation of a review action sequence is allowed to improve the quality of evidence. When the confidence level is sufficient and the conflict set is empty, a disposal action package is generated from the verifiable action template set and bound to the expected afterimage verification rule. After the action is executed, evidence is collected again, and the consistency of the action is judged according to the expected afterimage verification rule. If it fails, a rollback is triggered first. If the inconsistency is still not achieved after the rollback or an irreversible risk is determined, the system is locked and enters the next round. The log audit module associates and records evidence, conflict set, threshold, action and verification result to form an evidence chain.

2. The new energy production automation monitoring system based on non-intrusive AI according to claim 1, characterized in that, The set of interface anchor points includes static anchor points and dynamic anchor points. Static anchor points include at least any one of the following: page title area, menu area, table area, and alarm list area. Dynamic anchor points include at least any one of the following: pop-up window, color status change, icon flashing, and limit violation mark. View positioning and area constraints use the anchor point combination matching result as the boundary condition of the valid reading area.

3. The new energy production automation monitoring system based on non-intrusive AI according to claim 1, characterized in that, The structured extraction includes at least text and number reading extraction, table key-value pair extraction, alarm entry extraction, and trend feature extraction. For each reading, a corresponding reading source identifier and view identifier are generated for cross-view consistency verification of the same state quantity.

4. The new energy production automation monitoring system based on non-intrusive AI according to claim 1, characterized in that, The threshold band and cross threshold correspond to different state quantity categories. The discrete consistency criterion is used for alarm state quantities, the relative deviation criterion is used for continuous numerical state quantities, and the morphological similarity criterion is used for trend state quantities, so as to uniformly generate the conflict set under the same algorithm framework.

5. The new energy production automation monitoring system based on non-intrusive AI according to claim 1, characterized in that, The review action sequence is generated by the review action template set provided by the process orchestration module. The review action template set includes at least two of the following: refreshing the page, switching to the details view, zooming in on the key area, switching to the backup view, and repositioning the target area. The review action sequence terminates when the conflict set is empty or the confidence level reaches the threshold.

6. The new energy production automation monitoring system based on non-intrusive AI according to claim 1, characterized in that, Each template in the verifiable action template set includes a precondition, an operation sequence, a postcondition, and a prohibition condition; the prohibition condition includes at least one of the following: the conflict set is not empty, the confidence level is below the threshold, and the view positioning fails, so as to ensure that the action package is generated only under the premise of satisfying the triple consistency constraint.

7. The new energy production automation monitoring system based on non-intrusive AI according to claim 1, characterized in that, The action package is divided into low-risk actions, medium-risk actions and high-risk actions according to the risk level of the action. High-risk actions include at least one of alarm confirmation or silencing, action conclusion confirmation and work order submission. The generation and execution of high-risk actions must meet a confidence threshold higher than that of low-risk actions and meet the cross-view consistency test.

8. The new energy production automation monitoring system based on non-intrusive AI according to claim 1, characterized in that, The expected afterimage verification rules include at least one or more of the following: alarm status bit change, key parameters falling into the expected range, work order number generation, event record addition, and page status identifier change. The expected afterimage verification rules are bound one-to-one with verifiable action templates for action consistency determination.

9. The new energy production automation monitoring system based on non-intrusive AI according to claim 1, characterized in that, The rollback or lock includes undoing the operation, returning to the security page, restoring the default filtering conditions, stopping subsequent actions and locking the process, or any one or more of these actions. The threshold band, cross threshold, and number of consecutive frames N are adaptively updated based on the error distribution of the failed verification samples, and the log audit module records the parameters before and after the update and the corresponding evidence chain to support playback reproduction.

Images