AI video hierarchical monitoring system for new energy station

Abstract

The invention discloses an AI video hierarchical monitoring system for a new energy station, and relates to the technical field of artificial intelligence. The system comprises a new energy station end, a regional centralized control end and a headquarter monitoring end. The new energy field station end comprises a plurality of inspection devices and AI edge devices, the inspection devices are connected with the AI edge devices, the inspection devices are used for obtaining inspection videos of the new energy field station, and the AI edge devices are used for carrying out identification processing on the inspection videos through a preset new energy field station inspection identification model to obtain inspection result data of the new energy field station; the inspection result data is sent to the regional centralized control terminal and the headquarter monitoring terminal; the regional centralized control terminal is used for auditing the inspection result data to obtain an inspection auditing result and sending the inspection auditing result to the headquarters monitoring terminal; and the headquarter monitoring terminal is used for training and updating the new energy station inspection identification model according to the inspection audit result and the inspection result data. According to the invention, real-time, accurate and efficient monitoring of the new energy station is realized.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to an AI video hierarchical monitoring system for new energy power plants. Background Technology

[0002] New energy power plants refer to facilities that use renewable energy sources (such as solar, wind, geothermal, and tidal energy) to produce electricity or other forms of energy. They are usually equipped with corresponding power generation equipment, such as solar photovoltaic panels and wind turbines, to achieve a clean and sustainable energy supply, which helps reduce dependence on fossil fuels and reduce greenhouse gas emissions.

[0003] With the rapid development of new energy power stations, their safety management has become increasingly important. Currently, new energy power stations typically rely on traditional manual inspection methods for safety protection, depending on manual monitoring and post-event analysis.

[0004] However, manual monitoring is inefficient and makes it difficult to achieve real-time, accurate and efficient monitoring of new energy power plants. Summary of the Invention

[0005] In view of the shortcomings of the above-mentioned background technology, the purpose of this invention is to provide an AI video hierarchical monitoring system for new energy power plants, which can solve the technical problem of how to monitor new energy power plants in real time, accurately and efficiently.

[0006] To achieve the above objectives, this invention provides an AI video hierarchical monitoring system for new energy power stations, comprising a new energy power station terminal, a regional control terminal, and a headquarters monitoring terminal. The new energy power station terminal includes multiple inspection devices and AI edge devices. The inspection devices are connected to the AI ​​edge devices. The inspection devices are used to acquire inspection videos of the new energy power station. The AI ​​edge devices are used to process the inspection videos using a preset new energy power station inspection recognition model to obtain inspection result data of the new energy power station, and send the inspection result data to the regional control terminal and the headquarters monitoring terminal. The regional control terminal is used to review the inspection result data, obtain the inspection review result, and send the inspection review result to the headquarters monitoring terminal. The headquarters monitoring terminal is used to train and update the new energy power station inspection recognition model based on the inspection review result and the inspection result data.

[0007] By adopting the above technical solution, a three-tiered architecture consisting of the new energy power station terminal, the regional centralized control terminal, and the headquarters monitoring terminal is used to achieve hierarchical monitoring of new energy power stations. At the new energy power station terminal, AI edge devices and various inspection devices are deployed to collect and analyze on-site data in real time. AI models are used to identify and process inspection videos, automatically detecting safety hazards. This data and analysis results are then sent to the regional centralized control terminal, which is responsible for reviewing and initially processing this information, and reporting necessary data to the headquarters monitoring terminal. The headquarters monitoring terminal is responsible for formulating the overall monitoring strategy and continuously optimizing the AI ​​model. This multi-level monitoring approach ensures rapid information flow and processing, improves the real-time performance and accuracy of monitoring, and enhances monitoring efficiency through intelligent means. It overcomes the limitations of traditional manual monitoring in achieving comprehensive coverage and continuous monitoring, realizing real-time, accurate, and efficient monitoring of new energy power stations.

[0008] In some implementations, the AI ​​edge device is also used to determine the corresponding inspection safety event and the event reporting level based on the inspection result data, and to report the inspection safety event to the regional control terminal or the headquarters monitoring terminal according to the event reporting level. The event reporting level of the headquarters monitoring terminal is higher than that of the regional control terminal.

[0009] The technical solution adopted in the above embodiments improves the timeliness and pertinence of safety incident handling. By intelligently judging inspection results through AI edge devices, the system automatically determines safety incidents and their reporting levels, enabling it to quickly report urgent and high-risk incidents to higher-level monitoring centers. This automated hierarchical reporting mechanism not only improves the response speed of safety management but also optimizes resource allocation, ensuring that monitoring centers at all levels can promptly handle corresponding safety incidents according to their responsibilities and authority, thereby enhancing the ability of new energy power plants to respond to emergencies. In some implementations, the regional control terminal is used to upload the inspection and review results to the headquarters monitoring terminal according to preset reporting conditions.

[0010] The technical solution adopted in the above embodiments improves the utilization efficiency of inspection data and the decision-making quality of the monitoring center. The regional centralized control terminal uploads the reviewed inspection results to the headquarters monitoring terminal according to preset reporting conditions, enabling the headquarters to obtain key information in a timely manner and provide more macro-level decision support. This process ensures smooth information transmission, strengthens collaboration between superiors and subordinates, improves the operational efficiency of the entire monitoring system, and makes safety management decisions more scientific and accurate.

[0011] In some implementations, the inspection equipment includes at least one of video surveillance cameras, tracked robots, and safety and fire protection equipment.

[0012] The technical solutions employed in the above embodiments enhance the comprehensiveness and accuracy of inspection data. By utilizing various inspection equipment such as video surveillance cameras, track-mounted robots, and safety and fire-fighting equipment, new energy power plants can be monitored from different angles and dimensions, ensuring data diversity and integrity. This helps AI edge devices more comprehensively identify and analyze potential safety risks, improving the reliability and early warning capabilities of the monitoring system.

[0013] In some implementations, the new energy power station inspection and identification model is specifically used to identify the smoke size, flame color, and flame size from inspection videos, and to determine whether a fire safety incident has occurred at the new energy power station.

[0014] The technical solutions employed in the above embodiments improve the ability to identify and respond to fire safety incidents. The AI ​​inspection and identification model is specifically optimized for fire-related features, such as smoke intensity, flame color, and flame size, enabling the system to quickly and accurately determine whether there are potential fire hazards at the site. This targeted model enhances the sensitivity and accuracy of fire detection, facilitating timely firefighting measures and reducing losses caused by fires.

[0015] In some implementations, the new energy power station inspection and identification model is also used to identify from inspection videos whether the staff of the new energy power station are wearing safety equipment and work clothes correctly.

[0016] The technical solution adopted in the above embodiments improves the efficiency of supervising workers' personal protective equipment (PPE). The AI ​​inspection and recognition model, by analyzing inspection videos, can automatically identify whether workers are correctly wearing safety equipment and work clothing. This intelligent monitoring method not only improves the coverage and accuracy of supervision but also helps to correct violations in a timely manner, reducing the risk of accidents caused by personnel not wearing protective equipment as required.

[0017] In some implementations, the new energy power station inspection and identification model is also used to identify, from inspection videos, any unsafe or illegal actions taken by staff at the new energy power station.

[0018] The technical solution adopted in the above embodiments strengthens the safety management of worker behavior. The AI ​​inspection and recognition model can identify dangerous or illegal actions by workers, such as improper climbing and electric shock behaviors, and issue timely alarms. This real-time monitoring and analysis capability helps prevent accidents, improves the safety level of on-site operations, and protects the lives of workers.

[0019] In some implementations, the new energy power station inspection and identification model is also used to identify the identity information of the personnel leaving the power station from the inspection video, and to determine whether there is any unauthorized personnel illegally intruding into the new energy power station based on the electronic fence and identity information of the new energy power station.

[0020] The technical solutions described above enhance the physical security and access control of new energy power stations. The AI-powered inspection and identification model, combined with an electronic fence system, can accurately identify and determine the identity information of personnel at the station, promptly detecting unauthorized intrusions. This intelligent monitoring technology effectively improves the security management level of the station and prevents potential threats from illegal intrusions.

[0021] In some implementations, the new energy power station inspection and identification model is also used to determine whether the staff at the new energy power station are working normally based on inspection videos and preset work orders.

[0022] The technical solution adopted in the above embodiments improves the monitoring capability of operational compliance. The AI ​​inspection and recognition model, combined with work order information, can determine whether workers are performing operations according to established plans and safety procedures. This intelligent monitoring method helps to promptly detect operational deviations, ensure operational compliance, and reduce risks caused by violations.

[0023] In some implementations, a consensus knowledge base is also included, which is preset at the new energy power station, the regional control center, and the headquarters monitoring center. The AI ​​edge device is also used to: compare the identification results obtained by the new energy power station inspection and identification model with the consensus knowledge base to generate semantic descriptors for describing the core features of the event; and calculate the information value of the semantic descriptors, and according to the preset information value threshold, send the semantic descriptors with high information value as inspection result data to the regional control center and the headquarters monitoring center.

[0024] By adopting the technical solution of the above embodiments, based on the semantic communication paradigm of information value entropy, massive video data is transformed into semantic information with high decision-making value for transmission, which reduces network bandwidth consumption and manual review burden by orders of magnitude, and realizes efficient and low-cost flow of monitoring information.

[0025] In some implementations, the new energy power station inspection and identification model includes multiple sub-models and an AI scheduling and fusion module. The multiple sub-models include an environmental safety sub-model, a personnel compliance sub-model, a behavior analysis sub-model, and an identity recognition sub-model. The AI ​​scheduling and fusion module is used to schedule and manage the multiple sub-models and to logically integrate and understand the identification results returned by the multiple sub-models.

[0026] The technical solution adopted in the above embodiments deconstructs a single recognition model into a sophisticated architecture of "perception layer + cognition layer". This not only utilizes parallel specialized sub-models to achieve comprehensive and efficient perception of the site environment, personnel, and behavior, but more importantly, through the AI ​​scheduling and fusion module—the "central brain"—it deeply integrates and logically reasons with these fragmented perception information and business rules, achieving a qualitative leap from simple target recognition to complex event-level cognition and decision-making. This significantly improves the intelligence, accuracy, and automation level of monitoring.

[0027] In some implementations, the regional control terminal is used to estimate the maintenance time and maintenance items of new energy equipment in new energy power stations based on inspection results data.

[0028] The technical solutions adopted in the above embodiments improve the predictability and foresight of new energy equipment maintenance. By analyzing inspection data, the regional centralized control terminal can estimate equipment maintenance time and items. This data-driven predictive maintenance strategy helps optimize maintenance plans, reduce unexpected downtime, and improve the operational efficiency and reliability of new energy power plants. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the architecture of an AI video hierarchical monitoring system for new energy power stations according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the architecture of another AI video hierarchical monitoring system for new energy power stations according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the architecture of a new energy power station inspection and identification model in an embodiment of the present invention; Figure 4 This is a schematic diagram of the overall architecture of an AI video hierarchical monitoring system for new energy power stations according to an embodiment of the present invention. Detailed Implementation

[0030] The terminology used in the following embodiments of the present invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the specification of the invention, the singular expressions “a,” “an,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in the invention refers to any or all possible combinations comprising one or more of the listed items.

[0031] It should also be noted that, unless otherwise explicitly specified and limited, the terms "setting" and "connection" in the embodiments of the present invention should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be a connection within two components; it can be a wired communication connection or a wireless communication connection. Those skilled in the art can understand the specific meaning of the above terms in the present invention according to the specific circumstances. The embodiments of the present invention will be described in detail below.

[0032] This invention provides an AI video hierarchical monitoring system for new energy power plants, such as... Figure 1 As shown, it includes the new energy power station terminal 1, the regional centralized control terminal 2, and the headquarters monitoring terminal 3.

[0033] The new energy power station terminal 1 includes multiple inspection devices 12 and AI (artificial intelligence) edge devices. The inspection devices 12 are connected to the AI ​​edge devices 11. The inspection devices 12 are used to acquire inspection videos of the new energy power station. The AI ​​edge devices 11 are used to identify and process the inspection videos through a preset new energy power station inspection recognition model to obtain the inspection result data of the new energy power station, and send the inspection result data to the regional control terminal 2 and the headquarters monitoring terminal 3.

[0034] Regional centralized control terminal 2 is used to review the inspection result data, obtain the inspection review result, and send the inspection review result to the headquarters monitoring terminal 3.

[0035] The headquarters monitoring terminal 3 is used to train and update the inspection and identification model of new energy power stations based on the inspection and audit results and inspection result data.

[0036] This embodiment achieves hierarchical monitoring of new energy power stations through a three-tier architecture: new energy power station terminal 1, regional centralized control terminal 2, and headquarters monitoring terminal 3. At the new energy power station terminal 1, multiple inspection devices 12 collect on-site video data. This data is transmitted to an AI edge device 11, which analyzes the video using a preset recognition model, extracts inspection result data, and sends it to the regional centralized control terminal 2 and the headquarters monitoring terminal 3. The regional centralized control terminal 2 reviews this inspection result data, generates inspection review results, and further reports them to the headquarters monitoring terminal 3. The headquarters monitoring terminal 3 then trains and updates the AI ​​recognition model based on the review results and the original data to optimize the model's accuracy and efficiency. This hierarchical monitoring approach improves the response speed and processing efficiency of safety management. Automated AI analysis reduces human interference and enhances monitoring accuracy. The review mechanism of the regional centralized control terminal 2 allows for secondary verification of the data from the new energy power station terminal 1, enhancing the reliability of the monitoring results. The headquarters monitoring terminal 3, through continuous training and updating of the model, ensures the system can adapt to constantly changing environments and needs, improving the system's adaptability and foresight.

[0037] Overall, this multi-level monitoring approach ensures rapid information flow and processing, improves the real-time nature and accuracy of monitoring, and enhances monitoring efficiency through intelligent means. It overcomes the limitations of traditional manual monitoring in achieving comprehensive coverage and continuous monitoring, enabling real-time, accurate, and efficient monitoring of new energy power plants.

[0038] In this embodiment, the AI ​​edge device 11 is a smart hardware deployed at the network edge, i.e., near the data source. It integrates artificial intelligence technology and is capable of performing real-time data processing, analysis, and decision-making without sending data to a remote data center or cloud. These devices typically possess a certain level of computing power, enabling them to run machine learning algorithms and deep learning models, thereby quickly analyzing data locally, identifying patterns, anomalies, or trends, and responding instantly. The use of the AI ​​edge device 11 can reduce data transmission latency, improve data processing speed, enhance privacy protection, and provide necessary computing power in situations where the network is unstable or unable to connect to the cloud.

[0039] The inspection and identification model for new energy power plants can be trained using machine learning and deep learning techniques. The training process begins with collecting a large amount of video data from power plant inspections, including normal operating conditions and various potential safety incident scenarios. Then, annotation tools are used to carefully annotate this video data to identify and classify different objects and events, such as smoke, flames, unauthorized intrusion, and equipment malfunctions. With this annotated data, supervised learning algorithms can be used to train the model, enabling it to identify and distinguish different safety incidents.

[0040] During training, the model continuously adjusts its internal parameters to minimize the discrepancy between the predicted results and the actual labels. Through repeated training and validation, the model's accuracy and robustness gradually improve. Furthermore, data augmentation techniques, such as rotation, scaling, or adding noise, can be used to increase the diversity of training data and enhance the model's generalization ability.

[0041] Once the model achieves satisfactory performance on the training set, it is tested on the validation set to evaluate its performance on unseen data. Finally, the fully trained and validated model is deployed to the AI ​​edge device 11 for real-time analysis of inspection videos and identification of potential security incidents. To maintain the model's continued effectiveness, it also needs to be regularly updated and fine-tuned using the latest inspection data to adapt to changes in the site environment and operating conditions.

[0042] In some embodiments, such as Figure 2 As shown, the inspection equipment 12 includes at least one of a video surveillance camera 121, a track robot 122, and a safety and fire protection device 123.

[0043] Among them, video surveillance cameras 121 are an important component of the inspection equipment 12. They are deployed in key locations at new energy power plants, such as near wind turbines and solar panel arrays, to capture real-time activities and environmental conditions within the plant. These cameras typically have high-definition resolution and night vision capabilities, ensuring clear images under various lighting conditions and providing raw data for image analysis by the AI ​​edge device 11.

[0044] The track robot 122 is another type of inspection device 12. It can move autonomously on a preset track to conduct close-up inspections of equipment within the station. The track robot can be equipped with sensors and cameras, enabling it to inspect areas that are difficult for humans to reach or that are dangerous, collect equipment status information and potential safety hazards, and transmit the data to the AI ​​edge device 11 for further analysis.

[0045] Safety and fire protection equipment 123 refers to equipment used for monitoring and preventing safety accidents such as fires, including smoke detectors, temperature sensors, flame detectors, fire alarm equipment (including audible and visual alarms), and can be equipped with high-definition cameras or intelligent fire monitors for fire monitoring and response. These devices can monitor environmental parameters within the site in real time. Once an anomaly is detected, such as smoke, high temperature, or flame, they will immediately trigger an alarm and send the alarm signal to the AI ​​edge device 11. In this way, the system can quickly take emergency measures, such as activating the fire extinguishing system or notifying on-site personnel to evacuate, to protect personnel safety and the integrity of equipment.

[0046] In some embodiments, the AI ​​edge device 11 is further configured to determine the corresponding inspection safety event and the event reporting level based on the inspection result data, and report the inspection safety event to the regional control terminal 2 or the headquarters monitoring terminal 3 according to the event reporting level, wherein the event reporting level of the headquarters monitoring terminal 3 is higher than the event reporting level of the regional control terminal 2.

[0047] Specifically, the AI ​​edge device 11, by integrating image recognition and machine learning algorithms, can perform in-depth analysis of video data acquired from the inspection device 12, and monitor and identify various potential safety risks and anomalies in real time. For example, it can identify smoke and flames, determine their size and color, thereby detecting potential fire hazards; identify whether workers are wearing safety helmets and work clothes correctly, ensuring that they meet safety dress requirements; and monitor electronic fences and human characteristics to automatically detect security incidents such as unauthorized personnel intrusion.

[0048] Based on this, the AI ​​edge device 11 can automatically assign a reporting level to each identified security event according to preset rules and risk assessment standards. These rules and standards can be customized and adjusted according to the specific circumstances of the site and security management needs. For example, a slight smoke may be classified as a low-risk event and only needs to be reported to the regional control terminal 2; while a serious fire or a large-scale illegal intrusion is classified as a high-risk event and needs to be reported to the headquarters monitoring terminal 3 immediately.

[0049] Furthermore, the incident reporting level of headquarters monitoring terminal 3 is typically higher than that of regional centralized control terminal 2. This means that headquarters monitoring terminal 3 is responsible for handling more serious and urgent security incidents, while some relatively minor incidents are handled by regional centralized control terminal 2. This hierarchical reporting mechanism not only ensures that important security incidents are quickly known to senior management and that corresponding emergency measures are taken, but also improves the efficiency and response speed of the entire security monitoring system. This allows managers at all levels to make corresponding emergency responses and decisions quickly based on the severity of the incident, thereby greatly improving the safety management level and risk prevention and control capabilities of new energy power plants.

[0050] In some embodiments, the regional control terminal 2 is used to upload the inspection and review results to the headquarters monitoring terminal 3 according to preset reporting conditions.

[0051] Specifically, the regional control terminal 2 can collect inspection result data from various new energy power station terminals 1 and perform preliminary review and analysis. The regional control terminal 2 has a set of preset reporting conditions and rules, which may be set based on the type, severity, frequency of occurrence, or other key indicators of the event. For example, if a power station reports similar safety events multiple times in a row, or if the severity of an event exceeds a set threshold, the regional control terminal 2 will automatically mark these inspection review results as high priority and upload them to the headquarters monitoring terminal 3 in real time.

[0052] In addition, the regional control terminal 2 can decide whether cross-regional coordination is needed based on the nature and scope of the event, and report the relevant information to the headquarters monitoring terminal 3 so that the headquarters monitoring terminal 3 can have a comprehensive understanding of the situation and make higher-level decisions and responses.

[0053] This intelligent reporting mechanism based on preset conditions not only improves the efficiency of information transmission, but also ensures that important and urgent security incidents can be quickly noticed by senior management and dealt with in a timely and effective manner.

[0054] In some embodiments, the new energy power station inspection and identification model can be used to identify the smoke size, flame color and flame size from inspection videos, and determine that there is a fire safety incident at the new energy power station.

[0055] Specifically, the inspection and identification model for new energy power stations uses deep learning algorithms, especially convolutional neural networks (CNN), to identify the smoke size, flame color, and flame size from inspection videos.

[0056] The inspection and identification model is first trained on a large amount of video data containing various fire scenarios to learn the characteristics of smoke and flames. During training, the model analyzes visual information such as color, texture, shape, and motion patterns in video frames to identify the presence of smoke and flames. For example, the model can be trained to recognize the gray or black color of smoke and the yellow, orange, or red color of flames. Furthermore, the model learns to assess the spread and intensity of smoke and flames to determine the size of the fire. Once the model detects these features in the video, and their intensity and range exceed preset thresholds, it determines that a fire safety incident has occurred at the renewable energy power station and triggers corresponding alarms and emergency response measures to quickly contain the fire and protect the safety of personnel and equipment.

[0057] This AI-based fire detection method is faster and more accurate than traditional manual monitoring, greatly improving the safety management level of new energy power plants.

[0058] In some embodiments, the new energy power station inspection and identification model can also be used to identify from inspection videos whether the staff of the new energy power station are wearing safety equipment and work clothes correctly.

[0059] Specifically, the new energy power station inspection and identification model uses deep learning technology, especially image recognition algorithms, to accurately identify whether staff are wearing safety equipment and work clothes correctly from inspection videos.

[0060] During the training phase, the model utilizes a large amount of labeled video data, containing both correct and incorrect examples of workers wearing various safety equipment and clothing. Using this data, the model learns to recognize the specific colors, shapes, and patterns of personal protective equipment (PPE) such as helmets, safety belts, protective suits, safety glasses, earplugs, gloves, and safety shoes. In practical applications, the model analyzes each frame of the video stream, detecting the worker's attire and equipment, and determining whether it meets pre-set safety standards. If it detects that a worker is not wearing the required equipment, the model immediately issues an alarm, alerting management to take corrective measures, thereby ensuring the safety of on-site operations.

[0061] This AI-based intelligent monitoring method greatly improves the efficiency and accuracy of supervising the wearing of personal protective equipment by staff.

[0062] In some embodiments, the new energy power station inspection and identification model is also used to identify, from inspection videos, any dangerous or illegal actions taken by staff at the new energy power station.

[0063] Specifically, the new energy power station inspection and identification model uses computer vision and deep learning technologies to analyze and identify the movement patterns of staff from inspection videos in order to detect whether there are any dangerous or illegal actions.

[0064] During the model training phase, a large amount of video data labeled with safe and dangerous actions is used, covering various work scenarios and action types. Using this data, the model learns to identify standard actions that comply with safety procedures and dangerous actions that could lead to accidents, such as improper climbing, slipping, electric shock, and falls from heights. In addition, the model also learns to identify behaviors that violate operating procedures, such as unauthorized operations and improper use of tools. In practical applications, the model analyzes the actions of workers in the video stream in real time, quickly identifying any abnormal or dangerous actions by matching and comparing them with safety action templates learned during training. Once a potential safety risk is detected, the model immediately triggers an alarm, notifies on-site management personnel, or automatically initiates emergency response measures to prevent accidents. This AI-based video analytics technology significantly improves the efficiency and speed of monitoring worker behavior, providing strong support for the safety management of new energy power plants.

[0065] In some embodiments, the new energy power station inspection and identification model is specifically used to identify the identity information of the personnel leaving the power station from the inspection video, and to determine whether there is any unauthorized personnel illegally intruding into the new energy power station based on the electronic fence and identity information of the new energy power station.

[0066] Specifically, the new energy power station inspection and identification model integrates facial recognition technology and behavior analysis algorithms, enabling it to accurately identify the identity information of personnel present in inspection videos.

[0067] The model is first trained on a large amount of video data containing the facial features and identity information of different employees to learn to recognize each person's unique facial features. In practical applications, the model analyzes facial images of people in the video stream in real time and quickly identifies the individuals present by comparing them with employee facial data stored in the database. Simultaneously, the model analyzes the movement trajectories and behavioral patterns of personnel to determine whether they are active within the designated work area. Combined with the electronic fence system deployed at the new energy power station, which monitors and records personnel entry and exit, the model can further determine whether personnel have obtained the appropriate access permissions. If unauthorized personnel are detected entering a restricted area or the electronic fence alarm indicates illegal intrusion, the model will immediately trigger an alarm and notify security management personnel to intervene.

[0068] This AI-based video analytics and identity verification method significantly improves the accuracy of personnel identification monitoring and the speed of response to unauthorized intrusion incidents at new energy power plants, thereby effectively enhancing the physical security and access control capabilities of the plants.

[0069] In some embodiments, the new energy power station inspection and identification model is also used to determine whether the staff of the new energy power station are working normally based on the inspection video and the preset work order.

[0070] A work permit is a formal document widely used in industries such as manufacturing, power, and construction, especially for high-risk operations such as equipment repair, maintenance, or construction. It details the planned work content, time, location, personnel involved, safety measures, and emergency plans to ensure the safe and orderly conduct of the work. Work permits typically require a strict approval process and are only valid after being signed and confirmed by the relevant responsible person. During the work, the work permit serves as the basis for on-site safety supervision and execution; any deviation from the work permit's stipulations should be recorded and evaluated to ensure the safety of workers and the quality of the work. The use of work permits helps prevent accidents, improves work efficiency, and provides important evidence for post-operation analysis and auditing.

[0071] In this embodiment, the new energy power station inspection and identification model determines whether workers are performing their duties correctly by combining computer vision technology and preset work order information. First, the AI ​​edge device needs to access the work order system to obtain information such as planned work tasks, work areas, and workers within a specific time period.

[0072] The model then analyzes video streams from renewable energy power plants, using advanced target detection and behavior recognition algorithms to monitor and track the location and activities of workers in real time. These algorithms can identify workers' shapes, movements, and positions relative to equipment within the plant, thus understanding their movement paths and the type of work they are performing. The system's built-in database stores details of the tasks scheduled on the work order, including the work content, required safety measures, work location, and time. By matching and comparing real-time data obtained from video analysis with the information on the work order, the model can determine whether workers are performing the scheduled tasks at the correct time and place and whether they are following all necessary safety procedures. If a worker's behavior deviates from the plan on the work order, such as working in an unauthorized area or performing unplanned tasks, the model will flag this deviation and trigger an alarm, notifying monitoring personnel for further inspection and intervention. This intelligent monitoring method significantly improves the safety and compliance of the work site, reducing the risk of safety accidents caused by human error or negligence.

[0073] For example, if a work order specifies that a worker should be on a wind turbine for maintenance at a particular time, but video analysis shows that the worker is not present at the designated location or is not performing the relevant tasks, the model will flag this deviation.

[0074] In addition, the model can identify potential safety violations, such as not wearing appropriate personal protective equipment or using unauthorized tools. Once any deviation or violation is detected, the system will immediately notify management so that corrective actions can be taken to ensure compliance and safety of operations.

[0075] This method, based on video analytics and work order verification, provides an automated means of monitoring operations at new energy power plants, helping to improve operational efficiency and safety.

[0076] In some embodiments, AI edge devices can also apply NLP (Natural Language Processing) technology to analyze the conversations of staff in inspection videos, identify whether they are discussing safety procedures, and whether they are following the safety instructions on the work permit.

[0077] Among its features, the new energy power station inspection and identification model, employing natural language processing technology, can conduct in-depth analysis of staff conversations. It converts dialogues into text data through speech recognition, and then utilizes a pre-set keyword and phrase library, along with contextual analysis capabilities, to identify whether the conversation involves discussions of safe operating procedures. The model can capture and identify terms and instructions related to safety procedures, such as "safety helmet," "emergency evacuation," or "work permit number," and compare them with the safety measures specified on the work permit to determine whether the staff's communication complies with safe operating requirements.

[0078] In addition, natural language processing technology can not only analyze the content of staff conversations, but also assess the tone and urgency of the conversations through tone analysis and emotion recognition.

[0079] Therefore, the new energy power station inspection and identification model can identify whether there are expressions of anxiety, tension, or urgency in conversations. For example, it can determine whether there are potential risks or emergencies on site by observing faster speech, increased volume, or the use of specific urgent vocabulary. The model monitors these subtle changes in language and tone, and combines them with contextual information, such as known risks in a specific work area and the current task, to determine whether immediate action is needed. If an urgent tone or expression of urgency is detected, the new energy power station will automatically trigger an alarm, notifying management personnel to intervene, thereby quickly responding to potential safety incidents and ensuring the safety of staff and the smooth progress of operations. This assessment of tone and urgency allows the monitoring system to more accurately capture real-time dynamics on site, improving the ability to respond to emergencies.

[0080] It should be noted that the new energy power station inspection and identification model in this embodiment, as understood by those skilled in the art, does not narrowly refer to a single, end-to-end neural network model, but should be broadly understood as a unified AI algorithm system or algorithm cluster that implements the aforementioned multiple identification functions. Given that the identification tasks covered in this embodiment (such as fire detection, personnel attire, behavior analysis, identity recognition, and operational compliance judgment) belong to different sub-tasks in the field of computer vision, and the technical implementation paths of each individual sub-task are mature and diverse in current artificial intelligence engineering practice, this embodiment also covers these implementation methods.

[0081] One feasible implementation approach is a "multi-model deployment and result fusion" architecture. In this approach, the new energy power station inspection and identification model includes multiple specially trained and highly optimized sub-models, as well as an AI scheduling and fusion module.

[0082] The sub-model can specifically include: The environmental safety sub-model employs target detection architectures such as YOLOv8 or Faster R-CNN, specifically designed for real-time identification of static or dynamic targets such as smoke and flames from video streams.

[0083] The personnel compliance sub-model is also based on the object detection architecture, but its training data and optimization objectives focus on detecting wearable items such as safety helmets, work clothes, and seat belts from human images.

[0084] The behavior analysis sub-model may adopt an architecture such as OpenPose combined with time series networks (such as LSTM). First, it extracts human key points (pose estimation), and then analyzes the time series of key points to identify dangerous or illegal actions such as climbing, falling, and waving.

[0085] The identity recognition sub-model uses specialized face recognition networks such as ArcFace or FaceNet to extract facial feature vectors and compare them with a database to confirm the identity of the person.

[0086] These sub-models can process the input video stream in parallel and output their respective recognition results to the AI ​​scheduling and fusion module. The AI ​​scheduling and fusion module is responsible for performing advanced logical integration and scene understanding on the atomic recognition information from different sub-models. The AI ​​scheduling and fusion module plays a key role as the "central brain" or "on-site commander" in this system. It does not directly execute the underlying image recognition tasks, but is responsible for scheduling and managing the various sub-models and performing logical integration and scene understanding on the atomic and fragmented recognition results they return.

[0087] In some embodiments, the functions of the AI ​​scheduling and fusion module include, but are not limited to: task assignment, i.e., activating the corresponding sub-model for analysis based on current needs or video content; information fusion, i.e., aggregating the identification tags, coordinates, confidence scores, and other information output by all sub-models; and logical reasoning, i.e., piecing together these multi-source information into a complete and business-meaning event description based on preset business rules, security procedures, and interactions with external data systems (such as work ticket systems), and making a final decision.

[0088] The powerful capabilities of the AI ​​scheduling and fusion module are particularly evident in handling complex tasks such as "determining the compliance of operations based on work orders": First, the module will interface with an external work order database to obtain real-time work plans, such as "Zhang San (identity) should perform equipment inspection tasks (actions) in the No. 2 transformer area (location) from 10:00 to 11:00 am (time) and must wear a safety helmet (wearing)".

[0089] Then, the "AI Scheduling and Fusion Module" will collect and analyze the outputs from each sub-model in real time: It identifies the person currently in the area of ​​transformer No. 2 as "Zhang San" from the identity recognition model.

[0090] It determines from the output of the behavior analysis model that the action being performed by the person matches "equipment inspection".

[0091] It obtained from the personnel compliance model that the person was indeed wearing a safety helmet.

[0092] At the same time, it combines the system time to confirm that the current time is between 10:00 AM and 11:00 AM.

[0093] Through cross-validation and logical reasoning of the information from multiple sources and multiple modalities, the AI ​​scheduling and fusion module can ultimately determine that "the staff is working normally according to the preset work order." Conversely, if any link is mismatched, such as incorrect time, wrong location, incorrect personnel identity, failure to wear safety equipment, or execution of unplanned dangerous actions, the AI ​​scheduling and fusion module will immediately determine that the operation is abnormal and generate a corresponding alarm.

[0094] In this embodiment, a specific workflow example is as follows: The new energy power station inspection and identification model uses its internal sub-model components to initially identify specific elements (such as people, objects, and actions) in the video. Subsequently, the AI ​​scheduling and fusion module performs more complex judgments, a process particularly evident in handling the task of "judging the compliance of work based on work orders." First, the AI ​​scheduling and fusion module will interface with an external work ticket database to obtain real-time work plans.

[0095] Next, the AI ​​scheduling and fusion module will gather atomic information from other sub-models, such as: the identity recognition model reporting "Zhang San detected", the behavior analysis model reporting "action is inspection", and the personnel compliance model reporting "safety helmet is worn".

[0096] Finally, the AI ​​scheduling and fusion module performs cross-validation and logical reasoning, comparing this multi-source, multi-modal information with the rules obtained from the work order database. This series of complex internal processes ultimately enables the entire new energy power station inspection and identification model to make the final judgment: "The staff is working normally according to the preset work order."

[0097] Conversely, if any link is mismatched, such as incorrect time, wrong location, or mismatched personnel identity, the "AI scheduling and fusion module" will complete the abnormality logic judgment, and then the new energy power station inspection and identification model will output an "operational abnormality" alarm result.

[0098] To further clarify the internal collaboration mechanism of the new energy power station inspection and identification model in this embodiment, please refer to [link / reference]. Figure 3 . Figure 3 The internal functional architecture of the model under a preferred implementation is shown, which embodies the technical concept of moving from "multidimensional perception" to "intelligent cognition".

[0099] like Figure 3 As shown, the overall workflow of the new energy power station inspection and identification model can be divided into two core levels: (1) Parallelized Multidimensional Perception Layer: After the video stream is input as raw data, it is first distributed to multiple parallel, highly specialized sub-models for preliminary processing. These sub-models constitute the system's "perception organs," each responsible for analyzing the scene from different dimensions: Environmental safety sub-model: focuses on identifying physical safety hazards in the scene and outputs environmental status information such as "smoke / flame recognition results".

[0100] Personnel compliance sub-model: Focuses on analyzing personnel dress codes and outputs results such as "wearing compliance identification results" to determine whether they are wearing safety helmets, work clothes, etc.

[0101] Behavioral analysis sub-model: Focuses on identifying the dynamic behavior of the human body. Through posture estimation and temporal analysis, it outputs "action behavior recognition results" to determine whether there are dangerous or illegal actions such as climbing or falling.

[0102] Identity Recognition Sub-model: Focuses on confirming the identity information of individuals and outputs accurate "identity recognition results".

[0103] These sub-models work in parallel, ensuring that the system can efficiently extract atomic information about multiple dimensions such as environment, clothing, behavior, and identity from the same video simultaneously.

[0104] (2) Unified Intelligent Cognition and Decision-Making Layer: All atomized recognition results output from the perception layer are uniformly integrated into the core of the system—the AI ​​scheduling and fusion module. This module constitutes the "central brain" and cognitive decision-making core of the system. At this level, more advanced intelligent processing is performed: Integrating multi-source information: The AI ​​scheduling and fusion module not only integrates real-time visual recognition results from all internal sub-models, but more importantly, it can also actively interact with the external "work ticket database" to obtain non-visual structured business data such as work plans and safety procedures.

[0105] Performing cross-validation and logical reasoning: This is the key innovation of this invention. This module performs cross-validation and deep logical reasoning on real-time perceived multi-dimensional information (who, where, what, and whether the attire is compliant) with preset business rules (when, where, what, and how they should wear the attire).

[0106] Output the final decision result: After the above processing, the system no longer outputs a bunch of scattered labels, but forms a final identification result with clear business significance after scene understanding and logical judgment, such as "Zhang San is working normally in the No. 2 transformer area according to the work ticket" or "Unauthorized person Li Si was detected to be performing dangerous climbing operations in the restricted area".

[0107] In summary, Figure 3 The architecture shown clearly reveals the advantages of this embodiment: through the decoupled design of the "perception layer + cognition layer" described above, multiple dedicated AI recognition capabilities are deeply integrated with business logic rules, achieving a qualitative leap from simple "target recognition" to complex "event understanding and compliance judgment." This closed-loop processing mechanism, which automates logical reasoning of multi-source real-time visual information and structured business data, is the key technical support for this embodiment to achieve truly "intelligent, accurate, and efficient" monitoring of new energy power plants.

[0108] Another feasible implementation is the "Multi-Task Learning" architecture. In this approach, the system employs a unified, complex model with a shared backbone network (such as a Vision Transformer or a large convolutional network) and multiple separate task heads. The shared backbone network is responsible for extracting low-level visual features common to all tasks, while each independent task head focuses on a specific output. Even with this architecture, a similar logical processing unit is ultimately needed to integrate the outputs of the task heads and interact with external information systems such as work orders to complete complex compliance judgments.

[0109] In some embodiments, the regional control terminal 2 is used to estimate the maintenance time and maintenance items of new energy equipment in the new energy power station based on the inspection result data.

[0110] Specifically, the regional control terminal 2 analyzes the inspection results data uploaded from the AI ​​edge device 11 at the new energy power station terminal 1. Using data analysis and machine learning algorithms, it can predict the maintenance time and maintenance items for the new energy equipment. This data includes equipment performance indicators, fault records, wear and tear, environmental factors, etc. Based on this information, combined with the equipment's historical maintenance records and maintenance patterns of similar equipment, the system at the regional control terminal 2 builds a predictive model. This model can identify trends in equipment performance degradation, predict potential fault points, and recommend maintenance times and specific maintenance items accordingly, such as replacing worn parts, system calibration, and software updates.

[0111] In this way, the regional control terminal 2 can not only plan maintenance work in advance and reduce the risk of unexpected downtime, but also optimize the allocation of maintenance resources and improve the operating efficiency and reliability of new energy power plants.

[0112] The AI ​​video hierarchical monitoring system in this embodiment aims to improve the safety management level of new energy power stations through advanced artificial intelligence technology. The system consists of three levels: new energy power station terminal 1, regional centralized control terminal 2, and headquarters monitoring terminal 3, forming a complete monitoring and response system.

[0113] The new energy power station 1 is equipped with various inspection devices 12, including video surveillance cameras 121, track robots 122, and safety and fire protection equipment 123. These devices are responsible for collecting video and environmental data within the station. AI edge devices 11 perform real-time analysis of the received data, using pre-set recognition models to detect safety events such as fires, personnel violations, and unauthorized intrusions, and determine the reporting level based on the severity of the event. This design enables the station to respond quickly to various emergencies and take timely measures to prevent accidents from occurring or escalating.

[0114] Regional control terminal 2 is responsible for reviewing inspection results data from various new energy power plants and uploading the review results to headquarters monitoring terminal 3 according to preset reporting conditions. This process not only ensures timely information transmission but also improves the efficiency and accuracy of data processing. In addition, regional control terminal 2 can also estimate equipment maintenance time and items based on inspection data, realizing predictive maintenance, reducing unexpected downtime, and improving equipment operating efficiency and reliability.

[0115] The headquarters monitoring terminal 3 is responsible for higher-level monitoring and decision-making, handling high-level security incidents reported by the regional centralized control terminal 2, and training and updating the AI ​​recognition model based on the inspection and review results. This continuous learning and optimization mechanism enables the system to adapt to constantly changing environments and needs, improving the accuracy and efficiency of monitoring.

[0116] Overall, this embodiment achieves comprehensive monitoring of new energy power plants through intelligent means, improving the initiative and preventative nature of safety management. It can not only promptly detect and respond to various safety incidents, but also predict potential risks through data analysis, realizing a shift from passive defense to proactive prevention. Furthermore, the system considers equipment maintenance and optimization, reducing downtime through predictive maintenance and improving the operational efficiency of new energy power plants. This integrated approach, combining AI technology, the Internet of Things, and big data analytics, provides an innovative solution for the safety management of new energy power plants, with broad application prospects and significant practical value.

[0117] In some embodiments, the AI ​​edge device 11 further includes a digital twin simulator and an adversarial virtual sample generator; the digital twin simulator is used to simulate the operating conditions of the new energy power station; the adversarial virtual sample generator is used to generate virtual samples that are difficult for the new energy power station inspection and identification model to identify or are easily confused in the simulated operating conditions of the digital twin simulator, and to use the virtual samples to perform localized autonomous fine-tuning of the new energy power station inspection and identification model, so as to improve the model's generalization ability to cope with unknown scenarios.

[0118] This embodiment constructs a digital twin and adversarial evolution mechanism at the edge, enabling the AI ​​model to have predictive self-evolution capabilities that precede the occurrence of real risks, greatly improving the system's rapid response and autonomous adaptation to unknown and new security threats.

[0119] Specifically, an integrated autonomous evolution module is deployed in each AI edge device 11 at the new energy power station. This module works in collaboration between a digital twin simulator and an adversarial virtual sample generator.

[0120] First, the digital twin simulator is a high-fidelity virtual software environment built using historical operational data of the site, 3D geographic information, equipment physical models, and meteorological environmental parameters. This digital twin simulator can simulate various foreseeable operating conditions and environmental changes within the site in real time or at accelerated speed. Examples include the reflection state of photovoltaic panels under different lighting angles and cloud cover, the blade speed and vibration patterns of wind turbines at different wind speeds, and the interference effects of severe weather such as rain, snow, and fog on video acquisition. This provides a zero-risk, low-cost sandbox environment for model testing and evolution.

[0121] Secondly, the adversarial virtual sample generator is an intelligent agent based on generative adversarial networks (GANs) or reinforcement learning algorithms. The core task of the adversarial virtual sample generator is not to randomly generate data, but to identify the cognitive weaknesses of the deployed new energy power station inspection and identification model by continuously probing its decision boundaries.

[0122] In the digital twin simulator, the adversarial virtual sample generator selectively generates highly challenging virtual samples that have the lowest model recognition confidence, are most easily confused with normal samples, or are completely beyond the model's current knowledge scope. For example, it generates a pseudo-image that combines the flickering frequency of a flame and the reflective texture of water to challenge the model's robustness. After acquiring these high-value virtual samples, the system initiates a localized autonomous fine-tuning process. This localized autonomous fine-tuning is an online learning mechanism that uses these virtual samples as new training data to make small, rapid adjustments to specific level weights of the new energy power station inspection and identification model deployed on the current AI edge device. This process does not aim for a complete reconstruction of the model, but rather, like patching, precisely enhances the model's generalization ability to cope with such unknown or ambiguous scenarios, thereby enabling the model to have predictive self-evolutionary capabilities before real risks occur, without relying on headquarters intervention.

[0123] In some embodiments, the system further includes a consensus knowledge base preset in the new energy power station terminal, the regional control terminal, and the headquarters monitoring terminal; the AI ​​edge device is specifically used to: compare the identification results obtained by the new energy power station inspection and identification model with the consensus knowledge base to generate a semantic descriptor describing the core features of the event; and calculate the information value of the semantic descriptor, and according to a preset information value threshold, send the semantic descriptor with high information value as the inspection result data to the regional control terminal and the headquarters monitoring terminal.

[0124] This embodiment adopts a semantic communication paradigm based on information value entropy to transform massive video data into semantic information with high decision-making value for transmission. This reduces network bandwidth consumption and manual review burden by orders of magnitude, and realizes efficient and low-cost flow of monitoring information.

[0125] Specifically, a unified, cross-level consensus knowledge base is constructed and deployed, and a data communication and processing mechanism based on semantics is established on this basis.

[0126] The consensus knowledge base is a structured, multimodal database that pre-stores standardized definitions and feature models of all known objects, events, states, and their interrelationships within a renewable energy power station. For example, the consensus knowledge base includes digital descriptions of entities and their attributes such as "Model A inverter," "standard attire for maintenance personnel," and "infrared thermal image distribution of equipment during normal operation."

[0127] At new energy power stations, after the AI ​​edge device performs preliminary analysis of inspection videos using the new energy power station inspection and recognition model, its output is no longer a simple pixel coordinate or classification label, but rather enters a semantic encoding process. In this process, the AI ​​edge device performs deep comparison and matching between the recognition results and entries in the consensus knowledge base, abstracting and transforming the raw, high-dimensional visual information into a set of refined, machine-readable semantic descriptors. A semantic descriptor is a data structure containing multiple key-value pairs, such as {Event Type: Suspected Smoke; Location: Transformer B Zone; Size: Small, Color Feature Vector: [...], Motion Trajectory Model: [...], Confidence: 0.65}.

[0128] After generating semantic descriptors, the system proceeds to calculate information value. Information value is a quantitative indicator used to assess the importance and novelty of an event or piece of information to the recipient's decision-making. The calculation of information value integrates multiple dimensions, including the rarity of the event (i.e., the reciprocal of its frequency of occurrence in the consensus knowledge base), the uncertainty of the identification result (i.e., the confidence level of the model output), and the risk level of the context in which the event occurs (for example, the information value of a spark occurring in a fire-restricted area is much higher than that of a spark occurring in a welding area).

[0129] AI edge devices only use semantic descriptors whose information value exceeds a preset information value threshold as the final inspection result data, and send them to the regional control terminal and the headquarters monitoring terminal via the network.

[0130] This implementation method ensures that the core of the system transmission is no longer the redundant data itself, but rather the "essence of information" that has been intelligently refined and has high decision-making value, fundamentally resolving the contradiction between data storms and network bandwidth and the efficiency of manual review.

[0131] In this embodiment, the consensus knowledge base serves as the infrastructure for enabling semantic communication and cross-level intelligent collaboration. Its construction is a systematic and multi-stage project. This consensus knowledge base is not a simple database, but a structured knowledge system that integrates ontology, knowledge graphs, and multimodal pre-trained model technologies. It aims to provide AI models with a standardized, machine-readable understanding framework for the physical world and business processes of new energy power plants.

[0132] The construction of a consensus knowledge base begins with the knowledge extraction and modeling phase. The core task of this phase is defining the ontology of knowledge, that is, clarifying the types of objects (entities) to be described within this domain, their attributes, and the relationships between them. Data sources include, but are not limited to: national and industry standards documents for the new energy industry, technical manuals and operation and maintenance procedures provided by equipment manufacturers, site design drawings (CAD drawings), historical accident reports, and the experience and knowledge of expert operation and maintenance personnel. Through Natural Language Processing (NLP) technology, key entities, such as "photovoltaic inverter," "wind turbine blade," and "35kV circuit breaker," are automatically extracted from these unstructured and semi-structured texts, along with their technical parameter attributes (such as rated power, size, and normal operating temperature range) and their relationships (such as "located in," "connected to," and "belongs to"). This extracted knowledge is organized into a knowledge graph and stored in the form of "entity-relationship-entity" or "entity-attribute-value" triples.

[0133] Secondly, there is the stage of association and fusion of multimodal features. To ensure the knowledge base not only contains textual descriptions but can also be understood by visual AI models, it's necessary to associate entities in the knowledge graph with multimodal data. This stage involves collecting and labeling a large amount of multimodal data, including images, videos, infrared thermal images, and audio. For example, the entity "Model A Inverter" in the knowledge graph is bound to hundreds of optical images of that inverter model under different lighting conditions, angles, and cleanliness levels, infrared thermal images under normal and abnormal operating conditions, and audio spectrum data during operation. This process utilizes a large-scale multimodal pre-trained model (such as a variant of the CLIP model) to learn the deep mapping relationship between textual descriptions and visual / auditory features, thereby generating a "multimodal feature vector" for each entity that can be used for cross-modal retrieval and comparison. After this stage, the consensus knowledge base possesses the ability to bidirectionally connect abstract semantics (such as device names) with concrete perceptual features (such as device appearance).

[0134] Finally, there is the deployment and dynamic updating phase of the knowledge base. The completed consensus knowledge base is not static; its core components are compiled and compressed into a lightweight version, pre-installed on the servers of AI edge devices at each new energy power station, regional control terminals, and headquarters monitoring terminals, ensuring that all levels of the system have a unified "cognitive" benchmark for the same thing. Furthermore, the consensus knowledge base is designed with a dynamic update mechanism. When new equipment models, new operating procedures, or when the AI ​​model discovers new, undefined anomaly patterns through continuous learning, this new knowledge can be incrementally added to the main knowledge base at headquarters after manual review and simultaneously distributed to all nodes. Through this systematic construction and dynamic maintenance, the consensus knowledge base acts like a living, system-wide shared "encyclopedia," providing a solid foundation for achieving accurate semantic encoding, efficient predictive coding communication, and unambiguous collaboration between intelligent agents.

[0135] In some embodiments, to further improve system communication efficiency, a silent communication mechanism based on neuroscience predictive coding theory can be introduced to transform the traditional active reporting or polling communication mode into an efficient, anomaly-driven prediction error communication mode. The core of this silent communication mechanism is to transform the regional control terminal or headquarters monitoring terminal (hereinafter collectively referred to as the central terminal) from a passive data receiver into an active environmental state predictor.

[0136] In practice, the implementation of this silent communication mechanism includes the following steps: First, a prediction model with the same origin or similar structure as the new energy power station inspection and identification model on the AI ​​edge device at the new energy power station is deployed at the central end. The prediction model at the central end will use the global information it has, including but not limited to historical inspection data, the digital twin model of the power station, weather forecast data, and preset work plans (such as work tickets), to continuously and frequently (e.g., on a second or sub-second basis) generate "predictive feature maps" of the visual scene of each monitoring point at the power station at the next moment. This "predictive feature map" is not a pixel-level image, but a high-dimensional vector representation in a shared, pre-trained deep feature space, which can abstractly describe the key elements and states of the scene.

[0137] Secondly, at the new energy power station, the new energy power station inspection and identification model on the AI ​​edge device also generates an "actual feature map" describing the current real scene after analyzing the inspection video in real time. The key step is that the AI ​​edge device does not immediately report this "actual feature map". Instead, the AI ​​edge device receives the "predicted feature map" generated by it at the central end for the corresponding time and calculates the difference or error between the "actual feature map" and the "predicted feature map" locally. This error is called the "prediction error vector".

[0138] In most cases, when the site operates normally and as expected, the "actual feature map" and the "predicted feature map" closely match, and the norm or information entropy of the calculated "prediction error vector" will be lower than a preset quiescent threshold. Under these circumstances, the AI ​​edge device will remain "silent" and will not generate any uplink data communication, thereby greatly saving network bandwidth and energy consumption.

[0139] Only when an unexpected anomaly occurs at the site, such as sudden smoke, unplanned personnel intrusion, or subtle but significant changes in the equipment's appearance, will the "actual feature map" deviate significantly from the "predicted feature map." In this case, the calculated "prediction error vector" will exceed a threshold. Once this threshold is exceeded, the AI ​​edge device will be activated for communication, but it will not upload complete scene information or video; instead, it will only upload this highly condensed "prediction error vector" containing the "unexpected" information.

[0140] Finally, upon receiving the "prediction error vector," the central terminal can overlay it with its own "prediction feature map," thereby reconstructing the anomaly occurring at the depot with extremely high efficiency and accuracy, and triggering subsequent review, alarm, and decision-making processes. Through this silent communication mechanism, the system's data interaction volume is completely decoupled from the depot's normal operating status, and is only positively correlated with the frequency and intensity of abnormal events. This achieves ultimate optimization of communication resources and makes it possible to deploy high-dimensional, real-time intelligent monitoring systems under extreme network conditions.

[0141] In some embodiments, the new energy power station inspection and identification model deployed on the AI ​​edge device is a student model, and the headquarters monitoring terminal also deploys a teacher model; the specific method by which the headquarters monitoring terminal trains and updates the student model based on the inspection and audit results and the inspection result data is as follows: using the teacher model, a soft tag containing rich guidance information is generated through a knowledge distillation method, and the soft tag is combined with the inspection and audit results to train and update the student model.

[0142] This embodiment leverages knowledge distillation and a teacher-student model architecture to enable a lightweight student model deployed on resource-constrained edge devices to efficiently absorb the deep knowledge and reasoning capabilities of a powerful teacher model at headquarters. This solves the fundamental contradiction that a unified model is difficult to adapt to diverse site environments with extremely high cost-effectiveness.

[0143] Specifically, an asymmetric, clearly defined teacher-student model architecture is constructed, and knowledge distillation technology is used to migrate and empower high-level knowledge from the headquarters to lightweight models at the edge. In this architecture, the teacher model deployed at the headquarters monitoring end is one or more large-scale, complex foundational models trained using massive amounts of data and supercomputing power. This teacher model possesses strong generalization ability and a deep understanding of complex scenarios, capable of inferring potential causal relationships from subtle features, but it cannot be directly deployed at the edge due to its enormous computational resource consumption.

[0144] In contrast, the new energy power station inspection and identification models deployed on the AI ​​edge devices of various new energy power stations are defined as student models. Student models are lightweight models designed to adapt to the limited computing and storage resources of edge devices; their structure is relatively simple, and their initial cognitive capabilities are limited. The core innovation of the system lies in the specific method of training and updating the model at the headquarters monitoring end.

[0145] When the regional control terminal aggregates the inspection and review results (i.e., hard labels with precise annotations that have been manually confirmed) and the corresponding inspection result data (such as feature vectors or low-confidence samples uploaded from the edge side) to the headquarters monitoring terminal, the headquarters monitoring terminal does not directly use these hard labels to retrain the student model. Instead, the headquarters monitoring terminal first inputs this new data into the teacher model, utilizing the teacher model's powerful reasoning ability for processing. The output of the teacher model is not a simple classification result (such as "is it a flame" or "is not a flame"), but a soft label containing rich guidance information. The soft label is a probability distribution vector that not only indicates the correct answer but also reveals the teacher model's "thinking process" and "knowledge structure" across all possible categories. For example, the teacher model might output "95% probability is a flame, but there is also a 4% chance it is the reflection of red silk cloth because its flickering frequency is slightly lower than that of a typical flame."

[0146] Subsequently, the headquarters monitoring system combined this soft label, which embodies the "wisdom" of the teacher model, with the inspection and review results (hard labels) confirmed by manual review, to form a composite training objective with extremely high information density.

[0147] Finally, using this composite training objective, the student model is trained and updated through an optimized knowledge distillation algorithm. This implementation method enables the student model to not only know what (hard labels) but also why (soft labels) during learning, thereby absorbing the deep knowledge of the teacher model with extremely high efficiency and achieving a leap in performance to cope with the unique and complex environment of each site.

[0148] In some embodiments, the system is not limited to passive monitoring and information reporting; it can also achieve closed-loop active intervention of physical equipment, thus forming a complete autonomous emergency response system. To achieve this function, the software architecture deployed in the AI ​​edge devices at the new energy power station further integrates an AI commander model. The system registers various controllable inspection and security devices within the station, such as tracked robots, video surveillance cameras with gimbals, drones, intelligent fire monitors, audible and visual alarms, and intelligent access control systems, as schedulable agents within a multi-agent system. The AI ​​commander model is a decision-making agent trained offline in a digital twin simulator matched to the power station using reinforcement learning algorithms. This digital twin simulator presets various high-risk emergency scenarios, such as fire spread, sudden equipment failure, and unauthorized personnel intrusion. During training, the AI ​​commander model aims to learn how to generate optimal action sequence instructions in different scenarios to command and schedule the collaborative work of multiple agents within the station, thereby resolving simulated crises with the fastest speed and lowest risk.

[0149] In practice, when the new energy power station inspection and identification model detects a high-risk safety event, such as identifying an electric arc in a transformer area, this event information will serve as a trigger signal to directly activate the AI ​​commander model locally. After receiving initial parameters such as the event type, location, and severity, the AI ​​commander model will immediately execute a series of autonomous decision-making and dispatch instructions based on pre-learned strategies. For example, the AI ​​commander model may first issue an instruction to dispatch the tracked robot closest to the incident point and control the tracked robot to switch to an infrared thermal imaging sensor to quickly approach the target for secondary precise confirmation and data collection. After confirming the fire, the AI ​​commander model will link with the security system to activate the audible and visual alarms in the incident area to warn the surrounding area. At the same time, based on the precise fire source coordinates transmitted back by the tracked robot, it will automatically calculate the trajectory and instruct the intelligent fire monitor to carry out precise fire extinguishing operations. Throughout the entire autonomous handling process, the status of all intelligent agents, the instructions executed, and the handling results will be recorded in real time and reported to the regional centralized control terminal and the headquarters monitoring terminal in the form of structured summaries, along with key video clips, for remote management personnel to supervise, take over, or conduct post-event analysis.

[0150] In this way, the system tightly couples the three links of perception, decision-making and physical execution, realizing a qualitative change from "passively seeing problems" to "actively solving problems", and endowing new energy power stations with the core capabilities of second-level response and unmanned autonomous handling of emergencies.

[0151] In some embodiments, the application scope of this system extends from security and compliance monitoring to refined analysis and optimization of production efficiency. To achieve this function, the system deeply integrates and merges AI edge devices with the process control systems (such as SCADA / DCS systems) of new energy power plants through data interfaces. This allows the AI ​​model to not only acquire visual information but also simultaneously read related key performance indicators (KPIs), such as the real-time power generation of photovoltaic arrays, gearbox temperature of wind turbines, equipment vibration frequency, and power quality data. Based on this data fusion, the system aims to build a digital process twin, whose core task is no longer limited to identifying isolated safety events but rather learning and understanding the deep causal relationship between "visual features" and "production efficiency."

[0152] In practical implementation, the new energy power plant inspection and identification model undergoes targeted training to identify subtle visual features that affect production efficiency. For example, the model can accurately quantify the coverage of minute dust accumulation, bird droppings, or localized shading on the surface of photovoltaic panels using high-resolution images. It then correlates this visual data in real time with the power generation attenuation data of the photovoltaic array collected by the SCADA system, thereby establishing a dynamic mathematical model between the degree of visual pollution and the loss of power generation efficiency. Based on this model, the system can continuously calculate the amount of power generation loss caused by pollution and compare it with the cost of calling in cleaning equipment or personnel. This allows for the automatic generation and dispatch of cleaning work orders when the return on investment (ROI) is highest.

[0153] In another application scenario, this system can integrate the sound spectrum of equipment operation captured by acoustic sensors and the subtle vibration patterns of equipment captured by video surveillance, and perform correlation analysis with process data such as lubricant consumption rate and bearing temperature. Through long-term learning, the AI ​​model can discover a high correlation between specific "sound and vibration" visual patterns and early potential equipment failures (such as bearing wear and poor lubrication). Once such a precursor pattern is detected, the system will trigger a predictive maintenance alarm. This alarm not only estimates the timing of maintenance but also, based on the pattern type, can infer with high probability the specific component and cause of the failure, thereby greatly improving the accuracy of maintenance work and the efficiency of spare parts preparation, upgrading the traditional safety monitoring system into an intelligent optimization engine that drives productivity improvement.

[0154] Figure 4 This embodiment demonstrates the overall architecture of the AI ​​video hierarchical monitoring system, which is divided into three levels: headquarters monitoring terminal, regional centralized control terminal, and new energy power station terminal. Each level contains different devices and systems, forming a complete energy monitoring, analysis, and management network. a. Headquarters Monitoring Terminal Inspection system client: Headquarters operators monitor and manage the inspections of various sites through the client.

[0155] Inspection system database server: Used to store inspection data, supporting long-term data storage, retrieval, and analysis.

[0156] b. Regional control center side Inspection system client: Operators at the regional control center can also manage and monitor the inspection status of new energy power stations within the region through the client.

[0157] Inspection system database server: There is also an inspection data storage server on the regional control center side, which can speed up data access within the region.

[0158] Streaming media server: The regional control center also has video stream management and transmission functions.

[0159] c. New energy power station side Inspection system client: The station's operators use the client to conduct on-site monitoring and inspections.

[0160] AI edge devices: used to handle data analysis tasks within the site.

[0161] Station monitoring host: Used to control and manage all monitoring equipment and systems within the site.

[0162] Storage devices: Used for local storage of video data and other important monitoring data.

[0163] Video compression host: Used to compress video data to reduce the bandwidth used during transmission.

[0164] Inspection equipment, including: Video surveillance system: including various camera devices (high-definition, panoramic, meter reading cameras, etc.) for video surveillance of the site.

[0165] Tracked robot system: Tracked robots and wireless APs, etc., are used for automated inspection.

[0166] Online temperature measurement system: High-precision dual-channel temperature measurement equipment, temperature measuring pan-tilt unit, and infrared thermal imager, etc., are used for real-time temperature monitoring of equipment and environment.

[0167] Security and fire protection system: including facial recognition cameras, vehicle recognition, perimeter security cameras, etc., used for site security protection, and also includes fire alarm equipment.

[0168] This embodiment demonstrates a multi-level monitoring and management system from headquarters to the power plant, encompassing various functions such as video surveillance, robotic inspection, online temperature measurement, and security protection. Through these devices and systems, intelligent management and operation of new energy power plants can be achieved, ensuring their safe and efficient operation.

[0169] In this embodiment, each new energy power station deploys AI edge devices with corresponding capabilities based on the number of cameras. These AI edge devices are connected to the local area network and interconnected with the cameras. The AI ​​edge devices need to perform four main functions: 1. Read the camera video stream in real time, perform AI recognition, and upload the recognition alarm results to the data center server.

[0170] 2. Responsible for carrying out inspection tasks according to the area requirements, namely, retrieving video information from inspection cameras (or inspection robots), automatically analyzing the corresponding meter and equipment status information, and generating inspection reports.

[0171] 3. Provides collection of video data analysis from law enforcement recorders, and can perform on-site analysis of model annotation and algorithms.

[0172] 4. Upload the analysis results of the aforementioned functions to the regional control center and the new energy headquarters.

[0173] The cloud server is configured with a three-tier management architecture, corresponding to the site level, regional branch office level, and headquarters level. Different device and alarm management permissions are configured for different levels: the site level can only manage the devices and alarms within its own site, the branch office level can manage the devices and alarms of all subordinate sites, and the headquarters level can manage the devices and alarms of all sites and branch offices.

[0174] In terms of architecture, the AI ​​video recognition system includes a robust and flexible interface layer to ensure efficient communication and interaction between components. These interfaces must be based on standardized protocols (such as RESTful APIs, gRPC, etc.) and should support high-concurrency and low-latency data processing. The interface layer design should describe the following characteristics: Modular design: Interfaces should follow modular design principles, allowing each functional module to be developed, tested, and deployed independently. Furthermore, interfaces should be independent of each other so that replacing or upgrading individual modules in the future does not affect the overall system functionality.

[0175] Scalability: The design takes into account the need for future system function expansion, ensuring that the interface is easily extensible. For example, a plug-in architecture allows for the rapid addition of new data processing functions or support for new types of data.

[0176] Security: All interfaces must be designed with security mechanisms, such as authentication, authorization control, and data encryption, to ensure the security and integrity of data transmission.

[0177] The three-tier architecture of this embodiment is not a simple stacking of functions or a division of management levels, but rather forms a synergistic mechanism that deeply couples data governance and model iteration, resulting in an unexpected leap in overall technical performance. Specifically, this synergistic effect is reflected in: 1. The regional control terminal plays a crucial role as a "high-quality data filter" and a "domain knowledge accumulation node." Traditional "edge-cloud" direct connection architectures face the dilemma of massive amounts of edge data being difficult to verify and having low value density. Directly using this "dirty data" for headquarters model training not only consumes enormous communication bandwidth and computing resources but may even lead to model performance degradation. This invention introduces a regional control terminal, utilizing its professional personnel to review and transform the raw "inspection result data" identified by the front-end AI into "inspection review results" with precise manual annotations and business logic confirmation. This process is not merely simple information verification but also an efficient, distributed knowledge extraction process. It makes the implicit operational experience of front-line sites explicit and structured, providing continuous, high-quality "nutrients" for headquarters model training.

[0178] 2. "Precise targeted optimization" and "efficient closed-loop iteration" of model training were achieved. The headquarters monitoring terminal receives high-value data that has been filtered and refined by the regional control terminals. This makes model training no longer a "needle in a haystack," but rather enables precise and targeted optimization of the model's performance weaknesses in specific stations and scenarios. For example, when multiple regional control terminals report misjudgments regarding the icing situation of a certain type of equipment, headquarters can quickly gather these high-quality negative samples and efficiently train a more robust recognition model. The optimized model is then distributed to the new energy station terminals, improving their front-end recognition accuracy and thus reducing the review burden on the regional control terminals.

[0179] 3. It resolves the inherent contradiction between edge real-time performance and centralized intelligence. The new energy power station focuses on local real-time alarms and rapid response, ensuring immediate on-site safety; the headquarters monitoring station focuses on using global data for deep learning and strategy optimization, improving the system's long-term intelligence level. The regional centralized control station acts as a bridge, cleverly decoupling real-time data streams from training data streams through its "review-reporting" mechanism, enabling the entire system to "see quickly and react quickly," and "learn accurately and evolve rapidly."

[0180] In summary, the three-tier architecture of this embodiment forms a self-evolving system of "perception-cognition-decision-optimization" through functional complementarity between each tier and a refined closed loop of data flow. This synergistic effect—the improved data quality, significantly increased model training efficiency, and continuous optimization of overall system performance brought about by the architectural design—is unattainable with a simple two-tier architecture or when each part operates independently. This demonstrates the outstanding substantive features and significant progress of this invention.

[0181] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An AI-based video hierarchical monitoring system for new energy power plants, characterized in that, This includes the new energy power station terminal, the regional centralized control terminal, and the headquarters monitoring terminal; The new energy power station terminal includes multiple inspection devices and AI edge devices. The inspection devices are connected to the AI ​​edge devices. The inspection devices are used to acquire inspection videos of the new energy power stations. The AI ​​edge devices are used to identify and process the inspection videos through a preset new energy power station inspection recognition model to obtain the inspection result data of the new energy power stations, and send the inspection result data to the regional centralized control terminal and the headquarters monitoring terminal. The regional centralized control terminal is used to review the inspection result data, obtain the inspection review result, and send the inspection review result to the headquarters monitoring terminal; The headquarters monitoring terminal is used to train and update the new energy power station inspection and identification model based on the inspection and audit results and the inspection result data.

2. The system according to claim 1, characterized in that, The AI ​​edge device is also used to determine the corresponding inspection safety event and the event reporting level based on the inspection result data, and to report the inspection safety event to the regional control terminal or the headquarters monitoring terminal according to the event reporting level. The event reporting level of the headquarters monitoring terminal is higher than that of the regional control terminal.

3. The system according to claim 1, characterized in that, The regional centralized control terminal is used to upload the inspection and review results to the headquarters monitoring terminal according to preset reporting conditions.

4. The system according to claim 1, characterized in that, The inspection equipment includes at least one of video surveillance cameras, track robots, and safety and fire protection equipment.

5. The system according to claim 1, characterized in that, The new energy power station inspection and identification model is specifically used to identify the smoke size, flame color, and flame size from the inspection video to determine if there is a fire safety incident at the new energy power station; and to identify whether the staff at the new energy power station are wearing safety equipment and work clothes correctly from the inspection video.

6. The system according to claim 5, characterized in that, The new energy power station inspection and identification model is also used to identify, from the inspection video, any unsafe or illegal actions taken by the staff of the new energy power station; the new energy power station inspection and identification model is also used to identify the identity information of the personnel leaving the station from the inspection video, and, based on the electronic fence of the new energy power station and the identity information, to determine whether there is any unauthorized illegal intrusion into the new energy power station.

7. The system according to claim 5, characterized in that, The new energy power station inspection and identification model is also used to determine whether the staff of the new energy power station are working normally based on the inspection video and the preset work order.

8. The system according to any one of claims 1-7, characterized in that, It also includes a consensus knowledge base pre-set in the new energy power station terminal, the regional centralized control terminal, and the headquarters monitoring terminal; The AI ​​edge device is also used to: compare the identification results obtained by the new energy station inspection and identification model with the consensus knowledge base to generate a semantic descriptor for describing the core features of the event; and calculate the information value of the semantic descriptor, and according to a preset information value threshold, send the semantic descriptor with high information value as the inspection result data to the regional control terminal and the headquarters monitoring terminal.

9. The system according to claim 8, characterized in that, The new energy power station inspection and identification model includes multiple sub-models and an AI scheduling and fusion module. The multiple sub-models include an environmental safety sub-model, a personnel compliance sub-model, a behavior analysis sub-model, and an identity recognition sub-model. The AI ​​scheduling and fusion module is used to schedule and manage the multiple sub-models and to logically integrate and understand the identification results returned by the multiple sub-models.

10. The system according to claim 1, characterized in that, The regional centralized control terminal is used to estimate the maintenance time and maintenance items of new energy equipment in the new energy power station based on the inspection result data.

Images