Power grid target detection method and system based on size model cloud edge collaboration

Abstract

The application discloses a power grid target detection method and system based on a size model cloud edge cooperation, and relates to the technical field of power grid target detection. The method performs power grid target detection based on a cloud edge size model cooperation framework. The cloud edge size model cooperation framework comprises a lightweight detection model deployed on an edge end, a multi-modal large model deployed on a cloud end, and hierarchical cooperation across models through hierarchical routing, result review and semantic completion. The method comprises the following steps: acquiring a power grid image, wherein the power grid image comprises substation, power transmission line and power transmission tower related images; performing preliminary target detection based on the lightweight detection model of the edge end according to the power grid image to obtain a first detection result; and judging whether the power grid image needs to be further uploaded to the cloud end for detection and output by using a cloud edge cooperation scheduling module based on adaptive sustainable learning according to the confidence of the first detection result and the priority of the detection task. The application improves the power grid target detection accuracy.

Description

Technical Field

[0001] This invention relates to the field of power grid target detection technology, and specifically to a power grid target detection method and system based on cloud-edge collaboration using large and small models. Background Technology

[0002] With the continuous expansion of the power grid and the increasing dispersion of equipment, intelligent power grid inspection is gradually evolving from centralized computing to distributed intelligence through cloud-edge collaboration. While traditional cloud-based centralized inference models offer high model accuracy and multimodal understanding capabilities when processing large-scale image and video sensing data, they face challenges such as high latency and bandwidth bottlenecks in weak network environments. Meanwhile, edge devices offer advantages in real-time performance and proximity to data sources, but their ability to independently run complex machine learning models (such as large visual language models, VLMs) is limited by computing power, storage, and energy consumption. Therefore, in power grid inspection scenarios characterized by heterogeneous computing power and unstable networks, achieving efficient collaboration between high-precision cloud models and lightweight edge models has become a key technological challenge for the intelligent development of target detection in power grid inspection.

[0003] In view of the above, this application is hereby submitted. Summary of the Invention

[0004] The technical problem this invention aims to solve is the low accuracy of power grid target detection due to diverse tasks, heterogeneous computing power, or network limitations. The invention provides a power grid target detection method and system based on cloud-edge collaboration using large and small models. On one hand, it proposes a cloud-edge large-model collaboration mechanism based on the adaptive and sustainable learning cloud-edge collaborative scheduling module L-CSNet. This mechanism deploys lightweight detection models (such as YOLO models or lightweight large models) at the edge and large VLM models in the cloud. The traditional rule-based scheduling module is upgraded to the adaptive and sustainable learning cloud-edge collaborative scheduling module L-CSNet, and combined with an SLA (Service Level Agreement) grading mechanism that combines task semantics and real-time requirements. This allows the system to dynamically select the optimal cloud-edge collaboration strategy based on the current overall system operation and task priorities, significantly improving system performance and collaboration efficiency. This provides a cloud-edge collaboration solution with long-term autonomous evolution capabilities for complex and ever-changing power grid inspection scenarios. On the other hand, by combining cloud-edge collaborative inference mechanisms and optimizing the structure and early termination of large-scale multimodal models (such as VLM models), some feature extraction and preliminary semantic understanding can be pushed down to the edge, while only image features are transmitted to the cloud. This reduces the bandwidth pressure caused by simultaneous transmission of power grid images from multiple edge devices. Furthermore, cross-modal understanding is completed in the cloud, and the early termination strategy reduces redundant inference, thus lowering the inference cost of large models while ensuring the complete inference capability of VLM. In addition, in weak network environments, by introducing asynchronous collaboration and task priority scheduling (SLA hierarchical) mechanisms, inference paths can be flexibly selected according to alarm levels and task types (such as fire detection, foreign object identification, and routine inspections), further improving the robustness and real-time performance of the power grid inspection system.

[0005] This invention is achieved through the following technical solution:

[0006] Firstly, this invention provides a power grid target detection method based on cloud-edge collaboration using large and small models. This method performs power grid target detection based on a cloud-edge-large model collaboration framework. The framework includes a lightweight detection model deployed at the edge and a large multimodal model with strong computing power and multimodal semantic understanding capabilities deployed in the cloud. Hierarchical collaboration across models is achieved through hierarchical routing, result verification, and semantic completion, thus balancing the agile response of the lightweight detection model with the cognitive depth of the large multimodal model. The method includes:

[0007] Acquire images of the power grid, including images of substations, transmission lines, and transmission towers;

[0008] Based on the power grid image, preliminary target detection is performed using a lightweight detection model at the edge, and the first detection result is obtained.

[0009] Based on the confidence level of the first detection result and the priority of the detection task, a cloud-edge collaborative scheduling strategy based on adaptive sustainable learning is used to determine whether the power grid image needs to be further uploaded to the cloud for detection and output.

[0010] Furthermore, based on the power grid image, preliminary target detection is performed using a lightweight detection model at the edge, yielding the first detection result, including:

[0011] Based on the power grid image, a preliminary target detection is performed using a lightweight detection model at the edge, resulting in a first detection result. The first detection result represents the confidence level of the target detection, including high confidence, medium-low confidence, and extremely low confidence.

[0012] Furthermore, the detection task is divided according to an SLA grading mechanism that combines task semantics and real-time requirements, including:

[0013] Emergency tasks such as identifying equipment smoke and fire, and detecting cracks in insulator sheaths are classified as emergency alarm tasks P0.

[0014] Important tasks such as main transformer oil tank leakage detection and equipment overheating detection are classified as important inspection tasks P1;

[0015] Routine tasks such as indicator light and observation window inspections, and foreign object intrusion detection on equipment supports are classified as routine monitoring tasks (P2).

[0016] Furthermore, the cloud-edge collaborative scheduling module L-CSNet based on adaptive sustainable learning provides a comprehensive perception of the system's full-dimensional state, enabling intelligent decision-making for cloud-edge collaborative logic; the full-dimensional state of the system includes confidence distribution, task level, network state, and cloud-edge computing power.

[0017] The L-CSNet is trained offline using a rule-driven strategy as a prior. After offline training, it is continuously fine-tuned based on posterior feedback during system operation, giving the system the ability to adapt to different complex power grid scenarios.

[0018] Furthermore, the rule-driven strategy includes:

[0019] When the first detection result is of high confidence and the priority of the detection task is arbitrary, the first scheduling strategy is executed to directly complete the inference at the edge and output the first detection result, and the alarm information is reported to the front end.

[0020] When the first detection result is of medium to low confidence and the priority of the detection task is emergency alarm, the second scheduling strategy is executed to trigger the dual-channel system. The dual-channel trigger includes reporting warning information at the edge and parallel verification and confirmation in the cloud.

[0021] When the first detection result has a low to medium confidence level and the priority of the detection task is important inspection or routine monitoring, the third scheduling strategy is executed to upload the first detection result and key frames to a large model in the cloud for in-depth verification.

[0022] When the first detection result has an extremely low confidence level, it is considered that the probability of anomaly is low and no further processing is performed.

[0023] This method also includes: a cloud-edge collaborative inference mechanism based on VLM structure pruning, specifically:

[0024] The Visual Language Model (VLM) is split into shallow visual encoding and deep semantic reasoning:

[0025] The shallow visual encoding part at the front end is deployed at the edge to encode the power grid image into image features. Only the image features are transmitted to the cloud, reducing the bandwidth pressure caused by multiple edge ends transmitting the power grid image at the same time.

[0026] The latter completes cross-modal understanding in the cloud and combines a state difference-based early termination strategy to reduce redundant reasoning, ensuring the complete reasoning capability of VLM while reducing the reasoning cost of large models.

[0027] The early termination strategy involves inserting an early termination evaluation module between the decoder layers of the VLM. The early termination evaluation module determines whether the VLM model understanding is saturated by calculating the difference between the hidden states between layers. When the difference is less than a set threshold, the subsequent calculation is terminated in advance and the second detection result is directly output.

[0028] Furthermore, the cloud-based detection includes:

[0029] The image encoding output from the edge is obtained by cropping and compressing the power grid image that L-CSNet determines needs to be uploaded to the cloud for detection, and then processing it through shallow visual encoding of the visual language model VLM.

[0030] Based on the image encoding, secondary inference and further target detection are performed using a multimodal large model in the cloud to obtain a second detection result;

[0031] The lightweight detection model at the edge is trained based on the second detection result to obtain the parameters of the trained model. The trained parameters are then periodically updated to the lightweight detection model at the edge via network transmission.

[0032] Furthermore, the method also includes: network transmission between the edge and the cloud, specifically:

[0033] The system detects the network connectivity between the edge device and the cloud. If there are network fluctuations or disconnections, it reconnects to the network and retransmits the image encoding output from the edge device based on the retry strategy.

[0034] If retransmission fails, lightweight large model inference is performed at the edge for emergency alarm tasks; for important inspection or routine monitoring tasks, data is cached at the edge and retransmitted to the cloud periodically.

[0035] Furthermore, the network connectivity between the edge and the cloud is detected, including:

[0036] A dual mechanism, employing both active and passive methods, is used to detect network connectivity between the edge and the cloud.

[0037] The system employs both active and passive mechanisms: the active mechanism uses TCP Ping, HTTP health checks, and KubeEdge heartbeat to detect connection status in real time; the passive mechanism uses request success rate and response latency statistics to assist in the determination.

[0038] Secondly, this invention provides a power grid target detection system based on cloud-edge collaboration using large and small models. This system performs power grid target detection based on a cloud-edge-large model collaboration framework. The framework includes a lightweight detection model deployed at the edge and a large multimodal model with strong computing power and multimodal semantic understanding capabilities deployed in the cloud. Hierarchical collaboration across models is achieved through hierarchical routing, result verification, and semantic completion, thus balancing the agile response of the lightweight detection model with the cognitive depth of the large multimodal model. The system includes:

[0039] The acquisition unit is used to acquire power grid images, which include images of substations, transmission lines, and transmission towers, etc.

[0040] The first detection unit is used to perform preliminary target detection based on the power grid image and a lightweight detection model at the edge, and obtain the first detection result.

[0041] The second detection unit is used to determine, based on the confidence level of the first detection result and the priority of the detection task, whether the power grid image needs to be further uploaded to the cloud for detection and output using a cloud-edge collaborative scheduling module based on adaptive sustainable learning.

[0042] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0043] This invention relates to a power grid target detection method and system based on cloud-edge collaboration of large and small models. (1) The scheduling module is upgraded from the traditional rule execution module to an adaptive sustainable learning cloud-edge collaborative scheduling module L-CSNet, and combined with the SLA hierarchical mechanism that combines task semantics and real-time requirements. This enables it to dynamically select the optimal cloud-edge collaborative strategy based on the current overall system operation status and task priority, significantly improving system performance and collaborative efficiency. This provides a cloud-edge collaborative solution with long-term autonomous evolution capability for complex and ever-changing power grid inspection scenarios. (2) A large and small model collaborative framework is designed, which can deploy a lightweight detection model at the edge to achieve rapid response and local prediction. When L-CSNet determines that it needs to be uploaded to the cloud for re-inspection, the large cloud model is triggered to perform secondary inference, achieving a dynamic balance between accuracy and latency. This invention solves the problems of complex and ever-changing scenarios, diverse tasks, and heterogeneous computing power in power grid target detection scenarios, and improves the accuracy of power grid target detection. Attached Figure Description

[0044] The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and form part of this application, do not constitute a limitation thereof. In the drawings:

[0045] Figure 1 This is a diagram of the power grid inspection system architecture based on cloud-edge-size model collaboration of the present invention.

[0046] Figure 2 This is a diagram of the adaptive sustainable learning scheduling mechanism based on the L-CSNet module of this invention;

[0047] Figure 3 This is a flowchart of the power grid target detection method based on cloud-edge collaboration using a large-scale model, as described in this invention.

[0048] Figure 4 This is a block diagram of the power grid target detection system based on cloud-edge collaboration using a large-scale model, as described in this invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0050] To address the challenges of diverse tasks, heterogeneous computing power, limited network coverage, and low accuracy in power grid target detection scenarios, this invention designs a power grid target detection method, system, and medium based on cloud-edge collaboration using both large and small models. The overall system, named GridSCOPE, utilizes key technologies such as a hierarchical scheduling mechanism, a large-scale model collaboration framework, and a cloud-edge collaborative inference mechanism based on VLM structure pruning to construct an integrated cloud-edge-device intelligent sensing system for power grid inspection scenarios. Its core innovations are reflected in the following three aspects:

[0051] 1. The scheduling module is upgraded from the traditional rule execution module to the adaptive and sustainable learning cloud-edge collaborative scheduling module L-CSNet, and combined with the SLA hierarchical mechanism that combines task semantics and real-time requirements. This enables it to dynamically select the optimal cloud-edge collaborative strategy based on the current overall system operation status and task priority, significantly improving system performance and collaborative efficiency. This provides a cloud-edge collaborative solution with long-term autonomous evolution capability for complex and ever-changing power grid inspection scenarios.

[0052] 2. A collaborative framework for large and small models was designed, which can deploy lightweight detection models (such as YOLO models or lightweight large models) at the edge to achieve fast response and local prediction; when L-CSNet determines that it needs to be uploaded to the cloud for re-examination, the large model in the cloud is triggered to perform secondary inference, so as to achieve a dynamic balance between accuracy and latency.

[0053] 3. A cloud-edge collaborative inference mechanism based on VLM structure pruning was designed, splitting VLM into shallow visual encoding and deep semantic inference. The former is pushed down to the edge to extract compact features, only transmitting image features to the cloud, reducing the bandwidth pressure caused by simultaneous transmission of original images from multiple edge devices. The latter completes cross-modal understanding in the cloud and combines an early termination strategy to reduce redundant inference, thereby reducing the inference cost of large models while ensuring the complete inference capability of VLM.

[0054] In weak network scenarios, GridSCOPE further constructs a weak network collaborative mechanism that includes state awareness, retry rate limiting, model degradation, local caching, and breakpoint resumption. Through network state judgment, priority task guarantees, and data consistency maintenance, the system maintains robustness by ensuring that emergency tasks are not interrupted and ordinary tasks are not lost even under conditions of bandwidth fluctuations, long latency, or even network outages. Overall, GridSCOPE achieves a balance between real-time performance, cognitive capabilities, and resource efficiency by introducing hierarchical scheduling, prudent cloud-edge splitting, and adaptive collaborative inference, providing a complete architectural paradigm for building a scalable and highly reliable intelligent power grid inspection system.

[0055] Specifically, cloud-edge size model collaboration based on the adaptive continuous learning scheduling module L-CSNet:

[0056] To meet the dual requirements of real-time performance and accuracy for power grid inspection target detection tasks, such as Figure 1 As shown, this invention constructs a cloud-edge-large model collaborative framework. Lightweight detection models (such as YOLOv8n) are deployed at the edge to handle image-level real-time target detection and preliminary classification tasks. Larger models (such as VLM) with stronger computing power and multimodal semantic understanding capabilities are deployed in the cloud to perform deep analysis and semantic reasoning on complex, ambiguous, or high-risk scenarios. By adaptively and sustainably learning the cloud-edge collaborative scheduling module L-CSNet and combining it with an SLA-based hierarchical mechanism that considers both task semantics and real-time requirements, the system can dynamically select the optimal cloud-edge collaborative strategy based on the current overall system operation status and task priorities. This significantly improves system performance and collaborative efficiency, and provides the system with the ability to adapt to complex and ever-changing power grid inspection scenarios.

[0057] like Figure 1 and Figure 2 As shown, Figure 1 This is an architecture diagram of a power grid inspection system based on cloud-edge-scale model collaboration. Figure 2 This is a diagram of the cloud-edge collaboration mechanism based on the L-CSNet module in this invention.

[0058] Table 1 Lightweight Task Classifier and SLA Grading Mechanism

[0059]

[0060] Table 2 Joint Scheduling Strategy Based on Confidence Level and Task Grade

[0061]

[0062] At the decision-making level, this architecture integrates an SLA grading mechanism (as shown in Tables 1 and 2) with a dynamic, sustainable learning cloud-edge collaborative scheduling network (L-CSNet) to construct an end-to-end scheduling paradigm for task awareness, system state modeling, and intelligent decision-making. L-CSNet, as the decision-making core of the GridSCOPE architecture, breaks through the rigid limitations of traditional rule-based scheduling. It takes the full-dimensional operating state of the system as the modeling object, achieves low-latency inference through a lightweight network, and possesses adaptive capabilities for complex scenarios. Using rule-driven strategies as initial priors, it builds a basic model through offline training and continuously fine-tunes it based on posterior feedback during system operation, achieving a full lifecycle evolution of cold start, adaptation, and self-evolution. For the task characteristics of power grid inspection scenarios, the task classification and lightweight edge inference process are defined as follows:

[0063]

[0064]

[0065] in, The original image frame that was input; Task classifier, used to determine task priorities. Lightweight edge detection model. Preliminary results output. and confidence level .

[0066] The input to L-CSNet, used for system state-space modeling, is not a single image feature, but rather a time window. The system's full-dimensional state vector ,definition The system state set at time t is as follows:

[0067]

[0068] Each item is defined as follows: Edge model in window Statistical characteristics of confidence distribution within (including mean) ,variance (and confidence histogram distribution), reflecting the uncertainty of the current visual environment; Task SLA level embedding, represented using one-hot encoding. (urgent), (important), (Regular) Three types of task priorities; Network state feature vector, including uplink bandwidth Round-trip delay and its volatility ; Cloud load status, inferred from the cloud service's current queue length. Quantification; Edge computing power status, including CPU load. With memory usage This design integrates multi-dimensional system features such as edge confidence distribution, task priority, network status, cloud load, and edge computing power to construct a multi-dimensional mapping relationship between scenes and system states. L-CSNet can accurately quantify the uncertainty (scene difficulty) of the current visual environment and the real-time pressure on cloud and edge resources, achieving a leap from single feature judgment to global situational awareness, and ultimately enabling the scheduling network to have a comprehensive perception capability of scene complexity and system operating pressure.

[0069] The Neural Scheduling, Reasoning, and Decision-Making L-CSNet is designed as a lightweight neural network. It comprises a State Encoder layer, a Fusion Neck layer, and a Policy Head. The network integrates heterogeneous states... This is mapped to a probability distribution of coordinated actions. The scheduling action space is defined. ={ These correspond to local closed-loop, cloud takeover, and dual-channel verification, respectively. The scheduling process can be formalized as follows:

[0070]

[0071] Final decision action Take the maximum probability:

[0072]

[0073] Based on decision-making actions The final reasoning result of the system The generation logic is as follows:

[0074]

[0075] in, For edge YOLO model output, Output to cloud-based VLM. This mechanism enables result fusion (simultaneously preserving rapid alerts from the edge and accurate confirmation results from the cloud). Compared to traditional rule-based judgments based on a threshold τ, this mechanism can handle complex scenarios such as forcing local processing when confidence is high but network performance is poor, or leveraging a large model to improve accuracy for low-priority tasks when they are idle in the cloud. In the feature fusion stage, L-CSNet explicitly models the higher-order relationships between task risk, network cost, and computational pressure through cross-state interactive fusion of the Neck (using an attention mechanism), generating a contextual representation. Finally, a multi-head decision head decouples scheduling actions: Branch 1: Outputs "Single-channel / Dual-channel" (binary classification, Sigmoid activation). Branch 2: Outputs "Cloud-based / No-cloud-based" (binary classification, Sigmoid activation).

[0076] The L-CSNet proposed in this study has the ability to evolve throughout its entire lifecycle, mainly in the following two stages:

[0077] 1. Cold Start: Offline supervised learning is performed using data generated from historical expert rules to train L-CSNet to learn expert prior policies and initialize network parameters. This phase ensures the safety boundaries of initial actions and the reliability of scheduling actions. The objective is to minimize the cross-entropy loss between predicted and expert actions, using the loss function... Defined as:

[0078]

[0079] in, For the number of expert samples, For the first The system's full-dimensional state vector for each sample The corresponding expert's optimal action. This represents the action probability distribution output by L-CSNet.

[0080] 2. Online fine-tuning: During system operation, the cloud continuously collects posterior feedback data from the edge. ,in The comprehensive reward signal, calculated based on the final detection accuracy (with the large cloud-based model serving as the Ground Truth) and system overhead, is denoted by the following formula:

[0081]

[0082] in, For the first The detection accuracy of the second scheduling. For inference delay, For network transmission bandwidth usage, , These are the baseline thresholds for latency and bandwidth, respectively. Weighting coefficients ( During system operation, the cloud periodically aggregates posterior feedback data from each edge node, and uses an incremental training strategy to adjust the network parameters of L-CSNet. Iterative updates are performed, using a loss function with a regularization term during the update process. The definition is as follows:

[0083]

[0084] in, This represents the number of feedback samples within the sliding window. This represents the decision action currently output by the model. These are the model parameters before the update. This represents the regularization coefficient. The updated model parameters are efficiently distributed and synchronously deployed to distributed edge nodes via OTA (Over-The-Air), ensuring the timeliness and consistency of parameter updates. This mechanism enables the scheduler to dynamically adapt to complex dynamic evolution characteristics in power grid inspection scenarios, such as seasonal changes in sunlight, performance degradation due to equipment aging, and geographical migration, continuously optimizing decision-making accuracy and adaptability to complex environments, and ensuring the long-term effectiveness of the scheduling strategy.

[0085] By introducing L-CSNet, the GridSCOPE architecture successfully achieved a core paradigm shift from traditional static rule-driven scheduling to dynamic, sustainable learning scheduling, providing a cloud-edge collaborative solution with long-term autonomous evolution capabilities for complex and ever-changing power grid inspection scenarios. This mechanism deeply integrates SLA grading and multi-dimensional system state awareness, constructing a cloud-edge model-scale collaborative reasoning system based on task importance and full-dimensional operational status. While strictly meeting the three major requirements of power grid scenarios for real-time response, detection accuracy, and resource efficiency, it effectively solves the inherent defects of traditional scheduling mechanisms in terms of dynamism, adaptability, and generalization. This provides key technical support and a complete engineering implementation path for building an intelligent, hierarchical, autonomous, and highly reliable power sensing system.

[0086] Specifically, cloud-edge collaborative reasoning:

[0087] In power grid scenarios, single-edge inference often struggles to balance real-time performance and cognitive depth. To address this, this invention proposes a cloud-edge collaborative inference mechanism based on a pruned Visual-Language Model (VLM) to achieve distributed collaborative computing across model layers. This scheme splits the complete VLM network according to its feature extraction and semantic understanding functions: the shallow visual encoding part (low-level vision-transformer) is deployed at the edge to encode image features; the high-level cross-modal understanding and inference module is deployed in the cloud to handle complex semantic associations, text generation, and multimodal knowledge reasoning tasks. Through this structural segmentation, the edge only needs to upload compressed feature tensors or semantic embeddings, rather than the original image frames, significantly reducing data transmission bandwidth pressure while maintaining complete inference capabilities in the cloud. The definition is as follows:

[0088] ,

[0089] in, The VLM visual encoding layer deployed at the edge outputs compressed visual feature tensors. ; : The VLM semantic decoding layer deployed in the cloud.

[0090] To further improve cloud-based inference efficiency, this invention introduces an early-exit mechanism into the cloud-based VLM model. Specifically, it determines whether the model's understanding is saturated by calculating the difference in hidden states between VLM-decoder layers and decides whether to terminate early. Details are as follows: if the difference between the state vectors of layer i and layer i-1 is minimal, it indicates that the model's understanding of the image has saturated, and subsequent layers no longer provide new semantic increments; in this case, early termination is possible. Let the VLM Encoder output be... The decoder contains N Transformer Blocks. The hidden state of the i-th layer... The calculation is as follows:

[0091]

[0092] Define a state saturation detection function Cosine similarity is used:

[0093]

[0094] The final inference result in the cloud for:

[0095]

[0096] in, : The high-dimensional feature vector output by the i-th layer VLM-decoder. : Calculate the similarity of features between the two layers. : Saturation threshold. The final text generation or classification header is executed only once when exiting is confirmed.

[0097] In summary, this collaborative reasoning architecture implements the design principles of feature forwarding, hierarchical reasoning, and early result retrieval. The edge computing handles efficient, low-latency perceptual computation, while the cloud focuses on semantic enhancement and global reasoning, reducing network transmission and computational overhead while ensuring high-precision analysis. This solution demonstrates excellent scalability and robustness in multi-source video monitoring tasks for power grids, providing strong support for intelligent inspection, alarm verification, and multimodal knowledge fusion in complex environments.

[0098] Specifically, cloud-edge-weak network collaboration:

[0099] like Figure 1As shown, the weak network collaborative inference solution is built on a three-layer architecture of "cloud-edge-device," using KubeEdge and Sedna as its technical foundation. Through precise network status awareness and intelligent task scheduling, it addresses bandwidth fluctuations and intermittent disconnections in scenarios such as remote substations. Network detection employs a dual "active + passive" mechanism: actively probing connection status in real-time via TCP Ping, HTTP health checks, and KubeEdge heartbeats; passively using request success rate and response latency statistics to assist in judgment. Finally, weighted fusion results in three states: "normal," "fluctuating," and "disconnected." A 30-second stabilization period is set for state transitions to avoid frequent fluctuations. A large VLM model is deployed in the cloud for high-precision, complex example inference. The edge integrates modules for task classification, inference decision-making, and local caching, and is equipped with both lightweight (YOLOv5s) and heavyweight (YOLOv8x) models. The device layer collects data through cameras and sensors, forming a closed-loop collaborative system.

[0100] To address network fluctuation scenarios, the solution employs a triple-layer protection system of "retry + rate limiting + degradation." The retry mechanism uses an exponential backoff strategy, with retry intervals increasing in increments of 2 seconds, 4 seconds, and 8 seconds after a failure, up to a maximum of 5 retries. This is combined with adaptive timeouts (dynamically adjusted based on historical response times) and request compression (image quality reduced to 75%, reducing transmission volume by 50%) to improve success rate. Rate limiting uses a token bucket algorithm, reducing the cloud request threshold from 2 requests per second to 1 request per second during network fluctuations, while a priority queue ensures urgent tasks are transmitted first. If three retries still fail, degradation is automatically triggered, using a lightweight edge model as a temporary replacement. This is marked and then optimized again once the network recovers, ensuring task continuity.

[0101] In network outage scenarios, the solution implements differentiated processing based on task priority to ensure "emergency tasks are not interrupted and ordinary tasks are not lost." P0-level emergency tasks (equipment malfunction, fire detection, etc.) immediately switch to local inference using a heavy-duty edge model (such as YOLOv8x), with inference results triggering local alarms and being cached instantly. P1 / P2-level ordinary and batch tasks are cached locally using SQLite, storing original images, lightweight model results, and task metadata. A 10GB capacity limit and a 7-day expiration policy are used to manage the cache. The background service scans the data to be transmitted every 30 seconds, and after network recovery, automatically retransmits according to the rule of "P1 priority, batch of 10 records / time," combined with MD5 deduplication and breakpoint resumption to ensure data integrity. The adaptive transmission and degradation functions are defined as follows:

[0102] Input data Transformation function before transmission Processing to adapt to bandwidth limitations,

[0103]

[0104] : Degradation compression strategy, ∅: indicates physical link failure, unable to send data. The transformed data, Current network status. Service continuity and caching mechanisms. When... During a disconnection, the system guarantees uninterrupted service. Continuity guarantee output is defined. And caching actions. Real-time response continuity means that even if the network is down, the edge device must still provide a result (i.e., degradation mode), defined as follows:

[0105]

[0106] This outputs results at the edge. Data integrity is guaranteed based on task priority. Implement different caching strategies:

[0107]

[0108] The final caching strategy selected so far. To store edge-end output and alarm metadata. To store all data. Compensation formula after weak network recovery: When network status recovers ( When (Normal), queue compensation is triggered. ,

[0109]

[0110] in For indicator functions, For the current time, Let j be the time when the task is enqueued. The maximum compensation time window is set. This indicates that task j should be resent. It is 1 when the condition is met (triggered in the cloud), and 0 otherwise.

[0111] Example 1

[0112] like Figure 3 As shown, this invention presents a power grid target detection method based on cloud-edge-large model collaboration. This method utilizes a cloud-edge-large model collaboration framework for power grid target detection. The framework includes a lightweight detection model deployed at the edge and a large multimodal model with strong computing power and multimodal semantic understanding capabilities deployed in the cloud. Hierarchical collaboration across models is achieved through hierarchical routing, result verification, and semantic completion, thus balancing the agile response of the lightweight detection model with the cognitive depth of the large multimodal model. The method includes:

[0113] Acquire images of the power grid, including images of substations, transmission lines, and transmission towers;

[0114] Based on the power grid image, preliminary target detection is performed using a lightweight detection model at the edge, and the first detection result is obtained.

[0115] Based on the confidence level of the first detection result and the priority of the detection task, a cloud-edge collaborative scheduling module based on adaptive sustainable learning is used to determine whether the power grid image needs to be further uploaded to the cloud for detection and output.

[0116] In this embodiment, based on the power grid image, preliminary target detection is performed using a lightweight detection model at the edge to obtain a first detection result, including:

[0117] Based on the power grid image, preliminary target detection is performed using a lightweight detection model at the edge, resulting in a first detection result. The first detection result represents the confidence level of the target detection, including high confidence, medium-low confidence, and extremely low confidence.

[0118] In this embodiment, the detection task is divided according to an SLA grading mechanism that combines task semantics and real-time requirements, including:

[0119] Emergency tasks such as identifying equipment smoke and fire, and detecting cracks in insulator sheaths are classified as emergency alarm tasks P0.

[0120] Important tasks such as main transformer oil tank leakage detection and equipment overheating detection are classified as important inspection tasks P1;

[0121] Routine tasks such as indicator light and observation window inspections, and foreign object intrusion detection on equipment supports are classified as routine monitoring tasks (P2).

[0122] In this embodiment, the cloud-edge collaborative scheduling module L-CSNet based on adaptive sustainable learning provides a comprehensive perception of the system's full-dimensional state, enabling intelligent decision-making for cloud-edge collaborative logic; the full-dimensional state of the system includes confidence distribution, task level, network status, and cloud-edge computing power.

[0123] The L-CSNet is trained offline using a rule-driven strategy as a prior. After offline training, it is continuously fine-tuned based on posterior feedback during system operation, giving the system the ability to adapt to different complex power grid scenarios.

[0124] In this embodiment, the rule-driven strategy includes:

[0125] When the first detection result is of high confidence and the priority of the detection task is arbitrary, the first scheduling strategy is executed to directly complete the inference at the edge and output the first detection result, and the alarm information is reported to the front end.

[0126] When the first detection result is of medium to low confidence and the priority of the detection task is emergency alarm, the second scheduling strategy is executed to trigger the dual-channel system. The dual-channel trigger includes reporting warning information at the edge and parallel verification and confirmation in the cloud.

[0127] When the first detection result has a low to medium confidence level and the priority of the detection task is important inspection or routine monitoring, the third scheduling strategy is executed to upload the first detection result and key frames to a large model in the cloud for in-depth verification.

[0128] When the first detection result has an extremely low confidence level, it is considered that the probability of anomaly is low and no further processing is performed.

[0129] In this embodiment, the method further includes: a cloud-edge collaborative inference mechanism based on VLM structure pruning, specifically:

[0130] The Visual Language Model (VLM) is split into shallow visual encoding and deep semantic reasoning:

[0131] The shallow visual encoding part at the front end is deployed at the edge to encode the power grid image into image features. Only the image features are transmitted to the cloud, reducing the bandwidth pressure caused by multiple edge ends transmitting the power grid image at the same time.

[0132] The latter completes cross-modal understanding in the cloud and combines a state difference-based early termination strategy to reduce redundant reasoning, ensuring the complete reasoning capability of VLM while reducing the reasoning cost of large models.

[0133] The early termination strategy involves inserting an early termination evaluation module between the decoder layers of the VLM. The early termination evaluation module determines whether the VLM model understanding is saturated by calculating the difference between the hidden states between layers. When the difference is less than a set threshold, the subsequent calculation is terminated in advance and the second detection result is directly output.

[0134] In this embodiment, the cloud-based detection includes:

[0135] The image encoding output from the edge is obtained by cropping and compressing the power grid image that L-CSNet determines needs to be uploaded to the cloud for detection, and then processing it through shallow visual encoding of the visual language model VLM.

[0136] Based on the image encoding, secondary inference and further target detection are performed using a multimodal large model in the cloud to obtain a second detection result;

[0137] The lightweight detection model at the edge is trained based on the second detection result to obtain the parameters of the trained model. The trained parameters are then periodically updated to the lightweight detection model at the edge via network transmission.

[0138] In this embodiment, the method further includes: network transmission between the edge and the cloud, specifically:

[0139] The system detects the network connectivity between the edge device and the cloud. If there are network fluctuations or disconnections, it reconnects to the network and retransmits the image encoding output from the edge device based on the retry strategy.

[0140] If retransmission fails, lightweight large model inference is performed at the edge for emergency alarm tasks; for important inspection or routine monitoring tasks, data is cached at the edge and retransmitted to the cloud periodically.

[0141] In this embodiment, detecting the network connectivity between the edge and the cloud includes:

[0142] A dual mechanism, employing both active and passive methods, is used to detect network connectivity between the edge and the cloud.

[0143] The system employs both active and passive mechanisms: the active mechanism uses TCP Ping, HTTP health checks, and KubeEdge heartbeat to detect connection status in real time; the passive mechanism uses request success rate and response latency statistics to assist in the determination.

[0144] In practical implementation, before system operation, an SLA (Service Level Agreement) grading mechanism is first constructed based on the risk level, real-time requirements, and confidence level requirements of power grid inspection tasks. Then, expert rules and historical operational data are integrated to train the adaptive sustainable learning cloud-edge collaborative scheduling module L-CSNet. L-CSNet takes task semantics, edge model confidence, network status, and cloud-edge resource load as inputs to learn the optimal cloud-edge collaborative strategy for different inspection scenarios. This enables adaptive decision-making regarding inference paths, inference depth, and synchronous or asynchronous collaborative methods, providing a stable and reliable scheduling foundation for subsequent online detection.

[0145] During the online detection phase, the image data collected by the inspection terminal is first processed by a lightweight detection model at the edge to perform target detection or preliminary feature extraction, and outputs corresponding confidence information. L-CSNet dynamically determines whether to complete the decision solely at the edge, or to upload the compressed visual features to the cloud to trigger VLM inference, based on the current task's SLA level, edge detection results, and system operating status. When cloud access is required, cross-modal semantic understanding is performed in the cloud, and early termination mechanisms and asynchronous collaboration strategies are combined to reduce inference overhead. In weak network or high-load scenarios, the system can flexibly adjust the inference path according to task priority. Finally, detection results and alarm information are required to be returned and linked, while continuous feedback is fed back to update L-CSNet, achieving long-term autonomous evolution of the cloud-edge collaboration strategy.

[0146] 1. The dataset includes:

[0147] (1) Wildfire Dataset: D-Fire is a dedicated image dataset developed by the Gaia Solutions-on-demand team, focusing on fire and smoke target detection tasks. It is designed for training and evaluating machine learning and deep learning models. It includes pure fire and smoke scenes, as well as scenes where fire and smoke coexist. It can be used for developing intelligent fire early warning systems, fire identification in security monitoring, and optimizing fire detection algorithms for low-power devices. Images are uniformly resized to 680x480.

[0148] (2) Foreign Object Dataset: The dataset includes 2219 images of foreign object intrusions into power transmission lines and 421 images of power transmission lines without foreign objects. Foreign object types include bird nests, balloons, kites, and household waste. All images were uniformly resized to 680x480.

[0149] 2. Analysis of Experimental Results

[0150] Table 3 Comparison of F1 scores between the lightweight YOLO model and the VLM model

[0151]

[0152] Table 4 Comparison of inference time between the lightweight YOLO model and the VLM model (single A5000 card)

[0153]

[0154] Based on the results in Tables 3 and 4, traditional object detection models (YOLO series) show good precision (above 0.95) when the confidence threshold is set to 0.7, but their recall is relatively low. This indicates that while traditional detection models can accurately identify some obvious targets when the confidence threshold is set high, their ability to identify small or blurred targets in complex scenes is limited, easily leading to false negatives. In contrast, the Qwen3-VL series multimodal models significantly outperform traditional detection models in overall performance. Among them, Qwen3-VL-4B and Qwen3-VL-8B achieved high F1 scores on both datasets, demonstrating strong visual semantic understanding and cross-scene generalization capabilities. Notably, Qwen3-VL-8B has a higher recall on the wildfire dataset, while 4B performs more stably on the foreign object intrusion dataset. For the GridSCOPE cloud-edge collaborative model of this invention, targeting the wildfire dataset, we categorized the images detected by the yolov8n model into three confidence intervals: 0-0.2, 0.2-0.6, and 0.6-1. Images with a confidence level of 0-0.2 were considered to have no fire traces, while images with a confidence level of 0.6-1 were considered to have experienced a fire. Images in the 0.2-0.6 confidence interval were sent to the cloud-based VLM model for re-examination. During the experiment, images in the 0.2-0.6 confidence interval accounted for approximately one-quarter of the entire dataset, resulting in an inference speed approximately 400% faster than sending all images to the VLM model. In real-world applications, the proportion of images sent to the cloud is often even lower because our dataset contains many fire images, while real-world fires are relatively rare. Therefore, the inference time of the cloud-based VLM model is negligible. In this case, this invention uses a cloud-edge collaborative model for power grid target detection, balancing accuracy and real-time performance. Furthermore, as the YOLO model is incrementally trained in real-world scenarios, the number of low-confidence missed detections will decrease, and the accuracy of this invention will be further improved.

[0155] This invention utilizes a cloud-edge large-scale model collaborative framework to deploy lightweight detection models (such as YOLO models or lightweight large models) at the edge, enabling rapid response and local prediction. When confidence levels are low and tasks are complex, the large model in the cloud is triggered for secondary inference, achieving a dynamic balance between accuracy and latency. This invention addresses the challenges of diverse tasks and heterogeneous computing power in power grid target detection scenarios, thereby improving the accuracy of power grid target detection.

[0156] Example 2

[0157] like Figure 4As shown, the difference between this embodiment and Embodiment 1 is that this embodiment provides a power grid target detection system based on cloud-edge collaboration of large and small models. This system corresponds one-to-one with the power grid target detection method based on cloud-edge collaboration of large and small models in Embodiment 1. This system performs power grid target detection based on a cloud-edge-large model collaboration framework. The cloud-edge-large model collaboration framework includes a lightweight detection model deployed at the edge and a large multimodal model with strong computing power and multimodal semantic understanding capabilities deployed in the cloud. Through hierarchical routing, result verification, and semantic completion, it achieves cross-model hierarchical collaboration, thereby balancing the agile response of the lightweight detection model and the cognitive depth of the large multimodal model. The system includes:

[0158] The acquisition unit is used to acquire power grid images, which include images of substations, transmission lines, and transmission towers, etc.

[0159] The first detection unit is used to perform preliminary target detection based on the power grid image and a lightweight detection model at the edge, and obtain the first detection result.

[0160] The second detection unit is used to determine, based on the confidence level of the first detection result and the priority of the detection task, whether the power grid image needs to be further uploaded to the cloud for detection and output using a cloud-edge collaborative scheduling module based on adaptive sustainable learning.

[0161] The execution process of each unit can be carried out according to the steps of the power grid target detection method based on cloud-edge collaboration of size model in Example 1, and will not be described in detail in this example.

[0162] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A power grid target detection method based on cloud-edge collaboration using a large-scale model, characterized in that, This method is based on a cloud-edge-large-scale model collaborative framework for power grid target detection. The framework includes a lightweight detection model deployed at the edge and a multimodal large model deployed in the cloud, achieving hierarchical collaboration across models through hierarchical routing, result verification, and semantic completion. The method includes: Acquire power grid images, including images of substations, transmission lines, and transmission towers; Based on the power grid image, preliminary target detection is performed using a lightweight detection model at the edge, resulting in a first detection result. Based on the confidence level of the first detection result and the priority of the detection task, the cloud-edge collaborative scheduling module L-CSNet, which is based on adaptive sustainable learning, is used to determine whether the power grid image needs to be further uploaded to the cloud for detection and output. This method also includes: a cloud-edge collaborative inference mechanism based on VLM structure pruning, specifically: The Visual Language Model (VLM) is split into shallow visual encoding and deep semantic reasoning: The shallow visual encoding part at the front end is deployed at the edge to encode the power grid image into image features, and only the image features are transmitted to the cloud. The latter completes cross-modal understanding in the cloud and combines it with a state-difference-based early termination strategy for inference; The early termination strategy involves inserting an early termination evaluation module between the decoder layers of the VLM. The early termination evaluation module determines whether the VLM model understanding is saturated by calculating the difference between the hidden states between layers. When the difference is less than a set threshold, the subsequent calculation is terminated in advance and the second detection result is directly output. The method also includes: network transmission between the edge and the cloud, specifically: The system detects the network connectivity between the edge device and the cloud. If there are network fluctuations or disconnections, it reconnects to the network and retransmits the image encoding output from the edge device based on the retry strategy. If retransmission fails, lightweight large model inference is performed at the edge for emergency alarm tasks; for important inspection or routine monitoring tasks, data is cached at the edge and retransmitted to the cloud periodically.

2. The power grid target detection method based on cloud-edge collaboration using a large-scale model as described in claim 1, characterized in that, Based on the power grid image, preliminary target detection is performed using a lightweight detection model at the edge, yielding a first detection result, including: Based on the power grid image, preliminary target detection is performed using a lightweight detection model at the edge, resulting in a first detection result. The first detection result represents the confidence level of the target detection, including high confidence, medium-low confidence, and extremely low confidence.

3. The power grid target detection method based on cloud-edge collaboration using a large-scale model as described in claim 1, characterized in that, The detection tasks are divided according to an SLA grading mechanism that combines task semantics and real-time requirements, including: The emergency tasks of detecting smoke and fire from equipment and cracks in the insulator sheath are classified as emergency alarm tasks P0. The important tasks of detecting oil leakage in the main transformer tank and detecting equipment overheating are classified as important inspection tasks P1. Routine tasks such as inspecting indicator lights and observation windows, and detecting foreign object intrusion into equipment supports are classified as routine monitoring tasks P2.

4. The power grid target detection method based on cloud-edge collaboration using a large-scale model as described in claim 1, characterized in that, The cloud-edge collaborative scheduling module L-CSNet, based on adaptive sustainable learning, provides a comprehensive perception of the system's full-dimensional state, enabling intelligent decision-making for cloud-edge collaborative logic. The full-dimensional state of the system includes confidence distribution, task level, network status, and cloud-edge computing power. The L-CSNet is trained offline using a rule-driven strategy as a prior. After offline training, it is continuously fine-tuned based on posterior feedback during system operation, giving the system the ability to adapt to different complex power grid scenarios.

5. The power grid target detection method based on cloud-edge collaboration using a large-scale model according to claim 4, characterized in that, The rule-driven strategy includes: When the first detection result is of high confidence and the priority of the detection task is arbitrary, the first scheduling strategy is executed to directly complete the inference at the edge and output the first detection result, and the alarm information is reported to the front end. When the first detection result is of medium to low confidence and the priority of the detection task is emergency alarm, the second scheduling strategy is executed to trigger the dual-channel system. The dual-channel trigger includes reporting warning information at the edge and parallel verification and confirmation in the cloud. When the first detection result has a low to medium confidence level and the priority of the detection task is important inspection or routine monitoring, the third scheduling strategy is executed to upload the first detection result and key frames to a large model in the cloud for in-depth verification. When the first detection result has an extremely low confidence level, it is considered that the probability of anomaly is low and no further processing is performed.

6. The power grid target detection method based on cloud-edge collaboration using a large-scale model as described in claim 1, characterized in that, The cloud-based detection includes: The image encoding output from the edge is obtained by cropping and compressing the power grid image that L-CSNet determines needs to be uploaded to the cloud for detection, and then processing it through shallow visual encoding of the visual language model VLM. Based on the image encoding, secondary inference and further target detection are performed using a multimodal large model in the cloud to obtain a second detection result; The lightweight detection model at the edge is trained based on the second detection result to obtain the parameters of the trained model. The trained parameters are then periodically updated to the lightweight detection model at the edge via network transmission.

7. The power grid target detection method based on cloud-edge collaboration using a large-scale model according to claim 1, characterized in that, Detecting network connectivity between the edge and the cloud, including: A dual mechanism, employing both active and passive methods, is used to detect network connectivity between the edge and the cloud. The active and passive dual mechanisms include: the active mechanism is to detect the connection status in real time through TCP Ping, HTTP health checks and KubeEdge heartbeat; the passive mechanism is to assist in the determination through request success rate and response latency statistics.

8. A power grid target detection system based on cloud-edge collaboration using a large-scale model, characterized in that, This system performs power grid target detection based on a cloud-edge-large model collaborative framework. The framework includes a lightweight detection model deployed at the edge and a multimodal large model deployed in the cloud, achieving hierarchical collaboration across models through tiered routing, result verification, and semantic completion. The system includes: The acquisition unit is used to acquire power grid images, which include images of substations, transmission lines and transmission towers. The first detection unit is used to perform preliminary target detection based on the power grid image and a lightweight detection model at the edge to obtain a first detection result. The second detection unit is used to determine, based on the confidence level of the first detection result and the priority of the detection task, whether the power grid image needs to be further uploaded to the cloud for detection and output using the cloud-edge collaborative scheduling module L-CSNet based on adaptive sustainable learning. It also includes: a cloud-edge collaborative inference mechanism based on VLM structure pruning, specifically: The Visual Language Model (VLM) is split into shallow visual encoding and deep semantic reasoning: The shallow visual encoding part at the front end is deployed at the edge to encode the power grid image into image features, and only the image features are transmitted to the cloud. The latter completes cross-modal understanding in the cloud and combines it with a state-difference-based early termination strategy for inference; The early termination strategy involves inserting an early termination evaluation module between the decoder layers of the VLM. The early termination evaluation module determines whether the VLM model understanding is saturated by calculating the difference between the hidden states between layers. When the difference is less than a set threshold, the subsequent calculation is terminated in advance and the second detection result is directly output. This also includes: network transmission between the edge and the cloud, specifically: The system detects the network connectivity between the edge device and the cloud. If there are network fluctuations or disconnections, it reconnects to the network and retransmits the image encoding output from the edge device based on the retry strategy. If retransmission fails, lightweight large model inference is performed at the edge for emergency alarm tasks; for important inspection or routine monitoring tasks, data is cached at the edge and retransmitted to the cloud periodically.

Images