A power transmission channel hidden danger identification method based on size model cooperation and cloud edge fusion

Abstract

The application discloses a power transmission channel hidden danger identification method based on size model cooperation and cloud-edge fusion, and relates to the technical field of power inspection.The method comprises receiving suspected hidden danger images of a power transmission channel and corresponding first metadata from an end-side collection device.A lightweight small model generated through channel-level pruning, post-training quantization and operator fusion compression is deployed at the end side, and the original monitoring images are preliminarily screened by using differentiated low confidence thresholds, only suspected hidden danger images and structured metadata containing hidden danger types, confidence, bounding box coordinates, device identifiers, preset bits and timestamps are uploaded to the cloud side, while ensuring high recall rates of various hidden danger targets such as engineering machinery and mountain fire smoke, the communication bandwidth occupation and transmission delay are significantly reduced, the invalid inference of the cloud side on a large number of non-hidden danger images is avoided, the contradiction between limited end-side computing power and cloud-side inference delay is effectively solved, and the collaborative optimization configuration of cloud-edge resources is realized.

Description

Technical Field

[0001] This invention relates to the field of power line inspection technology, specifically to a method for identifying potential hazards in power transmission channels based on large and small model collaboration and cloud-edge fusion. Background Technology

[0002] As a critical infrastructure of the power system, the safe and stable operation of transmission channels directly affects the reliability of power grid supply and the safety of electricity consumption in society. With the continuous expansion of transmission line scale and the increasingly complex external environment of the corridors, external hazards such as illegal construction by engineering machinery and wildfire smoke pose increasingly prominent threats to the safety of transmission lines. Traditional manual inspection methods are no longer sufficient to meet the needs of large-scale, all-weather real-time monitoring. Intelligent hazard identification methods based on computer vision and deep learning technologies have become an important means of transmission channel operation and maintenance. How to achieve real-time, accurate, and low-false-alarm identification of external hazards under conditions of limited computing power at the end-side, limited communication bandwidth, and interference from complex scenarios is a pressing technical problem that needs to be solved in the intelligent operation and maintenance of transmission channels.

[0003] In existing technologies, the identification of potential hazards in power transmission channels mainly adopts two technical architectures: centralized cloud processing and single-model detection at the edge. For example, the centralized cloud analysis scheme uploads all monitoring images acquired by the front-end acquisition equipment to the cloud server in real time, and the heavy-duty detection model deployed in the cloud uniformly performs hazard identification and alarm decision-making. This method can achieve high identification accuracy by utilizing the abundant computing power of the cloud. The single-model detection scheme at the edge deploys a compressed lightweight detection model on embedded acquisition devices or edge gateways, and the edge directly infers from the original images and outputs the detection results. This method can reduce transmission latency and cloud computing pressure to a certain extent.

[0004] However, the aforementioned existing technologies have many shortcomings in practical applications. For cloud-based centralized solutions, due to the large number of front-end acquisition points and high sampling frequency, uploading all raw monitoring images to the cloud results in huge communication bandwidth consumption, high transmission costs, and large latency. Furthermore, the cloud needs to perform ineffective inference on a large number of images without potential hazards, leading to serious waste of computing resources and making it difficult to meet the timeliness requirements of real-time alarms. For edge-side single detection solutions, due to the limitations of computing power, storage, and power consumption of edge devices, the deployed lightweight models are usually deeply compressed, significantly weakening the feature extraction capability. This makes it difficult to effectively identify difficult samples such as small machinery at a distance and low-contrast smoke from early-stage wildfires, resulting in high false alarm and false negative rates. Furthermore, existing solutions mostly use fixed confidence thresholds for target identification, which cannot be adaptively adjusted according to the differences in imaging characteristics of different hazard types such as engineering machinery and wildfire smoke. They also lack the ability to perform spatial correlation analysis with the physical structure of the power transmission channel (such as conductors and tower protection zones), the ability to verify the temporal continuity of target movement status and repeated alarms, and the ability to distinguish real smoke from morning fog and light interference using multimodal semantic understanding. This results in low alarm accuracy and a heavy workload for manual review, which seriously restricts the practical application of intelligent operation and maintenance of power transmission channels. Summary of the Invention

[0005] The purpose of this invention is to provide a method for identifying potential hazards in power transmission channels based on large-scale model collaboration and cloud-edge fusion, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for identifying potential hazards in power transmission channels based on large-scale model collaboration and cloud-edge fusion, applied to a cloud-side analysis platform, the method comprising: Receive images of suspected potential hazards in the power transmission channel and corresponding first metadata from the end-side acquisition equipment; The suspected hazard images are determined by a lightweight small model deployed on the edge, which performs a low-confidence threshold initial screening on the original monitoring images. The first metadata includes hazard type, confidence level, bounding box coordinates, device identifier, preset position and timestamp. The low-confidence threshold is used to ensure the recall rate of hazard targets. The lightweight small model is used to undertake basic calculations and filter out images without hazards. The suspected hazard images are input into the cloud-based large model for secondary identification to obtain accurate identification results of the hazard targets; The cloud-based large model employs a collaborative hybrid allocation training mechanism, deploying multiple auxiliary detection heads in parallel during the encoder stage for one-to-many intensive supervision, while retaining only the main detection head during the inference stage to perform one-to-one label allocation. The cloud-based large model is used for accurate analysis of the initial screening results. Based on the accurate identification results, spatial filtering, temporal filtering, size filtering and multimodal filtering are performed sequentially to generate the final alarm decision for the power transmission channel. The spatial filtering is based on the BiRefNet segmentation network to extract pixel-level masks of conductors and towers, and calculates the dynamic alarm range based on the pixel-level masks, thereby determining the spatial overlap of the dynamic alarm range to which the hidden danger target domain belongs. The spatial filtering is used to filter low-risk targets in the protected area. The time-series filtering determines the target's motion state and repeated alarm interval based on the historical trajectory information of the potential hazard target. The time-series filtering is used to suppress stationary, risk-free targets and repeated alarms. The size filtering determines the actual target height based on the ratio of the target detection box to the image size, and compares it with the safe height threshold corresponding to the voltage level. The size filtering is used to filter low and risk-free targets. The multimodal filtering method calls a multimodal large model to analyze the dynamic evolution characteristics of smoke by combining dynamic prompts and continuous dual-image input. The time-division dynamic prompts are adaptively generated based on the differences in interference characteristics in the current time period. The multimodal filtering method is used to distinguish between real wildfire smoke and morning fog and light interference.

[0007] As a specific solution in this application, the lightweight small model is deployed on the edge in the following manner: The basic detection module is pruned at the channel level, retaining a pre-set proportion of convolutional channels before importance scoring, to obtain the pruned model. The pruned model is trained and then quantized to convert the floating-point weights into integer weights, resulting in the quantized model. The quantized model is then subjected to operator fusion, which combines convolution, normalization, and activation sequences into a single operator to obtain the lightweight small model.

[0008] As a specific solution in this application, the low confidence threshold includes: A first threshold for potential hazards in construction machinery is used to detect small machinery at long distances and ensure the recall rate of construction machinery. A second threshold is set for the potential danger of wildfire smoke. This second threshold is lower than the first threshold and is used to ensure the detection rate of initial smoke.

[0009] As a specific solution in this application, the step of inputting the suspected hidden danger image into a cloud-based large model for secondary identification includes: The suspected hazard images are input into a collaborative detection model based on an improved DET architecture; The main detection head of the collaborative detection model performs one-to-one label allocation and outputs the final detection result. During the training phase, one-to-many intensive supervision is performed through multiple auxiliary detection heads of the collaborative detection model, and multiple positive samples are assigned to each real target to accelerate encoder feature learning. During the inference phase, the multiple auxiliary detection heads are completely removed.

[0010] As a specific solution in the technical solution of this application, the spatial filtering includes: The suspected hazard image is input into the BiRefNet segmentation network. The multi-scale contextual information of the conductor and tower is captured by the fusion localization module, the edge details are restored by the reconstruction module, and the global semantics and local textures are fused by the bilateral reference mechanism to generate a pixel-level segmentation mask. The set of conductor centerlines is extracted based on the pixel-level segmentation mask, and the dynamic alarm range is calculated based on camera parameters and the width of the line protection zone. Calculate the intersection-union ratio (IUU) between the target detection box and the dynamic alarm range, and retain the alarm if the IUU is greater than a preset threshold.

[0011] As a specific solution in the technical solution of this application, the timing filtering includes: For potential hazards related to construction machinery, calculate the standard deviation of the centroid displacement of the same target in the current frame and historical frames. If the standard deviation is less than a preset pixel threshold and the time since the last valid alarm is less than a preset time interval, suppress the current alarm. For potential hazards such as wildfire smoke, the current frame is compared with historical images of the same location at the same time the previous day. If the smoke pattern does not change, it is determined to be interference from a fixed light source and an alarm is issued.

[0012] As a specific solution in this application, the size filtering includes: Calculate the ratio of the target detection bounding box height to the image height; Establish a mapping relationship between the ratio and the actual target height based on the camera installation height and focal length; If the actual target height is less than the line safety height threshold for the corresponding voltage level, the current alarm is filtered.

[0013] As a specific solution in this application, the multimodal filtering includes: The corresponding time-sharing dynamic prompt words are determined based on the current time period. The time-sharing dynamic prompt words include high recall prompt words for daytime periods, negative light source prompt words for nighttime periods, and morning fog differentiation prompt words for early morning periods. The current hazard image and the previous hazard image are concatenated into a dual-channel tensor according to the time dimension, and then input into the multimodal large model. The dynamic evolution characteristics of the smoke region are analyzed using the multimodal large model, and a smoke dynamic score is output. An alarm is triggered when the smoke dynamic score exceeds a preset threshold. As a specific solution in this application, the method further includes: Uncertainty assessment is performed on the identification results, high-entropy samples are automatically marked and added to the incremental sample library, and the high-entropy samples are used to indicate suspected missed or false detection data; The cloud-side large model is incrementally trained using the incremental sample library, and the intermediate layer features of the cloud-side large model are used as supervision signals to optimize the lightweight small model on the edge through feature distillation. The optimized lightweight mini-model is downloaded to the edge acquisition device during off-peak hours via over-the-air download technology to complete the hot replacement of the model.

[0014] As a specific solution in the technical solution of this application, applied to an end-side data acquisition device, the method includes: A lightweight small model is used to perform a low-confidence threshold initial screening on the original monitoring images to identify suspected potential hazard images and their corresponding first metadata. The lightweight small model is used to perform basic calculations and filter out images without potential hazards, while the low-confidence threshold is used to ensure the recall rate of potential hazard targets. Send the suspected hazard image and the first metadata to the cloud-based analysis platform; The suspected hazard images are used to generate a final alarm decision after secondary identification and multiple filtering on the cloud side.

[0015] As a specific solution in this application, the lightweight small model is generated by compression through at least one of channel-level pruning, post-training quantization, and operator fusion. The number of parameters of the lightweight small model is less than a preset threshold, and the inference speed meets the real-time requirements.

[0016] As a specific solution in the technical solution of this application, applied to a cloud-based analysis platform, the method includes: Receive suspected hidden danger images and first metadata from the end-side acquisition device. The suspected hidden danger images are determined by the end-side lightweight small model performing a low-confidence threshold initial screening on the original monitoring images. The suspected hazard images are then subjected to secondary identification using a cloud-based large model to obtain accurate identification results. Spatial filtering is performed based on BiRefNet segmentation results, temporal filtering is performed based on historical trajectory information, size filtering is performed based on target bounding box size, and smoke verification filtering is performed based on multimodal large model to generate the final alarm decision.

[0017] As a specific solution in this application, the step of performing smoke verification filtering based on a multimodal large model includes: The corresponding time-sharing dynamic prompt dictionary is invoked according to the current time period. The time-sharing dynamic prompt dictionary includes differentiated semantic constraints for daytime, nighttime and early morning time periods. Two consecutive frames of hazard images are input into a multimodal large model to analyze the trend of smoke motion vector and regional area change; Whether to trigger an alarm is determined based on the output of the multimodal large model.

[0018] A hazard identification device includes a processor coupled to a memory, the memory storing a computer program; the processor is used to call part or all of the computer program in the memory to implement any of the methods described in the above embodiments, or to implement any of the methods described in the above embodiments, or to implement any of the methods described in the above embodiments.

[0019] A computer-readable storage medium storing a computer program or instructions, which, when executed by a hazard identification device, implements any of the methods described in the above embodiments, or implements any of the methods described in the above embodiments, or implements any of the methods described in the above embodiments.

[0020] A computer program product includes a computer program or instructions, which, when executed by a hazard identification device, implements any of the methods described in the above embodiments, or implements any of the methods described in the above embodiments, or implements any of the methods described in the above embodiments.

[0021] A chip, the chip including a processor, the processor being configured to implement any of the methods described in the above embodiments, or to implement any of the methods described in the above embodiments, or to implement any of the methods described in the above embodiments.

[0022] A power transmission channel hazard identification system, the system comprising an end-side acquisition device and a cloud-side analysis platform; wherein, the end-side acquisition device is used to execute the method described in any one of the above embodiments, and the cloud-side analysis platform is used to execute the method described in any one of the above embodiments, or to execute the method described in any one of the above embodiments.

[0023] Compared with the prior art, the beneficial effects of the present invention are: This invention provides a method for identifying potential hazards in power transmission channels based on collaborative small and large models and cloud-edge fusion. By deploying a lightweight small model on the edge side, generated through channel-level pruning, post-training quantization, and operator fusion compression, and using differentiated low-confidence thresholds to initially screen the original monitoring images, only images of suspected hazards and structured metadata containing hazard type, confidence level, bounding box coordinates, equipment identifier, preset position, and timestamp are uploaded to the cloud side. While ensuring a high recall rate for various hazard targets such as construction machinery and wildfire smoke, this method significantly reduces communication bandwidth usage and transmission latency, avoids invalid inference on massive amounts of images without hazards on the cloud side, effectively solves the contradiction between limited computing power on the edge side and inference latency on the cloud side, and achieves collaborative optimization of cloud-edge resources.

[0024] Meanwhile, this invention deploys a collaborative detection model based on an improved DET architecture on the cloud side. During the training phase, multiple auxiliary detection heads are deployed in parallel to perform one-to-many intensive supervision. Multiple positive samples are assigned to each real target to accelerate the encoder's feature learning for multi-scale and multi-form hidden danger targets. During the inference phase, the auxiliary detection heads are completely removed, and only the main detection head is retained to perform one-to-one label assignment. Without increasing inference latency and computational overhead, this invention significantly improves the accurate identification capability of small targets, occluded targets, and low-contrast hidden danger targets in complex backgrounds, and effectively reduces the false alarm rate and false positive rate.

[0025] This invention employs a four-level cascaded filtering process—spatial filtering, temporal filtering, size filtering, and multimodal filtering—based on accurate identification results. This process deeply verifies the identification results from multiple dimensions, including spatial location rationality, target motion state, physical size compliance, and cross-modal semantic consistency. Specifically, it extracts pixel-level masks of conductors and towers using a BiRefNet segmentation network and calculates dynamic alarm ranges, effectively filtering low-risk targets outside protected areas. It determines target motion state based on historical trajectory information and suppresses repetitive alarms, effectively filtering stationary, risk-free targets. It maps the actual target height based on the ratio of the target detection box to the image size, combined with camera parameters, and compares it against a safety height threshold corresponding to the voltage level, effectively filtering low-lying, risk-free targets. Finally, it utilizes a multimodal large model, adaptively generated time-sharing dynamic prompts based on the differences in interference characteristics during the current time period, and verifies the dynamic evolution characteristics of smoke through continuous dual-image input, effectively distinguishing real wildfire smoke from morning fog and light interference. This significantly improves the accuracy and reliability of alarm decisions and drastically reduces the cost of manual review. Attached Figure Description

[0026] Figure 1 This is a flowchart illustrating the method for identifying potential hazards in power transmission channels according to the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] The terms "first," "second," etc., in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. For example, the first cluster and the second cluster mentioned below belong to different clusters. It should be understood that such names can be used interchangeably where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or modules is not necessarily limited to those steps or modules explicitly listed, but may include other steps or modules not explicitly listed or inherent to these processes, methods, products, or devices. The division of modules in the embodiments of this application is merely a logical division. In actual applications, there may be other division methods. For example, multiple modules may be combined into or integrated into another system, or some features may be ignored or not performed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interface, and the indirect coupling or communication connection between modules may be electrical or other similar forms. None of these are limited in the embodiments of this application. Furthermore, the modules or sub-modules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed among multiple circuit modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiments of this application.

[0029] To improve the accuracy of power transmission channel hazard identification, reduce cloud-side computing load and communication bandwidth consumption, and resolve the contradiction between limited edge computing power and cloud-side inference latency, this application provides a power transmission channel hazard identification system based on big-small model collaboration and cloud-edge fusion. The identification system includes edge-side data acquisition equipment and a cloud-side analysis platform. Optionally, the system also includes edge-side cache nodes and alarm response devices.

[0030] The end-side acquisition equipment can be image acquisition and computing units deployed on power transmission channel towers, inspection drones, or fixed monitoring points to acquire raw monitoring images of the power transmission channel and perform initial screening using low-confidence thresholds. The cloud-side analysis platform can be a high-performance computing node deployed on a cloud server, data center, or cluster to perform secondary identification of large models and multi-level filtering decisions.

[0031] It should be noted that, in the embodiments of this application, the term "suspected hidden danger image" refers to an image frame that is determined by the lightweight small model on the edge to be a target that may have a hidden danger, but whose confidence level does not reach the high confidence confirmation threshold; the term "first metadata" refers to the structured information that accompanies the upload of the suspected hidden danger image, which includes at least the hidden danger type, confidence level, bounding box coordinates, device identifier, preset position and timestamp; the term "low confidence threshold" refers to the judgment threshold set below the conventional confirmation threshold in order to ensure the recall rate of hidden danger targets.

[0032] The method for identifying potential hazards in power transmission channels provided in this application will be described below with reference to the accompanying drawings. It is understood that in the embodiments of this application, the interactive entities are the end-side data acquisition device and the cloud-side analysis platform. For the meanings of the end-side data acquisition device and the cloud-side analysis platform, please refer to the preceding description.

[0033] Option 1 In the first approach, a lightweight small model is deployed on the edge to perform low-confidence threshold initial screening on the original monitoring images. A large model is deployed on the cloud-based analysis platform to perform secondary identification on the suspected hidden danger images obtained from the initial screening, and spatial filtering, temporal filtering, size filtering and multimodal filtering are performed in sequence to generate the final alarm decision.

[0034] Figure 1 This is a flowchart illustrating a method for identifying potential hazards in power transmission channels, provided in an embodiment of this application. The end-side data acquisition device and cloud-side analysis platform in this embodiment can be... Figure 1 The method involves edge-side data acquisition devices and a cloud-side analysis platform. The steps include:

[0035] Step 101: The end-side acquisition device acquires the original monitoring image of the power transmission channel.

[0036] The end-side acquisition device samples the raw monitoring images of the power transmission channel using a preset image sensor at a set sampling period (e.g., 1 frame per second or 1 frame per 5 seconds). The raw monitoring images can be visible light images, infrared images, or fused visible light and infrared images; this application embodiment does not limit this.

[0037] Step 102: The edge acquisition device performs a low-confidence threshold initial screening on the original monitoring images using a lightweight small model deployed on the edge to identify suspected hidden danger images and their corresponding first metadata.

[0038] Suspected hazard images are identified by a lightweight mini-model deployed on the edge, which performs a low-confidence threshold initial screening on the original monitoring images. The first metadata includes hazard type, confidence level, bounding box coordinates, device identifier, preset position, and timestamp. The low-confidence threshold ensures high recall of hazard targets, while the lightweight mini-model handles basic computation and filters out images without hazards.

[0039] Based on the above solution, by deploying a lightweight small model on the edge and performing initial screening with a low confidence threshold, a large number of images without potential hazards can be filtered on the edge, and only images with suspected potential hazards and first-level metadata can be uploaded to the cloud. This significantly reduces the transmission bandwidth usage and the overhead of invalid inference on the cloud. At the same time, the low confidence threshold setting can ensure a high recall rate for potential hazard targets and avoid missed detections.

[0040] As one possible approach, the lightweight small model is deployed on the edge as follows: Channel-level pruning is performed on the basic detection module, retaining a preset proportion (e.g., the top 70% or top 80%) of convolutional channels before importance scoring to obtain the pruned model; post-training quantization is performed on the pruned model to convert floating-point weights to integer weights (e.g., INT8) to obtain the quantized model; operator fusion is performed on the quantized model to fuse convolution, normalization, and activation sequences into a single operator to obtain the lightweight small model.

[0041] Based on the above scheme, redundant feature channels are removed by channel-level pruning, weight storage precision is reduced by post-training quantization, and memory access and computation stages in the inference process are reduced by operator fusion. This significantly reduces the number of parameters, computation, and storage overhead of the edge model, making it adaptable to the computing power and power consumption constraints of edge devices (such as embedded cameras and edge gateways).

[0042] As one possible implementation method, the low confidence threshold includes: a first threshold for potential hazards to construction machinery, which is used to capture small machinery at a distance and ensure the recall rate of construction machinery; and a second threshold for potential hazards to wildfire smoke, which is lower than the first threshold and is used to ensure the detection rate of initial smoke.

[0043] Based on the above scheme, setting differentiated low-confidence thresholds for different types of hazards can ensure recall while taking into account the differences in imaging characteristics of different hazard targets. For example, construction machinery has blurred features when it is far from the image, so setting a relatively high first threshold can avoid missed detections. Wildfire smoke has a small diffusion range and low contrast in the early stages, so setting a lower first threshold can improve the sensitivity of early smoke detection.

[0044] Step 103: The end-side acquisition device sends images of suspected potential hazards and first metadata to the cloud-side analysis platform. Correspondingly, the cloud-side analysis platform receives images of suspected potential hazards and corresponding first metadata from the end-side acquisition device for the power transmission channel.

[0045] Optionally, the edge-side acquisition device can perform lossy or lossless compression on the suspected hazard images before transmission to further reduce transmission bandwidth. Optionally, information such as device identifiers and preset positions in the first metadata can be used by the cloud-side analysis platform to quickly locate the physical location of the hazard.

[0046] Step 104: The cloud-based analysis platform inputs the suspected hazard images into the cloud-based large model for secondary identification, and obtains accurate identification results of the hazard targets.

[0047] The cloud-based large-scale model employs a collaborative hybrid assignment training mechanism. During the encoder phase, multiple auxiliary detection heads are deployed in parallel for one-to-many intensive supervision, while during the inference phase, only the main detection head is retained to perform one-to-one label assignment. This cloud-based large-scale model is used for precise analysis of the initial screening results.

[0048] Based on the above scheme, the cloud-based large model introduces multiple auxiliary detection heads for one-to-many intensive supervision during the training phase, which can allocate multiple positive samples to each real target, thereby accelerating the encoder's feature learning of multi-scale and multi-form hidden danger targets. During the inference phase, the auxiliary detection heads are completely removed, and only the main detection head is retained to perform one-to-one label allocation. This can improve the accuracy of the initial screening results and reduce the false alarm rate without increasing the inference latency and computational overhead.

[0049] As one possible approach, suspected hazard images are input into a collaborative detection model based on an improved DET (DetectionTransformer) architecture. The main detection head of the collaborative detection model performs one-to-one label assignment and outputs the final detection result. During the training phase, one-to-many dense supervision is performed through multiple auxiliary detection heads of the collaborative detection model, and multiple positive samples are assigned to each real target to accelerate encoder feature learning. During the inference phase, multiple auxiliary detection heads are completely removed.

[0050] Based on the above scheme, the collaborative detection model with improved DET architecture can use the global modeling capability of Transformer to capture the contextual relationship between the hidden danger target and the background of the power transmission channel (such as conductors, towers, and vegetation), thereby improving the identification accuracy of hidden dangers such as obstruction, small targets and complex backgrounds.

[0051] Step 105: Based on the accurate identification results, the cloud-based analysis platform sequentially performs spatial filtering, temporal filtering, size filtering, and multimodal filtering to generate the final alarm decision for the power transmission channel.

[0052] Spatial filtering extracts pixel-level masks of conductors and towers using the BiRefNet segmentation network, calculates the dynamic alarm range based on the pixel-level masks, and then determines the spatial overlap of the dynamic alarm range to which the hazard target domain belongs. Spatial filtering is used to filter low-risk targets in the protected area. Temporal filtering determines the target's motion state and repeated alarm interval based on the historical trajectory information of the hazard target. Temporal filtering is used to suppress stationary, risk-free targets and repeated alarms. Size filtering determines the actual target height based on the ratio of the target detection box to the image size and compares it with the safe height threshold corresponding to the voltage level. Size filtering is used to filter low, risk-free targets. Multimodal filtering calls a multimodal large model to analyze the dynamic evolution characteristics of smoke by combining dynamic prompts and continuous dual-image input. Time-division dynamic prompts are adaptively generated based on the differences in interference characteristics in the current time period. Multimodal filtering is used to distinguish between real wildfire smoke and morning fog and light interference.

[0053] Based on the above scheme, through four levels of filtering—spatial, temporal, dimensional, and multimodal—false alarm targets can be eliminated from multiple dimensions, such as spatial location rationality, target motion state, physical size compliance, and cross-modal semantic consistency. This significantly improves the accuracy and reliability of alarm decisions and reduces the cost of manual review.

[0054] As one possible implementation method, in spatial filtering, the BiRefNet segmentation network performs pixel-level segmentation of the conductors and towers in the suspected hazard image to obtain conductor masks and tower masks; based on the geometric topological relationship of the conductor masks and tower masks, the dynamic alarm range is determined (e.g., a buffer area of ​​N meters on both sides of the conductor and M meters around the tower); if the spatial overlap between the bounding box of the hazard target and the dynamic alarm range is lower than a preset overlap threshold, the hazard target is determined to be a low-risk target in the protected area and is filtered out.

[0055] Based on the above scheme, by calculating the dynamic alarm range through pixel-level masking, the alarm area can be associated with the actual physical structure (conductors, towers) of the power transmission channel, thus avoiding the misreporting of targets appearing in non-protected areas (such as background mountains far from the conductors) as potential hazards.

[0056] As a possible implementation method, in time-series filtering, the cloud-based analysis platform maintains the historical trajectory information of the potential target; determines the target's motion state (stationary, moving, or disappearing) within a preset time window based on the historical trajectory information; if the target is stationary and the duration exceeds the stationary judgment threshold, it is judged as a stationary, risk-free target and filtered; if the time interval between the current alarm and the previous alarm for the same target is less than the repeated alarm suppression interval, it is judged as a repeated alarm and suppressed.

[0057] Based on the above solution, time-series filtering can effectively suppress static repetitive alarms caused by camera shake, fixed obstructions, or periodic changes in light and shadow, thereby improving the timeliness and effectiveness of alarms.

[0058] As one possible implementation method, in size filtering, the actual target height of the potential hazard is calculated based on the ratio of the pixel height of the target detection box to the total height of the image, combined with camera calibration parameters (such as focal length, installation height, and pitch angle). The actual target height is then compared with the safety height threshold corresponding to the voltage level of the current power transmission channel (e.g., 5 meters for 110kV lines and 7 meters for 220kV lines). If the actual target height is lower than the safety height threshold, it is determined to be a low-lying, risk-free target and filtered out.

[0059] Based on the above scheme, by mapping pixel size to actual physical height and comparing it with voltage level safety regulations, low targets (such as low shrubs and small animals) that do not pose a substantial threat to power transmission safety can be filtered out, reducing invalid alarms.

[0060] As a possible implementation method, in multimodal filtering, the cloud-based analysis platform calls a multimodal large model (such as a visual-language large model) to verify suspected smoke hazards; the input includes a continuous double image consisting of the current frame image (first image) and the previous frame image (second image), as well as time-divided dynamic prompt words adaptively generated according to the current time period (such as early morning, night, noon); the multimodal large model combines the prompt words to analyze the dynamic evolution characteristics of the smoke area in the double image (such as diffusion trend, concentration change, morphological irregularity), and distinguishes real wildfire smoke from morning fog and light interference.

[0061] Based on the above scheme, by introducing a multimodal large model and time-division dynamic prompt words, cross-modal semantic understanding capabilities and continuous frame temporal information can be used to effectively distinguish real wildfire smoke from common interference sources such as morning fog, vehicle lights, and streetlights, significantly reducing the false alarm rate of smoke-related hazards.

[0062] The second option The second approach primarily describes the differentiated deployment of lightweight, small-scale end-side models and the adaptive configuration of low-confidence thresholds. This approach can be used in conjunction with the first approach to further optimize end-side initial screening performance.

[0063] Step 201: The edge data acquisition device determines the type of potential hazard to be detected based on the current monitoring scenario.

[0064] For example, hazard types include hazards from construction machinery and hazards from wildfire smoke. The edge data acquisition equipment can determine the type of hazard to be monitored based on preset location information, shooting time period, or user configuration.

[0065] Step 202: The end-side data acquisition device loads the corresponding low confidence threshold according to the type of hidden danger.

[0066] If the current focus is on monitoring potential hazards related to construction machinery, then a first threshold is applied; if the current focus is on monitoring potential hazards related to wildfire smoke, then a second threshold is applied. The second threshold is lower than the first threshold.

[0067] Based on the above scheme, by configuring differentiated low confidence thresholds for different types of hidden dangers, the ability to detect specific high-risk hidden dangers can be prioritized even under the condition of limited computing power on the edge.

[0068] Step 203: The edge acquisition device compresses the basic detection model into a lightweight small model through channel-level pruning, post-training quantization, and operator fusion.

[0069] Specifically, channel-level pruning is performed on convolutional channels based on the scaling factor or gradient importance score of the batch normalization layer; post-training quantization adopts a layer-by-layer or channel-by-channel quantization strategy to map FP32 weights to INT8 integers; operator fusion combines continuous operators such as Conv-BN-ReLU or Conv-BN-SiLU into a single computational kernel to reduce memory access overhead during inference.

[0070] Based on the above scheme, the three-stage compression pipeline of pruning-quantization-fusion can compress the model volume to 10% to 25% of the original model while maintaining the basic detection accuracy, and reduce the inference latency by more than 50%, thus meeting the real-time processing requirements of the edge.

[0071] The third option The third approach primarily describes the collaborative hybrid allocation training mechanism and the cascaded execution logic of multi-level filtering for the large cloud-side model. This approach can be used in conjunction with the first approach to further improve the accuracy of secondary identification and alarm decision-making on the cloud side.

[0072] Step 301: After receiving the suspected hidden danger image, the cloud-based analysis platform inputs it into the collaborative detection model based on the improved DET architecture.

[0073] This collaborative detection model includes an encoder, a main detection head, and multiple auxiliary detection heads used during the training phase. The encoder employs a multi-layer Transformer structure to extract global and local detail features from images suspected of hazard exposure.

[0074] Step 302: During the training phase, one-to-many intensive supervision is performed using multiple auxiliary detection heads; during the inference phase, only the main detection head is retained to perform one-to-one label assignment.

[0075] For example, there are 3-5 auxiliary detection heads, set at different levels of the encoder, used to assign multiple positive samples to each real target (e.g., through center point neighborhood or IoU threshold matching), thereby enhancing the encoder's feature learning ability for difficult samples (such as small targets or occluded targets). During the inference phase, all auxiliary detection heads are removed, and only the main detection head performs one-to-one label assignment based on optimal transport or Hungarian matching, outputting the final detection result.

[0076] Based on the above scheme, the label allocation strategy of decoupling training and inference can provide sufficient positive sample supervision signals during training and avoid the additional overhead caused by NMS post-processing and multi-head prediction during inference, thus achieving the optimal balance between accuracy and efficiency.

[0077] Step 303: Based on the accurate identification results output by the main detection head, the cloud-based analysis platform sequentially triggers spatial filtering, temporal filtering, size filtering, and multimodal filtering.

[0078] For example, the four-level filtering is executed in a cascading manner: the target with potential risks will only enter the next level of filtering after the target has passed the previous level of filtering; if any level of filtering determines that the target is low risk or interference, the target will be discarded directly and no further filtering or alarms will be triggered.

[0079] Based on the above scheme, the cascaded filtering architecture can eliminate obvious false alarms as early as possible while ensuring filtering depth, thereby reducing the waste of computing resources in subsequent filtering and alarm modules.

[0080] Example 1: Construction Machinery Hazard Identification Scenario. Model Input: An excavator is shown below a power line in a suspected hazard image. Spatial Filtering: BiRefNet extracts the power line mask, and the dynamic alarm range is calculated to be 10 meters on both sides of the power line. The excavator's bounding box overlaps with the dynamic alarm range by 85%, exceeding the threshold, so it is retained. Temporal Filtering: The excavator has been continuously moving within the past 10 minutes, classifying it as a moving target, so it is retained. Size Filtering: The actual height is calculated to be 4.2 meters based on the ratio of the detection box to the image size. The current line voltage level is 110kV (safe height threshold 5 meters), which is below the threshold, so it is filtered. No alarm is ultimately generated.

[0081] Example 2: Wildfire Smoke Hazard Identification Scenario. Model Input: Two consecutive images show a grayish-white updraft in a certain area. Spatial Filtering: The area is located within the guide corridor and is retained. Temporal Filtering: The airflow has been continuously spreading over the past 5 minutes, classifying it as a dynamic target and retaining it. Size Filtering: The calculated height of the spreading area exceeds 8 meters and is retained. Multimodal Filtering: A multimodal large model, combined with early morning alert analysis, determines that the airflow has a clear spreading trend and irregular edges, consistent with the characteristics of real wildfire smoke, confirming it as a real hazard and generating a final alarm.

[0082] Based on any of the implementation methods of the first to third aspects mentioned above, as a possible implementation method, after generating the final alarm decision, the cloud-side analysis platform can also push the alarm information to the mobile terminal of the operation and maintenance personnel or the production command system. The alarm information includes at least the type of hidden danger, confidence level, bounding box coordinates, equipment identification, preset position, timestamp and filtering pass record, so that the operation and maintenance personnel can quickly locate and deal with it.

[0083] It is understood that the sequence number of each step in the above embodiments does not imply the order of execution; the execution order of each process should be determined by its function and internal logic. This application embodiment can divide the edge-side acquisition device and cloud-side analysis platform into functional units based on the above method examples. For example, each function can be divided into its own functional unit, or two or more functions can be integrated into one unit.

[0084] The embodiments of this application can be implemented as software functional modules and sold or used as independent products, based on the above method examples. Alternatively, they can be integrated into a computer-readable storage medium and implemented through computer program instructions. The computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media.

[0085] In the various embodiments of this application, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions of different embodiments are consistent and can be referenced by each other. The technical features of different embodiments can be combined to form new embodiments according to their inherent logical relationship.

[0086] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended embodiments and their equivalents.

Claims

1. A method for identifying potential hazards in power transmission channels based on large-scale model collaboration and cloud-edge fusion, characterized in that, Applied to a cloud-based analytics platform, the method includes: Receive images of suspected potential hazards in the power transmission channel and corresponding first metadata from the end-side acquisition equipment; The suspected hazard images are determined by a lightweight small model deployed on the edge, which performs a low-confidence threshold initial screening on the original monitoring images. The first metadata includes hazard type, confidence level, bounding box coordinates, device identifier, preset position and timestamp. The low-confidence threshold is used to ensure the recall rate of hazard targets. The lightweight small model is used to undertake basic calculations and filter out images without hazards. The suspected hazard images are input into the cloud-based large model for secondary identification to obtain accurate identification results of the hazard targets; The cloud-based large model employs a collaborative hybrid allocation training mechanism, deploying multiple auxiliary detection heads in parallel during the encoder stage for one-to-many intensive supervision, while retaining only the main detection head during the inference stage to perform one-to-one label allocation. The cloud-based large model is used for accurate analysis of the initial screening results. Based on the accurate identification results, spatial filtering, temporal filtering, size filtering and multimodal filtering are performed sequentially to generate the final alarm decision for the power transmission channel. The spatial filtering is based on the BiRefNet segmentation network to extract pixel-level masks of conductors and towers, and calculates the dynamic alarm range based on the pixel-level masks, thereby determining the spatial overlap of the dynamic alarm range to which the hidden danger target domain belongs. The spatial filtering is used to filter low-risk targets in the protected area. The time-series filtering determines the target's motion state and repeated alarm interval based on the historical trajectory information of the potential hazard target. The time-series filtering is used to suppress stationary, risk-free targets and repeated alarms. The size filtering determines the actual target height based on the ratio of the target detection box to the image size, and compares it with the safe height threshold corresponding to the voltage level. The size filtering is used to filter low and risk-free targets. The multimodal filtering method calls a multimodal large model to analyze the dynamic evolution characteristics of smoke by combining dynamic prompts and continuous dual-image input. The time-division dynamic prompts are adaptively generated based on the differences in interference characteristics in the current time period. The multimodal filtering method is used to distinguish between real wildfire smoke and morning fog and light interference.

2. The method for identifying potential hazards in power transmission channels based on large-scale model collaboration and cloud-edge fusion as described in claim 1, characterized in that, The lightweight mini-model is deployed on the edge in the following manner: The basic detection module is pruned at the channel level, retaining a pre-set proportion of convolutional channels before importance scoring, to obtain the pruned model. The pruned model is trained and then quantized to convert the floating-point weights into integer weights, resulting in the quantized model. The quantized model is then subjected to operator fusion, which combines convolution, normalization, and activation sequences into a single operator to obtain the lightweight small model.

3. A method for identifying potential hazards in power transmission channels based on large-scale model collaboration and cloud-edge fusion, as described in claim 1 or 2, characterized in that... The low confidence threshold includes: A first threshold for potential hazards in construction machinery is used to detect small machinery at long distances and ensure the recall rate of construction machinery. A second threshold is set for the potential danger of wildfire smoke. This second threshold is lower than the first threshold and is used to ensure the detection rate of initial smoke.

4. A method for identifying potential hazards in power transmission channels based on large-scale model collaboration and cloud-edge fusion, as described in any one of claims 1 to 3, characterized in that... The step of inputting the suspected hazard image into the cloud-based large model for secondary identification includes: The suspected hazard images are input into a collaborative detection model based on an improved DET architecture; The main detection head of the collaborative detection model performs one-to-one label allocation and outputs the final detection result. During the training phase, one-to-many intensive supervision is performed through multiple auxiliary detection heads of the collaborative detection model, and multiple positive samples are assigned to each real target to accelerate encoder feature learning. During the inference phase, the multiple auxiliary detection heads are completely removed.

5. A method for identifying potential hazards in power transmission channels based on large-scale model collaboration and cloud-edge fusion, as described in any one of claims 1 to 4, characterized in that... The spatial filtering includes: The suspected hazard image is input into the BiRefNet segmentation network. The multi-scale contextual information of the conductor and tower is captured by the fusion localization module, the edge details are restored by the reconstruction module, and the global semantics and local textures are fused by the bilateral reference mechanism to generate a pixel-level segmentation mask. The set of conductor centerlines is extracted based on the pixel-level segmentation mask, and the dynamic alarm range is calculated based on camera parameters and the width of the line protection zone. Calculate the intersection-union ratio (IUU) between the target detection box and the dynamic alarm range, and retain the alarm if the IUU is greater than a preset threshold.

6. A method for identifying potential hazards in power transmission channels based on large-scale model collaboration and cloud-edge fusion, as described in any one of claims 1 to 5, characterized in that... The time-series filtering includes: For potential hazards related to construction machinery, calculate the standard deviation of the centroid displacement of the same target in the current frame and historical frames. If the standard deviation is less than a preset pixel threshold and the time since the last valid alarm is less than a preset time interval, suppress the current alarm. For potential hazards such as wildfire smoke, the current frame is compared with historical images of the same location at the same time the previous day. If the smoke pattern does not change, it is determined to be interference from a fixed light source and an alarm is issued.

7. A method for identifying potential hazards in power transmission channels based on large-scale model collaboration and cloud-edge fusion, as described in any one of claims 1 to 6, characterized in that... The size filtering includes: Calculate the ratio of the target detection bounding box height to the image height; Establish a mapping relationship between the ratio and the actual target height based on the camera installation height and focal length; If the actual target height is less than the line safety height threshold for the corresponding voltage level, the current alarm is filtered.

8. A method for identifying potential hazards in power transmission channels based on large-scale model collaboration and cloud-edge fusion, as described in any one of claims 1 to 7, characterized in that... The multimodal filtering includes: The corresponding time-sharing dynamic prompt words are determined based on the current time period. The time-sharing dynamic prompt words include high recall prompt words for daytime periods, negative light source prompt words for nighttime periods, and morning fog differentiation prompt words for early morning periods. The current hazard image and the previous hazard image are concatenated into a dual-channel tensor according to the time dimension, and then input into the multimodal large model. The dynamic evolution characteristics of the smoke region are analyzed by the multimodal large model, and a smoke dynamic score is output. An alarm is triggered when the smoke dynamic score is greater than a preset threshold.

9. A method for identifying potential hazards in power transmission channels based on large-scale model collaboration and cloud-edge fusion, as described in any one of claims 1 to 8, characterized in that... The method further includes: Uncertainty assessment is performed on the identification results, high-entropy samples are automatically marked and added to the incremental sample library, and the high-entropy samples are used to indicate suspected missed or false detection data; The cloud-side large model is incrementally trained using the incremental sample library, and the intermediate layer features of the cloud-side large model are used as supervision signals to optimize the lightweight small model on the edge through feature distillation. The optimized lightweight mini-model is downloaded to the edge acquisition device during off-peak hours via over-the-air download technology to complete the hot replacement of the model.

10. The method for identifying potential hazards in power transmission channels based on large-scale model collaboration and cloud-edge fusion as described in claim 1, characterized in that, Applied to end-side data acquisition devices, the method includes: A lightweight small model is used to perform a low-confidence threshold initial screening on the original monitoring images to identify suspected potential hazard images and their corresponding first metadata. The lightweight small model is used to perform basic calculations and filter out images without potential hazards, while the low-confidence threshold is used to ensure the recall rate of potential hazard targets. Send the suspected hazard image and the first metadata to the cloud-based analysis platform; The suspected hazard images are used to generate a final alarm decision after secondary identification and multiple filtering on the cloud side.

11. The method for identifying potential hazards in power transmission channels based on large-scale model collaboration and cloud-edge fusion as described in claim 10, characterized in that, The lightweight small model is generated by compression through at least one of channel-level pruning, post-training quantization, and operator fusion. The number of parameters in the lightweight small model is less than a preset threshold, and the inference speed meets the real-time requirements.

12. The method for identifying potential hazards in power transmission channels based on large-scale model collaboration and cloud-edge fusion as described in claim 1, characterized in that, Applied to a cloud-based analytics platform, the method includes: Receive suspected hidden danger images and first metadata from the end-side acquisition device. The suspected hidden danger images are determined by the end-side lightweight small model performing a low-confidence threshold initial screening on the original monitoring images. The suspected hazard images are then subjected to secondary identification using a cloud-based large model to obtain accurate identification results. Spatial filtering is performed based on BiRefNet segmentation results, temporal filtering is performed based on historical trajectory information, size filtering is performed based on target bounding box size, and smoke verification filtering is performed based on multimodal large model to generate the final alarm decision.

13. The method for identifying potential hazards in power transmission channels based on large-scale model collaboration and cloud-edge fusion as described in claim 12, characterized in that, The smoke verification filtering based on a multimodal large model includes: The corresponding time-sharing dynamic prompt dictionary is invoked according to the current time period. The time-sharing dynamic prompt dictionary includes differentiated semantic constraints for daytime, nighttime and early morning time periods. Two consecutive frames of hazard images are input into a multimodal large model to analyze the trend of smoke motion vector and regional area change; Whether to trigger an alarm is determined based on the output of the multimodal large model.

14. A hazard identification device, characterized in that, The device includes a processor coupled to a memory storing a computer program; the processor is configured to invoke part or all of the computer program in the memory to implement the method of any one of claims 1 to 9, or the method of any one of claims 10 to 11, or the method of any one of claims 12 to 13.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program or instructions, which, when executed by the hazard identification device, implement the method of any one of claims 1 to 9, or the method of any one of claims 10 to 11, or the method of any one of claims 12 to 13.

16. A computer program product, characterized in that, It includes a computer program or instructions that, when executed by the hazard identification device, implement the method of any one of claims 1 to 9, or the method of any one of claims 10 to 11, or the method of any one of claims 12 to 13.

17. A chip, characterized in that, The chip includes a processor for implementing the method of any one of claims 1 to 9, or the method of any one of claims 10 to 11, or the method of any one of claims 12 to 13.

18. A power transmission channel hazard identification system, characterized in that, The system includes an edge-side data acquisition device and a cloud-side analysis platform; wherein the edge-side data acquisition device is used to perform the method as described in any one of claims 10 to 11, and the cloud-side analysis platform is used to perform the method as described in any one of claims 1 to 9, or to perform the method as described in any one of claims 12 to 13.

Images