Power transmission line multi-objective real-time detection and positioning method and system based on edge lightweight artificial intelligence model

Abstract

The application discloses a power transmission line multi-target real-time detection and positioning method and system based on an edge lightweight artificial intelligence model. The method comprises the following steps: based on hardware perception neural structure search, a lightweight multi-scale feature fusion model is automatically generated under multi-dimensional constraints; the model is compiled into a cross-platform unified model package containing hierarchical meta information; the model is loaded in block form in a memory mapping manner according to the hierarchical meta information, and is instantiated into a model service container supporting differential hot updating and automatic rollback; a runtime manager dynamically configures memory reuse, heterogeneous operator scheduling, batch processing and calculation precision strategies to optimize the energy efficiency ratio according to the device state; low-delay inference is performed, and structured detection results are output; accurate positioning and identification of facilities are realized by associating with a device account, and difficult sample-driven cloud continuous evolution is returned. Efficient real-time detection and positioning of multi-target power transmission lines are realized, and the detection efficiency, energy efficiency ratio and system reliability of the edge device are significantly improved.

Description

[0001] This case claims priority to the invention patent filed on July 14, 2026, with application number 2026110432288, entitled "Method and System for Real-time Detection and Positioning of Multiple Targets in Transmission Lines Based on Lightweight Artificial Intelligence Model at the Edge End", the entire text of which is incorporated herein by reference. Technical Field

[0002] This invention belongs to the field of computer vision and edge computing technology, specifically relating to a method and system for real-time detection and localization of multiple targets in power transmission lines based on a lightweight artificial intelligence model at the edge. Background Technology

[0003] As a critical infrastructure of the power system, the safe and stable operation of transmission lines is directly related to the normal functioning of the national economy. Traditional manual inspection methods suffer from significant problems such as low efficiency, high cost, and high risk, making them unsuitable for the operation and maintenance needs of large-scale power grids. With the rapid development of artificial intelligence technology, deep learning-based visual inspection methods are gradually being applied to transmission line inspection tasks. By analyzing image data collected by drones or fixed monitoring equipment, automatic detection and positioning of multiple targets, such as towers, insulators, conductors, and auxiliary hardware, can be achieved.

[0004] However, existing technologies still have the following significant shortcomings in practical applications:

[0005] First, there is a significant contradiction between model complexity and limited resources on edge devices. Existing high-precision target detection models typically have a large number of parameters and high computational complexity, with stringent requirements for memory, computing power, and power consumption, making them difficult to deploy directly on edge computing devices with limited computing power and sensitive power consumption.

[0006] Second, the cross-platform compatibility of model deployment is poor. Different edge chips have different instruction sets, operator libraries, and inference frameworks, making model porting cumbersome and lacking a unified compilation and encapsulation mechanism, resulting in long development and deployment cycles and high costs for algorithms.

[0007] Third, edge-end model updates and maintenance are difficult. Power inspection equipment is usually deployed at high altitudes or in remote areas, and physical restarts are costly and risky. Traditional model update methods require service interruption and equipment restart, and lack hot updates and fault self-healing capabilities.

[0008] Fourth, insufficient adaptability to the runtime environment. The load, power consumption, and temperature of edge devices change dynamically, and existing systems lack a mechanism to dynamically adjust the inference strategy based on the device status, making it difficult to maximize energy efficiency while ensuring detection accuracy.

[0009] Fifth, the models lack the ability to continuously evolve. Existing solutions are mostly one-way processes, which cannot effectively utilize the difficult samples encountered at the edge to feed back into model optimization, making it difficult to form a closed-loop iterative mechanism from data collection to model upgrade.

[0010] To address the aforementioned technical challenges, there is an urgent need for a technical solution that enables efficient, reliable, and evolving real-time multi-target detection and positioning of power transmission lines on resource-constrained edge devices. Summary of the Invention

[0011] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method and system for real-time detection and localization of multiple targets in power transmission lines based on a lightweight artificial intelligence model at the edge, thereby solving the technical problems of excessive model complexity, difficulty in cross-platform deployment, lack of hot update mechanism, low energy efficiency during operation, and lack of continuous evolution capability in the prior art.

[0012] In a first aspect, embodiments of this application provide a method for real-time detection and localization of multiple targets in transmission lines based on a lightweight edge-end artificial intelligence model, the method comprising:

[0013] Based on hardware-aware neural architecture search, a lightweight multi-scale feature fusion model is automatically generated under multi-dimensional constraints, which integrates a partially convolutional backbone network with cross-modal attention.

[0014] The model is exported as a standard intermediate representation, and mixed-precision quantization and automatic operator kernel tuning are performed for the target platform. The model is then compiled to generate a cross-platform unified model package containing hierarchical meta-information.

[0015] Based on the hierarchical metadata in the unified model package, the model functions are divided into blocks and loaded on demand using memory mapping, and instantiated into model service containers that support differential hot updates and automatic rollback, and the model handle of the container is returned.

[0016] Based on the model handle, the runtime manager monitors the device load, memory, power consumption and temperature status in real time, and dynamically coordinates the configuration of memory reuse, heterogeneous operator scheduling, batch processing and calculation accuracy strategies to optimize the energy efficiency ratio.

[0017] Based on the configuration optimization strategy, the model handle-driven acceleration unit is invoked to perform low-latency forward inference, and the post-processed output is a structured detection result containing category, location, confidence and configuration identifier.

[0018] Based on the structured detection results and associated equipment ledger information, facilities can be accurately located and identified. Difficult samples are sent back to the cloud to drive model retraining, forming a continuous evolutionary closed loop from edge to cloud.

[0019] Secondly, embodiments of this application provide a real-time multi-target detection and localization system for transmission lines based on a lightweight edge-end artificial intelligence model, applied to the real-time multi-target detection and localization method for transmission lines based on a lightweight edge-end artificial intelligence model as described in the first aspect. The system includes:

[0020] Model generation module: configured to automatically generate a lightweight multi-scale feature fusion model that uses a partially convolutional backbone network and cross-modal attention fusion under multi-dimensional constraints based on hardware-aware neural architecture search.

[0021] Model compilation module: configured to export the model as a standard intermediate representation, perform mixed-precision quantization and automatic operator kernel tuning for the target platform, and compile to generate a cross-platform unified model package containing hierarchical meta-information;

[0022] Model deployment module: configured to load and instantiate model functions into blocks on demand using memory mapping based on the hierarchical meta information in the unified model package, and return the model handle of the container;

[0023] Runtime management module: Configured to monitor device load, memory, power consumption and temperature status in real time based on the model handle, and dynamically coordinate the configuration of memory reuse, heterogeneous operator scheduling, batch processing and calculation accuracy strategies to optimize energy efficiency ratio;

[0024] Inference execution module: Configured with a configuration-based optimization strategy, it calls the model handle to drive the acceleration unit to perform low-latency forward inference, and outputs a structured detection result containing category, location, confidence and configuration identifier after post-processing;

[0025] Continuous Evolution Module: Configured to accurately locate and identify facilities by associating the structured detection results with equipment ledger information, and to send difficult samples back to the cloud to drive model retraining, forming a continuous evolution closed loop from edge to cloud.

[0026] Thirdly, embodiments of this application provide an electronic device, including:

[0027] processor;

[0028] Memory used to store processor-executable instructions;

[0029] The processor is configured to implement, when executing the instructions, the real-time detection and localization method for multiple targets on power transmission lines based on a lightweight edge-end artificial intelligence model as described in the first aspect.

[0030] Fourthly, embodiments of this application provide a computer-readable storage medium storing a program that instructs a device to execute the method for real-time detection and localization of multiple targets in transmission lines based on a lightweight edge-end artificial intelligence model as described in the first aspect.

[0031] Compared with existing technologies, this invention has the following advantages: It automatically generates lightweight models through hardware-aware neural architecture search, incorporating latency, energy consumption, and memory usage into optimization objectives, significantly reducing model complexity; it employs a partial convolutional backbone network and channel-selective fusion mechanism, greatly reducing computational overhead while maintaining detection accuracy. Through hybrid precision quantization and automatic operator kernel tuning, it compiles and generates a unified model package containing hierarchical metadata, supporting direct loading and execution on various edge chips, solving the fragmentation problem of model porting. Through block memory mapping loading, differential hot updates, and atomic switching mechanisms, it ensures that model updates do not interrupt inference services; it supports pre-warm-up verification and automatic rollback, ensuring automatic system recovery in case of update failure. The runtime manager monitors device status in real time and dynamically coordinates memory reuse, heterogeneous operator scheduling, batch processing, and computational precision strategies, maximizing the processing power per unit of energy consumption while meeting detection accuracy and latency constraints, extending the battery life of battery-powered devices. Through difficult sample discrimination, desensitization and backhaul, cloud incremental learning, and differential update distribution, it forms a complete closed loop from edge to cloud, enabling the model to be continuously optimized in practical applications, improving detection performance for long-term deployment. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of a method for real-time detection and localization of multiple targets in power transmission lines based on a lightweight artificial intelligence model at the edge, provided as an embodiment of this application.

[0033] Figure 2 This is an architecture diagram of a real-time multi-target detection and localization system for power transmission lines based on a lightweight edge-end artificial intelligence model, provided as an embodiment of this application.

[0034] Figure 3 A schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0035] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0036] It should be noted that in the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the specification of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application.

[0037] Based on the embodiments described in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0038] Example 1

[0039] Figure 1 This is a schematic flowchart illustrating a method for real-time multi-target detection and localization of power transmission lines based on a lightweight edge-end artificial intelligence model, provided as an embodiment of this application. Figure 1 As shown, a method for real-time detection and localization of multiple targets in power transmission lines based on a lightweight edge-end artificial intelligence model includes:

[0040] S1. Based on hardware-aware neural architecture search, a lightweight multi-scale feature fusion model is automatically generated under multi-dimensional constraints, employing a partially convolutional backbone network and cross-modal attention fusion. Under hardware resource constraints, a lightweight neural network model that balances accuracy and efficiency is automatically searched and generated, capable of processing both visible light and infrared image inputs simultaneously.

[0041] The hardware-sensing neural architecture search includes the following steps:

[0042] Step 1.1: Construct a hardware performance predictor for the target edge chip to evaluate the latency, power consumption and memory usage of candidate substructures. The predictor is trained using substructure performance data sampled in advance on the target chip.

[0043] The system first constructs a hardware performance predictor for the target edge chip. This hardware performance predictor is a lightweight neural network model that rapidly estimates the performance of any candidate neural network substructure on a specific chip without actual deployment and operation. For example, when the target chip is the Cambricon MLU220, developers pre-sample a large number of different substructures on the chip in a laboratory environment, recording the latency (time required to complete one inference operation), energy consumption (power consumed during inference), and memory usage (memory space occupied during runtime) of each substructure during inference tasks. This sampled data constitutes the training dataset used to train the hardware performance predictor. After training, the predictor can quickly provide estimates of latency, energy consumption, and memory usage for new candidate substructures, thus avoiding the high cost of actual deployment and operation for each evaluation and significantly accelerating the subsequent search process.

[0044] Step 1.2: Define a multi-dimensional search space, which includes the convolution kernel size, channel expansion factor, attention mechanism type and location, and skip connection configuration, and traverse the search space using a gradient-optimized differentiable search strategy.

[0045] The system defines a search space with multiple dimensions. Specifically, the search space includes the following dimensions: kernel size, i.e., the side length of the filter in the convolution operation, which can be 3×3, 5×5, or 7×7; channel expansion factor, i.e., the proportion by which the number of feature map channels is amplified in the module, which can be 1x, 2x, or 4x; attention mechanism type and location, i.e., whether an attention module is introduced and its insertion position in the network, with selectable attention mechanisms including SE modules, CBAM modules, or coordinate attention modules; and skip connection configuration, i.e., the connection method across layers, which can be standard residual connections, dense connections, or no connections. The gradient-optimized differentiable search strategy refers to relaxing the discrete choices in the search space (e.g., choosing between 3×3 and 5×5 convolution kernels) into continuously variable parameters through mathematical transformations, allowing these parameters to be optimized using the gradient descent algorithm. Using this strategy, the system can efficiently traverse the entire search space through backpropagation during the search process, avoiding the exponential computational overhead of exhaustive search.

[0046] To improve search efficiency and reduce supernet training overhead, the differentiable search strategy also introduces a progressive shrinkage mechanism. In the initial search phase, the search space remains intact, and all candidate operations participate in supernet training. As the number of search iterations increases, the system dynamically prunes candidate operation paths whose contribution is lower than a preset pruning threshold based on the structural parameter amplitude of each candidate operation, gradually shrinking the search space. Specifically, after a preset number of iterations, the system calculates the absolute value of the structural parameters of all current candidate operations and eliminates operation paths whose contribution is less than a certain proportion of the maximum structural parameter value. At the end of the search phase, the system retains only the top-ranked candidate operations and performs fine-tuning based on these. Furthermore, the hardware performance predictor uses a meta-learning framework for pre-training. By learning a general hardware-aware representation from multi-source performance data sampled from various edge chips, it quickly adapts to small-sample data collected on the target chip. This allows for usable prediction accuracy with only a small number of substructure performance samples collected on the target chip, significantly reducing the time cost of hardware performance evaluation.

[0047] Step 1.3: Set up a multi-objective optimization function. The optimization function consists of a weighted sum of three normalized penalty terms: detection accuracy loss, latency, energy consumption, and memory usage. Each penalty term is configured with an adjustable weight coefficient to balance the priority relationship between accuracy and hardware efficiency.

[0048] The system employs a multi-objective optimization function to guide the search process. This function is a mathematical expression that simultaneously optimizes multiple potentially conflicting objectives. It consists of a weighted sum of four parts: the first part represents the model's detection accuracy loss on the validation set, reflecting its detection performance; the second part is the inference latency penalty term on the target chip; the third part is the energy consumption penalty term during inference; and the fourth part is the memory usage penalty term during model runtime. The normalization penalty term scales different units of measurement (e.g., latency in milliseconds, energy consumption in joules) to dimensionless values ​​between 0 and 1 for weighted summation. Each penalty term is configured with an adjustable weight coefficient; for example, developers can set the accuracy loss weight to 1.0, the latency weight to 0.5, the energy consumption weight to 0.3, and the memory weight to 0.2. By adjusting these weighting coefficients, developers can flexibly balance the priority between detection accuracy and hardware efficiency according to actual application needs. For example, in battery-powered scenarios, the energy consumption weight can be increased to extend battery life, while in scenarios with high real-time requirements, the latency weight can be increased to ensure response speed.

[0049] Step 1.4: Under the constraints of the multi-objective optimization function, the structural parameters of the search supernet are updated iteratively through gradient descent. After the search is completed, the Pareto optimal substructure is selected as the final model architecture.

[0050] Under the constraints of the aforementioned multi-objective optimization function, the system iteratively updates the structural parameters of the search supernet using the gradient descent algorithm. The search supernet is a large neural network containing all possible substructure choices, and its parameters determine the probability distribution of sampling specific substructures from the search space. Gradient descent is an optimization algorithm that finds the optimal solution by calculating the derivative of the objective function with respect to the parameters and gradually adjusting the parameters in the reverse direction of the derivative. In each iteration, the system samples substructures from the supernet, evaluates their latency, energy consumption, and memory usage using a hardware performance predictor, and calculates the comprehensive optimization objective in conjunction with the detection accuracy loss. After multiple iterations, the search process converges, and the system outputs several substructures located on the Pareto front. Pareto optimality refers to a state where no objective can be further improved without compromising other objectives. These substructures on the Pareto front achieve the optimal balance between accuracy and hardware efficiency; improvement in any one metric will lead to deterioration in others. Finally, based on the specific needs of the actual deployment scenario, such as prioritizing detection accuracy or prioritizing power consumption reduction, the system selects a substructure from the Pareto optimal set as the final model architecture.

[0051] The lightweight multi-scale feature fusion model includes:

[0052] Step 1.5: Construct the backbone network using partial convolution. The partial convolution only performs spatial convolution operations on a portion of the input feature map, while the remaining channels maintain an identity mapping. Cross-channel information fusion is achieved through pointwise convolution to form a T-shaped convolution structure.

[0053] The system employs partial convolution to construct its backbone network. Partial convolution is a computationally intensive convolution method. Its core idea is that while traditional convolution performs convolution operations on all channels, partial convolution selects only a subset (e.g., a quarter) of the channels for spatial convolution, leaving the remaining channels unchanged. For example, with a 64-channel input feature map, partial convolution performs 3×3 convolutions on only 16 channels, outputting the remaining 48 channels without any processing. Then, a 1×1 pointwise convolution fuses the information from all channels. Pointwise convolution refers to a 1×1 kernel, which linearly combines features from different channels. This combination of partial and pointwise convolution is called a T-shaped convolution structure, named for its T-shaped computation path. Using this structure, the computational cost of the backbone network is reduced by approximately 75% compared to traditional convolution, while the feature representation capability is not significantly reduced because the original information of the unconvolutioned channels is preserved.

[0054] The partially convolutional backbone network consists of multiple stacked T-shaped convolutional blocks. Each T-shaped convolutional block includes a partially convolutional layer, an identity mapping branch, and a pointwise convolutional layer. In the partially convolutional layers, the system does not use the same channel selection ratio across all layers. Instead, it employs a hierarchical channel allocation strategy: in the shallow layers of the backbone network, where the feature map spatial resolution is high and the number of channels is low, a lower partially convolutional ratio is used to retain more spatial detail information; in the deeper layers, where the feature map spatial resolution decreases and the number of channels increases, a higher partially convolutional ratio is used to maximize computational compression. Specifically, the partially convolutional ratio is set to half in the shallow layers, one-third in the intermediate layers, and one-quarter in the deep layers. Furthermore, the channels selected for spatial convolution are not fixed but adaptively selected based on the content of the input features during each forward propagation, ensuring that computational resources are always focused on the feature channels with the richest information.

[0055] Step 1.6: Use a double ReLU activation structure in the shallow layers of the backbone network to enhance the response of edge and texture details, and use SiLU or Smish activation functions in the deep layers to maintain smooth gradient propagation.

[0056] The system employs differentiated activation function strategies at different depths of the backbone network. An activation function is a non-linear mathematical function introduced into a neural network that determines whether a neuron is activated. In the shallow layers of the backbone network (those near the input), the system uses a double ReLU activation structure. The ReLU activation function is defined as follows: the output equals the input when the input is positive, and the output is zero when the input is negative. The double ReLU structure refers to the combination of two consecutive convolutions followed by ReLU activation. This design enhances the responsiveness to image edges and texture details, allowing the shallow network to more sensitively capture the high-frequency features of power transmission line equipment, such as the outline of insulator strings and the edges of conductors. In the deeper layers of the backbone network (those far from the input), the system uses either SiLU or Smish activation functions. The SiLU activation function is characterized by its smoothness and non-monotonicity, maintaining stable gradient propagation in deep networks and avoiding the gradient vanishing problem. The Smish activation function also has smoothness characteristics and outperforms SiLU in certain tasks. This layered activation strategy, with shallow layers responsible for capturing details and deeper layers responsible for maintaining gradient flow, collectively improves the model training effect.

[0057] Step 1.7: Embed a cross-modal attention fusion module in the feature fusion network. The module receives multi-scale feature maps extracted from visible light and infrared images respectively, calculates the correlation weights of cross-modal features through a cross-attention mechanism, and realizes adaptive weighted fusion.

[0058] The system embeds a cross-modal attention fusion module into the feature fusion network. Cross-modal refers to data modalities from different sources; in this scheme, it includes visible light images and infrared thermal imaging images. Visible light images provide rich texture and color information, but their effectiveness is poor at night or in inclement weather; infrared images provide thermal radiation information, unaffected by lighting conditions, but with less texture detail. The two modalities are complementary. This module receives multi-scale feature maps extracted from each modality. Multi-scale feature maps refer to feature representations at different resolutions; high-resolution feature maps retain detailed information, while low-resolution feature maps contain semantic information. The module employs a cross-attention mechanism, using visible light features to query the relevance of infrared features, and vice versa, to calculate cross-modal relevance weights. These weights represent the mutual importance of the two modalities at different spatial locations. Finally, based on these weights, the features of the two modalities are adaptively weighted and fused, assigning higher weights to features that are important and lower weights to features that are unimportant, thus achieving complementary advantages.

[0059] The cross-modal attention fusion module employs a bidirectional cross-attention mechanism to mutually enhance visible light and infrared features. Specifically, the module retrieves relevant information from infrared features using visible light features as queries, and vice versa, with the computations in both directions executed in parallel. When calculating attention weights, the system uses a multi-head design, setting up multiple parallel attention heads to capture cross-modal correlations in different subspaces. Each attention head independently calculates its own attention distribution, and the outputs of all heads are finally concatenated and fused through a projection layer. During fusion, the system configures learnable scalar fusion coefficients for each modality. These coefficients are automatically adjusted during model training to adaptively balance the contributions of visible light and infrared information in different scenarios. For example, the fusion coefficient of the visible light channel automatically increases in daytime scenarios, while the fusion coefficient of the infrared channel automatically increases in nighttime or foggy scenarios, achieving dynamic optimization of modal weights without manual intervention.

[0060] Step 1.8: Introduce a channel-selective partial convolutional fusion module at the output of the feature fusion network. After hierarchical upsampling of multi-level feature maps, use a differentiable gating function to dynamically select feature channels with importance higher than a preset threshold for convolutional fusion, thereby suppressing redundant calculations and preserving key geometric information.

[0061] The system introduces a channel-selective partial convolutional fusion module at the output of the feature fusion network. This module first performs hierarchical upsampling on multi-level feature maps. Multi-level feature maps refer to features from different depths of the network; shallow features have high resolution but weak semantics, while deep features have low resolution but strong semantics. Hierarchical upsampling means applying different magnification factors to features at different levels, for example, magnifying shallow features by 2 times and deep features by 4 times, so that all feature maps are unified to the same resolution. A differentiable gating function is a learnable filtering mechanism that calculates an importance score for each feature channel. Channels with scores higher than a preset threshold (e.g., 0.5) are retained and subjected to convolutional fusion, while channels with scores lower than the threshold are skipped. This dynamic filtering mechanism can identify and retain the feature channels most important to the detection task, while suppressing redundant and low-information feature channels, thereby reducing unnecessary computation. At the same time, because key geometric information (such as target boundaries and shape features) is retained, the detection accuracy does not decrease significantly due to channel filtering. This module is one of the key technologies that ensures high detection accuracy even after the model is lightweighted.

[0062] The importance threshold in the differentiable gating function is not fixed but employs an adaptive threshold mechanism. During training, the system statistically analyzes the distribution of importance scores for all channels. After each preset training epoch, the threshold is updated to the median or mean of the importance scores for all channels in the current batch, automatically adjusting the proportion of selected channels as training progresses. After model deployment, the threshold can also be dynamically adjusted based on the current load of the edge device: when the device load is light, the threshold is lowered to retain more channels for convolution fusion, aiming for higher detection accuracy; when the device load is heavy or the battery is low, the threshold is raised to select fewer channels for convolution operations, aiming for lower computational power consumption. The dynamic adjustment of the threshold is issued in real time through the runtime manager, without recompiling the model, achieving a flexible trade-off between accuracy and efficiency at runtime.

[0063] S2. Export the model as a standard intermediate representation, perform mixed-precision quantization and automatic operator kernel tuning for the target platform, and compile to generate a cross-platform unified model package containing hierarchical meta-information. Convert the designed model into a unified intermediate format, perform precision allocation and computational optimization for the target hardware, and package it into an independent file package containing all the information required for operation.

[0064] The hybrid precision quantization and compilation to generate a cross-platform unified model package includes the following steps:

[0065] Step 2.1: Export the model as an ONNX standard intermediate representation. The system first exports the trained model as a standard intermediate representation of the Open Neural Network Exchange Format (ONNX). ONNX is an open-source deep learning model representation standard developed by companies such as Microsoft and Facebook. This format defines a unified set of model data structures, including network layer types, connection relationships, weight parameters, etc. The advantage of this format is that the model can be freely transferred between different deep learning frameworks. For example, a model trained with PyTorch can be exported as ONNX and then loaded and used by other frameworks or compilation tools that support ONNX. In this solution, exporting to ONNX format is the starting point for the subsequent compilation process, laying the foundation for cross-platform deployment.

[0066] Step 2.2: Perform quantization sensitivity analysis on each layer of the model to identify layers sensitive to low precision. These sensitive layers include the query, key-value projection, layer normalization, and upsampling layers of the attention mechanism. The system performs quantization sensitivity analysis on each layer of the model. Quantization sensitivity analysis assesses the degree of decrease in overall model detection accuracy when a layer is converted from high-precision floating-point numbers (e.g., 32-bit floating-point) to low-precision integers (e.g., 8-bit integers). Layers with high sensitivity will experience a significant decrease in accuracy after low-precision quantization; these layers are called sensitive layers. Experimental analysis revealed that the following types of layers are particularly sensitive to low-precision quantization: the query, key-value projection, and upsampling layers in the attention mechanism, which are responsible for converting input features into the three matrices required for attention calculations, and whose numerical range varies considerably; the layer normalization layer, which normalizes the mean and variance of the input and requires high precision; and the upsampling layer, which is responsible for amplifying the feature map resolution and involves interpolation calculations, where low precision can lead to blurred edges. After identifying these sensitive layers, the system can adopt different processing strategies for them in subsequent quantization steps.

[0067] Step 2.3: Execute a mixed-precision quantization strategy based on the sensitivity analysis results. Regular convolutional layers and activation layers are quantized to 8-bit integer precision, while the sensitive layers are retained to 16-bit or 32-bit floating-point precision. The system executes the mixed-precision quantization strategy based on the sensitivity analysis results. Mixed-precision quantization refers to using multiple different numerical precisions simultaneously in the same model, rather than uniformly using the same precision. Specifically, for regular convolutional layers and activation layers, which are not precision-sensitive, the system quantizes them from 32-bit floating-point to 8-bit integer precision. 8-bit integer precision uses one byte to represent a value, occupying three-quarters of the storage space compared to 32-bit floating-point numbers, and is also faster in computation. For the sensitive layers identified in Step 2.2, including attention projection layers, layer normalization layers, and upsampling layers, the system retains them to 16-bit or 32-bit floating-point precision. 16-bit floating-point precision (FP16) uses two bytes, while 32-bit floating-point precision (FP32) uses four bytes. Both provide higher numerical precision than 8-bit integers, thus ensuring the computational accuracy of sensitive layers. This mixed precision strategy maximizes the efficiency gains of quantization while maintaining the overall accuracy of the model.

[0068] Step 2.4: For the processor architecture of the target edge chip, perform kernel automatic tuning of the convolution operators in the model. The tuning includes selecting the optimal implementation from multiple convolution implementation algorithms. The multiple convolution implementation algorithms include at least direct convolution, memory-rearranged convolution, and Winograd convolution.

[0069] The system automatically optimizes the convolution operator kernel for the processor architecture of the target edge chip. Automatic kernel optimization refers to automatically selecting or generating the most efficient operator implementation code for a specific hardware platform. For convolution, the core computational operation in deep learning, there are various implementation algorithms. Direct convolution is the simplest and most intuitive implementation, calculating directly according to the definition of convolution, suitable for small convolution kernels or sparse scenarios. Memory-rearranged convolution, also known as Im2Col plus a general matrix multiplication method, transforms convolution operations into matrix multiplication to utilize a highly optimized matrix operation library, suitable for large convolution kernels or batch processing scenarios. Winograd convolution is a fast algorithm that reduces the number of multiplications through mathematical transformations, suitable for small convolution kernels such as 3×3, significantly improving speed while maintaining computational accuracy. The system actually runs test cases for each algorithm on the target chip, measures its execution time, and then selects the fastest algorithm for each convolutional layer in the model to optimize the overall inference speed.

[0070] Step 2.5: Compile and generate a cross-platform unified model package containing hierarchical meta-information, which includes at least: a description of computation subgraph dependencies, a performance profile of each operator on heterogeneous computing units, a memory access pattern map, a definition of model function block boundaries, and corresponding on-demand loading strategies.

[0071] The system compiles and generates a cross-platform unified model package containing hierarchical metadata. Hierarchical metadata refers to multi-level auxiliary data describing the model's structure and operational characteristics. Specifically, it includes: a description of computational subgraph dependencies, i.e., a dependency graph between various computational nodes in the model, used to determine the execution order; performance profiles of each operator on heterogeneous computing units, i.e., execution latency and energy consumption data for each operator on different computing units such as CPU, NPU, and GPU, used for runtime scheduling decisions; a memory access pattern graph, i.e., the read and write patterns of feature maps at each layer during model execution, used to optimize memory reuse; model functional block boundary definitions, i.e., the segmentation point information for dividing the model into backbone network blocks, neck network blocks, and detection head blocks; and corresponding on-demand loading strategies, i.e., the loading timing and dependencies of each block. This hierarchical metadata, together with the model's executable instructions, constitutes a complete model package, enabling the runtime system to understand the model structure and make intelligent optimization decisions.

[0072] Step 2.6: Package the hierarchical metadata and the model's executable binary code together to form a unified model package in a single file format. The system packages the hierarchical metadata and the model's executable binary code together to form a unified model package in a single file format. Executable binary code refers to the sequence of machine instructions generated after the model is compiled, which can be directly run on the target chip, similar to an EXE file generated after traditional software compilation. The packaging process integrates the metadata and binary code into a single file. This file has a custom format structure; for example, the file header stores version information and a directory index, the middle section stores the hierarchical metadata, and the end section stores the binary code of each block. This unified model package in a single file format has the following advantages: easy distribution and transmission, requiring only one file to manage; easy version management, with complete and self-contained file content; easy loading and parsing, as the system reads all the necessary information from a single file during runtime. This model package supports direct loading and running on various edge chip platforms such as Cambricon, Ascend, and NVIDIA Jetson series, achieving true cross-platform deployment.

[0073] S3. Based on the hierarchical metadata in the unified model package, the model functions are divided into blocks and loaded on demand using memory mapping, and instantiated into model service containers that support differential hot updates and automatic rollback. The model handle of the container is returned. The model file is split into multiple modules according to function, and necessary modules are selectively loaded using efficient memory mapping technology. An inference service that supports non-stop updates and fault recovery is started, and the access credentials for the service are returned.

[0074] The step of loading model functions into blocks on demand using memory mapping based on the hierarchical meta-information in the unified model package includes the following steps:

[0075] Step 3.1: Parse the hierarchical meta-information in the unified model package and extract the boundary definitions of the model function blocks. The blocks include at least a backbone network block responsible for basic feature extraction, a neck network block responsible for multi-scale feature fusion, and at least one detection head block responsible for specific category detection.

[0076] The system first parses the hierarchical meta-information in the unified model package to extract the boundary definitions of model functional blocks. Hierarchical meta-information, auxiliary data embedded in the model package during the model compilation phase, describes the internal structural division of the model. Functional block division refers to dividing the complete neural network model into multiple relatively independent modules according to their functional responsibilities. Specifically, this scheme defines three types of blocks: the backbone network block, responsible for extracting basic features from the original image and serving as the computational core of the model; the neck network block, responsible for fusing features at different scales to enhance the model's ability to detect targets of different sizes; and the detection head block, responsible for outputting the final detection results based on the feature map, including target category, location, and confidence score. The system can configure multiple different detection head blocks, such as a head block for detecting towers, a head block for detecting insulators, and a head block for detecting conductors, each head block focusing on the detection of a specific category of targets.

[0077] Step 3.2: Perform page boundary alignment preprocessing on the unified model package file to ensure that the starting address of each functional block is aligned with the operating system memory page size, so as to support zero-copy memory mapping loading.

[0078] The system performs page boundary alignment preprocessing on the unified model package file. Page boundary alignment refers to adjusting the starting position of each functional block in the file to an integer multiple of the operating system's memory page size. The basic unit for managing memory in the operating system is called a memory page, typically 4KB in size. When the starting address of a block in the file falls exactly at the beginning of a memory page, that block can be directly mapped into memory without additional data copying. Zero-copy is an efficient data loading technique where data is directly mapped from the file system to the process's memory space without the need for copying through user-space buffers. By performing page boundary alignment preprocessing on the model package file, the system can utilize the operating system's memory mapping mechanism to directly map model blocks from the file into memory, significantly reducing loading time and memory usage.

[0079] Step 3.3: Upon edge device startup, the backbone network block and neck network block are loaded into memory space via a memory mapping mechanism. The memory mapping mechanism, provided by the operating system, maps file content to the process's memory address space, typically implemented through system calls. After the system call returns, the application can access the file content like regular memory, while the operating system handles the background reads data from disk into memory as needed. This solution treats the backbone network block and neck network block as resident modules, loading them immediately upon device startup. These two modules are fundamental modules shared by all detection tasks; whether subsequent detection tasks involve towers or insulators, the same backbone and neck networks are required. Preloading these modules avoids reloading them each time a detection task is switched, thus reducing latency.

[0080] Step 3.4: Based on the current inspection task requirements, dynamically load the corresponding inspection head block as needed. When the task requirements change, unload the current inspection head block and load a new one, achieving dynamic switching of inspection heads. The system dynamically loads the corresponding inspection head block as needed based on the current inspection task requirements. For example, when the inspection task requires detecting whether a tower is tilted or damaged, the system loads the tower inspection head block; when the task changes to detecting whether insulators have experienced flashover or damage, the system unloads the tower inspection head block, releasing the memory space it occupies, and then loads the insulator inspection head block. The core advantage of this dynamic switching mechanism lies in memory optimization: although the backbone network block and neck network block are relatively large, they are only loaded once; although each inspection head block is relatively small, it can be loaded as needed according to the task, without having to reside in memory simultaneously. For edge devices with limited memory resources, this mechanism allows a single device to support multiple different inspection tasks without having to accommodate all model parameters simultaneously.

[0081] Step 3.5: Create a model service container instance, register the memory addresses of each loaded functional block to the container management table, and generate a unique model handle to return to the caller. Multiple callers can share the same model handle to reuse the loaded backbone and neck network blocks. The system creates a model service container instance and registers the memory addresses of each loaded functional block to the container management table. The model service container is a software abstraction responsible for managing the model's lifecycle, including loading, inference, updating, and unloading operations. The container management table is an internal data structure that records the starting address and size of the backbone, neck, and detection header blocks in memory. After loading, the system generates a unique model handle and returns it to the caller. The model handle is similar to an ID number or pointer; all subsequent inference requests from the caller need to carry this handle, and the system uses the handle to find the corresponding model service container instance. Multiple callers can share the same model handle; for example, multiple applications or multiple threads within the same application can use the handle simultaneously for inference. Since the backbone network block and neck network block are read-only, multiple callers can safely reuse them without having to load a separate copy for each caller, thus saving memory resources.

[0082] The model service container that supports differential hot updates and automatic rollback includes:

[0083] Step 3.6: The model service container maintains two independent model handles: the currently running version and the version to be updated. The currently running version handle points to the model being used, while the version to be updated handle is used to receive the chunked loading of new models.

[0084] The model service container internally maintains two independent model handles. A model handle is a reference to the model service container, used by the caller to access the model for inference. The currently running version handle points to the older model that is currently in use, and all ongoing inference requests are executed through this handle. The version to be updated handle is a reserved handle slot, initially empty, specifically used to receive the loading of new models. This dual-handle design isolates the running model from the model being loaded, ensuring that the loading process of a new model does not interfere with the currently running inference service, laying the foundation for non-downtime updates.

[0085] Step 3.7: When a model update instruction is received, the model service container obtains the changed blocks of the new model by downloading. The changed blocks are differential data packets calculated based on the differences in hierarchical metadata between the old and new models. Only the changed blocks are transmitted to reduce network transmission volume.

[0086] When the cloud management system issues a model update command, the model service container initiates the update process. The system first compares the hierarchical metadata of the old and new models to calculate the differences between them. A differential data packet is an incremental update file that only contains the differences between the old and new models, rather than the complete model file. For example, the new model might only adjust a few convolutional layer parameters in the detection head block, while keeping the backbone and neck blocks unchanged. In this case, the differential data packet only contains the changed parts of these convolutional layers. The cloud server pre-generates this differential data packet, and the model service container downloads it over the network. Because differential data packets are typically much smaller than the complete model file, this mechanism can significantly reduce network transmission time and bandwidth consumption, which is particularly important for power line inspection equipment deployed in remote areas with limited network conditions.

[0087] Step 3.8: Load the downloaded changed blocks into the reserved area in memory to be updated using memory mapping, and combine them with the loaded unchanged blocks to form a complete new model, thus completing the construction of the handle for the version to be updated.

[0088] The model service container loads the downloaded change chunks into a reserved update area in memory. Memory mapping is a technique that directly maps file content to the process's memory space without explicit data copying. The reserved update area is a memory space pre-allocated during the model service container's initialization, specifically for receiving update data. After loading, the system concatenates the change chunks with the unchanged chunks already loaded in memory. For example, if the backbone network block of the new model is identical to the old model, this memory can be directly reused; if only the detection header block has changed, the system combines the newly downloaded detection header block with the original backbone network block and neck network block to form a complete new model. After concatenation, the update version handle is constructed, pointing to the memory structure of this new model, but the new model is not yet in use.

[0089] Step 3.9: Perform pre-warm-up inference verification on the handle to be updated. Monitor its detection accuracy and inference latency during a preset number of consecutive inference tasks. If the detection accuracy does not decrease beyond a preset threshold and there are no inference timeouts or crashes during the verification period, then perform an atomic switch operation. The system performs pre-warm-up inference verification on the handle to be updated. Pre-warm-up inference verification involves running the new model with a small number of test samples before the formal switch to verify its normal operation. The system monitors the detection accuracy and inference latency of the new model during a preset number of consecutive inference tasks (e.g., 10 or 20 times). Detection accuracy is calculated by comparing it with pre-labeled verification samples, and inference latency is measured by timing. If the detection accuracy does not decrease beyond a preset threshold during the verification period (e.g., a decrease of no more than 1 percentage point relative to the old model), and there are no inference timeouts or program crashes, then the new model can work normally, and the system performs an atomic switch operation. This verification step is crucial to ensuring system reliability and preventing defective models from being incorrectly deployed to the production environment.

[0090] Step 3.10: The atomic switching operation is implemented by updating the active handle pointer of the model service container within a single instruction cycle, replacing the current running version handle with the version handle to be updated, and the switching process does not interrupt the ongoing inference request.

[0091] Atomic switching operations are implemented by updating the active handle pointer of the model service container. An atomic switch means that the switching operation is completed within a single, indivisible instruction cycle; it either succeeds completely or fails completely, without any intermediate states. Specifically, the model service container maintains an active handle pointer that points to the currently used model handle. All inference requests entering the service container first read this pointer and then perform inference based on the model pointed to by the pointer. The atomic switch operation modifies this pointer from the address of the currently running version handle to the address of the version handle to be updated. Since pointer assignment is atomic on most processor architectures, the entire switching process is completed in a very short time. Ongoing inference requests are unaware of the switch, thus achieving hot updates without interrupting service.

[0092] Before performing the atomic switch operation, the system first executes a request drainage process to ensure that no inference requests being processed are lost during the switch. Specifically, upon receiving the switch instruction, the model service container immediately returns a "switching in progress" status flag to the caller, notifying the caller to pause sending new inference requests or automatically queue new requests. The container waits for all currently running inference tasks to complete before atomically switching the active handle pointer from the current running version to the version to be updated. This process has a maximum waiting time; if the wait times out, the switch is forcibly completed, and a retry instruction is returned for the timed-out request. After the switch is complete, the container changes the "switching in progress" status to the "running" status and resumes receiving new requests. For requests received before the switch but not yet processed, the system uses a request replay mechanism to resubmit these requests to the new model handle for inference execution, ensuring no requests are lost. The entire process is completely transparent to the caller; upper-layer applications only perceive a very brief state transition.

[0093] Step 3.11: If the version handle to be updated experiences an accuracy drop exceeding a preset threshold, inference timeout, or crash during the verification period, the version handle to be updated will be automatically discarded and the memory resources it occupies will be released. The model service container will continue to use the currently running version handle, thus achieving automatic rollback.

[0094] If an anomaly occurs in the handle to be updated during the pre-verification period, the system automatically performs a rollback operation. Anomalies include: a decrease in detection accuracy exceeding a preset threshold, indicating that the new model's performance is substandard; inference timeouts, indicating that the new model's computational efficiency is problematic; and crashes, indicating that the new model has serious defects causing program errors. When any of these situations occur, the system automatically discards the handle to be updated, releases all resources it occupies in memory, and then resets the slot for the handle to be updated to empty. The model service container continues to use the currently running handle to process all inference requests, and the service remains unaffected. This automatic rollback mechanism requires no manual intervention, can promptly block problematic model updates, and ensures the continuous and stable operation of edge devices.

[0095] Step 3.12: The model service container will report the success or failure status information to the cloud management system. In case of failure, the reason for the exception and the rollback operation record will be attached.

[0096] The model service container reports the update results to the cloud management system. Regardless of whether the update is successful or fails, the system generates a status report. If the update is successful, the status information includes the new model's version number, the timestamp of the switch completion, and statistics on pre-warm-up verification. If the update fails, the status information, in addition to the failure identifier, includes the reason for the anomaly (e.g., accuracy drop exceeding a threshold, inference timeout, or crash) and a detailed record of the rollback operation. After receiving this status information, the cloud management system can record it in the operations log for technical personnel to analyze. If a new model repeatedly fails to update on multiple edge devices, the cloud system can mark the model version as problematic, stop further distribution, and prevent the problem from spreading.

[0097] S4. Based on the model handle, the runtime manager monitors the device load, memory, power consumption, and temperature status in real time, and dynamically coordinates and configures memory reuse, heterogeneous operator scheduling, batch processing, and computational accuracy strategies to optimize energy efficiency. It continuously monitors the device's computational load, remaining memory, power level, and temperature, and dynamically adjusts memory usage, computational task allocation, request processing mode, and computational accuracy to achieve maximum processing capacity per unit of energy consumption. This includes the following steps:

[0098] Step 4.1: Real-time collection of CPU and NPU utilization, available memory, remaining power, and chip temperature.

[0099] The runtime manager collects the dynamic context state of edge devices in real time through operating system interfaces and hardware driver layers. The runtime manager is a background system service module responsible for monitoring device status and making optimization decisions. Dynamic context state refers to various indicators that change over time and reflect the current operating status of the device. Specific indicators collected include: CPU utilization, reflecting the busyness of the computing cores; NPU utilization, reflecting the usage of the AI ​​acceleration chip; available memory capacity, reflecting the remaining memory space; remaining battery percentage, reflecting the power status; and chip junction temperature, reflecting the chip's heat generation. These indicators are obtained through system call interfaces provided by the operating system and sensor reading interfaces provided by the hardware driver layer. The collection frequency can be set according to actual needs, such as once per second.

[0100] Step 4.2: Input the collected status into the multidimensional optimization strategy library.

[0101] The runtime manager inputs the collected dynamic context state into the built-in multidimensional optimization strategy library. This library is a pre-generated lookup table or decision model that stores the optimal configuration schemes for different combinations of device states. The library is generated as follows: developers simulate various combinations of device states (e.g., low battery + high temperature, high load + normal temperature) in a laboratory environment, testing the performance of different configuration schemes and recording the configuration scheme with the highest energy efficiency under each state. The configuration schemes cover four dimensions: memory reuse strategy, which determines how memory resources are managed; heterogeneous operator scheduling strategy, which determines how computing tasks are allocated to different computing units; batch processing parameters, which determine how multiple inference requests are merged; and computational precision strategy, which determines which numerical precision to use for computation. Based on the currently collected state, the runtime manager searches the strategy library for a matching configuration scheme.

[0102] Step 4.3: Execute the memory reuse strategy, allocate and reclaim memory blocks from the pre-allocated memory pool, and reuse the same address for intermediate feature tensors with the same shape.

[0103] The runtime manager configures memory reuse strategies. Memory reuse strategies are management methods that reduce the number of memory allocations and deallocations by reusing allocated memory blocks. The system pre-allocates a memory pool, a large contiguous block of memory, specifically for storing intermediate feature tensors during inference. The feature map size refers to the height, width, and number of channels of the feature map. When performing an inference task, the system allocates a memory block of the required size from the memory pool based on the current feature map size. For intermediate feature tensors with the same shape in multiple consecutive inferences, the system reuses the same memory address; that is, it is not released after the previous inference is completed, and the same address is used again in the next inference. This reuse mechanism avoids frequent memory allocation and deallocation operations, reduces memory management overhead, and also reduces memory fragmentation.

[0104] Step 4.4: Execute the heterogeneous operator scheduling strategy to dynamically allocate operator tasks among CPU, NPU and GPU based on load and temperature.

[0105] The runtime manager configures the heterogeneous operator scheduling strategy. Heterogeneous operator scheduling refers to allocating different computational tasks in a neural network to different types of processors for execution. Edge devices are typically equipped with multiple processors: the central processing unit (CPU) is suitable for general-purpose computing and serial tasks; the neural network acceleration unit (NPU) is specifically optimized for matrix operations and has the highest energy efficiency; and the graphics processing unit (GPU) is suitable for highly parallel computing tasks. The runtime manager dynamically determines which processor executes each operator based on the current load of each processor and the chip temperature. When the temperature of a chip exceeds a preset temperature threshold (e.g., 85 degrees Celsius), the system schedules the computationally intensive operators on that chip to a lower-temperature computing unit to prevent overheating from causing performance degradation or hardware damage.

[0106] Step 4.5: Configure batch processing parameters to dynamically merge concurrent requests for batch execution and reduce batch size when battery is low.

[0107] The runtime manager configures batch processing parameters. Batch processing refers to merging multiple independent inference requests into a single batch and submitting them to the acceleration unit for execution all at once. This fully utilizes the hardware's parallel computing capabilities and improves throughput. The runtime manager dynamically determines the batch size based on the arrival rate of current inference requests and the length of the waiting queue. When the device has sufficient battery power, a larger batch size is used to achieve the highest processing efficiency. When the battery level is detected to be below a first battery threshold (e.g., 30%), the system reduces the batch size because smaller batches consume less power and can extend battery life. When the battery level is below a second battery threshold (e.g., 15%), the system distributes the inference tasks of multiple requests over time instead of merging them into batches to avoid short-term power spikes and further extend device runtime.

[0108] Step 4.6: Implement the calculation precision strategy. Under normal conditions, use INT8 precision. When the power is low or the temperature is high, downgrade to a lower bit precision or skip non-critical detection heads.

[0109] The runtime manager configures the computational precision strategy. The computational precision strategy refers to dynamically adjusting the numerical precision used during inference based on the device's state. Eight-bit integer precision (INT8) is the default precision, used when battery power is sufficient and temperature is normal, providing good detection accuracy and fast inference speed. When battery power is low or temperature is too high, the system degrades to lower bit precision quantization inference, such as four-bit integer precision (INT4) or two-bit integer precision (INT2). This precision results in less computation and lower power consumption, but detection accuracy will also decrease. Furthermore, in extreme cases, the system can choose to skip non-critical detection head modules, such as performing only tower detection while skipping insulator detection, to reduce the computational load. This dynamic precision adjustment mechanism allows the device to maintain basic detection functions even when battery power is running low or temperature is too high, rather than shutting down and ceasing service.

[0110] Step 4.7: The runtime manager combines the configuration items of the above four dimensions to generate a joint optimization strategy for the current device state, so as to maximize the number of inference tasks that can be processed per unit of energy consumption while meeting the preset minimum detection accuracy and maximum inference latency constraints.

[0111] The runtime manager collaboratively combines configuration items across four dimensions to generate a joint optimization strategy. Collaborative combination means that the configurations across the four dimensions are not chosen independently, but rather influence and jointly determine each other. For example, reducing batch size can reduce power consumption but may increase inference latency; reducing computational precision can reduce computational load but may affect detection accuracy. The runtime manager needs to optimize while meeting two hard constraints: a preset minimum detection precision (e.g., average precision not less than 90%) and a maximum inference latency (e.g., single-frame inference latency not exceeding 50 milliseconds). Under these constraints, the system aims to maximize the number of inference tasks that can be processed per unit of energy consumption, i.e., how many images can be processed per joule of power consumed. This optimization objective directly reflects the system's energy efficiency ratio and is crucial for battery-powered edge devices.

[0112] When combining strategies across four dimensions—memory reuse, heterogeneous operator scheduling, batch processing, and computational precision—conflicts may arise between configuration objectives across different dimensions. For example, high-precision mode requires a higher bit depth for computational precision strategies, conflicting with the need to reduce precision under low-power conditions; large batch processing can increase throughput but increases single-inference latency, conflicting with low-latency constraints. To resolve such conflicts, the system pre-defines priority levels for each dimension in the strategy library: detection precision requirements are the highest priority and cannot be sacrificed; inference latency requirements are the second highest priority and must be strictly guaranteed; energy efficiency optimization is the third priority. When strategy conflicts occur, the runtime manager arbitrates according to this priority level, prioritizing ensuring that detection precision does not fall below the preset minimum standard, then ensuring that inference latency does not exceed the preset maximum latency, and maximizing energy efficiency under these conditions. The specific arbitration method is as follows: the system pre-stores multiple candidate strategy combinations and their performance indicators under different device states in the strategy library. When a conflict is detected, the system filters out all candidate combinations that meet the hard constraints of precision and latency, and then selects the combination with the highest processing power per unit of energy consumption for execution, ensuring the determinism and reproducibility of the conflict resolution process.

[0113] Step 4.8: The runtime manager applies the configured optimization strategy to the model service container pointed to by the model handle, and records the current state and adjustment action to the local log after each strategy adjustment for subsequent offline analysis and optimization of the strategy library.

[0114] The runtime manager applies the configured optimization strategies to the model service container pointed to by the model handle. Upon receiving the new configuration, the model service container executes it according to the new configuration in subsequent inference tasks. Simultaneously, after each strategy adjustment, the runtime manager records the current device state and the adjustment action to a local log. Each record in the local log includes at least: a timestamp, the collected state metrics, the selected configuration scheme, and the reason for the adjustment (e.g., which threshold condition was triggered). This log data is stored locally on the edge device and uploaded to the cloud periodically or on demand. Developers can use this log data to perform offline analysis and optimization of the strategy library, such as discovering that the current strategy is not optimal under certain state combinations or that certain threshold settings are unreasonable, thereby updating the strategy library to make subsequent strategy decisions more accurate and efficient.

[0115] S5. Based on the configuration-based optimization strategy, the model handle-driven acceleration unit is invoked to perform low-latency forward inference. After post-processing, a structured detection result containing category, location, confidence level, and configuration identifier is output. According to the current optimization strategy, computing hardware is invoked to perform fast inference, decoding and filtering the original output to generate a standardized detection report containing target type, spatial location, recognition reliability, and configuration information for this run.

[0116] The configuration-based optimization strategy invokes the model handle-driven acceleration unit to perform low-latency forward inference, and outputs a structured detection result containing category, location, confidence level, and configuration identifier after post-processing, including the following steps:

[0117] Step 5.1: Invoke the model handle and send the multimodal input data to the acceleration unit for inference according to the optimization strategy. The model handle is a string of identification information held by the caller; the model service container uses this handle to locate the corresponding model instance. Multimodal input data refers to preprocessed visible light images and infrared thermal imaging images, which have been registered and aligned. The system sends this input data to the Neural Processing Unit (NPU) for forward inference computation according to the optimization strategy previously configured by the runtime manager. Forward inference computation refers to the process of data being passed forward layer by layer from the network input layer, and the output result obtained after computation at each layer. The NPU is specifically hardware optimized for neural network computation, enabling efficient execution of operations such as convolution and pooling.

[0118] Step 5.2: The runtime manager dynamically allocates different operators to the CPU, NPU or GPU for parallel execution according to the heterogeneous operator scheduling configuration, and adopts a double buffering mechanism to achieve overlap between data loading and computation.

[0119] During inference execution, the runtime manager dynamically allocates different operators in the model to different processors for parallel execution based on the heterogeneous operator scheduling configuration. Operators are the basic computational units in neural networks, such as convolution operators, pooling operators, and activation operators. Different types of operators are suitable for execution on different processors: for example, large-scale convolution operations are suitable for execution on an NPU, small-scale convolutions or special operators are suitable for execution on a CPU, and highly parallel matrix operations are suitable for execution on a GPU. The runtime manager simultaneously allocates operators that can be executed in parallel to different processors, achieving multi-processor collaborative computing. Simultaneously, the system employs a double-buffering mechanism, using two buffers to alternately load and compute data: while data in one buffer is being computed, the other buffer simultaneously loads the next batch of data, thus achieving overlapping execution of data loading and computation, reducing processor idle waiting time.

[0120] Step 5.3: Lock resources for the critical path operator and increase its priority to ensure that the delay is controlled within the upper bound of deterministic delay.

[0121] For critical path operators in the model, the runtime manager locks computing resources and elevates their execution priority. Critical path operators are those on the longest computational path in the model; their execution time determines the end-to-end latency of the entire inference task. After identifying these critical path operators, the runtime manager locks the necessary computing resources for them (e.g., reserving a certain amount of NPU computing power) and elevates their scheduling priority in the operating system, ensuring that these operators can obtain computing resources preferentially without being blocked by other tasks. The deterministic latency upper bound refers to the maximum inference latency promised by the system, such as 50 milliseconds. Through this resource locking and priority elevation mechanism, the system can guarantee that even under high device load, the execution time of critical path operators remains controllable, thus keeping the end-to-end inference latency within the preset deterministic latency upper bound.

[0122] Step 5.4: Read the original output tensor after inference is complete.

[0123] After inference, the system reads the raw output tensor from the output buffer of the acceleration unit. The output buffer is a storage area within the acceleration unit used to temporarily store computation results. The raw output tensor refers to the raw values ​​directly output by the last layer of the network, without any post-processing transformation. These raw outputs include three parts: bounding box coordinate offset, which is the positional offset of the bounding box predicted by the network relative to the preset anchor box, rather than absolute coordinates; target confidence score, representing the probability that the network judges that a target exists at that location; and probability distribution for each category, representing the probability that the target belongs to each category (such as tower, insulator, conductor). These raw output values ​​are usually not directly usable detection results and need to be decoded and post-processed to be converted into intuitive detection boxes.

[0124] Step 5.5: Perform anchor box decoding to generate a set of candidate detection boxes.

[0125] The system performs the first level of post-processing, namely anchor box decoding. Anchor boxes are a predefined set of reference boxes with different sizes and aspect ratios, used to assist in bounding box prediction. The network outputs not the absolute coordinates of the bounding box, but rather its offset relative to the anchor boxes. The anchor box decoding process converts the offsets into absolute coordinates in the original image coordinate system, including the coordinates of the top-left and bottom-right corners of the bounding box, or the center point coordinates and dimensions. The resulting bounding boxes are called candidate detection boxes. Each candidate detection box is associated with a confidence score and a class probability distribution. Since the network may output multiple candidate detection boxes for the same target, the resulting set of candidate detection boxes contains a large number of overlapping and redundant boxes, requiring further filtering.

[0126] Step 5.6: Perform confidence threshold filtering and non-maximum suppression, and output the final set of detection boxes.

[0127] The system performs a second level of post-processing, including confidence threshold filtering and non-maximum suppression. Confidence threshold filtering is the first step: the system presets a confidence threshold (e.g., 0.5), and only candidate boxes with confidence scores higher than this threshold are retained; those below the threshold are considered false detections and are directly discarded. Non-maximum suppression is the second step: for the retained candidate boxes, the system sorts them by confidence score from highest to lowest, selects the box with the highest confidence score as the baseline, and calculates the intersection-union ratio (IU) (the ratio of overlapping area to total area) of the other boxes with this baseline. If the IU exceeds a preset threshold (e.g., 0.45), these boxes are considered to detect the same target as the baseline box and are suppressed (discarded). The system repeats this process until all candidate boxes have been processed. The final set of retained candidate boxes is the final detection result, with one detection box corresponding to each target.

[0128] Step 5.7: Generate structured detection results containing category labels, bounding box coordinates, confidence scores, and configuration identifiers.

[0129] The system generates structured detection results. Structured detection results refer to detection data organized according to a fixed format, facilitating program parsing and use. The detection result for each target includes at least the following fields: category label, such as "tower," "insulator," or "conductor," represented as a string or numeric code; bounding box coordinates, typically represented by four integers: the x-coordinate of the top-left corner, the y-coordinate of the top-left corner, the x-coordinate of the bottom-right corner, and the y-coordinate of the bottom-right corner; and a confidence score, a decimal between 0 and 1, indicating the reliability of the detection result. In addition, the result includes a configuration identifier used in this inference. This identifier is a unique string or numeric code used to identify the combination of optimization strategies configured by the runtime manager (including the values ​​of memory reuse strategies, operator scheduling strategies, batch processing parameters, and computational precision strategies). With this identifier, it is possible to trace the configuration under which each detection result was generated.

[0130] Step 5.8: Encapsulate into a standardized message format and return.

[0131] The system encapsulates the structured detection results into a standardized message format and returns it to the caller. This standardized message format is a universal data exchange format, such as JSON or Protobuf, facilitating data exchange between different software modules. In addition to the detection results themselves, the system also appends a timestamp and device identification information to the message. The timestamp records the completion time of this inference, used for subsequent time-series analysis and logging. The device identification information is a unique identifier for each edge device, used to distinguish data sources in a multi-device system. After encapsulation, the system returns the message to the caller via a model handle. The caller can be an upper-layer application, such as inspection software or an operations and maintenance management platform, which receives the detection results and performs subsequent processing, such as generating inspection reports or triggering alarms.

[0132] S6. Based on the structured detection results and associated equipment ledger information, accurate facility location and identification are achieved. Difficult samples are then sent back to the cloud to drive model retraining, forming a continuous evolutionary closed loop from edge to cloud. The detection results are matched with equipment file information to achieve accurate target identification and location. Samples with poor model performance are selected, processed securely, and sent back to the cloud for model retraining. The optimized model is then redeployed to the edge to complete the closed-loop iteration.

[0133] The process of accurately locating and identifying facilities by associating structured detection results with equipment ledger information, and then sending difficult samples back to the cloud to drive model retraining, forming a continuous evolutionary closed loop from edge to cloud, includes the following steps:

[0134] Step 6.1: Analyze the structured detection results and extract the category labels, bounding box coordinates, and confidence scores.

[0135] The system first receives structured detection results from the inference execution module. These structured results are detection data organized in a fixed format for easy parsing and use by the program. From these results, the system extracts the following key information: a category label indicating the type of the detected target, such as "tower," "insulator," or "conductor"; bounding box coordinates, indicating the target's position in the image, usually given in pixel coordinates; and a confidence score, a decimal between 0 and 1, indicating the reliability of the detection result. In addition, the system extracts the configuration identifier used in this inference (used to trace the combination of optimization strategies employed at that time) and timestamp information (recording the moment the detection was completed). This information forms the basis for subsequent association ledgers and the identification of difficult samples.

[0136] Step 6.2: Match the test results with the equipment ledger information to achieve accurate correspondence between the test targets and the poles, insulators, conductors and fittings.

[0137] The system correlates and matches the parsed detection results with pre-set transmission line equipment ledger information. The equipment ledger information is a pre-established structured database that records basic information about all facilities along the transmission line, including: tower number (uniquely identifying each tower), insulator type (such as the specific specifications of composite or glass insulators), conductor specifications (such as conductor cross-sectional area and material), hardware type (metal fittings connecting and securing conductors), and the latitude and longitude coordinates of each facility. Since drones or monitoring equipment record GPS location and gimbal angles during image capture, the system can convert the pixel coordinates of the detection boxes in the image into actual geospatial coordinates, and then map and align them with the latitude and longitude coordinates in the ledger. This achieves precise correspondence between the detected target and the specific facility, such as determining which tower a detected insulator belongs to.

[0138] Step 6.3: Generate a standard inspection report and output it to the operation and maintenance management system.

[0139] The system generates a standard inspection report based on the association matching results. A standard inspection report is a structured document or data record that summarizes all the findings of this inspection. The report content includes at least: inspection time (i.e., the timestamp of inference completion), equipment identifier (the edge device number that performed the inspection), target category (e.g., tower, insulator, or conductor), location coordinates (the geographical location of the target, which can be latitude and longitude or tower number plus offset), confidence score (the reliability of the inspection results), and defect type determination (e.g., insulator damage, tower tilt, conductor strand breakage, etc.). The system can output the report in a preset format (e.g., JSON or CSV table) or directly upload it to the operation and maintenance management system via the communication interface for operation and maintenance personnel to view and process.

[0140] Step 6.4: When the detection confidence is lower than the confidence threshold, deviates significantly from historical statistical patterns, or is inconsistent with multiple consecutive outputs, it is marked as a difficult sample.

[0141] The system performs hard sample classification on structured detection results. Hard samples are those that the model struggles to identify correctly and require focused learning. The system employs three classification methods: First, if the confidence score of the detection result is below a preset confidence threshold (e.g., 0.7), it indicates that the model is not confident enough in its current prediction. Second, if the detection result deviates significantly from historical statistical patterns (e.g., the insulators of a certain tower were all previously detected as normal, but this time they are classified as damaged with low confidence), it indicates a possible misjudgment or a newly emerging defect. Third, if the model outputs inconsistent category classifications for the same target in multiple consecutive inferences (e.g., the first classification is "insulator normal," the second is "insulator damaged"), it indicates that the model is confused about the target. If any of these conditions are met, the system marks the input sample and its corresponding detection result as a hard sample.

[0142] Step 6.5: Cache difficult samples, de-identify them after reaching the batch or time period, and send them back to the cloud through an encrypted channel.

[0143] The system caches the marked difficult samples locally. Local caching means temporarily storing these samples in the local storage space of the edge device. The system sets two trigger conditions: first, the number of cached samples reaches a preset batch size (e.g., 100 difficult samples have accumulated); second, a preset time period is reached (e.g., every 24 hours, regardless of the number of cached samples, an upload is triggered). When either condition is met, the system performs desensitization processing on the cached difficult samples. Desensitization processing refers to removing or blurring sensitive information in the image to prevent privacy leaks. Desensitization operations include at least: erasing faces from the image (if people are accidentally captured in the image), erasing license plate numbers, and erasing facility logos involving national security. After desensitization, only information related to power transmission line detection is retained in the image.

[0144] Step 6.6: The anonymized hard samples are sent back to the cloud server through an encrypted transmission channel. The sent data includes at least the original input image, the structured detection results output by the model, the manual review and annotation information, and the device operating context status.

[0145] The system transmits the anonymized hard samples back to the cloud server via an encrypted transmission channel. An encrypted transmission channel refers to a network connection established using a secure transmission protocol, such as an HTTPS connection encrypted with TLS, to prevent data from being stolen or tampered with during transmission. The transmitted data includes at least four parts: the original input image, i.e., the image that caused the model's poor performance; the structured detection results output by the model, including category, coordinates, confidence score, etc.; manually verified annotation information, i.e., the correct annotations added by maintenance personnel after subsequent verification of the samples (if already verified); and the device operating context status, i.e., information such as load, power consumption, and temperature of the edge device during inference. These data collectively constitute effective training material in the cloud.

[0146] Step 6.7: Incorporate the training dataset into the cloud and retrain the model using incremental learning or federated learning.

[0147] After receiving the returned difficult samples, the cloud server incorporates them into the training dataset and triggers hardware-aware neural architecture search for model retraining. Incremental learning is a method that updates and trains an existing model using only new samples, eliminating the need for retraining from scratch. It retains the knowledge already learned by the original model while performing targeted optimizations for difficult samples. Federated learning is a distributed learning framework where multiple edge devices compute model updates locally, uploading only updated parameters and not the original data, thus better protecting data privacy. Regardless of the method used, the cloud retains the capabilities of the original model (avoiding catastrophic forgetting) while optimizing for difficult samples, enabling the model to correctly identify similar samples the next time it encounters them.

[0148] Step 6.8: Deploy the optimized model to the edge device through differential update, triggering hot update and automatic rollback mechanisms to complete the upgrade.

[0149] After the cloud server completes model retraining, the optimized model is distributed to edge devices via differential updates. Differential updates transmit only the differences between the old and new models, rather than the complete model file, significantly reducing network traffic. Upon receiving the differential data packets, the edge device triggers the differential hot update and automatic rollback mechanism described in claim 6: the system loads the changed blocks into the area to be updated, concatenates them with the unchanged blocks to form a new model, performs pre-verification, and, upon successful verification, performs an atomic switch without interrupting ongoing inference services. Thus, a complete closed loop is formed, from collecting difficult samples at the edge, retraining in the cloud, to upgrading the model at the edge. As this closed loop iterates continuously, the model's performance in real-world applications will continuously improve, forming a continuous evolutionary system from edge to cloud.

[0150] Example 2

[0151] like Figure 2 As shown, this application provides an architecture diagram of a real-time multi-target detection and localization system for transmission lines based on a lightweight edge-end artificial intelligence model. This system is applied to the real-time multi-target detection and localization method for transmission lines based on a lightweight edge-end artificial intelligence model as described in Embodiment 1, and includes:

[0152] Model generation module 210: configured to automatically generate a lightweight multi-scale feature fusion model that uses a partially convolutional backbone network and cross-modal attention fusion under multi-dimensional constraints based on hardware-aware neural structure search.

[0153] Model compilation module 220: configured to export the model as a standard intermediate representation, perform mixed-precision quantization and automatic operator kernel tuning for the target platform, and compile to generate a cross-platform unified model package containing hierarchical meta-information.

[0154] Model deployment module 230: configured to load and instantiate model functions into blocks on demand using memory mapping based on the hierarchical metadata in the unified model package, and return the model handle of the container.

[0155] Runtime management module 240: Configured to monitor device load, memory, power and temperature status in real time based on the model handle, and dynamically coordinate memory reuse, heterogeneous operator scheduling, batch processing and calculation accuracy strategies to optimize energy efficiency ratio.

[0156] Inference execution module 250: Configured with a configuration-based optimization strategy, it calls the model handle to drive the acceleration unit to perform low-latency forward inference, and outputs a structured detection result containing category, location, confidence and configuration identifier after post-processing.

[0157] Continuous Evolution Module 260: Configured to accurately locate and identify facilities by associating the structured detection results with the equipment ledger information, and to send difficult samples back to the cloud to drive model retraining, forming a continuous evolution closed loop from the edge to the cloud.

[0158] Figure 3 This is an electronic device provided in one embodiment of this application. For example... Figure 3 As shown, the electronic device includes at least the following components: processor 301 and memory 300, communication interface 303, and bus 302.

[0159] In this embodiment of the application, memory 300 is used to store executable instructions of processor 301, which, when configured to execute instructions, implements the method as described in the first aspect.

[0160] In embodiments of this application, a computer-readable storage medium includes instructions that instruct a device to perform the method as described in the first aspect. For example, the instructions instruct the device to perform... Figure 1 The method is shown in the process steps.

[0161] In one embodiment of this application, the program operating in the electronic device may be a program that controls a central processing unit (CPU) or similar device to achieve the functions of the above-described embodiments of the present invention (a program that enables the computer to function). Information processed by these systems is then temporarily stored in random access memory (RAM) during processing, and subsequently stored in various ROMs such as read-only memory (FlashROM) and hard disk drives (HDDs), and read, corrected, and written by the CPU as needed.

[0162] It should be noted that a portion of the electronic device described above can also be implemented using a computer. In this case, the program for implementing the control function can be recorded on a computer-readable recording medium, and the program recorded on the recording medium can be read into the computer and executed.

[0163] It should be noted that the computer mentioned here refers to a computer built into an electronic device, employing hardware including an operating system and peripheral devices. Furthermore, computer-readable recording media refers to removable media such as floppy disks, magneto-optical disks, ROMs, and CD-ROMs, as well as storage systems such as hard drives built into the computer.

[0164] Furthermore, computer-readable recording media can include: media that dynamically stores programs for short periods of time, such as communication lines used when transmitting programs via networks like the Internet or communication lines like telephone lines; and media that store programs for fixed periods of time, such as volatile memory inside a computer that serves as a server or client in this case. In addition, the aforementioned program can be a program used to implement the above-mentioned functions, or it can be a program that can implement the above-mentioned functions by combining them with programs already recorded in the computer.

[0165] Furthermore, the electronic device in the above embodiments can also be implemented as an assembly (system group) composed of multiple systems. Each system constituting the system group can possess some or all of the functions or functional blocks of the electronic device in the above embodiments. As a system group, it is sufficient to have all the functions or functional blocks of the electronic device.

[0166] Those skilled in the art should recognize that the above embodiments are only used to illustrate this application and are not intended to limit this application. Any appropriate changes and variations made to the above embodiments within the essential spirit and scope of this application fall within the scope of protection claimed in this application.

Claims

1. A method for real-time detection and localization of multiple targets in power transmission lines based on a lightweight edge-end artificial intelligence model, characterized in that, Includes the following steps: Based on hardware-aware neural architecture search, a lightweight multi-scale feature fusion model is automatically generated under multi-dimensional constraints, which integrates a partially convolutional backbone network with cross-modal attention. The model is exported as a standard intermediate representation, and mixed-precision quantization and automatic operator kernel tuning are performed for the target platform. The model is then compiled to generate a cross-platform unified model package containing hierarchical meta-information. Based on the hierarchical metadata in the unified model package, the model functions are divided into blocks and loaded on demand using memory mapping, and instantiated into model service containers that support differential hot updates and automatic rollback, and the model handle of the container is returned. Based on the model handle, the runtime manager monitors the device load, memory, power consumption and temperature status in real time, and dynamically coordinates the configuration of memory reuse, heterogeneous operator scheduling, batch processing and calculation accuracy strategies to optimize the energy efficiency ratio. Based on the configuration optimization strategy, the model handle-driven acceleration unit is invoked to perform low-latency forward inference, and the post-processed output is a structured detection result containing category, location, confidence and configuration identifier. Based on the structured detection results and associated equipment ledger information, facilities can be accurately located and identified. Difficult samples are sent back to the cloud to drive model retraining, forming a continuous evolutionary closed loop from edge to cloud.

2. The method according to claim 1, characterized in that, The hardware-sensing neural architecture search includes the following steps: A hardware performance predictor for the target edge chip is constructed to evaluate the latency, power consumption and memory usage of candidate substructures. The predictor is trained using substructure performance data pre-sampled on the target chip. A multi-dimensional search space is defined, which includes the kernel size, channel expansion factor, attention mechanism type and location, and skip connection configuration. A gradient-optimized differentiable search strategy is used to traverse the search space. A multi-objective optimization function is set up, which is composed of a weighted sum of three normalized penalty terms: detection accuracy loss, latency, energy consumption, and memory usage. Each penalty term is configured with an adjustable weight coefficient to balance the priority relationship between accuracy and hardware efficiency. Under the constraints of the multi-objective optimization function, the structural parameters of the search supernet are updated iteratively through gradient descent. After the search is completed, the Pareto optimal substructure is selected as the final model architecture.

3. The method according to claim 1, characterized in that, The lightweight multi-scale feature fusion model includes: A backbone network is constructed using partial convolution. The partial convolution only performs spatial convolution operations on a portion of the input feature map, while the remaining channels maintain an identity mapping. Cross-channel information fusion is achieved through pointwise convolution to form a T-shaped convolution structure. In the shallow layers of the backbone network, a double ReLU activation structure is used to enhance the response of edge and texture details, while in the deep layers, SiLU or Smish activation functions are used to maintain smooth gradient propagation. A cross-modal attention fusion module is embedded in the feature fusion network. The module receives multi-scale feature maps extracted from visible light and infrared images respectively, and calculates the correlation weights of cross-modal features through a cross-attention mechanism to achieve adaptive weighted fusion. A channel-selective partial convolutional fusion module is introduced at the output of the feature fusion network. After hierarchical upsampling of multi-level feature maps, feature channels with importance higher than a preset threshold are dynamically selected by a differentiable gating function for convolutional fusion, which suppresses redundant calculations and preserves key geometric information.

4. The method according to claim 1, characterized in that, The hybrid precision quantization and compilation to generate a cross-platform unified model package includes the following steps: Export the model as an ONNX standard intermediate representation; A quantitative sensitivity analysis was performed on each layer of the model to identify the attention projection layer, layer normalization layer, and upsampling layer as sensitive layers. Regular convolutional and activation layers are quantized to INT8, while sensitive layers are retained as FP16 or FP32. Automatically optimize the convolution operator kernel for the target chip, selecting the optimal implementation from direct convolution, Im2Col, and Winograd convolution. The compilation generates hierarchical meta-information that includes computational subgraph dependencies, operator performance profiles, memory access pattern graphs, and model functional block boundaries, which is then packaged together with the executable binary code into a unified model package.

5. The method according to claim 1, characterized in that, The step of loading model functions into blocks on demand using memory mapping based on the hierarchical meta-information in the unified model package includes the following steps: Parse the hierarchical metadata to extract the block boundaries of the backbone network block, neck network block, and detection head block; The model package undergoes page boundary alignment preprocessing, supporting zero-copy memory mapping loading; At startup, the backbone network block and neck network block are loaded into memory through a memory mapping mechanism; Dynamically load the corresponding detection header block as needed according to task requirements, and support dynamic switching of detection headers; Create a model service container and generate a unique model handle to return to the caller. Multiple callers can share the same handle to reuse the loaded backbone network block and neck network block.

6. The method according to claim 1, characterized in that, The model service container that supports differential hot updates and automatic rollback includes: Maintain two independent handles: the currently running version and the version to be updated; During updates, differential data packets generated based on differences in hierarchical metadata are obtained, and only the changed blocks are transmitted; The changed blocks are loaded into the area to be updated and then combined with the unchanged blocks to form a complete new model. Perform pre-warm-up inference verification on the handle to be updated. Once the verification is successful, perform an atomic switch without interrupting the inference request. When verification fails, the handle to be updated is automatically discarded and resources are released, and the current version is continued to be used to achieve automatic rollback. The status information of success or failure will be reported to the cloud management system.

7. The method according to claim 1, characterized in that, The process of optimizing the energy efficiency ratio includes the following steps: Real-time monitoring of CPU and NPU utilization, available memory, remaining power, and chip temperature; Input the collected data into the multidimensional optimization strategy library; The memory reuse strategy is implemented to allocate and reclaim memory blocks from the pre-allocated memory pool, and to reuse the same address for intermediate feature tensors with the same shape. Implement a heterogeneous operator scheduling strategy to dynamically allocate operator tasks among CPU, NPU, and GPU based on load and temperature; Configure batch processing parameters to dynamically merge concurrent requests for batch execution, and reduce batch size when battery is low; The calculation precision strategy is implemented: under normal conditions, INT8 precision is used, and when the power is low or the temperature is high, it is downgraded to a lower bit precision or non-critical detection heads are skipped. The four-dimensional strategies are combined in a coordinated manner to maximize the processing capacity per unit of energy consumption while meeting the constraints of detection accuracy and latency, and the strategies are then applied to the model service container.

8. The method according to claim 1, characterized in that, The output includes structured detection results with category, location, confidence level, and configuration identifier, and includes the following steps: The model handle is invoked, and multimodal input data is sent to the acceleration unit for inference according to the optimization strategy; The runtime manager dynamically allocates different operators to the CPU, NPU, or GPU for parallel execution based on the heterogeneous operator scheduling configuration, and uses a double buffering mechanism to achieve overlap between data loading and computation; Lock resources and increase priority for critical path operators to ensure that latency is kept within the upper bound of deterministic latency; Read the original output tensor after inference is complete; Perform anchor box decoding to generate a set of candidate detection boxes; Perform confidence threshold filtering and non-maximum suppression, and output the final set of detection boxes; Generate structured detection results containing category labels, bounding box coordinates, confidence scores, and configuration identifiers, and return them after encapsulating them into a standardized message format.

9. The method according to claim 1, characterized in that, The process of sending difficult samples back to the cloud to drive model retraining, forming a continuous evolutionary closed loop from edge to cloud, includes the following steps: Analyze the structured detection results to extract category labels, bounding box coordinates, and confidence scores; By linking and matching the test results with the equipment ledger information, the test targets can be accurately matched with the towers, insulators, conductors and fittings. Generate standard inspection reports and output them to the operation and maintenance management system; When the detection confidence level is lower than the confidence threshold, there is a significant deviation from historical statistical patterns, or the output is inconsistent with the judgment in multiple consecutive outputs, it is marked as a difficult sample. Difficult samples are cached, de-identified after batch or time period, and then sent back to the cloud through an encrypted channel; The training dataset is incorporated into the cloud, and the model is retrained using incremental learning or federated learning. The optimized model is distributed to edge devices through differential updates, triggering hot updates and automatic rollback mechanisms to complete the upgrade.

10. A real-time multi-target detection and localization system for transmission lines based on a lightweight edge-end artificial intelligence model, applied to the method described in any one of claims 1 to 9, characterized in that, The system includes: Model generation module: configured to automatically generate a lightweight multi-scale feature fusion model that uses a partially convolutional backbone network and cross-modal attention fusion under multi-dimensional constraints based on hardware-aware neural architecture search. Model compilation module: configured to export the model as a standard intermediate representation, perform mixed-precision quantization and automatic operator kernel tuning for the target platform, and compile to generate a cross-platform unified model package containing hierarchical meta-information; Model deployment module: configured to load and instantiate model functions into blocks on demand using memory mapping based on the hierarchical meta information in the unified model package, and return the model handle of the container; Runtime management module: Configured to monitor device load, memory, power consumption and temperature status in real time based on the model handle, and dynamically coordinate the configuration of memory reuse, heterogeneous operator scheduling, batch processing and calculation accuracy strategies to optimize energy efficiency ratio; Inference execution module: Configured with a configuration-based optimization strategy, it calls the model handle to drive the acceleration unit to perform low-latency forward inference, and outputs a structured detection result containing category, location, confidence and configuration identifier after post-processing; Continuous Evolution Module: Configured to accurately locate and identify facilities by associating the structured detection results with equipment ledger information, and to send difficult samples back to the cloud to drive model retraining, forming a continuous evolution closed loop from edge to cloud.

Images