Lightweight AI Model Adaptive Deployment System for Edge Computing

Abstract

This invention belongs to the field of edge computing technology and discloses a lightweight AI model adaptive deployment system for edge computing. It constructs a dependency strength matrix by accurately quantifying data dependencies between model layers, monitors network stability in real time and identifies abnormal periods, calculates communication penalty factors and bandwidth attenuation factors based on latency peak distribution characteristics, and then constructs a communication overhead prediction model for collaborative computing between edge devices. The system innovatively generates a multi-granularity model segmentation candidate scheme set and identifies segmentation boundaries that trigger resource contention by monitoring computational load fluctuations and memory usage changes. It dynamically adjusts the model partitioning granularity and mapping strategy based on the gradient transmission volume and activation value size at the segmentation boundaries. This invention reduces cross-device communication overhead, improves the balance of computational resource utilization, and ensures real-time response capabilities in latency-sensitive scenarios.

Description

Technical Field

[0001] This invention relates to the field of edge computing technology, and more specifically, to a lightweight AI model adaptive deployment system for edge computing. Background Technology

[0002] With the rapid development of IoT, 5G, and AI technologies, edge computing, as a new computing paradigm that moves computing power from the cloud to near the data source, is rapidly becoming a key infrastructure for AI application deployment. In scenarios such as autonomous driving, smart cities, and industrial IoT, the demand for edge AI inference is exploding, making lightweight AI model deployment technology a research hotspot. Current mainstream edge AI deployment solutions mainly include model compression, model quantization, and model segmentation. Among these, model segmentation technology, by dividing a large AI model into multiple sub-models and distributing them across edge device clusters, fully utilizes distributed computing resources and effectively solves the problem of insufficient computing power of a single edge device, and is considered an important development direction for edge intelligence.

[0003] However, the core challenge facing AI model deployment in current edge computing environments lies in the severe inadequacy of determining model partition boundaries. Most existing technologies employ simple heuristics or predefined static partitioning points, failing to accurately identify the optimal model partitioning location, leading to excessive inter-layer communication overhead. Especially in complex heterogeneous edge environments, existing methods lack precise quantitative analysis of data dependencies within the model's computation graph, and cannot construct mathematical models reflecting the strength of inter-layer communication-computation coupling, leaving partitioning decisions without theoretical support. For example, in intelligent traffic monitoring systems, when deep neural network models need to collaboratively process high-definition video streams across multiple roadside computing units, inappropriate partitioning point selection leads to frequent transmission of large amounts of feature map data in bandwidth-constrained edge networks, extending system response time from milliseconds to seconds, failing to meet real-time early warning requirements. Simultaneously, existing technologies neglect nonlinear data dependency structures in the model (such as residual connections and attention mechanisms), often producing suboptimal solutions when processing such complex topologies, causing a surge in data transmission volume and insufficient computational load for some devices. More critically, existing segmentation strategies fail to simultaneously consider the different data flow characteristics of forward inference and backpropagation, leading to a significant decrease in segmentation efficiency in edge training scenarios. For example, in distributed edge learning, the communication volume for gradient updates surges, consuming limited bandwidth resources. Furthermore, most existing methods treat model segmentation and device mapping separately, failing to optimize them collaboratively. They ignore the impact of heterogeneous computing capabilities of different edge nodes on the optimal segmentation point. Consequently, in industrial IoT environments, even if the model segmentation point seems reasonable, improper resource matching can lead to problems such as computational overload, memory overflow, or communication congestion on some devices, ultimately causing the overall system performance to fall far short of theoretical expectations.

[0004] In view of this, the present invention proposes a lightweight AI model adaptive deployment system for edge computing to solve the above problems. Summary of the Invention

[0005] To overcome the aforementioned shortcomings of existing technologies and achieve the above objectives, this invention provides the following technical solution: a lightweight AI model adaptive deployment system for edge computing, comprising:

[0006] The network monitoring module is used to acquire the network latency jitter sequence between each device node in the edge device cluster, and to identify periods of abnormal network stability based on the autocorrelation of latency jitter within adjacent time windows.

[0007] The communication evaluation module is used to calculate the latency penalty factor and bandwidth utilization attenuation factor for cross-device communication based on the latency peak distribution characteristics during periods of abnormal network stability.

[0008] The overhead prediction module is used to construct a communication overhead prediction model for collaborative computing between edge devices based on the latency penalty factor and the bandwidth utilization attenuation factor.

[0009] The data acquisition module is used to acquire the computational graph structure data of the AI ​​model to be deployed and the heterogeneous computing resource distribution data of the edge device cluster;

[0010] The dependency analysis module is used to construct the inter-layer data dependency strength matrix based on the computation graph structure data and the heterogeneous computing resource distribution data.

[0011] The scheme generation module is used to generate a set of multi-granularity model segmentation candidate schemes based on the communication overhead prediction model and the data dependency strength matrix between model layers.

[0012] The resource monitoring module is used to identify the segmentation boundaries that trigger resource contention by monitoring the computing load fluctuations and memory usage changes of each edge device in real time.

[0013] The dynamic adjustment module is used to dynamically adjust the granularity level and mapping strategy of the model partitioning based on the gradient transfer amount and activation value size at the segmentation boundary.

[0014] The deployment and execution module is used to control the model deployment process based on the adjusted model partitioning granularity level and mapping strategy, enabling efficient distributed inference of AI models on edge device clusters.

[0015] The technical effects and advantages of this invention's lightweight AI model adaptive deployment system for edge computing:

[0016] This invention improves the inference efficiency of AI models on heterogeneous edge device clusters. Through an optimal partitioning strategy, it significantly reduces cross-device communication overhead, enabling complex models that were previously limited by bandwidth bottlenecks to run smoothly in resource-constrained environments. Even in the face of severe network fluctuations, it maintains stable service quality, achieving balanced utilization of computing and memory resources in edge device clusters and improving the return on hardware investment. In latency-sensitive scenarios, this invention ensures the real-time responsiveness of AI applications, providing reliable support for critical business decisions. Furthermore, the adaptive characteristics of this invention effectively reduce deployment complexity and maintenance costs, and through refined resource scheduling and energy efficiency optimization, it achieves green and low-carbon development of edge computing. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the lightweight AI model adaptive deployment system for edge computing according to the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] This application provides a lightweight AI model adaptive deployment system for edge computing. The system's execution entities include, but are not limited to, edge servers, edge gateways, IoT terminals, and embedded devices that can be considered as general-purpose computing nodes in this application.

[0020] Please see Figure 1 In this embodiment of the invention, the lightweight AI model adaptive deployment system for edge computing includes: a network monitoring module, a communication evaluation module, an overhead prediction module, a data acquisition module, a dependency analysis module, a solution generation module, a resource monitoring module, a dynamic adjustment module, and a deployment execution module.

[0021] First, the network monitoring module acquires the network latency jitter sequence among the device nodes in the edge device cluster. Based on the autocorrelation of latency jitter within adjacent time windows, it identifies periods of abnormal network stability. The network latency jitter sequence directly reflects the communication quality between edge devices and is crucial for deployment strategies. The system performs sliding window segmentation on the latency jitter sequence, calculates the autocorrelation function, extracts the attenuation rate index, and identifies candidate abnormal periods through fluctuation value anomaly detection. Finally, it combines kurtosis and skewness coefficient analysis to determine the periods of abnormal network stability, laying the foundation for communication overhead assessment.

[0022] Based on the peak latency distribution characteristics during periods of network instability, the communication evaluation module calculates the latency penalty factor and bandwidth utilization attenuation factor for cross-device communication. These two coefficients directly determine the degree of loss in communication efficiency between edge devices under network fluctuations. The calculation process first extracts the latency peak sequence, constructs a probability density distribution, fits a generalized extreme value distribution model to obtain shape and scale parameters, calculates the long-tail effect strength as the latency penalty factor, and calculates the bandwidth utilization attenuation factor based on the coefficient of variation, providing key parameters for constructing a communication overhead prediction model.

[0023] The overhead prediction module constructs a communication overhead prediction model for collaborative computation between edge devices based on latency penalty factors and bandwidth utilization attenuation factors. This model describes the relationship between communication costs and performance between edge devices under network fluctuations, considering the combined impact of latency fluctuations and bandwidth attenuation. The model construction process includes directed graph modeling of communication paths between devices, assignment of basic communication overhead weights, dynamic adjustment, and edge weight updates. Ultimately, it generates a complete model capable of predicting the expected communication latency and throughput limits between any two edge devices, providing a theoretical basis for generating multi-granularity model segmentation schemes.

[0024] The data acquisition module obtains the computational graph structure data of the AI ​​model to be deployed and the heterogeneous computing resource distribution data of the edge device cluster. The computational graph structure data includes key information such as the network layer topology of the AI ​​model, the computational characteristics of each layer, and memory requirements, which is extracted in real time using model parsing tools. The heterogeneous computing resource distribution data includes resource characteristics such as processor computing power, memory capacity, and network bandwidth of each device in the edge device cluster, obtained through device monitoring interfaces. This data provides a foundation for formulating subsequent deployment strategies, ensuring the targeting and effectiveness of the deployment plan.

[0025] The dependency analysis module constructs a data dependency strength matrix between model layers based on computation graph structure data and heterogeneous computing resource distribution data. This matrix quantifies the density of data interactions and resource sensitivity between different layers of the AI ​​model, reflecting the computational and communication coupling relationships within the model. The inter-layer data dependency strength matrix is ​​derived by analyzing the relationship between inter-layer data transfer volume and the computational complexity of each layer, providing a decision-making basis for subsequent model segmentation and device mapping optimization. High dependency strength regions in the matrix will serve as key reference points for model segmentation, ensuring the rational utilization of system resources.

[0026] The scheme generation module generates a set of candidate segmentation schemes for multi-granularity models based on the communication overhead prediction model and the data dependency strength matrix between model layers. Multi-granularity model segmentation is a core technology for achieving efficient deployment of AI models on heterogeneous edge devices, meeting diverse deployment needs through model segmentation at different granularities. The generation process first identifies an initial set of candidate segmentation points based on data dependency strength, generates preliminary segmentation schemes, and evaluates computational and memory balance, quantifies communication overhead, and uses a multi-objective optimization algorithm to solve for the Pareto optimal segmentation scheme set. Then, a multi-granularity model segmentation candidate scheme set covering coarse / fine granularity ranges is generated through granularity expansion operations.

[0027] The resource monitoring module identifies resource contention boundaries triggered by real-time monitoring of computational load fluctuations and memory usage changes across edge devices. Resource contention is a critical factor affecting edge deployment performance, and accurately identifying these contention boundaries is essential for optimizing deployment strategies. The monitoring process includes collecting CPU and GPU utilization time series data, calculating cross-correlation functions to extract time offsets, determining resource usage synergy to mark contention-risk periods, detecting sudden changes in memory usage, and determining the contention boundaries through time-series matching, providing feedback for subsequent adjustments.

[0028] The dynamic adjustment module dynamically adjusts the granularity and mapping strategy of model partitions based on the gradient transmission volume and activation value size at the partition boundaries. This process forms a closed-loop feedback control, ensuring continuous optimization of the deployment scheme. The adjustment process includes calculating the activation values ​​and gradient tensor size at the partition boundaries, determining the communication density index, determining the adjustment direction through normalization and threshold comparison, selecting to merge adjacent partitions or further split the current partition based on the index value, and recalculating the optimal mapping relationship, so that computing resources can more accurately match the model requirements and improve inference efficiency.

[0029] The deployment execution module controls the model deployment process based on the adjusted granularity level and mapping strategy of model partitioning, enabling efficient distributed inference of AI models on edge device clusters. This is the final execution step of the entire system, translating the optimized deployment scheme into actual physical deployment. Based on received instructions, the deployment control deploys the AI ​​model to the edge device cluster according to the optimized partitioning strategy, coordinating data transmission and synchronization mechanisms between devices to ensure the consistency and efficiency of the distributed inference process, ultimately achieving low-latency, high-throughput edge AI inference capabilities.

[0030] In this embodiment of the invention, the detailed implementation steps for identifying periods of abnormal network stability based on the autocorrelation of delay jitter within adjacent time windows include:

[0031] A sliding window segmentation method was applied to the network latency jitter sequence to obtain multiple latency jitter subsequences. Sliding window segmentation is a fundamental method for analyzing the local characteristics of time-series data, capable of capturing the dynamic features of latency variations. The segmentation process employs a standard sliding window technique with a fixed window length and a fixed step size. The window length is typically 5-30 seconds, and the step size is 1 / 4 to 1 / 2 of the window length, ensuring sufficient overlap between adjacent windows to capture continuous variations. Each subsequence contains complete latency jitter data within a specific time period, facilitating subsequent targeted analysis.

[0032] The autocorrelation function (ACF) of each time-delay jitter subsequence is calculated, and the first zero-crossing point of the ACF is extracted as an indicator of the correlation decay rate. The ACF reflects the similarity between the signal and its time-shifted version, and is an important tool for evaluating the intrinsic structure of time-series data. The calculation uses the standard ACF formula and achieves efficient computation through a Fast Fourier Transform (FFT). The first zero-crossing point represents the shortest time shift by which the signal loses its autocorrelation, directly reflecting the rate of signal change. A smaller correlation decay rate indicates faster time delay fluctuations and poorer network stability. This indicator is measured in seconds or sampling points, typically ranging from a few to tens of sampling points.

[0033] The rate of change of the correlation decay rate index between adjacent time windows is recorded as the latency stability fluctuation value. The latency stability fluctuation value quantifies the drastic nature of network state changes and is a key indicator for anomaly identification. The calculation formula is the difference between the decay rate index of the current window and the previous window, divided by the decay rate index of the previous window, yielding the relative rate of change. This index is dimensionless and typically fluctuates within the range of [-0.5, 0.5]. A larger absolute value indicates a more drastic change in network state. The statistical characteristics of the fluctuation value sequence can reflect the overall stability trend of the network, providing an important basis for anomaly detection.

[0034] Anomaly detection is performed on latency stability fluctuations, and time periods where fluctuations exceed a dynamic threshold are marked as candidate anomaly periods. Anomaly detection is a crucial step in screening out time periods that may have stability issues. The detection method employs an adaptive threshold technique, with the dynamic threshold typically set as the mean of historical fluctuation data plus twice the standard deviation, adaptable to baseline levels in different network environments. The detection process considers continuity, requiring at least three consecutive sampling points to exceed the threshold before marking a period as a candidate anomaly, avoiding misjudgments of isolated fluctuation points. Candidate anomaly periods represent preliminary screening results and require further validation to determine the true network stability anomalies.

[0035] The kurtosis and skewness coefficients of latency jitter within candidate abnormal time periods are calculated. Time periods with kurtosis coefficients greater than a preset kurtosis threshold and skewness coefficients greater than a preset skewness threshold are identified as periods of network stability anomalies. Kurtosis and skewness coefficients are high-order statistics describing the distribution pattern of data and can effectively distinguish between normal and abnormal fluctuations. Kurtosis reflects the kurtosis of the distribution; normal network latency is usually close to a Gaussian distribution (kurtosis approximately 3), while abnormal latency typically exhibits a peaked distribution (kurtosis > 5). Skewness reflects the asymmetry of the distribution; a larger absolute value indicates a greater deviation from normality in the latency distribution. Abnormal latency is usually positively skewed (skewness > 0.5). The preset kurtosis threshold is typically set to 5-7, and the preset skewness threshold is typically set to 0.5-1.0. This dual-indicator screening can accurately identify true network stability anomalies, eliminate the interference of occasional fluctuations, and provide a reliable foundation for subsequent analysis.

[0036] In this embodiment of the invention, the detailed implementation steps for calculating the latency penalty factor and bandwidth utilization attenuation factor for cross-device communication based on the latency peak distribution characteristics during periods of abnormal network stability include:

[0037] Delay peak sequences during periods of network instability are extracted, and their probability density distributions are constructed. These delay peak sequences reflect extreme network fluctuations and are crucial for assessing communication risks. The extraction process employs a local maximum detection algorithm to identify all points in the delay jitter sequence that satisfy the condition of being "greater than their two neighboring points and exceeding the average value by a certain multiple," thus forming the delay peak sequence. Peak identification considers the signal-to-noise ratio (SNR), typically requiring peak values ​​to be at least 1.5 times higher than the local mean before inclusion in the sequence. The probability density distribution is constructed using a kernel density estimation method, employing a Gaussian kernel function and adaptively optimizing the bandwidth parameter to obtain a smooth and continuous distribution curve that intuitively reflects the statistical regularity of the peak values.

[0038] Based on the tail decay characteristic of the probability density distribution, a generalized extreme value distribution model is fitted to obtain shape and scale parameters. The generalized extreme value distribution is a specialized statistical model describing extreme events, particularly suitable for analyzing extreme data such as network latency peaks. The fitting employs maximum likelihood estimation, using a numerical optimization algorithm to solve for the optimal parameter combination. The shape parameter determines the tail characteristics of the distribution: positive values ​​indicate a heavy-tailed distribution (high probability of extreme values), negative values ​​indicate a light-tailed distribution, and zero corresponds to exponential tail decay. The scale parameter determines the dispersion of the distribution; a larger value indicates a wider range of peak fluctuations. Parameter estimation considers sample size factors, and bias correction techniques are used to improve fitting accuracy for small sample cases.

[0039] The long-tail effect strength is calculated based on shape parameters to determine the peak delay, and the product of the long-tail effect strength and the scale parameter is used as the delay penalty factor. The long-tail effect strength quantifies the risk level of extreme delays and is a core indicator for communication reliability assessment.

[0040] The calculation formula is: ,in, For the strength of the long-tail effect, The shape parameter is positive, indicating a long-tail effect strength greater than 0, signifying the presence of heavy-tailed characteristics. When the shape parameter is negative, the long-tail effect strength approaches -1, indicating limited extreme risk. The latency penalty factor comprehensively considers both long-tail risk and volatility, serving as a quantifiable indicator of communication latency uncertainty. A higher value indicates higher latency risk in cross-device communication, requiring a more conservative communication strategy.

[0041] Based on the coefficient of variation and delay penalty factor of the peak delay sequence, a bandwidth utilization attenuation factor is calculated using a nonlinear mapping function. The bandwidth utilization attenuation factor reflects the degree of effective bandwidth loss under network fluctuations and directly affects communication throughput prediction. The coefficient of variation is calculated as the standard deviation of the peak delay sequence divided by its mean, reflecting the relative degree of fluctuation. The nonlinear mapping uses a sigmoid function.

[0042] ;

[0043] in, This is the bandwidth utilization attenuation factor. The coefficient of variation is 1. As a delay penalty factor, , , The parameter is typically calibrated based on historical data. This mapping function outputs in the range (0,1), with values ​​closer to 1 indicating more severe bandwidth loss. The attenuation factor considers the impact of latency fluctuations on the window size of protocols such as TCP, accurately predicting the actual decrease in network throughput and providing a quantitative basis for subsequent communication cost assessment.

[0044] In this embodiment of the invention, the detailed implementation steps for constructing a communication overhead prediction model for collaborative computing between edge devices based on the latency penalty factor and the bandwidth utilization attenuation factor include:

[0045] Based on the topology of the edge device cluster, a directed graph model of communication paths between devices is constructed. The directed graph model is a mathematical expression describing the communication relationships between devices; nodes represent edge devices, and directed edges represent communication links. The construction process first collects network connection information of the device cluster, including physical connection types, network hierarchy relationships, and routing configurations. Then, a directed graph data structure conforming to the actual topology is established. The graph model considers multi-hop communication scenarios and fully expresses all possible communication paths between any two devices, facilitating subsequent optimal path calculation and communication cost evaluation.

[0046] In a directed graph model, each edge is assigned a basic communication cost weight, which is determined by the physical link bandwidth and transmission distance. This basic communication cost weight reflects the baseline of communication costs under ideal network conditions and serves as a reference value for evaluating actual communication costs. The weighting process considers the inverse relationship between physical bandwidth and the direct relationship between transmission distance, and the calculation formula is as follows:

[0047] Basic communication overhead weight = α × transmission distance / physical link bandwidth;

[0048] Here, α is the normalization coefficient, used to adjust the dimensional relationship between different units. Transmission distance for wired connections is typically calculated based on the physical cable length, while for wireless connections, the equivalent distance is estimated using a signal strength attenuation model. The base weight is a positive real number; a larger value indicates a higher communication infrastructure cost.

[0049] The latency penalty factor is applied as a multiplicative factor to the basic communication overhead weights to obtain dynamically adjusted communication overhead weights. This dynamic adjustment reflects the impact of network fluctuations on communication costs, enabling the model to adapt to real-world network conditions. The adjustment calculation uses simple multiplication:

[0050] Dynamically adjusted communication overhead weight = basic communication overhead weight × (1 + latency penalty factor);

[0051] This design assigns higher overhead weights to links with high latency risks, naturally reducing their priority during path selection. The adjustment process takes into account the differences between links, calculating adjustment coefficients independently for each edge to accurately reflect the impact of local network characteristics.

[0052] The effective bandwidth estimate between adjacent devices is corrected based on a bandwidth utilization attenuation factor, and the edge weights in the directed graph model are updated. Effective bandwidth directly determines the data transmission rate and is a key factor affecting communication latency and throughput. The correction calculation formula is as follows:

[0053] Effective bandwidth = physical link bandwidth × (1 - bandwidth utilization attenuation factor);

[0054] The update process recalculates the weight value of each edge, substituting the corrected effective bandwidth into the basic weight calculation formula to obtain new weight values ​​that take bandwidth attenuation into account. This update ensures that the edge weights accurately reflect the communication performance under actual network conditions, improving the accuracy of the prediction model.

[0055] A communication cost prediction model is constructed by integrating dynamically adjusted communication cost weights and updated edge weights. This model outputs the expected communication latency and throughput ceiling between any two edge devices. The integration process uses a weighted average method to merge the two weights according to a preset ratio, forming a comprehensive weight. The prediction model is implemented based on the shortest path algorithm of a directed graph. For any two nodes, the path with the minimum comprehensive weight is calculated as the optimal communication path, and the sum of the comprehensive weights of the paths is the expected communication cost. The model output includes two key indicators: the expected communication latency is calculated by multiplying the comprehensive weight by the reference latency, and the throughput ceiling is determined by the minimum effective bandwidth in the path (the weakest link principle). The model supports batch query and caching mechanisms, providing efficient communication cost prediction services and a reliable basis for model segmentation decisions.

[0056] In this embodiment of the invention, the detailed implementation steps for generating a multi-granularity model segmentation candidate scheme set based on the communication overhead prediction model and the inter-model layer data dependency strength matrix include:

[0057] Based on the data dependency strength matrix between model layers, the layer indices corresponding to local minima of data dependency strength are identified and used as the initial candidate set of segmentation points. Local minima of data dependency strength correspond to natural boundary points within the model and are ideal locations for segmentation. The identification process employs a local minimum detection algorithm, traversing the elements near the diagonal of the matrix to find elements that satisfy the condition of being "less than neighboring elements and lower than the global mean by a certain proportion," and recording the corresponding layer indices. The detection algorithm considers the impact of noise and enhances detection stability through smoothing filtering preprocessing. The initial candidate set of segmentation points typically contains 5-15 candidate points, providing a basic selection space for subsequent fine-grained segmentation.

[0058] For each candidate split point in the initial split point candidate set, the model structure is divided with that point as the boundary, generating a corresponding preliminary splitting scheme. The computational load balance and memory usage balance of each sub-model under this preliminary splitting scheme are then calculated. Balance is a crucial indicator for evaluating the quality of the splitting scheme and directly impacts the efficiency of distributed inference. The partitioning process divides the computation graph according to the layer index of the candidate split points, forming two or more sub-models. The computational load balance is calculated using the inverse of the coefficient of variation of the computational load of each sub-model, and the memory usage balance is calculated using the inverse of the coefficient of variation of the memory requirements of each sub-model. The balance index ranges from (0,1], with values ​​closer to 1 indicating a more balanced distribution. The preliminary splitting scheme encompasses all possible single-point splitting scenarios, providing a candidate space for multi-objective optimization.

[0059] Based on the communication overhead prediction model, the total cross-device communication overhead of each preliminary segmentation scheme under different edge device mapping combinations is quantitatively evaluated, resulting in a communication overhead evaluation value for each preliminary segmentation scheme. Communication overhead evaluation is a crucial step in balancing computational distribution and communication costs. The evaluation process first enumerates all possible device mapping combinations. For each combination, the necessary data transmission volume between sub-models is calculated. Then, the corresponding communication latency and bandwidth usage are obtained by querying the communication overhead prediction model, and the weighted total overhead is calculated. The evaluation considers the communication requirements of both forward and backward propagation stages, comprehensively quantifying the communication efficiency of the segmentation scheme. The evaluation value is a positive real number; the smaller the value, the higher the communication efficiency, and the more superior the corresponding scheme.

[0060] The optimization objectives are computational balance, memory balance, and communication overhead. A multi-objective optimization algorithm is used to find the set of non-dominated solutions that achieve the optimal balance across all three objectives, which constitutes the Pareto-optimal partitioning scheme set. Multi-objective optimization can find the optimal balance between conflicting objectives and is an effective solution to complex decision-making problems. The optimization process employs a non-dominated sorting genetic algorithm (NSGA-II), setting the three objective functions as maximizing computational balance, maximizing memory balance, and minimizing communication overhead. The algorithm iteratively performs population evolution, gradually converging to the Pareto front through crossover, mutation, and non-dominated sorting. The final solution set contains 10-30 non-dominated solutions, each representing a partitioning scheme that cannot be surpassed by any other solution in all three objectives, collectively forming the Pareto-optimal partitioning scheme set.

[0061] For each partitioning scheme in the Pareto-optimal partitioning scheme set, a granular expansion operation is performed: adjacent partitions are merged to reduce communication frequency, or individual partitions are refined to improve parallel granularity, thereby generating a multi-granularity model partitioning candidate scheme set covering both coarse and fine granularity. Granularity expansion enriches the diversity of candidate schemes, enabling the system to adapt to different scenario requirements. The merging operation targets schemes with high communication overhead, merging adjacent small partitions into larger computational units to reduce cross-device communication; the refining operation targets schemes with unbalanced computation, further splitting computationally heavy partitions to improve parallelism. The expansion process adaptively selects the expansion strategy based on the characteristics of the original scheme and controls the expansion depth, typically expanding each original scheme into 2-5 variants with different granularities. The final multi-granularity model partitioning candidate scheme set covers the full spectrum from coarse-grained (low communication, high computation) to fine-grained (high communication, low computation), providing rich choices for real-time deployment decisions.

[0062] In this embodiment of the invention, the detailed implementation steps for identifying the segmentation boundaries that trigger resource contention in a set of multi-granularity model segmentation candidate schemes by real-time monitoring of computational load fluctuations and memory usage changes of each edge device include:

[0063] The system collects CPU and GPU utilization time series data from various edge devices during inference tasks. These utilization time series data directly reflect the resource usage status of the devices and are fundamental data for identifying resource contention. The collection process is implemented through a lightweight monitoring agent, recording the processor activity status of the devices at a high sampling frequency of 10-100ms. CPU utilization is obtained from the operating system performance counters, summarizing the average utilization of all cores; GPU utilization is read through the hardware driver interface, recording the activity level of the computing units. The time series data synchronously records task execution timestamps to ensure temporal consistency in data analysis. The collection mechanism considers monitoring overhead and employs an asynchronous buffering strategy to minimize interference with normal task execution.

[0064] The cross-correlation function of CPU utilization time series and GPU utilization time series is calculated, and the time offset corresponding to the correlation peak is extracted. Cross-correlation analysis can reveal the temporal relationship between two resource usage patterns and is an effective method for judging synergy. The calculation adopts the standard cross-correlation function formula, and the calculation process is accelerated by Fourier transform. The correlation peak represents the most similar alignment point of the two series, and the corresponding time offset reflects the order of resource usage. Under normal collaborative operation, CPU and GPU utilization should show a certain time difference (usually tens to hundreds of milliseconds) and high correlation; while under resource contention, they are often used synchronously (offset close to zero) and have low correlation. This analysis can reveal hidden problems in resource scheduling and provide key clues for competition identification.

[0065] Based on the time offset, the degree of temporal coordination between CPU and GPU resource usage is determined, and consecutive time periods with a coordination level below a preset synchronization threshold are marked as resource contention risk periods. Resource coordination level is a quantitative expression of the efficiency of CPU and GPU cooperation, directly affecting system performance. The determination process first calculates the standardized coordination index:

[0066] Synergy index = peak cross-correlation value × (1 - |time offset| / maximum allowable offset);

[0067] The maximum allowable offset is typically set to 10%-20% of the expected processing time. The coordination index ranges from [0,1], with higher values ​​indicating better resource cooperation. The preset synchronization threshold is typically set to 0.6-0.8. When the coordination index is below the threshold and persists for a certain period (usually 3-5 sampling points), this period is marked as a resource competition risk period and used as the target area for further analysis.

[0068] During periods of resource contention risk, the memory usage time series of each edge device is acquired, and abrupt changes are detected, with the occurrence time of each abrupt change recorded. Changes in memory usage are a crucial indicator of transitions during model execution, and abrupt changes typically correspond to critical computational boundaries. The detection employs the Cumulative and Change Point Detection (CUSUM) algorithm, which can sensitively identify sudden changes in the sequence. The algorithm accumulates the deviation between the sequence and a reference level; when the accumulated deviation exceeds a preset threshold, it is identified as an abrupt change. The threshold setting takes into account the signal-to-noise ratio, typically being 3-5 times the standard deviation of normal fluctuations. The detection process focuses on ascending abrupt changes (increased memory usage), simultaneously recording the magnitude and occurrence time of the abrupt changes to provide precise time points for subsequent boundary matching.

[0069] The timing of mutation points is matched with the execution times of each segmentation boundary in the multi-granularity model segmentation candidate scheme set. If the execution time of a segmentation boundary coincides with the occurrence time of any mutation point or falls within a preset time tolerance window, and the memory usage change corresponding to the mutation point exceeds a preset mutation threshold, then the segmentation boundary is identified as a segmentation boundary that triggers resource contention. Timing matching is a key step in associating memory mutations with model segmentation, enabling precise location of problem boundaries. The matching process uses a time window method, setting a preset time tolerance window (usually 50-200ms), and checking whether the execution time of each segmentation boundary coincides with the time of any mutation point within the window range. Simultaneously, the mutation magnitude condition is verified; the preset mutation threshold is usually set to 2%-5% of the device's total memory to ensure that only significant changes are considered. Segmentation boundaries that meet both timing matching and magnitude conditions are identified as problem points triggering resource contention. These boundaries will be prioritized for adjustment in subsequent deployment optimization to eliminate resource contention risks and improve overall system performance.

[0070] In this embodiment of the invention, the detailed implementation steps for dynamically adjusting the granularity level and mapping strategy of the model partitioning based on the gradient transfer amount and activation value size at the segmentation boundary include:

[0071] The size of the activation tensor during forward propagation and the size of the gradient tensor during backpropagation at the segmentation boundary are calculated. Tensor size is a direct indicator of communication requirements at the segmentation boundary and is crucial for optimizing communication efficiency. The calculation process is based on the computational graph structure and data flow analysis of the AI ​​model, extracting the tensor specifications transmitted between the two computational nodes connected at the segmentation boundary. The size of the activation tensor during forward propagation is calculated based on the dimension and data type of the output feature map, typically in MB or KB units. The size of the gradient tensor during backpropagation is usually the same as the forward tensor structure, but may differ in some model structures. Size calculations consider data compression and quantization factors, reflecting actual transmission requirements rather than theoretical storage space.

[0072] The communication density index of the partition boundary is determined based on the ratio of the activation value tensor size to the gradient tensor size. The communication density index reflects the data exchange characteristics of the partition boundary and is a key reference value for determining the partitioning strategy. The calculation formula is as follows:

[0073] Communication density index = (activation value tensor size + gradient tensor size) / min (computation cost of front partition, computation cost of back partition);

[0074] This metric comprehensively considers the balance between communication and computational overhead, and is measured in bytes per FLOP or a similar communication-to-computation ratio. A higher metric value indicates a greater relative demand for communication compared to computational demands at the boundary, resulting in lower segmentation efficiency. This metric accurately reflects the efficiency differences at different segmentation points, providing a quantitative basis for granularity adjustment.

[0075] The communication density index is normalized and compared with preset multi-level thresholds to determine the direction of partition granularity adjustment for the segmentation boundary. Normalization makes the indexes of different model structures comparable, which is the basis for fair evaluation. The processing adopts the Min-Max normalization method to map the original index to the [0,1] interval. The preset multi-level thresholds usually include an upper threshold (0.7-0.8) and a lower threshold (0.2-0.3), dividing the index space into high, medium, and low regions. The comparison results directly determine the adjustment direction: high regions correspond to merging operations, low regions correspond to splitting operations, and the middle regions remain unchanged. This multi-level decision-making mechanism avoids the limitations of simple dichotomy and makes the adjustment more flexible and accurate.

[0076] When the normalized communication density index exceeds the upper threshold, adjacent partitions connected by the segmentation boundary are merged to reduce the frequency of cross-device communication. When it falls below the lower threshold, the current partition is further split along weak data dependency points to generate finer-grained sub-partitions; this yields the adjusted partition granularity. Granularity adjustment is the core operation of model deployment optimization and directly impacts system performance. The merging operation targets communication bottleneck boundaries, integrating adjacent partitions originally distributed across different devices into the same device, eliminating high-cost communication. The splitting operation targets computational bottleneck partitions, further splitting them at weak data dependency points (identified by the dependency strength matrix) to improve parallelism. The adjustment process considers device resource constraints, ensuring that the merged partitions do not exceed device capabilities, and that insufficiently fine-grained partition granularity after splitting leads to excessive scheduling overhead.

[0077] Based on the adjusted partition granularity, the optimal mapping relationship between each partition and edge device is recalculated, generating an updated mapping strategy. Mapping optimization is a crucial step in rationally allocating partitions to heterogeneous devices, directly determining execution efficiency. The calculation process is modeled as a bipartite graph optimal matching problem, with partitions and devices as nodes on opposite sides, and edge weights representing matching efficiency. Weight calculation comprehensively considers three factors: the matching degree between device computing power and partition computing needs, memory resource satisfaction, and communication proximity, obtaining a comprehensive score through weighted summation. The optimal matching is solved using the Kuhn-Munkres (Hungarian) algorithm, yielding the partition-device mapping scheme with the highest overall efficiency. Mapping optimization pays special attention to the previously identified resource contention partition boundaries, avoiding mapping related partitions to the same device or resource-complementary device combinations through reasonable scheduling, fundamentally eliminating resource contention risks. The final generated mapping strategy is output in the form of a configuration table to guide the subsequent deployment and execution process.

[0078] In this embodiment of the invention, the detailed implementation steps for constructing the inter-layer data dependency strength matrix of the model based on computation graph structure data and heterogeneous computing resource distribution data include:

[0079] The computational graph structure data is analyzed to extract the input tensor dimensions, output tensor dimensions, and weight parameter sizes for each network layer, and to record the topological connections between layers in the computational graph. This computational graph structure data contains the complete computational logic of the AI ​​model and forms the basis for analyzing data dependencies. The analysis process employs a graph traversal algorithm, visiting each computational node in topological order starting from the input node to extract its computational characteristics. Input and output tensor dimensions are typically recorded in formats such as [batch size, number of channels, height, width], while the weight parameter sizes record the total number and storage size of model parameters. Topological connections are represented using adjacency lists or adjacency matrices, recording the predecessor and successor layer indices for each layer, thus fully describing the data flow path. The analysis results constitute a structured description of the model, providing detailed references for subsequent analysis.

[0080] This process acquires processor computing power and memory bandwidth metrics for each edge device node from heterogeneous computing resource distribution data, and allocates one or more candidate deployment device nodes for each network layer. Resource distribution data reflects the hardware capabilities differences in the edge computing environment and serves as the basis for rationally allocating computing tasks. Processor computing power metrics are typically measured in FLOPS (floating-point operations per second), distinguishing the computing characteristics of different processor types such as CPUs, GPUs, and NPUs; memory bandwidth metrics are measured in GB / s, describing the upper limit of a device's memory access speed. Candidate deployment device allocation is based on an evaluation of the match between network layer type and device characteristics. For example, convolutional layers prioritize GPUs / NPUs, while fully connected layers can consider CPUs, providing 2-5 suitable candidate devices for each layer to increase deployment flexibility.

[0081] For each network layer, the theoretical number of floating-point operations is calculated based on its layer type. This theoretical number of operations is then converted into the normalized computation time on the target device by considering the processor computing power of the candidate deployment nodes. Simultaneously, the normalized memory access time of the layer on the target device is estimated by combining the weight parameter size and the memory bandwidth of the candidate deployment nodes. The weighted sum of the normalized computation time and normalized memory access time is then used as the comprehensive computational complexity of the layer in a heterogeneous environment. Computational complexity is a comprehensive indicator for evaluating the computational cost of a layer, reflecting actual execution efficiency. The theoretical number of floating-point operations is calculated based on the layer type and parameter size; for example, convolutional layers consider factors such as input size, kernel size, and stride. The normalized computation time is obtained by dividing the theoretical computational cost by the device's computing power, reflecting pure computation time. The normalized memory access time considers the bandwidth requirements for parameter loading and intermediate result retrieval, capturing the impact of memory bottlenecks. The weighted sum uses an adaptive weighting strategy that emphasizes computation time for computationally intensive layers and access time for memory-intensive layers, comprehensively reflecting the layer's execution characteristics. Complexity is expressed as a dimensionless relative value, which facilitates cross-level comparisons.

[0082] Based on the topological connections between adjacent layers, the total amount of feature map data transferred during forward propagation and the total amount of gradient data transferred during backpropagation are calculated. The sum of these two amounts is taken as the inter-layer data transfer volume between the adjacent layers. Inter-layer data transfer volume is a direct indicator for evaluating the cost of inter-layer communication and has a significant impact on segmentation decisions. The forward propagation data volume is calculated using the size and data type of the output feature map, taking into account the impact of batch size. The backpropagation data volume includes gradient transfer and parameter updates, and is typically similar to the forward data structure but in the opposite direction. The data volume is measured in bytes, accurately reflecting the actual communication requirements. For complex topologies with multiple paths (such as residual connections), the data volume on all paths is accumulated to comprehensively capture communication relationships.

[0083] Dividing the inter-layer data transfer volume by the sum of the individual computational complexities of the two adjacent layers yields the inter-layer coupling coefficient for that pair. This coefficient quantifies the strength of communication-computation coupling between two layers due to data interaction under current heterogeneous computing resource constraints. The inter-layer coupling coefficient is an innovative metric that comprehensively reflects the cost-effectiveness of partitioning. The calculation formula is essentially a variation of the Communication-Computation Ratio, but it specifically considers the characteristics of heterogeneous environments. A larger coupling coefficient indicates a higher relative communication overhead compared to computational overhead between adjacent layers, resulting in lower partitioning efficiency at that point. This metric can accurately identify the "natural dividing point" in the model—the location where communication cost is minimal relative to computational cost—providing a crucial basis for subsequent partitioning decisions.

[0084] Based on the inter-layer coupling coefficients calculated for all adjacent layers, an N×N dimensional model inter-layer data dependency strength matrix is ​​constructed in layer index order. The element in the i-th row and j-th column corresponds to the inter-layer coupling coefficient between the i-th and j-th layers. If two layers have no direct data dependency, the element value is set to 0. The model inter-layer data dependency strength matrix reflects the density of data interaction and resource sensitivity between different layers of the model under the resource distribution conditions of heterogeneous edge device clusters, serving as the core basis for subsequent model segmentation and device mapping collaborative optimization. The dependency strength matrix is ​​the core output of the entire analysis process, providing a global perspective for model segmentation. The matrix typically exhibits sparse characteristics, with non-zero values ​​mainly near the diagonal, corresponding to the local connectivity characteristics of the forward computation graph. Matrix visualization is usually represented by heatmaps, with highly coupled regions displayed as hotspots, intuitively reflecting the internal structural characteristics of the model. Matrix analysis techniques, including eigenvalue decomposition and community detection, can further uncover the modular structure and natural segmentation points of the model, providing a mathematical basis for adaptive deployment. The construction of the dependency strength matrix completes cross-scale modeling from micro-level computational characteristics to macro-level model structural characteristics, forming the theoretical foundation of the entire system.

[0085] In this embodiment of the invention, the detailed implementation steps for calculating the kurtosis coefficient and skewness coefficient of the delay jitter during the candidate abnormal time period include:

[0086] The latency jitter data within the candidate anomaly periods are standardized to obtain a standardized sequence with zero mean and unit variance. Standardization eliminates the influence of data scale on statistical calculations, ensuring the comparability of results across different network environments. The Z-score standardization method is used, subtracting the mean from the original latency jitter data and dividing by the standard deviation to convert it into a reference form of a standard normal distribution. The standardized sequence has a mean of 0 and a standard deviation of 1, facilitating direct calculation and comparison of subsequent higher-order moments. This step is particularly important because the calculation of kurtosis and skewness is sensitive to data scale; standardization ensures the statistical validity of the results.

[0087] Calculate the fourth central moment of the standardized sequence and divide it by the square of the variance to obtain the kurtosis coefficient. The kurtosis coefficient quantifies the kurtosis of the distribution, reflecting the frequency of extreme values. The calculation employs an unbiased estimation method, considering the influence of sample size on higher-order moment estimation; the formula is as follows:

[0088] ;

[0089] in, For the sample size, For standardized values, The mean is 0. The standard deviation is (1). This formula includes a small sample correction term, which improves the estimation accuracy. The kurtosis coefficient is a dimensionless quantity. The kurtosis of a normal distribution is 3. A value greater than 3 indicates a peaked and heavy-tailed distribution, while a value less than 3 indicates a flat and light-tailed distribution. In network analysis, kurtosis usually corresponds to intermittent congestion patterns and is an important characteristic of network instability.

[0090] The third central moment of the standardized sequence is calculated and divided by the cube of the standard deviation to obtain the skewness coefficient PX. The skewness coefficient quantifies the asymmetry of the distribution, reflecting the tendency of the data to skew towards higher or lower values. The calculation also uses an unbiased estimation method, with the following formula:

[0091] ;

[0092] The parameters are defined as above. The skewness coefficient is a dimensionless quantity; the skewness of a normal distribution is 0, a positive value indicates a right-skewed distribution (long tail on the right), and a negative value indicates a left-skewed distribution (long tail on the left). In network analysis, positive skewness usually corresponds to a sudden increase in latency, while negative skewness may indicate a sudden change in link capacity; both are characteristics of network anomalies.

[0093] Excess kurtosis correction is applied to the kurtosis coefficient to obtain the corrected kurtosis coefficient. Excess kurtosis correction makes the kurtosis index more intuitive and facilitates the setting of reasonable thresholds. The correction formula is:

[0094] Corrected kurtosis coefficient = kurtosis coefficient - 3;

[0095] This simple transformation adjusts the reference value for the normal distribution from 3 to 0. A positive value directly indicates a more spiky distribution than normal, while a negative value indicates a flatter distribution. The corrected index is easier to interpret; an excess kurtosis greater than 2 is generally considered a significant peak characteristic, corresponding to obvious network anomalies. This correction is common in statistical practice and improves the index's usability.

[0096] The corrected kurtosis and skewness coefficients are used as quantitative indicators to determine the degree of network stability anomalies. These two indicators together describe the morphological characteristics of the latency distribution, effectively distinguishing between normal fluctuations and abnormal patterns. The judgment process employs a dual-indicator threshold method, requiring both the kurtosis coefficient to be greater than a preset kurtosis threshold (typically 2-3) and the absolute value of the skewness coefficient to be greater than a preset skewness threshold (typically 0.5-1.0) for a significant anomaly to be identified. This combined criterion improves the specificity of the detection and avoids misjudgments that may arise from a single indicator. In practical applications, the thresholds can be dynamically adjusted according to the characteristics of the network environment to adapt to the baseline characteristics of different networks. These quantitative indicators are ultimately used to determine the periods of network stability anomalies, providing a basis for period selection for subsequent communication overhead assessment.

[0097] In this embodiment of the invention, the method for solving the Pareto optimal partitioning scheme set includes:

[0098] For each initial partitioning scheme to be optimized, the computational load distribution of each sub-partition is calculated based on the sum of the comprehensive computational complexity of the network layers contained in each sub-partition. The reciprocal of the coefficient of variation of the computational load distribution is defined as the computational balance of the scheme. Simultaneously, the memory load distribution of each sub-partition is calculated based on the sum of the normalized memory usage assessment values ​​of the network layers contained in each sub-partition. The reciprocal of the coefficient of variation of the memory load distribution is defined as the memory balance of the scheme. The balance index quantifies the balance of resource allocation and is a key dimension for evaluating the quality of the partitioning scheme. The computational load distribution is obtained by summing the comprehensive computational complexity of all network layers within each sub-partition, reflecting the distribution of computational resource requirements; similarly, the memory load distribution is obtained by summing the normalized memory usage assessment values, reflecting the distribution of memory resource requirements. The coefficient of variation is calculated as the standard deviation divided by the mean, quantifying the relative dispersion of the distribution. Taking the reciprocal ensures a consistent direction for the index (the larger the value, the better) and limits it to the range (0,1], facilitating multi-objective optimization. The balance index accurately reflects the resource utilization efficiency of the partitioning scheme, providing a reliable evaluation standard for optimization.

[0099] A multi-objective optimization function is constructed, comprising three optimization objectives: maximizing computational balance, maximizing memory balance, and minimizing the total communication overhead assessment value calculated based on a communication overhead prediction model. Multi-objective optimization is a systematic approach to handling conflicting objectives, finding the optimal balance between them. The optimization function is represented in vector form, with each component corresponding to an objective function. To unify the optimization direction, the total communication overhead assessment value is inverted or its reciprocal, transforming it into a maximization objective. The three objectives often conflict: improving balance often requires finer-grained partitioning, which increases communication overhead; the heterogeneity of different devices also makes it difficult to simultaneously achieve optimal computational and memory balance. The design of the multi-objective optimization function considers the trade-offs between objectives, does not pre-set fixed weights, but seeks a Pareto optimal solution set, providing a comprehensive perspective for decision-making.

[0100] A non-dominated sorting genetic algorithm (NSGA-II) is used to solve the multi-objective optimization function. The initial population is initialized, with each individual corresponding to a preliminary splitting scheme and carrying its corresponding computational balance, memory balance, and total communication overhead evaluation values ​​as a fitness vector. NSGA-II is a classic method for solving multi-objective optimization problems, capable of efficiently exploring the Pareto front. The initialization process creates an initial population of 30-100 individuals, each encoded as a splitting scheme, including the splitting point location and device mapping information. Fitness evaluation calculates the performance vector of each individual across the three objectives, providing a selection basis for subsequent evolutionary operations. The initial population typically employs a partially randomized strategy, combined with heuristic rules to generate diverse candidate schemes, ensuring broad coverage of the search space.

[0101] In each generation of evolution, crossover and mutation operations are performed on parent individuals to generate a new offspring population with different splitting schemes. Individuals on the non-dominated frontier are then ranked and selected based on crowding distance to maintain a balance between population diversity and convergence. Evolutionary operations are the core mechanism of the genetic algorithm, simulating the selection and mutation processes of biological evolution. Crossover selects two parent schemes, exchanging some of their splitting points or mapping strategies to generate offspring schemes that incorporate characteristics of both. Mutation randomly alters certain attributes of individual schemes, increasing population diversity and avoiding getting trapped in local optima. Non-dominated ranking stratifies individuals in the population according to Pareto dominance: the first stratum represents non-dominated solutions, the second stratum represents solutions dominated by the first stratum, and so on. Crowding distance measures the dispersion of solutions in the target space, prioritizing the retention of highly crowded individuals on the non-dominated frontier to ensure a uniform distribution of the solution set. The selection process employs an elitist strategy, directly retaining excellent individuals from the previous generation to accelerate algorithm convergence.

[0102] The evolutionary process described above is iteratively executed until a preset convergence condition is met. All partitioning schemes corresponding to non-dominated solutions in the final population are extracted, forming a Pareto-optimal partitioning scheme set for subsequent granular expansion operations. Convergence conditions typically include a maximum number of iterations (usually 50-200 generations) or a stable front across multiple generations. During algorithm execution, the Pareto front gradually moves towards optimization in the objective space, eventually converging to an approximation of the true Pareto front. The final extracted set of non-dominated solutions typically contains 10-30 different partitioning schemes, each embodying a different trade-off strategy among the three objectives. These schemes collectively constitute the Pareto-optimal partitioning scheme set, representing solutions that cannot be further optimized under the current constraints, providing a comprehensive candidate space for subsequent granular expansion and deployment decisions.

[0103] This invention achieves adaptive deployment of lightweight AI models on edge device clusters through key technologies such as inter-layer data dependency analysis, network stability monitoring, multi-granularity model segmentation, and resource contention identification. The dynamic optimization feature of this invention can adjust deployment parameters in real time according to the heterogeneous characteristics of edge devices and changes in the network environment, significantly improving inference efficiency and reducing system resource consumption.

[0104] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0105] It should be noted that all formulas in this manual are calculated by removing dimensions and taking their numerical values. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0106] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A lightweight AI model adaptive deployment system for edge computing, characterized in that, include: The network monitoring module is used to acquire the network latency jitter sequence between each device node in the edge device cluster, and to identify periods of abnormal network stability based on the autocorrelation of latency jitter within adjacent time windows. The communication evaluation module is used to calculate the latency penalty factor and bandwidth utilization attenuation factor for cross-device communication based on the latency peak distribution characteristics during the abnormal network stability period. The overhead prediction module is used to construct a communication overhead prediction model for collaborative computing between edge devices based on the latency penalty factor and the bandwidth utilization attenuation factor. The data acquisition module is used to acquire the computational graph structure data of the AI ​​model to be deployed and the heterogeneous computing resource distribution data of the edge device cluster; The dependency analysis module is used to construct an inter-layer data dependency strength matrix based on the computation graph structure data and the heterogeneous computing resource distribution data. The scheme generation module is used to generate a set of multi-granularity model segmentation candidate schemes based on the communication overhead prediction model and the data dependency strength matrix between model layers. The resource monitoring module is used to identify the segmentation boundaries that trigger resource contention in the multi-granularity model segmentation candidate schemes by real-time monitoring of the computing load fluctuations and memory usage changes of each edge device. The dynamic adjustment module is used to dynamically adjust the granularity level and mapping strategy of the model partitioning based on the gradient transmission amount and activation value size at the segmentation boundary. The deployment execution module is used to control the model deployment process based on the adjusted granularity level and mapping strategy of the model partitions.

2. The lightweight AI model adaptive deployment system for edge computing according to claim 1, characterized in that, The method of identifying abnormal network stability periods based on the autocorrelation of delay jitter within adjacent time windows includes: The network delay jitter sequence is segmented by a sliding window to obtain multiple delay jitter subsequences; Calculate the autocorrelation function of each of the time delay jitter subsequences, and extract the first zero-crossing position of the autocorrelation function as an indicator of the correlation decay rate; The rate of change of the correlation decay rate index between adjacent time windows is recorded as the time delay stability fluctuation value. Anomaly detection is performed on the time delay stability fluctuation value, and the time period when the fluctuation value exceeds the dynamic threshold is marked as a candidate abnormal period. Calculate the kurtosis coefficient and skewness coefficient of latency jitter within the candidate abnormal time period, and determine the time period in which the kurtosis coefficient is greater than a preset kurtosis threshold and the absolute value of the skewness coefficient is greater than a preset skewness threshold as the abnormal network stability time period.

3. The lightweight AI model adaptive deployment system for edge computing according to claim 1, characterized in that, The calculation of the latency penalty factor and bandwidth utilization attenuation factor for cross-device communication based on the latency peak distribution characteristics during the abnormal network stability period includes: Extract the latency peak sequence during the period of network instability and construct the probability density distribution of latency peaks; Based on the tail decay characteristics of the probability density distribution, a generalized extreme value distribution model is fitted to obtain the shape parameters and scale parameters; The long-tail effect intensity of the delay peak is calculated based on the shape parameters, and the product of the long-tail effect intensity and the scale parameter is used as the delay penalty factor. The bandwidth utilization attenuation factor is calculated using a nonlinear mapping function based on the coefficient of variation of the peak delay sequence and the delay penalty factor.

4. The lightweight AI model adaptive deployment system for edge computing according to claim 1, characterized in that, The step of constructing a communication overhead prediction model for collaborative computation between edge devices based on the latency penalty factor and the bandwidth utilization attenuation factor includes: Based on the topology of the edge device cluster, a directed graph model of the communication paths between devices is constructed; In the directed graph model, each edge is assigned a basic communication overhead weight, which is determined by the physical link bandwidth and transmission distance. The delay penalty factor is applied as a multiplicative factor to the basic communication overhead weight to obtain the dynamically adjusted communication overhead weight. Based on the bandwidth utilization attenuation factor, the effective bandwidth estimate between adjacent devices is corrected, and the edge weights in the directed graph model are updated. The communication overhead prediction model is constructed by integrating the dynamically adjusted communication overhead weights with the updated edge weights.

5. The lightweight AI model adaptive deployment system for edge computing according to claim 1, characterized in that, The step of generating a multi-granularity model segmentation candidate scheme set based on the communication overhead prediction model and the inter-layer data dependency strength matrix includes: Based on the inter-layer data dependency strength matrix of the model, identify the layer index corresponding to the local minimum of data dependency strength, and use it as the initial candidate set of split points; For each candidate segmentation point in the initial segmentation point candidate set, the model structure is divided with that point as the boundary, a corresponding preliminary segmentation scheme is generated, and the computational balance and memory usage balance of each sub-model under the preliminary segmentation scheme are calculated. Based on the communication overhead prediction model, the total cross-device communication overhead of each preliminary segmentation scheme under different edge device mapping combinations is quantitatively evaluated to obtain the communication overhead evaluation value of each preliminary segmentation scheme. The computational balance, memory usage balance, and communication overhead evaluation value are used as optimization objectives. The results are input into a multi-objective optimization algorithm to find the set of non-dominated solutions that satisfy the comprehensive optimum of the three objectives. This set of solutions is then used as the Pareto optimal partitioning scheme set. For each segmentation scheme in the Pareto-optimal segmentation scheme set, a granularity expansion operation is performed: by merging adjacent partitions or refining a single partition, a multi-granularity model segmentation candidate scheme set is generated.

6. The lightweight AI model adaptive deployment system for edge computing according to claim 1, characterized in that, The method of identifying the segmentation boundaries that trigger resource contention within the multi-granularity model segmentation candidate scheme set by real-time monitoring of computational load fluctuations and memory usage changes of each edge device includes: Collect CPU utilization time series and GPU utilization time series of each edge device when performing inference tasks; Calculate the cross-correlation function between the CPU utilization time series and the GPU utilization time series, and extract the time offset corresponding to the correlation peak; Based on the time offset, the degree of coordination between CPU and GPU resource usage in terms of timing is determined, and continuous time periods with a coordination degree lower than a preset synchronization threshold are marked as resource contention risk periods. During the resource contention risk period, the memory occupancy time series of each edge device is obtained, abrupt changes that occur are detected, and the occurrence time of each abrupt change is recorded. The occurrence time of the mutation point is matched with the execution time of each segmentation boundary in the multi-granularity model segmentation candidate scheme set. If the execution time corresponding to a certain segmentation boundary coincides with the occurrence time of any mutation point or falls within a preset time tolerance window, and the change in memory occupancy rate corresponding to the mutation point exceeds a preset mutation threshold, then the segmentation boundary is identified as a segmentation boundary that triggers resource contention.

7. The lightweight AI model adaptive deployment system for edge computing according to claim 1, characterized in that, The step of dynamically adjusting the granularity level and mapping strategy of model partitioning based on the gradient transfer amount and activation value size at the segmentation boundary includes: Calculate the size of the activation value tensor during forward propagation and the size of the gradient tensor during backpropagation at the segmentation boundary; The communication density index of the segmentation boundary is determined based on the ratio of the activation value tensor size to the gradient tensor size. The communication density index is normalized and compared with a preset multi-level threshold to determine the direction of partition granularity adjustment for the segmentation boundary; When the normalized communication density index is higher than the upper threshold, the adjacent partitions connected by the segmentation boundary are merged; when it is lower than the lower threshold, the current partition is further split along the weak data dependency connection points to generate sub-partitions with finer granularity; that is, the adjusted partition granularity is obtained. Based on the adjusted partition granularity, the optimal mapping relationship between each partition and the edge device is recalculated, and an updated mapping strategy is generated.

8. The lightweight AI model adaptive deployment system for edge computing according to claim 1, characterized in that, The step of constructing the inter-layer data dependency strength matrix based on the computation graph structure data and the heterogeneous computing resource distribution data includes: The computation graph structure data is parsed to extract the input tensor dimension, output tensor dimension, and weight parameter scale of each network layer, and the topological connection relationship of each layer in the computation graph is recorded. Obtain the processor computing power and memory bandwidth indicators of each edge device node in the heterogeneous computing resource distribution data, and allocate one or more candidate deployment device nodes for each network layer; For each network layer, the theoretical number of floating-point operations is calculated based on its layer type, and combined with the processor computing power index of its candidate deployment device nodes, the theoretical number of floating-point operations is converted into the normalized computation time of that layer on the target device. At the same time, combined with the weight parameter scale and the memory bandwidth index of the candidate deployment device nodes, the normalized memory access time of that layer on the target device is estimated. The normalized computation time and the normalized memory access time are weighted and summed to obtain the comprehensive computational complexity of that layer in a heterogeneous environment. Based on the topological connection between two adjacent layers, the total amount of feature map data transmitted during the forward propagation and the total amount of gradient data transmitted during the backward propagation of the adjacent layer pair are calculated, and the sum of the two is taken as the inter-layer data transmission amount between the adjacent layer pair. Divide the inter-layer data transfer amount by the sum of the comprehensive computational complexity of each of the two adjacent layers to obtain the inter-layer coupling coefficient of the adjacent layer pair. Based on the inter-layer coupling coefficients calculated from all adjacent layer pairs, an N×N dimensional model inter-layer data dependency strength matrix is ​​constructed in layer index order.

9. The lightweight AI model adaptive deployment system for edge computing according to claim 2, characterized in that, The calculation of the kurtosis and skewness coefficients of the delay jitter within the candidate abnormal time period includes: The time delay jitter data within the candidate abnormal time period is standardized to obtain a standardized sequence with zero mean and unit variance; The fourth central moment of the standardized sequence is calculated and divided by the square of the variance to obtain the kurtosis coefficient. The third central moment of the standardized sequence is calculated and divided by the cube of the standard deviation to obtain the skewness coefficient. The kurtosis coefficient is adjusted by excess kurtosis correction to obtain the corrected kurtosis coefficient; The corrected kurtosis coefficient and the skewness coefficient are used as quantitative indicators to judge the degree of network stability anomalies.

10. The lightweight AI model adaptive deployment system for edge computing according to claim 5, characterized in that, The methods for finding the Pareto optimal partitioning scheme set include: For each preliminary partitioning scheme to be optimized, the computational load distribution of each sub-partition under the scheme is calculated based on the sum of the comprehensive computational complexity of the network layers contained in each sub-partition, and the reciprocal of the coefficient of variation of the computational load distribution is defined as the computational balance of the scheme; at the same time, the memory load distribution of each sub-partition under the scheme is calculated based on the sum of the normalized memory usage evaluation values ​​of the network layers contained in each sub-partition, and the reciprocal of the coefficient of variation of the memory load distribution is defined as the memory balance of the scheme. A multi-objective optimization function is constructed, which includes three optimization objectives: maximizing the computational balance, maximizing the memory balance, and minimizing the total communication overhead evaluation value calculated based on the communication overhead prediction model. The multi-objective optimization function is solved using a non-dominated sorting genetic algorithm to initialize the population, where each individual corresponds to an initial splitting scheme and carries its corresponding computational balance, memory balance, and total communication overhead evaluation values ​​as a fitness vector; In each generation of evolution, crossover and mutation operations are performed on the parent individuals to generate a new segmentation scheme for the offspring population. The non-dominant frontier individuals are ranked and selected based on the crowding distance to maintain a balance between population diversity and convergence. The above evolutionary process is iteratively executed until the preset convergence condition is met. The partitioning schemes corresponding to all non-dominated solutions in the final population are extracted to form the Pareto optimal partitioning scheme set.

Images