Distribution task adaptive load prediction method and system oriented to edge computing, and medium

Abstract

The invention relates to an edge computing-oriented distribution task self-adaptive load prediction method and system and a medium, and aims to solve the problems that a traditional resource configuration mode is difficult to meet the quality requirement of content distribution along with explosive increase of mobile data traffic, large-scale invalid data transmission is often caused, and the service quality is influenced. According to the task-sending adaptive load prediction system based on edge computing, content caching and intelligent resource allocation are carried out at network edge nodes, so that the data transmission delay is remarkably reduced, and the response speed is increased. The system drives local workload prediction through incremental expression of historical service data, introduces a position information coding mode based on a Transform architecture, and effectively identifies and processes key features in load changes, so that the resource utilization rate of edge devices is improved, and efficient management of the edge devices is realized.

Description

Technical Field

[0001] The present invention relates to the field of load prediction technology, and in particular to a method, system and medium for adaptive load prediction of distributed tasks for edge computing. Background Art

[0002] In the growing mobile edge computing environment, efficient content distribution plays a vital role. The current explosive growth of mobile data traffic will lead to a low content hit rate under unreasonable resource allocation, which often leads to large-scale invalid data transmission, and then leads to a decline in the service quality of content distribution. To improve this situation, the industry has begun to pay extensive attention to and deeply explore the two core elements of caching strategy and intelligent routing mechanism. Among them, by using the advantages of cloud computing technology and edge computing, implementing content caching on network edge nodes close to end users can significantly reduce the data transmission distance, improve response speed and hit rate, and thus optimize user experience. However, due to the dynamic changes in content popularity and the limited cache capacity, an effective prediction scheme should be designed to reduce the risk of downtime caused by insufficient cache. Summary of the invention

[0003] In order to achieve the above-mentioned object of the present invention, the present invention provides a distributed task adaptive load prediction system for edge computing, comprising:

[0004] Data preprocessing module, preprocessing the input data;

[0005] Data engineering module, which checks and repairs missing values, incorrect values, and relationship errors in data;

[0006] The sequence partitioning module partitions the input data and specifies the data length used to predict a data value;

[0007] Model training module, various feature data are input into the model for training;

[0008] The prediction module uses the training results for load prediction;

[0009] The drawing module presents the load forecast results in an intuitive and easy-to-understand chart format. By comparing the forecast data with the actual data, the performance of the forecast is predicted and the model is optimized.

[0010] In a preferred embodiment of the present invention, the data preprocessing module includes:

[0011] Get hardware configuration data: including CPU frequency, memory capacity, memory speed, and number of core threads;

[0012] Obtain server resource status data: including the starting values ​​of CPU utilization, memory utilization, and bandwidth utilization, and continuously monitor changes in memory utilization and CPU utilization, as well as I / O operation volume and bandwidth usage, and record the specific time required for the server to process specific data;

[0013] Obtain system request data: including the number of queries per second, average response time, the actual number of requests received, and the actual request data size;

[0014] The data collection granularity is refined to a time interval of 10 seconds to 30 seconds to ensure the real-time and accuracy of the data.

[0015] In a preferred embodiment of the present invention, the data engineering module includes:

[0016] Missing value processing: Check whether there are missing values ​​in the time series data, and use different filling strategies according to the different missing situations. For the missing time part, use the average value to fill it, and for the missing business features, use the linear interpolation method;

[0017] Outlier detection: Using the average value of the entire sequence as the standard, the thresholds are set at 2 times, 2.5 times, and 3 times to mark outliers;

[0018] Consistency check: ensure that data is consistent with each other and with other known facts or rules, sort the original data according to different period standards, and repair the missing and errors in the time sequence. In particular, for periodic data, confirm the consistency of period boundaries;

[0019] Relationship modeling: Model the relationship between device load and load, and load and business in edge devices. By exploring the relationship between various feature historical data, you can identify normal business behavior patterns based on hidden information obtained from data analysis.

[0020] In a preferred embodiment of the present invention, the sequence division module comprises:

[0021] For example, sample, interpolate or truncate the data to ensure that all data points are distributed at a uniform time interval, divide the normalized time series into a series of continuous time segments with a defined sliding window, extract relevant training sets for each sliding window, each training set covers historical observation values, related time series labels and business feature attributes, mark the real result sequence to be predicted for the future time period corresponding to each window, set it as the target sequence, and finally, integrate the corresponding training sets and their respective target sequences to construct a training batch.

[0022] In a preferred embodiment of the present invention, the model training module includes:

[0023] The relative position load difference information between information nodes enables the model to understand and capture the relative order and dependency between elements in the load sequence data, thereby improving the model's modeling ability for sequence data. The attention score between two load node vectors is calculated as follows:

[0024]

[0025] in Defined as:

[0026]

[0027] q m represents the query vector, k n is the key vector, is the similarity score between the query vector and the key vector, which is used to measure the degree of match between them;

[0028] d k is the dimension of the vector, r is a hyperparameter, m and n are the position indexes in the sequence, indicating the relative distance between two elements in the position;

[0029] For a two-dimensional vector x, if it is rotated counterclockwise around the origin by m radians, it is represented by the rotation matrix R m The role of the rotation matrix R m Usually has the following form:

[0030]

[0031] That is, for the unit rotation transformation matrix R m and R n , which represent the operation of rotating m radians and n radians counterclockwise around the origin respectively. At the same time, according to the combined properties of the rotation matrix, when a vector is first rotated by R m Rotate, then press R n Rotate, its effect is equivalent to one-time Rotate, and the result of this matrix product is actually the rotation matrix R of (nm) radians n-m ;

[0032] For a vector in two-dimensional space, suppose there are two vectors x and y. If we first rotate vector x by m radians, we get R m x, and rotate the vector y by n radians to get R n y, and then perform a dot multiplication on the two rotated vectors, that is, calculate (R m x) T ·(R ny), which represents two different time nodes in edge computing. According to the above matrix multiplication rule, if we directly rotate x by (nm) radians, we can get R n-m x, and multiply this vector by the unrotated y, that is, (R n-m ·x)y, the dot product results obtained in these two ways will be the same. This is because the combined operation of the rotation matrix maintains the invariance of the dot product operation, which provides a method to simplify the problem based on the relative value of the rotation angle, namely:

[0033]

[0034] (R m x) T and (R n y) represents the vector x and the vector y rotated by m and n radians respectively around the far point;

[0035] Let x0, x1 be the element values ​​in vector x, and y0, y1 be the element values ​​in vector y, and we can get:

[0036]

[0037] Q m m rotation degree, k n Rotate n degrees, then by

[0038]

[0039] Available,

[0040] score(q m ,k n )=(R m q m ) T ·(R n k n ),

[0041] (R m q m ) T Represents the query vector q after rotation m The transpose of (R n k n ) represents the key vector k after rotation n .

[0042] In a preferred embodiment of the present invention, the prediction module comprises:

[0043] The known sequence vector is abstracted into the query matrix and key matrix of the self-attention mechanism through feature evolution from different angles. The dot product of the query matrix Q and the key matrix K is usually used to calculate the attention score:

[0044] AttnScore=Q·K T ,

[0045] Among them, AttnScore∈R N×N , N is the sequence length. In order to find the most important query, the scores of each row are accumulated and summed to obtain a new vector Q sum :

[0046]

[0047] Q sum [i] represents the sum of the attention scores of the ith query vector and all key vectors, which is a vector consisting of the cumulative sum of each row. AttnScore[i][j] represents the attention score between the ith query vector and the jth key vector.

[0048] Then, sort Q in descending order sum , and select the largest front The query index corresponding to the score is recorded as set I Q , based on set I Q , reconstruct the self-attention score matrix as AttnScore′, with a size of

[0049] AttnScore′[i][:]=AttnScore′[i][I Q ],i=1,2,...,N,

[0050] AttnScore′[i][:] is all the elements in the i-th row of the reconstructed new sparse attention score matrix. Q ] is the i-th row and column index I in the original attention score matrix Q All elements of , select specific columns from the original attention score matrix to reconstruct the matrix AttnScore′, that is, keep the selected columns, which can reduce the size of the matrix and thus reduce the computational complexity;

[0051] For the reconstructed attention score matrix AttnScore′, we change the perspective again and calculate the key vector K sum Perform cumulative summation to obtain:

[0052]

[0053] Similarly, sort K in descending order sum , K sum [i] represents the cumulative sum of the i-th column vector, and selects the largest front The key index corresponding to the score is recorded as set IK ;

[0054] Finally, according to set I K Reconstruct the attention score matrix again as AttnScore′, with a size of ln 2 N:

[0055] AttnScore′[:,j]=AttnScore′[I K ][j],j=1,2,...,lnN,

[0056] Through the above steps, the complexity of the original self-attention is reduced from O(N 2 ) is reduced to O(ln 2 N), thus greatly improving the calculation efficiency;

[0057] AttnScore′[:,j] represents all elements in the jth column of the reconstructed attention score matrix AttnScore′. K ][j] represents selecting set I from the reconstructed attention score matrix AttnScore′ K The element of the j-th column corresponding to the row specified in ;

[0058] Combined with the rotation position encoding, the relative position can be integrated into the self-attention mechanism and improve the performance of the Transformer architecture. For the input sequence of the business data stream length N:

[0059]

[0060] Among them, ω i represents the i-th token in the input sequence. Token refers to the basic unit in the input sequence. Embedding is a technique for mapping discrete symbols to continuous vector space. The input sequence S N The corresponding embedding can be expressed as:

[0061]

[0062] where x i represents the i-th token, ω i Corresponding to the d-dimensional embedding vector;

[0063] S N Represents the set of all tokens in the input sequence, E N Represents the input sequence S N The set of embedding vectors corresponding to each token in ;

[0064] Then, before doing self-attention, the query vector, key vector, and value vector are calculated and the position information is added. The function formula is expressed as follows:

[0065] q m =f q (x m ,m),

[0066] k n =f k (x n ,n),

[0067] v n =f v (x n ,n),

[0068] Among them, f q (*), f k (*) and f v (*) are functions that generate query vectors, key vectors, and value vectors, respectively, and accept input embedding vectors and position information as input;

[0069] q m Represents the load vector x corresponding to the mth token m The query vector after integrating the position information m, and k n and v n It represents the load vector x corresponding to the nth token n The key and value vectors after integrating the position information n;

[0070] Calculate the mth load embedding vector x m The corresponding self-attention output is q m and other k n Calculate an attention score, and then multiply the attention score by the corresponding v n Then sum to get the output vector o m :

[0071]

[0072] is the transpose of the query vector, k n is the key vector with index n, k j It is a general representation when traversing all key vectors, j traverses all key vectors from 1 to N; a m,n is the query vector q m Key vector k n The attention score, v n is a value vector, and the calculated attention score am,n As the weight, d is the dimension, and the weighted sum is performed on all n to obtain the output vector o m .

[0073] In a preferred embodiment of the present invention, the drawing module includes:

[0074] The predicted data is presented in the form of intuitive and easy-to-understand charts. By comparing the distribution and trend of the predicted data with the actual observed real data, the degree of agreement and potential deviation between the two can be clearly seen. At the same time, the drawing module can quantitatively analyze the prediction error, thereby evaluating the quality of the model's prediction performance and providing key basis for further optimization of the model.

[0075] The present invention also discloses a computer system, comprising:

[0076] processor;

[0077] a memory for storing processor-executable instructions;

[0078] Wherein, the processor is configured to implement the distributed task adaptive load prediction system for edge computing when executing the executable instructions.

[0079] The present invention also discloses a computer-readable storage medium, comprising:

[0080] a memory having a computer program stored thereon;

[0081] A processor is used to execute the program in the memory to implement a distributed task adaptive load prediction system for edge computing.

[0082] The present invention also discloses a distributed task adaptive load prediction method for edge computing, comprising the following steps:

[0083] S1, preprocess the input data;

[0084] S2, check and repair missing values, wrong values ​​and relationship errors in the data;

[0085] S3, divide the input data and specify the data length used to predict a data value;

[0086] S4, various feature data are input into the model for training;

[0087] S5, using the training results for load prediction;

[0088] S6 presents the load forecast results in an intuitive and easy-to-understand chart format. By comparing the predicted data with the actual data, the performance of the prediction is evaluated and the model is optimized.

[0089] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0090] Significantly reduce data transmission delay and improve response speed. The system drives local workload prediction through incremental representation of historical business data, and introduces a location information encoding method based on the Transformer architecture to effectively identify and process key features in load changes, thereby improving edge device resource utilization and achieving efficient management of edge devices.

[0091] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0093] Figure 1 It is a system block diagram of the present invention.

[0094] Figure 2 It is a schematic diagram of edge computing system.

[0095] Figure 3 This is a diagram of the task scheduling process in the edge computing system.

[0096] Figure 4 It is a diagram showing the connection between two different time points before dimensionality reduction.

[0097] Figure 5 It is a diagram showing the connection between two different time points after dimensionality reduction.

[0098] Figure 6 This is a diagram of the loss information at the end of training.

[0099] Figure 7 It is an overall comparison chart of the model prediction value and the actual data. DETAILED DESCRIPTION

[0100] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0101] Scenario modeling: When users participate in or watch real-time interactive live content, such as online education, game live broadcasts, and large-scale conference live broadcasts, CDN edge nodes pre-cache part of the live content, and at the same time combine edge computing capabilities to achieve real-time transcoding, editing, and personalized recommendations for audio and video streams. The media content uploaded by the anchor first reaches the nearest edge node, and after being quickly processed by the edge node, the content is quickly distributed to each CDN node, ensuring that viewers can get a low-latency, high-quality viewing experience no matter where they are.

[0102] like Figure 1 to Figure 7 As shown:

[0103] 1. Data preprocessing module, used to preprocess the input data;

[0104] When building and analyzing system performance, the application data involved mainly includes the following aspects:

[0105] 1.1 Hardware configuration data: including the basic hardware parameters of the server, such as the CPU frequency, memory capacity, memory speed, and core thread number. The total number of servers participating in the data sampling is also recorded to ensure the representativeness of the sample. In addition, it also includes information such as the CPU speed that reflects the server's computing power, the bandwidth size of the network communication capability, the specific network card model, and the hard disk storage specifications.

[0106] 1.2 Server resource status data: The collected initial status information of the server resources, such as the starting values ​​of core performance indicators such as CPU utilization, memory utilization, and bandwidth utilization. As time goes by, the changes in memory utilization and CPU utilization, as well as the amount of I / O operations and bandwidth usage are continuously monitored, and the specific time required for the server to process specific data is recorded.

[0107] 1.3 Request data: system request-related data, such as the number of queries per second (QPS), average response time, the actual number of requests received, the actual request data size, and other information.

[0108] 1.4 Data collection granularity: In order to accurately capture and deeply analyze changes in system performance, the data collection granularity is set to the minute level, or refined to a shorter time interval of 10 seconds to 30 seconds to ensure the real-time and accuracy of the data, thereby better guiding system optimization and improvement work.

[0109] During the in-depth data preprocessing stage, necessary filling operations were performed to address the irregularities of the data to ensure data integrity and consistency, including missing values. Padding technology was used to handle the length inconsistency in the sequence data, thereby solving the possible discontinuity problem between time series data points. In addition, in order to integrate information from different dimensions, data at the cluster and device levels were associated and matched to ensure that the data of each device can accurately correspond to the cluster to which it belongs.

[0110] 2. Data engineering module. In the data preprocessing stage, missing value processing, outlier detection, consistency check and relationship modeling are crucial. First, by checking the missing values ​​in the time series and adopting different strategies according to the missing type, such as filling the time missing with the average value or using linear interpolation to handle the missing business features, analysis bias can be effectively avoided. Secondly, outlier detection and correction are performed using the set threshold to reduce the impact of outliers on model accuracy. Furthermore, consistency checks are used to ensure the logical coherence of the data and the correctness of the timestamps, especially for periodic data, to ensure its boundary consistency. Finally, by modeling the relationship between equipment loads and businesses, exploring the correlation in historical data helps to identify normal behavior patterns and establish predictive models. We further elaborated on the data engineering part.

[0111] 2.1 Missing value processing: Missing value processing is an important part of data cleaning. Check whether there are missing values ​​in the time series data, such as incomplete records at certain time points, missing business features, etc. Different filling strategies are used according to different missing situations. The missing time part is filled with the average value, and the missing business features are filled with linear interpolation. Missing value processing can solve the problem of data blanks or missing data in the data set due to various reasons, and avoid the deviation of analysis results, the decline of model fitting effect, and even model failure caused by missing values.

[0112] 2.2 Outlier detection: Outlier detection is to identify and process observations in the data set that are significantly deviated from the normal expected range. By taking the average value of the entire series as the standard, the thresholds are set at 2 times, 2.5 times, and 3 times to mark the outliers. In the process of processing outliers, the original time series is reconstructed by correcting the anomalies to a reasonable range to reduce the impact of outliers on the model and avoid distorting the results of data analysis and the accuracy of the model.

[0113] 2.3 Consistency check: Consistency check focuses on the internal logic and integrity of data, ensuring that the data is consistent with each other and other known facts or rules. Ensuring the consistency and correctness of timestamps cannot be ignored for sequence prediction dependency modeling. The original data is sorted by different period standards, and missing and errors in the time sequence are repaired. Especially for periodic data, the consistency of period boundaries (such as data sorted by day, week, month, etc.) is confirmed.

[0114] 2.4 Relationship modeling: Model the relationship between device load and load, and load and business in edge devices. By exploring the relationship between each feature historical data, we can identify normal business behavior patterns based on hidden information obtained from data analysis, and thus establish relevant models.

[0115] 3. Sequence division module divides the input multimedia data and specifies the length of data used to predict a certain data value. In order to intuitively show the characteristics of the data, it is represented by 5-minute and daily granularity, and the time span represented is two days and one month. The data set consists of business load information at each timestamp (5-minute intervals), and the length is 8842. We use the first 80% of the data set as the training set and the remaining 20% ​​as the test set.

[0116] In order to construct an effective training set, the original time series data is first preprocessed, such as sampling, interpolating or truncating the data, to ensure that all data points are distributed at a uniform time interval. The normalized time series is divided into a series of continuous time segments with a defined sliding window. At this time, there is certain dynamic pattern information between adjacent time steps in each window. For each sliding window, a training set is extracted. Each training set covers historical observation values, related time series labels and business feature attributes. The real result sequence to be predicted for the future time period corresponding to each window is marked and set as the target sequence. Finally, the corresponding training sets and their respective target sequences are integrated to construct a training batch.

[0117] 4. Model training and prediction module

[0118] 4.1 Model building,

[0119] That is, the business characteristics at different times (bandwidth, number of requests, frequency (bandwidth divided by number of requests) and equipment load characteristics).

[0120] 4.2 Model Training

[0121] Mobile edge computing (MEC) reduces the edge network traffic burden and improves data processing speed and response time by migrating computing and data processing tasks from the cloud to the network edge close to where data is generated.

[0122] However, in edge networks, servers equipped with edge networks and edge nodes have relatively small storage and computing capabilities. At the same time, due to the dynamic, complex, and heterogeneous nature of content in the network, efficient resource management and scheduling are still quite challenging in edge networks. Position encoding in the Transformer architecture is an important technical means to capture the features between elements in a sequence.

[0123] First, relative information encoding is introduced in position encoding. The encoding of relative information introduces the concept of time, which can capture the characteristic changes brought by the distance of the time series, so as to better capture the potential correlation between the load time nodes in the sequence. In the encoding matrix corresponding to the time series, the mth position of the load vector matrix Q (Query) and the nth position of the load vector matrix K (Key) are taken, denoted as q m and k n , which is different from the traditional edge computing that only relies on static location information. m and k n When calculating the attention score between two load node vectors, we not only consider their absolute position information, but also introduce the concept of relative position index, i.e., |mn|. m With k n The relative position index between them is used to enhance the model's ability to understand sequence data. At the same time, this value reflects the m With k n The relative distance in the sequence, where n is chosen as the reference. That is, for q m and k n , their attention scores are not only based on the load characteristics of the current moment, but also incorporate the relative position load difference information between different load information nodes, so that the model can understand and capture the relative order and dependency between elements in the load sequence data, thereby improving the model's modeling ability for sequence data. The attention score between two load node vectors is calculated as:

[0124]

[0125] in Defined as:

[0126]

[0127] q m represents the query vector, k n is the key vector, is the similarity score between the query vector and the key vector, which is used to measure the degree of match between them;

[0128] d k is the dimension of the vector, r is a hyperparameter, m and n are the position indexes in the sequence, indicating the relative distance between two elements in the position;

[0129] Here r·|mn| represents the relative position embedding, which is a predefined vector used to reflect q m and k n The relative position relationship between them. In order to introduce the information characteristics of the relative distance between loads, the rotation matrix is ​​used to transform the load vector. Taking the two-dimensional vector as the standard, when there are two load node vectors x and y, in the vector coordinate system, when the vectors rotate around the origin by a certain angle, although their respective directions will change, the key is that the relative position relationship between them, that is, the angle formed by the line connecting the two, will change accordingly. That is, in edge computing load prediction, the dot product of the rotated load vector actually contains the relative time distance change information between the loads at different time points before and after the rotation. The specific operating principle is further given - using matrices to implement this rotation operation. For a two-dimensional vector x, if it rotates m radians counterclockwise around the origin, it is expressed as passing through the rotation matrix R m The rotation matrix R m Usually has the following form:

[0130]

[0131] That is, for the unit rotation transformation matrix R m and R n , which represent the operation of rotating m radians and n radians counterclockwise around the origin respectively. At the same time, according to the combined properties of the rotation matrix, when a vector is first rotated by R m Rotate, then press R n Rotate, its effect is equivalent to one-time Rotate, and the result of this matrix product is actually the rotation matrix R of (nm) radians n-m .

[0132] Furthermore, for vectors in two-dimensional space, suppose there are two vectors x and y. If we first rotate vector x by m radians to get R m x, and rotate the vector y by n radians to get R n y, and then perform a dot multiplication on the two rotated vectors, that is, calculate (R m x) T ·(R n y), which represents two different time nodes in edge computing. According to the above matrix multiplication rule, if we directly rotate x by (nm) radians, we get R n-m x, and multiply this vector by the unrotated y, that is, (R n-m·x)y, the dot product results obtained in these two ways will be the same, because the combined operation of the rotation matrix maintains the invariance of the dot product operation. This provides a method to simplify the problem based on the relative value of the rotation angle, namely:

[0133]

[0134] That is, for the unit rotation transformation matrix R m and R n , which represent the operation of rotating m radians and n radians counterclockwise around the origin respectively. At the same time, according to the combined properties of the rotation matrix, when a vector is first rotated by R m Rotate, then press R n Rotate, its effect is equivalent to one-time Rotate, and the result of this matrix product is actually the rotation matrix R of (nm) radians n-m ;

[0135] For a vector in two-dimensional space, suppose there are two vectors x and y. If we first rotate vector x by m radians, we get R m x, and rotate the vector y by n radians to get R n y, and then perform a dot multiplication on the two rotated vectors, that is, calculate (R m x) T ·(R n y), which represents two different time nodes in edge computing. According to the above matrix multiplication rule, if we directly rotate x by (nm) radians, we can get R n-m x, and multiply this vector by the unrotated y, that is, (R n-m ·x)y, the dot product results obtained in these two ways will be the same. This is because the combined operation of the rotation matrix maintains the invariance of the dot product operation, which provides a method to simplify the problem based on the relative value of the rotation angle, namely:

[0136]

[0137] (R m x) T and (R n y) represents the vector x and the vector y rotated by m and n radians respectively around the far point;

[0138] Let x0, x1 be the element values ​​in vector x, and y0, y1 be the element values ​​in vector y, and we can get:

[0139]

[0140] Q m Rotate m degrees, k nRotate n degrees, then by

[0141]

[0142] Available,

[0143] score(q m ,k n )=(R m q m ) T ·(R n k n )

[0144] (R m q m ) T Represents the query vector q after rotation m The transpose of (R n k n ) represents the key vector k after rotation n .

[0145] At this time, the encoding provides relative information. For absolute position encoding, the introduction of relative information can not only reduce the model's dependence on absolute position information, but also help reduce the number of model parameters and improve the generalization ability of the model.

[0146] like Figure 2 and Figure 3 Figure 1 shows the task scheduling process in a mobile edge computing (MEC) system, which receives requests from different service providers (such as YouTube, Twitter, Amazon, TikTok, Facebook, and Netflix) and performs load distribution and task scheduling through intelligent algorithms. The request first flows into the system, passes through preset rules and load evaluation, and then the reinforcement learning algorithm determines the optimal task scheduling strategy. At this time, the load is assigned to the edge server or a more remote cloud data center for processing. At this time, the load volatility is affected by many factors, including but not limited to network conditions, user behavior patterns, and service quality requirements. In addition, the time complexity of the prediction algorithm will also affect the performance and response time of the system.

[0147] In order to solve these problems, the DoubleSimple method is proposed. By modifying the self-attention mechanism, the computational complexity can be reduced, thereby optimizing the system performance and response time.

[0148] In the edge computing environment, the dual simplification of self-attention is used to optimize the load prediction and resource scheduling on edge devices. With the introduction of relative encoding, the relationship between two different time points in the time series is displayed as follows Figure 4 and Figure 5 shown.

[0149] By observing the output image, we can find that not all dot products between Query (Q) and Key (K) can produce meaningful or important results. Especially in edge networks, considering the limited computing resources and large load fluctuations, it is unnecessary to spend a lot of computing resources to process each dot product, especially considering that some elements may only have a strong correlation with a few of their elements, and the correlation with most other elements is not obvious. In order to improve computing efficiency and adapt to the characteristics of edge computing, it is necessary to focus on these dot product values ​​with significant characteristics.

[0150] First, for a sequence of length N, the score of each row obtained by the self-attention mechanism is accumulated and summed, where each element represents the original N 2 The importance scores of the corresponding rows in the attention matrix are calculated, and this new sequence is called Qscore. Then, the first lnN elements with the largest values ​​in Qscore and their corresponding indices are selected, which effectively converts the original self-attention matrix from N to 2 The dimension is reduced to NlnN. In this process, special attention is paid to those loads that frequently interact in the edge network or have a significant impact on the system status, so as to ensure that computing resources are allocated to these critical loads first.

[0151] A similar operation is performed on this reduced attention matrix. This time, the scores of the columns (i.e., each key) of the matrix are horizontally added to obtain a new sequence Kscore, and the first lnN elements with the largest values ​​and their indices are also selected from them. In this way, the original attention matrix is ​​further simplified to ln 2 N size. In this way, the computational complexity of the self-attention mechanism is reduced from O(N 2 ) is reduced to O(ln 2 N), thereby greatly reducing the amount of calculation and improving the calculation efficiency, and increasing the response speed and resource utilization of edge devices. Finally, after extracting the most critical elements for edge network load scheduling from the self-attention mechanism during the training process, the corresponding connections between two different time points are shown as follows:

[0152] This process can be described as:

[0153] The known sequence vector is abstracted into the query matrix and key matrix of the self-attention mechanism through feature evolution from different angles. The dot product of the query matrix Q and the key matrix K is usually used to calculate the attention score:

[0154] AttnScore=Q·K T

[0155] Among them, AttnScore∈R N×N, N is the sequence length. In order to find the most important query, the scores of each row are accumulated and summed to obtain a new vector Q sum :

[0156]

[0157] Q sum [i] represents the sum of the attention scores of the ith query vector and all key vectors, which is a vector consisting of the cumulative sum of each row. AttnScore[i][j] represents the attention score between the ith query vector and the jth key vector.

[0158] Then, sort Q in descending order sum , and select the largest front The query index corresponding to the score is recorded as set I Q , based on set I Q , reconstruct the self-attention score matrix as AttnScore′, with a size of

[0159] AttnScore′[i][:]=AttnScore′[i][I Q ],i=1,2,...,N

[0160] AttnScore′[i][:] is all the elements in the i-th row of the reconstructed new sparse attention score matrix. Q ] is the i-th row and column index I in the original attention score matrix Q All elements of , select specific columns from the original attention score matrix to reconstruct the matrix AttnScore′, that is, keep the selected columns, which can reduce the size of the matrix and thus reduce the computational complexity;

[0161] For the reconstructed attention score matrix AttnScore′, we change the perspective again and calculate the key vector K sum Perform cumulative summation to obtain:

[0162]

[0163] Similarly, sort K in descending order sum , K sum [i] represents the cumulative sum of the i-th column vector, and selects the largest front The key index corresponding to the score is recorded as set I K ;

[0164] Finally, according to set I K Reconstruct the attention score matrix again as AttnScore′, with a size of ln2 N:

[0165] AttnScore′[:,j]=AttnScore′[I K ][j],j=1,2,...,lnN

[0166] Through the above steps, the complexity of the original self-attention map is reduced from O(N 2 ) is reduced to O(ln 2 N), thus greatly improving the calculation efficiency;

[0167] AttnScore′[:,j] represents all elements in the jth column of the reconstructed attention score matrix AttnScore′. K ][j] represents selecting set I from the reconstructed attention score matrix AttnScore′ K The element of the j-th column corresponding to the row specified in ;

[0168] It can achieve efficient and fast load prediction and resource allocation in edge computing environments, while reducing unnecessary computing overhead to adapt to the resource-constrained characteristics of edge computing environments.

[0169] Combined with the rotation position encoding, the relative position can be integrated into the self-attention mechanism and improve the performance of the Transformer architecture. For the input sequence of the business data stream length N:

[0170]

[0171] Among them, ω i represents the i-th token in the input sequence. Token refers to the basic unit in the input sequence. Embedding is a technique for mapping discrete symbols to continuous vector space. The input sequence S N The corresponding embedding can be expressed as:

[0172]

[0173] where x i represents the i-th token, ω i Corresponding to the d-dimensional embedding vector;

[0174] S N Represents the set of all tokens in the input sequence, E N Represents the input sequence S N The set of embedding vectors corresponding to each token in ;

[0175] Then, before doing self-attention, the query vector, key vector, and value vector are calculated and the position information is added. The function formula is expressed as follows:

[0176] q m =f q (x m ,m)

[0177] k n =f k (x n ,n)

[0178] v n =f v (x n ,n)

[0179] Among them, f q (*), f k (*) and f v (*) are functions that generate query vectors, key vectors, and value vectors, respectively, and accept input embedding vectors and position information as input;

[0180] q m Represents the load vector x corresponding to the mth token m The query vector after integrating the position information m, and k n and v n It represents the load vector x corresponding to the nth token n The key and value vectors after integrating the position information n;

[0181] Calculate the mth load embedding vector x m The corresponding self-attention output is q m and other k n Calculate an attention score, and then multiply the attention score by the corresponding v n Then sum to get the output vector o m :

[0182]

[0183] is the transpose of the query vector, k n is the key vector with index n, k j It is a general representation when traversing all key vectors, j traverses all key vectors from 1 to N; a m,n is the query vector q m Key vector k n The attention score, v n is a value vector, and the calculated attention score am,n As the weight, d is the dimension, and the weighted sum is performed on all n to obtain the output vector o m .

[0184] 4.3 Model prediction, an important part of data analysis and machine learning, is used to predict the data trend at a certain point in time or time period in the future. Once the model is trained and its performance is good, the trained model can be used to estimate future data points. Using the latest time series data as input, the prediction model can generate continuous or discrete numerical prediction results within a certain period of time in the future.

[0185] Denormalization module,In the data preprocessing stage, normalization technology is often used to make data of different scales or distributions comparable.,For this purpose, a denormalization module is specially designed to restore the previously processed,predicted values ​​at the same scale to their actual numerical levels in the original data set.

[0186] The drawing module presents the predicted data in an intuitive and easy-to-understand chart. By comparing the distribution and trend of the predicted data with the actual observed real data, the degree of agreement and potential deviation between the two can be clearly seen. At the same time, the drawing module can quantitatively analyze the prediction error, thereby evaluating the performance of the model prediction and providing a key basis for further optimizing the model.

[0187] The experimental data of the present invention uses the data set of Baishan Cloud Technology Company, which contains 108 edge cache clusters, and the number of services in each cluster ranges from 1 to 139. At the code implementation level, through feature processing and data standardization, the core data is effectively converted, and its numerical range is standardized to the interval of (-1,1), thereby successfully eliminating the influence of abnormal data. Then, through neural network modeling and instantiation, in 50 iterations of training, Figure 6 The final loss and other metrics are shown as they change over epochs.

[0188] The traditional LSTM content popularity prediction method has relatively weak long-term dependency capabilities when processing sequence information, and sometimes it may be difficult to capture longer-term influencing factors. The dependencies in LSTM limit the ability of parallel computing, which is particularly important in edge computing environments because edge devices are often limited by computing power and storage resources, which may lead to slower training and reasoning. At the same time, the internal working mechanism of LSTM is relatively complex and not easy to explain. For a given target, the Attention mechanism generates a weight coefficient to perform a weighted sum of the input to identify which features in the input are important to the target and which features are not important. The Transformer model uses a self-attention mechanism to establish connections on a global scale and assign weights to each element by comparing the relationship between each element in the sequence and other elements. This mechanism is very suitable for edge computing scenarios because it can effectively process large amounts of data while reducing unnecessary computing burdens.

[0189] The use of relative position encoding provides the ability to describe the relative position changes between vectors. This encoding method helps the model understand and process the dynamic structure in the data, thereby improving the learning and expression capabilities of complex sequence patterns, allowing the model to flexibly process relative position information without adding additional complexity. This is particularly beneficial for edge computing, simplifying the calculation process, while enabling the model architecture to better handle input sequences of arbitrary lengths, adding position information to each token embedding, and making the model more adaptable and generalizable when facing data of different lengths or structures. This is critical for edge devices because they need to handle a variety of data types and lengths. For the problem of high time complexity of the self-attention mechanism, the DoubleSimple strategy is improved to reduce the dimension of Qscore and Kscore to find out the query and key with higher importance, effectively reducing unnecessary computing resource consumption, thereby reducing the complexity of the self-attention mechanism from O(N^2) to O(ln^2N), greatly improving the training speed and computing efficiency. In the edge computing environment, it is conducive to the rapid response and efficient resource management of edge device systems.

[0190] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. An adaptive load prediction system for distributed tasks for edge computing, characterized in that: include: Data preprocessing module, preprocessing the input data; Data engineering module, which checks and repairs missing values, incorrect values, and relationship errors in data; The sequence partitioning module partitions the input data and specifies the data length used to predict a data value; Model training module, various feature data are input into the model for training; The prediction module uses the training results for load prediction; The drawing module presents the load forecast results in an intuitive and easy-to-understand chart format. By comparing the predicted data with the actual data, the performance of the forecast is evaluated and the model is optimized.

2. The distributed task adaptive load prediction system for edge computing according to claim 1 is characterized in that: The data preprocessing module comprises: Get hardware configuration data: including CPU frequency, memory capacity, memory speed, and number of core threads; Obtain server resource status data: including the starting values ​​of CPU utilization, memory utilization, and bandwidth utilization, and continuously monitor changes in memory utilization and CPU utilization, as well as I / O operation volume and bandwidth usage, and record the specific time required for the server to process specific data; Obtain system request data: including the number of queries per second, average response time, the actual number of requests received, and the actual request data size; The data collection granularity is refined to a time interval of 10 seconds to 30 seconds to ensure the real-time and accuracy of the data.

3. The distributed task adaptive load prediction system for edge computing according to claim 1 is characterized in that: The data engineering module includes: Missing value processing: Check whether there are missing values ​​in the time series data, and use different filling strategies according to the different missing situations. For the missing time part, use the average value to fill it, and for the missing business features, use the linear interpolation method; Outlier detection: Using the average value of the entire sequence as the standard, the thresholds are set at 2 times, 2.5 times, and 3 times to mark outliers; Consistency check: ensure that data is consistent with each other and with other known facts or rules, sort the original data according to different period standards, and repair the missing and errors in the time sequence. In particular, for periodic data, confirm the consistency of period boundaries; Relationship modeling: Model the relationship between device load and load, and load and business in edge devices. By exploring the relationship between various feature historical data, you can identify normal business behavior patterns based on hidden information obtained from data analysis.

4. The distributed task adaptive load prediction system for edge computing according to claim 1 is characterized in that: The sequence division module comprises: For example, sample, interpolate or truncate the data to ensure that all data points are distributed at a uniform time interval, divide the normalized time series into a series of continuous time segments with a defined sliding window, extract relevant training sets for each sliding window, each training set covers historical observation values, related time series labels and business feature attributes, mark the real result sequence to be predicted for the future time period corresponding to each window, set it as the target sequence, and finally, integrate the corresponding training sets and their respective target sequences to construct a training batch.

5. The distributed task adaptive load prediction system for edge computing according to claim 1 is characterized in that: The model training module includes: The relative position load difference information between information nodes enables the model to understand and capture the relative order and dependency between elements in the load sequence data, thereby improving the model's modeling ability for sequence data. The attention score between two load node vectors is calculated as follows: in Defined as: q m represents the query vector, k n is the key vector, is the similarity score between the query vector and the key vector, which is used to measure the degree of match between them; d k is the dimension of the vector, r is a hyperparameter, m and n are the position indexes in the sequence, indicating the relative distance between two elements in the position; For a two-dimensional vector x, if it is rotated counterclockwise around the origin by m radians, it is represented by the rotation matrix R m The role of the rotation matrix R m Usually has the following form: That is, for the unit rotation transformation matrix R m and R n , which represent the operation of rotating m radians and n radians counterclockwise around the origin respectively. At the same time, according to the combined properties of the rotation matrix, when a vector is first rotated by R m Rotate, then press R n Rotate, its effect is equivalent to one-time Rotate, and the result of this matrix product is actually the rotation matrix R of (nm) radians n-m ; For a vector in two-dimensional space, suppose there are two vectors x and y. If we first rotate vector x by m radians, we get R m x, and rotate the vector y by n radians to get R n y, and then perform a dot multiplication on the two rotated vectors, that is, calculate (R m x) T ·(R n y), which represents two different time nodes in edge computing. According to the above matrix multiplication rule, if we directly rotate x by (nm) radians, we can get R n-m x, and multiply this vector by the unrotated y, that is, (R n-m ·x)y, the dot product results obtained in these two ways will be the same. This is because the combined operation of the rotation matrix maintains the invariance of the dot product operation, which provides a method to simplify the problem based on the relative value of the rotation angle, namely: (R m x) T and (R n y) represents the vector x and the vector y rotated by m and n radians respectively around the far point; Let the element values ​​in vector x be x0, x1, and the element values ​​in vector y be y0, y1, and we can get: Q m m rotation degree, k n Rotate n degrees, then by Available, score(q m ,k n )=(R m q m ) T ·(R n k n ), (R m q m ) T Represents the query vector q after rotation m The transpose of (R n k n ) represents the key vector k after rotation n .

6. The distributed task adaptive load prediction system for edge computing according to claim 1 is characterized in that: The prediction module comprises: The known sequence vector is abstracted into the query matrix and key matrix of the self-attention mechanism through feature evolution from different angles. The dot product of the query matrix Q and the key matrix K is usually used to calculate the attention score: AttnScore=Q·K T , Among them, AttnScore∈R N×N , N is the sequence length. In order to find the most important query, the scores of each row are accumulated and summed to obtain a new vector Q sum : Q sum [i] represents the sum of the attention scores of the ith query vector and all key vectors, which is a vector consisting of the cumulative sum of each row. AttnScore[i][j] represents the attention score between the ith query vector and the jth key vector. Then, sort Q in descending order sum , and select the largest front The query index corresponding to the score is recorded as set I Q , based on set I Q , reconstruct the self-attention score matrix as AttnScore′, with a size of AttnScore′[i][:]=AttnScore′[i][I Q ],i=1,2,...,N, AttnScore′[i][:] is all the elements in the i-th row of the reconstructed new sparse attention score matrix. Q ] is the i-th row and column index I in the original attention score matrix Q All elements of , select specific columns from the original attention score matrix to reconstruct the matrix AttnScore′, that is, keep the selected columns, which can reduce the size of the matrix and thus reduce the computational complexity; For the reconstructed attention score matrix AttnScore′, we change the perspective again and calculate the key vector K sum Perform cumulative summation to obtain: Similarly, sort K in descending order sum , K sum [i] represents the cumulative sum of the i-th column vector, and selects the largest front The key index corresponding to the score is recorded as set I K ; Finally, according to set I K Reconstruct the attention score matrix again as AttnScore′, with a size of ln 2 N: AttnScore′[:,j]=AttnScore′[I K ][j],j=1,2,...,lnN, Through the above steps, the complexity of the original self-attention is reduced from O(N 2 ) is reduced to O(ln 2 N), thus greatly improving the calculation efficiency; AttnScore′[:,j] represents all elements in the jth column of the reconstructed attention score matrix AttnScore′. K ][j] represents selecting set I from the reconstructed attention score matrix AttnScore′ K The element of the j-th column corresponding to the row specified in ; Combined with the rotation position encoding, the relative position can be integrated into the self-attention mechanism and improve the performance of the Transformer architecture. For the input sequence of the business data stream length N: Among them, ω i represents the i-th token in the input sequence. Token refers to the basic unit in the input sequence. Embedding is a technique for mapping discrete symbols to continuous vector space. The input sequence S N The corresponding embedding can be expressed as: where x i represents the i-th token, ω i Corresponding to the d-dimensional embedding vector; S N Represents the set of all tokens in the input sequence, E N Represents the input sequence S N The set of embedding vectors corresponding to each token in ; Then, before doing self-attention, the query vector, key vector, and value vector are calculated and the position information is added. The function formula is expressed as follows: q m =f q (x m ,m), k n =f k (x n ,n), v n =f v (x n ,n), Among them, f q (*), f k (*) and f v (*) are functions that generate query vectors, key vectors, and value vectors, respectively, and accept input embedding vectors and position information as input; q m Represents the load vector x corresponding to the mth token m The query vector after integrating the position information m, and k n and v n It represents the load vector x corresponding to the nth token n The key and value vectors after integrating the position information n; Calculate the mth load embedding vector x m The corresponding self-attention output is q m and other k n Calculate an attention score, and then multiply the attention score by the corresponding v n Then sum to get the output vector o m ： is the transpose of the query vector, k n is the key vector with index n, k j It is a general representation when traversing all key vectors, j traverses all key vectors from 1 to N; a m,n is the query vector q m Key vector k n The attention score, v n is a value vector, and the calculated attention score a m,n As the weight, d is the dimension, and the weighted sum is performed on all n to obtain the output vector o m .

7. The distributed task adaptive load prediction system for edge computing according to claim 1 is characterized in that: The drawing module comprises: The predicted data is presented in the form of intuitive and easy-to-understand charts. By comparing the distribution and trend of the predicted data with the actual observed real data, the degree of agreement and potential deviation between the two can be clearly seen. At the same time, the drawing module can quantitatively analyze the prediction error, thereby evaluating the quality of the model's prediction performance and providing key basis for further optimization of the model.

8. A computer system, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to implement the distributed task adaptive load prediction system for edge computing described in any one of claims 1 to 7 when executing the executable instructions.

9. A computer-readable storage medium, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the program in the memory to implement the distributed task adaptive load prediction system for edge computing as described in any one of claims 1 to 7.

10. A method for adaptive load prediction of distributed tasks for edge computing, characterized in that: The following steps are involved: S1, preprocess the input data; S2, check and repair missing values, wrong values ​​and relationship errors in the data; S3, divide the input data and specify the data length used to predict a data value; S4, various feature data are input into the model for training; S5, using the training results for load prediction; S6 presents the load forecast results in an intuitive and easy-to-understand chart format. By comparing the predicted data with the actual data, the performance of the prediction is evaluated and the model is optimized.

Images