Real-time link anomaly detection and sampling method and device based on edge AI drive

Abstract

The invention provides a real-time link anomaly detection and sampling method and device based on edge AI driving, and is applied to the technical field of artificial intelligence. Segmenting and aggregating the original link tracking data in each time window through a dynamic time window division mechanism to generate a statistical feature vector of link topology in the time window, the original link tracking data including a link topology relationship, a service calling level and a performance index; inputting the statistical feature vector into an anomaly detection model obtained by pre-training for analysis to obtain a path anomaly probability score; and dynamically adjusting the sampling strategy of the next time window according to the path anomaly probability score. Through edge AI calculation nodes, dynamic time window division, segmented aggregation and dynamic adjustment of a sampling strategy, the real-time performance of anomaly detection and the reliability and practicability of anomaly detection are improved, adaptive sampling is realized, and the sampling precision and efficiency are ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence and system monitoring, in particular to a real-time link anomaly detection and sampling method and device based on edge AI driving. BACKGROUND

[0002] Traditional link anomaly detection systems mostly rely on a central centralized processing architecture, and data needs to be transmitted to a central server for processing and analysis. This results in a large delay in the data transmission process, especially when the data volume is large or the network condition is poor. The time interval from the occurrence of link anomalies to the detection of anomalies is long, which cannot meet the requirements of real-time business scenarios such as financial transaction systems and industrial automation production lines. In these scenarios, timely detection and processing of anomalies are crucial, and any delay may cause serious consequences. In a financial transaction system, if the link is abnormal, it may cause transaction data transmission delay or error, which may result in huge financial losses.

[0003] Existing anomaly detection methods, whether simple threshold judgment or detection methods based on traditional machine learning algorithms, are difficult to accurately identify link anomalies in complex and variable business environments. The threshold judgment method is limited by the fixed threshold setting and cannot adapt to dynamic changes in business, which easily produces a large number of false positives and false negatives. The limitations of traditional machine learning algorithms in feature extraction and model training make it difficult for them to fully exploit the complex features and potential anomaly patterns in link data, making it difficult to accurately locate abnormal points and abnormal reasons and reducing the reliability and practicality of the detection results. During e-commerce promotion activities, business traffic will fluctuate greatly, and traditional detection methods may frequently misreport or miss report anomalies due to their inability to adapt to traffic changes in a timely manner.

[0004] In the data collection stage, the fixed sampling rate sampling strategy cannot dynamically adjust the sampling rate according to the actual importance and abnormality of the link, resulting in a large amount of unimportant data being collected and transmitted, occupying valuable network bandwidth and storage resources. In large-scale distributed systems, if the vast amount of link data generated daily is not effectively processed and optimized, it will result in a large amount of network bandwidth being occupied, a sharp increase in storage costs, and a reduction in data processing efficiency and speed. SUMMARY

[0005] At least one embodiment of the present application provides a real-time link anomaly detection and sampling method and device based on edge AI driving, which solves the problem of false positives or false negatives in the prior art due to link anomaly detection in complex and variable environments, and cannot effectively sample and optimize.

[0006] To solve the above technical problems, the present application is implemented as follows:

[0007] In a first aspect, embodiments of this application provide a real-time link anomaly detection and sampling method based on edge AI, including:

[0008] After receiving the raw link tracing data generated in real time by the service node, the raw link tracing data in each time window is segmented and aggregated through a dynamic time window division mechanism to generate a statistical feature vector of the link topology within the time window. The raw link tracing data includes: link topology relationships, service call levels, and performance indicators.

[0009] The statistical feature vector is input into the pre-trained anomaly detection model for analysis to obtain a path anomaly probability score.

[0010] Based on the path anomaly probability score, the sampling strategy for the next time window is dynamically adjusted.

[0011] Preferably, in the method described above, the step of segmenting and aggregating the original link tracing data within each time window using a dynamic time window division mechanism to generate a statistical feature vector of the link topology within the time window includes:

[0012] In the path time aggregation layer, the response time series of all Spans in the same service call path are statistically analyzed to obtain the first feature vector. The first feature vector includes multiple preset delay quantiles and the kurtosis statistics of the response time series.

[0013] In the error code distribution layer, clustering statistical analysis is performed on the responses of the service nodes based on the Hyper Text Transfer Protocol (HTTP) status codes to obtain a second feature vector. The second feature vector includes the frequency distribution of at least two error status codes and the obtained error entropy value.

[0014] In the topology layer, a third feature vector is obtained based on the service dependency matrix constructed based on the call frequency of each node within the time window. The third feature vector includes: the maximum eigenvalue, the maximum eigenvector, and the asymmetry of the service dependency matrix.

[0015] The first feature vector, the second feature vector, and the third feature vector are merged to obtain the statistical feature vector.

[0016] Specifically, in the method described above, the dynamic time window segmentation mechanism includes:

[0017] An initial time window is constructed based on a first duration, and adjacent time windows have a first ratio of overlap.

[0018] Get the current cumulative number of time windows and the traffic fluctuation coefficient;

[0019] If the number of time windows and the traffic fluctuation coefficient meet the first preset condition, then the service traffic change rate is obtained based on the difference in the number of requests and the time interval between adjacent time windows.

[0020] The window duration of the next time window is dynamically adjusted based on the flow rate change.

[0021] Preferably, in the method described above, dynamically adjusting the window duration of the next time window based on the flow rate change includes:

[0022] If the rate of change of the business traffic is greater than or equal to the first threshold, then the window duration of the next time window will be shortened to the second duration, which is within the range of the first preset duration.

[0023] If the rate of change of the business traffic is less than or equal to the second threshold, the window duration of the next dynamic time window will be extended to a third duration, the third duration being within the range of the second preset duration, and the second threshold being less than the first threshold.

[0024] Preferably, in the method described above, the step of inputting the statistical feature vector into the pre-trained anomaly detection model for analysis to obtain a link anomaly probability score includes:

[0025] The statistical feature vectors corresponding to the first preset number of consecutive time windows are input into the time series analysis channel of the anomaly detection model to obtain the response time surge index. The time series analysis channel is a three-layer long short-term memory network architecture.

[0026] In the topology analysis channel, attention-weighted aggregation is performed based on the edge features of each service node and its neighboring nodes to obtain the topology anomaly score of each call path. The topology analysis channel is a graph attention network, and the edge features include average response latency and error rate.

[0027] Based on the response time surge index and the topology anomaly score, the results are concatenated and weighted to obtain the path anomaly probability score corresponding to each of the calling paths.

[0028] Optionally, the method described above also includes:

[0029] According to the first preset cycle, it periodically receives global model parameters issued by the central analysis platform;

[0030] The cosine similarity is obtained based on the global model parameters and the local model parameters obtained by the anomaly detection model through local updates.

[0031] When the cosine similarity is less than the first threshold, the model parameters of the anomaly detection model are updated based on federated learning.

[0032] Furthermore, the method described above also includes:

[0033] If there is a target call path whose path anomaly probability score is greater than the first score threshold, then according to the second preset period, the anomaly detection model is periodically optimized by gradient descent based on the most recent first number of target call paths stored in the sliding window buffer and the preset learning rate.

[0034] Preferably, in the method described above, the step of dynamically adjusting the sampling strategy for the next time window based on the path anomaly probability score includes:

[0035] For critical abnormal paths whose path anomaly probability score is greater than or equal to the second scoring threshold, high-precision sampling is performed.

[0036] For non-critical paths whose path anomaly probability score is less than the second score threshold, data aggregation based on the prefix tree compression algorithm is performed, and the sampling rate is adjusted based on a pre-built control model of resource consumption and sampling accuracy.

[0037] Specifically, as described above, for non-critical paths with an anomaly probability score less than the second score threshold, data aggregation based on a prefix tree compression algorithm is performed, including:

[0038] The non-critical paths are converted to a dot-separated string representation.

[0039] Construct a multi-branch tree structure based on a depth-first traversal strategy according to the string representation;

[0040] The frequency of occurrence of each subtree in the multi-branch tree structure is statistically analyzed, and the subtrees that meet the preset merging conditions are determined as candidate subtrees;

[0041] The candidate subtrees are compressed to obtain compressed aggregated data, wherein the aggregated data is encoded using structured markup.

[0042] Furthermore, as described above, the step of compressing the candidate subtree to obtain compressed aggregated data includes:

[0043] If there is a target Span data in the candidate subtree whose time consumption is greater than the preset quantile or contains a preset error status code, then the target Span data is forcibly retained according to the forced retention strategy.

[0044] Preferably, in the method described above, adjusting the sampling rate based on a pre-built control model of resource consumption and sampling accuracy further includes:

[0045] When the resource consumption meets the preset conditions, a negative feedback adjustment mechanism is activated to perform a linear decrease adjustment on the sampling rate corresponding to the non-critical path, wherein the sampling rate is greater than or equal to the minimum sampling rate.

[0046] Preferably, the method described above further includes:

[0047] The service dependency matrix within each time window is processed into an upper triangular matrix to obtain an upper triangular element matrix, and the maximum eigenvalue of the upper triangular element matrix is ​​obtained.

[0048] The first topological hash fingerprint is obtained based on the upper triangular element matrix, the maximum eigenvalue, and the cyclic redundancy check function;

[0049] A consistency check is performed based on the first topological hash fingerprint and the reference hash fingerprint corresponding to the historical baseline to obtain the check result;

[0050] If the verification results are consistent, then the indicator increment mode is entered, and the indicator difference of each call path in the current time window is transmitted to the central analysis platform through the difference transmission engine.

[0051] If the verification result is inconsistent, the system enters the structure update mode and transmits the unique identifier list of the newly added path subtree and the deleted path to the central analysis platform through the difference transmission engine.

[0052] Optionally, the method described above further includes:

[0053] If a reverse retransmission request is received from the central analysis platform, a complete topology snapshot and compression operation log are reported for data repair. The reverse retransmission request is sent by the central analysis platform when the second topology hash fingerprint obtained by reconstructing the link topology structure based on the received incremental data of the nodes is inconsistent with the first topology hash fingerprint.

[0054] Preferably, the method described above further includes:

[0055] If a preset extreme resource pressure scenario is detected, a circuit breaker mechanism is activated, which includes an emergency compression mode and / or a lossy transmission mode.

[0056] If the preset extreme resource pressure scenario is detected to be exited, the circuit breaker mechanism will be exited, and data repair will be performed based on historical link observation data of the same time period, and a data repair report will be generated.

[0057] Secondly, embodiments of this application provide a device for real-time link anomaly detection and sampling based on edge AI, comprising:

[0058] The first processing module is used to receive the raw link tracing data generated in real time by the service node, and then use a dynamic time window division mechanism to segment and aggregate the raw link tracing data in each time window to generate a statistical feature vector of the link topology in the time window. The raw link tracing data includes: link topology relationships, service call levels and performance indicators.

[0059] The second processing module is used to input the statistical feature vector into the pre-trained anomaly detection model for analysis, and obtain a path anomaly probability score.

[0060] The third processing module is used to dynamically adjust the sampling strategy for the next time window based on the path anomaly probability score.

[0061] Thirdly, embodiments of this application provide a terminal, including: a processor, a memory, and a program stored in the memory and executable on the processor. When the program is executed by the processor, it implements the steps of the real-time link anomaly detection and sampling method based on edge AI as described above.

[0062] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the edge AI-driven real-time link anomaly detection and sampling method described above.

[0063] Fifthly, embodiments of this application provide a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the edge AI-driven real-time link anomaly detection and sampling method described above.

[0064] Compared with existing technologies, the real-time link anomaly detection and sampling method and apparatus based on edge AI driven by the embodiments of this application can reduce the intermediate links in data transmission to the central platform and improve the real-time performance of anomaly detection by using edge AI computing nodes for data acquisition and processing based on a dynamic time window division mechanism. The received raw link tracing data is processed by segmentation and aggregation to obtain statistical feature vectors, and anomaly detection analysis is performed on the pre-trained anomaly detection model to obtain a path anomaly probability score. Based on this, the sampling strategy for the next time window is dynamically adjusted to achieve adaptive sampling, ensuring sampling accuracy and efficiency, and improving the reliability and practicality of anomaly detection. Attached Figure Description

[0065] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0066] Figure 1 This is one of the flowcharts illustrating the real-time link anomaly detection and sampling method based on edge AI in this application;

[0067] Figure 2 This is the second flowchart illustrating the real-time link anomaly detection and sampling method based on edge AI in this application.

[0068] Figure 3 This is the third flowchart illustrating the real-time link anomaly detection and sampling method based on edge AI in this application;

[0069] Figure 4 This is the fourth flowchart illustrating the real-time link anomaly detection and sampling method based on edge AI in this application;

[0070] Figure 5 This is the fifth flowchart illustrating the real-time link anomaly detection and sampling method based on edge AI in this application;

[0071] Figure 6 This is the sixth flowchart illustrating the real-time link anomaly detection and sampling method based on edge AI in this application;

[0072] Figure 7 This is the seventh flowchart illustrating the real-time link anomaly detection and sampling method based on edge AI in this application.

[0073] Figure 8 This is the eighth flowchart illustrating the real-time link anomaly detection and sampling method based on edge AI in this application.

[0074] Figure 9 This is a schematic diagram of the device for real-time link anomaly detection and sampling based on edge AI in this application;

[0075] Figure 10 This is a schematic diagram of the structure of an electronic device in one embodiment of this application. Detailed Implementation

[0076] Exemplary embodiments of the present application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the scope of the present application to those skilled in the art.

[0077] The terms “first,” “second,” etc., used in the specification and claims of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus. The terms “and / or” in the specification and claims indicate at least one of the connected objects.

[0078] The following description provides examples and is not intended to limit the scope, applicability, or configuration set forth in the claims. Changes may be made to the function and arrangement of the elements discussed without departing from the spirit and scope of this disclosure. Various procedures or components may be appropriately omitted, substituted, or added to the examples. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Furthermore, features described with reference to certain examples may be combined in other examples.

[0079] Please refer to Figure 1 This application provides a real-time link anomaly detection and sampling method based on edge AI, comprising:

[0080] Step S101: After receiving the raw link tracing data generated in real time by the service node, the raw link tracing data in each time window is segmented and aggregated through a dynamic time window division mechanism to generate a statistical feature vector of the link topology within the time window. The raw link tracing data includes: link topology relationships, service call levels, and performance indicators.

[0081] Step S102: Input the statistical feature vector into the pre-trained anomaly detection model for analysis to obtain a path anomaly probability score;

[0082] Step S103: Based on the path anomaly probability score, dynamically adjust the sampling strategy for the next time window.

[0083] In this embodiment, when performing real-time link anomaly detection and sampling in a distributed system, this step is applied to edge artificial intelligence (AI) computing nodes. The edge AI computing nodes receive raw link tracing data (e.g., Span Data) generated in real-time by distributed service nodes and aggregate the raw link tracing data within each time window based on a dynamic time window segmentation mechanism. This generates a statistical feature vector of the link topology within the corresponding time window, facilitating subsequent analysis and processing. Specifically, the raw link tracing data includes link topology relationships, service call levels, and performance metrics. Link topology relationships refer to the complete path graph of inter-service calls; service call levels represent the depth and parent-child relationship of services in the call chain, i.e., which service calls which service, i.e., the call order and sequence. Key metrics include call depth, number of calls, and call frequency; performance metrics are used to quantify data for each Span or service call, with core metrics including latency, error rate, throughput, and success rate.

[0084] Furthermore, the statistical feature vector generated by segmented aggregation is input into the pre-trained anomaly detection model to perform anomaly detection on each call path in the link topology within the time window, thereby obtaining the path anomaly probability score corresponding to each call path. In this case, anomaly detection is performed at the edge AI computing nodes, which reduces the intermediate links in the data transmission to the central platform, thereby improving the real-time performance of anomaly detection. Moreover, detection based on the statistical feature vector generated by segmented aggregation can accurately locate anomaly points, improve the reliability and practicality of anomaly detection, and can adapt to situations with large amounts of data.

[0085] Then, based on the path anomaly probability score, the sampling strategy for the next time window can be dynamically adjusted, which facilitates adaptive sampling, ensures sampling accuracy and efficiency, avoids sampling unimportant data that occupies valuable network bandwidth and storage resources, and timely adjustment of the sampling strategy can ensure that enough key link data is sampled, thus balancing the relationship between data sampling cost and anomaly detection accuracy.

[0086] In summary, this application utilizes edge AI computing nodes for data acquisition and processing based on a dynamic time window partitioning mechanism. This reduces intermediate steps in data transmission to the central platform, improving the real-time performance of anomaly detection. By processing the received raw link tracing data through segmented aggregation, statistical feature vectors are obtained. Based on a pre-trained anomaly detection model, anomaly detection analysis is performed to obtain a path anomaly probability score. Based on this score, the sampling strategy for the next time window is dynamically adjusted to achieve adaptive sampling, ensuring sampling accuracy and efficiency, and improving the reliability and practicality of anomaly detection.

[0087] See Figure 2 Preferably, in the method described above, the step of segmenting and aggregating the original link tracing data within each time window using a dynamic time window division mechanism to generate a statistical feature vector of the link topology within the time window includes:

[0088] Step S201: In the path time aggregation layer, perform statistical analysis on the response time series of all Spans in the same service call path to obtain a first feature vector. The first feature vector includes multiple preset delay quantiles and the kurtosis statistics of the response time series.

[0089] Step S202: In the error code distribution layer, cluster statistical analysis is performed on the response of the service node based on the HTTP status code to obtain a second feature vector. The second feature vector includes the frequency distribution of at least two error status codes and the obtained error entropy value.

[0090] Step S203: In the topology layer, based on the service dependency matrix constructed according to the call frequency of each node within the time window, a third feature vector is obtained. The third feature vector includes: the maximum eigenvalue, the maximum eigenvector, and the asymmetry of the service dependency matrix.

[0091] Step S204: The first feature vector, the second feature vector, and the third feature vector are merged to obtain the statistical feature vector.

[0092] In this implementation, the step of segmenting and aggregating data to obtain statistical feature vectors is illustrated. At the end of each time window, a data flushing operation is forcibly triggered, immediately performing feature extraction and format normalization on the aggregated data within the current window. The maximum delay of this process is strictly limited to within 5% of the window duration to ensure the real-time performance and robustness of the entire system. To achieve a structured representation of the link's operational status, three-layer segmented aggregation is performed on the original link tracing data within the time window to generate corresponding topological statistical feature vectors.

[0093] The first layer is a path time aggregation layer, which statistically analyzes the response time sequence of all spans in the same service call path and calculates multiple preset latency quantiles to evaluate normal and tail performance. In a specific embodiment, the preset latency quantiles include P50, P90 and P99. At the same time, the kurtosis of the sequence is calculated to characterize the sharpness of the latency distribution. By extracting multiple paths in parallel, a 12-dimensional first feature vector can be formed.

[0094] The second layer is the error code distribution layer, which clusters and statistically analyzes the responses of each service node according to HTTP status codes. For example, it extracts the proportion of 5xx errors to measure the degree of server failure, calculates the growth rate of 4xx errors to reflect the abnormal trend of client requests, and calculates the error entropy value based on the frequency distribution of each status code. The formula for calculating the error entropy value can be expressed as: ,in status code The proportion of the total response, which measures the dispersion and uncertainty of error types, ultimately constitutes the 8-dimensional second feature vector.

[0095] The third layer is the topology layer, which is based on building a service dependency matrix within the window. ,in, The matrix element represents the number of active service nodes within the current time window. Indicates service node For nodes The frequency of calls is determined by calculating the spectral radius. That is, matrix The maximum eigenvalue is used to evaluate the complexity of the call chain, and then its maximum eigenvector is extracted to identify the centrality of high-load paths. Finally, the asymmetry is calculated. The difference between the dependency matrix and its transpose (Frobenius norm) measures whether there is a significant directional bias in service calls. The above three items together constitute a 6-dimensional third feature vector.

[0096] Merging the feature sub-vectors generated by the three-layer aggregation and using them as input to the anomaly detection model in the edge AI computing node helps to accurately identify link risks caused by high latency, abnormal error distribution, or topology imbalance.

[0097] See Figure 3 Specifically, in the method described above, the dynamic time window segmentation mechanism includes:

[0098] Step S301: Construct an initial time window based on the first duration, and adjacent time windows have a first ratio of overlap.

[0099] Step S302: Obtain the current cumulative number of time windows and the traffic fluctuation coefficient;

[0100] Step S303: If the number of time windows and the traffic fluctuation coefficient meet the first preset condition, then obtain the service traffic change rate based on the difference in the number of requests and the time interval between adjacent time windows.

[0101] Step S304: Dynamically adjust the window duration of the next time window based on the flow rate change rate.

[0102] This embodiment illustrates a time window segmentation mechanism, which employs a sliding overlap mechanism. Adjacent time windows maintain a 20% to 30% overlap, effectively mitigating boundary effects caused by cross-window data segmentation and improving the continuity and accuracy of statistical feature vectors. During the initial cold start phase, a fixed first duration (e.g., 60 seconds) is used to construct initial time windows, and the number of time windows is accumulated, along with the accumulated traffic fluctuation coefficient. Only if the accumulated number of time windows and the traffic fluctuation coefficient meet a first preset condition is the aforementioned dynamic adjustment strategy based on traffic changes activated, ensuring that the system intervenes in dynamic response only after a stable observation period. The first preset condition can be that the accumulated number of time windows is greater than 5 and the traffic fluctuation coefficient is greater than 0.3. The dynamic adjustment strategy involves obtaining the business traffic change rate based on the difference in the number of requests and the time interval between adjacent time windows, and dynamically adjusting the window duration of the next time window based on this traffic change rate. The business traffic change rate can be expressed as: ,in This represents the difference in the number of requests between the current window and the previous window. This represents the corresponding time interval.

[0103] In a specific embodiment, preferably, the method described above, wherein dynamically adjusting the window duration of the next time window based on the flow rate change includes:

[0104] If the rate of change of the business traffic is greater than or equal to the first threshold, then the window duration of the next time window will be shortened to the second duration, which is within the range of the first preset duration.

[0105] If the rate of change of the business traffic is less than or equal to the second threshold, the window duration of the next dynamic time window will be extended to a third duration, the third duration being within the range of the second preset duration, and the second threshold being less than the first threshold.

[0106] In this embodiment, the step of dynamically adjusting the window duration of the next time window based on the traffic change rate is illustrated. If the traffic change rate is greater than or equal to a first threshold (e.g., 50% / min), the window duration of the next time window is shortened to a second duration within a first preset duration range (e.g., 10±2 seconds) to improve the response speed to sudden anomalies. If the traffic change rate is less than or equal to a second threshold (e.g., 10% / min), the window duration of the next time window is extended to a third duration within a second preset duration range (e.g., 300±30 seconds) to enhance the data aggregation capability under stable load.

[0107] SeeFigure 4 Preferably, in the method described above, the step of inputting the statistical feature vector into the pre-trained anomaly detection model for analysis to obtain a link anomaly probability score includes:

[0108] Step S401: Input the statistical feature vectors corresponding to the first preset number of consecutive time windows into the time series analysis channel of the anomaly detection model to obtain the response time surge index. The time series analysis channel is a three-layer long short-term memory network architecture.

[0109] Step S402: In the topology analysis channel, attention weighted aggregation is performed based on the edge features of each service node and its neighboring nodes to obtain the topology anomaly score of each call path. The topology analysis channel is a graph attention network, and the edge features include average response latency and error rate.

[0110] Step S403: Based on the response time surge index and the topology anomaly score, perform concatenation and weight fusion to obtain the path anomaly probability score corresponding to each of the calling paths.

[0111] This embodiment illustrates the steps for analyzing the anomaly detection model. The anomaly detection model in the edge AI computing node employs a dual-channel neural network architecture, consisting of a temporal analysis channel and a topology analysis channel, to fuse multi-dimensional features and achieve high-precision link anomaly identification. The temporal analysis channel uses a three-layer Long Short-Term Memory Network (LSTM). The input is a path time aggregation feature vector extracted within a first preset number of time windows (e.g., 10). Its memory gating structure captures the historical dependencies of latency fluctuations, and finally, through linear transformation and a sigmoid activation function, it outputs a response time surge exponent. ,in This represents the hidden state of the LSTM at the current moment. This is the weight matrix. For bias terms, This is the Sigmoid function, which is used to quantify the intensity of latency anomalies in the current link.

[0112] The topology analysis channel employs a Graph Attention Network (GAT), constructing a graph with service nodes as nodes and edges composed of call relationships and their edge features. Edge features include average response latency and error rate. In GAT, each node performs attention-weighted aggregation based on the edge features of its neighboring nodes, outputting a topology anomaly score for the entire graph. ,in For the service dependency matrix, The topological anomaly score, which is the edge feature tensor, is used to capture structural anomalies or load imbalances in the call path.

[0113] The integrated decision-making level will and After concatenation, the data is fed into a fully connected layer to calculate the final path anomaly probability score, which is in the form of... ,in, This indicates the result of feature concatenation. To fuse the weight matrix, To incorporate the bias term, the output is... Mapped to the [0, 1] interval. This dual-channel architecture significantly improves the sensitivity and positioning accuracy of link anomaly detection through collaborative modeling from both temporal and structural perspectives.

[0114] join Figure 5 Optionally, the method described above also includes:

[0115] Step S501: Periodically receive global model parameters issued by the central analysis platform according to the first preset cycle;

[0116] Step S502: Obtain the cosine similarity based on the global model parameters and the local model parameters obtained by the anomaly detection model through local update;

[0117] Step S503: When the cosine similarity is less than the first threshold, update the model parameters of the anomaly detection model based on federated learning.

[0118] In one specific embodiment, to ensure the model maintains adaptability and coordination in a complex and ever-changing link environment, the dual-channel neural network introduces a federated learning mechanism. Specifically, at each first preset interval, for example, 24 hours, the central analysis platform distributes unified global model parameters to each edge AI computing node. The edge AI computing node receives the global model parameters. Then, the local model parameters will be independently updated based on the local observation data. and the global model parameters And calculate the cosine similarity between the two. The cosine similarity Used to assess the degree of model drift. If cosine similarity... If the value is less than a first threshold (e.g., 0.7), it indicates a significant difference between the local model and the global model in feature distribution or parameter gradient, and the federated learning process is immediately triggered to update the model parameters of the anomaly detection model. In one specific embodiment, the triggered federated learning process involves the edge AI computing nodes updating the parameter gradient difference... The local data distribution characteristics extracted within the current window are reported to the central analysis platform, which then returns new aggregation parameters based on a hybrid update strategy. ,in, This indicates that the global model has dominant weights, thus taking into account both system stability and the individual differences of edge environments.

[0119] Furthermore, the method described above also includes:

[0120] If there is a target call path whose path anomaly probability score is greater than the first score threshold, then according to the second preset period, the anomaly detection model is periodically optimized by gradient descent based on the most recent first number of target call paths stored in the sliding window buffer and the preset learning rate.

[0121] In addition to the periodic detection based on the first period, detection will also be performed based on the first scoring threshold, whereby an abnormal path probability will be considered. When a high-confidence anomalous sample with a score greater than the first scoring threshold (e.g., 0.9) is identified as a target call path, an incremental learning mechanism is triggered in real time. This mechanism uses a sliding window buffer to store the most recent number (e.g., 1000) of target call paths and a preset learning rate (e.g., ...). Every second preset period (e.g., 2 hours), a small batch of gradient descent model optimization is performed to maintain the model's rapid adaptability to local anomalies. This strategy achieves a dynamic balance between local independent learning and global collaborative updating of the edge intelligent model, improving the system's long-term stability and real-time diagnostic capabilities in real-world deployment environments.

[0122] Preferably, in the method described above, the step of dynamically adjusting the sampling strategy for the next time window based on the path anomaly probability score includes:

[0123] For critical abnormal paths whose path anomaly probability score is greater than or equal to the second scoring threshold, high-precision sampling is performed.

[0124] For non-critical paths whose path anomaly probability score is less than the second score threshold, data aggregation based on the prefix tree compression algorithm is performed, and the sampling rate is adjusted based on a pre-built control model of resource consumption and sampling accuracy.

[0125] This embodiment illustrates how to adjust the sampling strategy based on path anomaly probability scores. The anomaly probability scores of each path are judged based on a second scoring threshold (e.g., 0.85). If the path anomaly probability score is greater than or equal to the second scoring threshold, the path is marked as a critical anomaly path, and high-precision sampling is performed on it, preferably 100% sampling. If the path anomaly probability score is less than the second scoring threshold, the path is marked as a non-critical path, and data aggregation based on a prefix tree compression algorithm is performed. The data is aggregated, and the sampling rate is adjusted based on a pre-built control model of resource consumption and sampling accuracy to ensure that the sampling rate is compatible with resources and sampling accuracy. This avoids over-sampling of unimportant data, preventing resource waste and effectively balancing the relationship between data acquisition costs and detection accuracy.

[0126] See Figure 6 Specifically, as described above, for non-critical paths with an anomaly probability score less than the second score threshold, data aggregation based on a prefix tree compression algorithm is performed, including:

[0127] Step S601: Convert the non-critical path to obtain a dot-separated string representation;

[0128] Step S602: Construct a multi-branch tree structure based on the depth-first traversal strategy according to the string representation;

[0129] Step S603: Statistical analysis is performed on the occurrence frequency of each subtree in the multi-branch tree structure, and the subtrees that meet the preset merging conditions are determined as candidate subtrees;

[0130] Step S604: The candidate subtree is compressed to obtain compressed aggregated data, wherein the aggregated data is encoded in a structured tag format.

[0131] In this implementation, an example of the data aggregation method based on the prefix tree compression algorithm is provided. The prefix tree compression algorithm is introduced to perform aggregation encoding on service call paths. Specifically, each non-critical path is first converted into a dot-separated string representation; for example, path A calling B and then C is represented as "ABC". Then, a multi-branch tree structure based on a depth-first traversal strategy is constructed using this format to represent the hierarchical relationship of service calls.

[0132] To identify compressible repetitive structures, the frequency of occurrence of each subtree is statistically analyzed. If the preset merging condition ① is that the frequency of occurrence of a subtree in the current time window exceeds 5% of the total requests, and the preset merging condition ② is that the depth of the subtree is greater than or equal to 3 and all leaf nodes have the same identifier, then the preset merging conditions ① and ② are met, and the subtree is selected as a candidate subtree for compression.

[0133] The compressed aggregated data is encoded using structured markup. The root node uses the first 4 bytes of a SHA-1 hash to represent the complete path signature, internal wildcard nodes are placed using "", and leaf nodes record the average response time of the path. Error count The expression is formed as follows: This effectively reduces the bandwidth consumption of redundant structures during data transmission. Through this mechanism, while ensuring link observability, efficient data compression and transmission optimization can be achieved in low-probability anomaly regions, further supporting the incremental focusing capability of differential transmission.

[0134] Furthermore, as described above, the step of compressing the candidate subtree to obtain compressed aggregated data includes:

[0135] If there is a target Span data in the candidate subtree whose time consumption is greater than the preset quantile or contains a preset error status code, then the target Span data is forcibly retained according to the forced retention strategy.

[0136] In this embodiment, in order to retain key diagnostic information, special handling is performed for special cases in the data compression step. For example, a forced retention strategy is set for the original Span data in all compression paths that takes more than a preset quantile (e.g., P99) or contains a preset error status code (e.g., 5xx error status code), so that the sampling rate is increased to 100% and potential hidden dangers are not lost due to compression.

[0137] Preferably, in the method described above, adjusting the sampling rate based on a pre-built control model of resource consumption and sampling accuracy includes:

[0138] When the resource consumption meets the preset conditions, a negative feedback adjustment mechanism is activated to perform a linear decrease adjustment on the sampling rate corresponding to the non-critical path, wherein the sampling rate is greater than or equal to the minimum sampling rate.

[0139] This embodiment illustrates the steps of adjusting the sampling rate based on a control model. By pre-constructing a control model relating resource consumption and sampling accuracy, dynamic control of link data acquisition and system load balancing can be achieved. The pre-constructed control model is as follows: ,in, This represents the resource consumption of an edge AI computing node per unit of time. For non-critical path sampling rate, This refers to the amount of sampled data for non-critical paths. and These are weighted coefficients for sampling overhead and data volume, respectively, used to reflect the impact of sampling rate adjustments on the overall system load in resource-constrained scenarios. In a specific embodiment, the utilization rate of the central processing unit (CPU) of the edge AI computing node is used. This is used to determine resource consumption. When the sampling rate exceeds 80% of the threshold, it indicates excessive resource consumption. At this point, a negative feedback adjustment mechanism is activated, implementing a linear decrease strategy for the sampling rate of non-critical paths. The adjustment range is determined by the formula... Calculate, where, The adjustment coefficient is preferably [value missing]. , This indicates the portion exceeding the utilization rate, expressed as a negative growth amount. Dynamically reduce the data acquisition frequency of low-priority paths to control resource pressure and ensure that critical paths can still maintain high-precision observations.

[0140] To avoid excessive downsampling leading to data unavailability, this embodiment also introduces a lower sampling rate constraint. Specifically, the sampling rate for non-critical paths must not be lower than the minimum sampling rate, which is expressed as: ,in This formula represents the total number of service call paths within the current time window. While ensuring minimum data retention, it automatically lowers the lower limit as the number of paths increases, controlling the overall sample density and thus improving the system's scalability in large-scale service scenarios.

[0141] In summary, by combining local resource awareness, adaptive control, and global coordination when dynamically adjusting the sampling rate, the operational efficiency and data representativeness of the link anomaly detection system under edge computing power constraints can be guaranteed.

[0142] It should also be noted that the metadata generated during each round of sampling decision-making will be synchronized to the central analysis platform as an independent data stream. This metadata includes: path identifiers, sampling rate adjustment records, and the current load status of edge AI computing nodes. It is used to perform sampling consistency correction during subsequent end-to-end data aggregation to ensure the temporal integrity and semantic comparability of the link data.

[0143] See Figure 7 Preferably, the method described above further includes:

[0144] Step S701: Perform upper triangularization on the service dependency matrix within each time window to obtain an upper triangular element matrix, and obtain the maximum eigenvalue of the upper triangular element matrix.

[0145] Step S702: Obtain the first topological hash fingerprint based on the upper triangular element matrix, the maximum eigenvalue, and the cyclic redundancy check function;

[0146] Step S703: Perform consistency verification based on the first topological hash fingerprint and the reference hash fingerprint corresponding to the historical baseline to obtain the verification result;

[0147] Step S704: If the verification result is consistent, then enter the indicator incremental mode, and transmit the indicator difference of each call path in the current time window to the central analysis platform through the difference transmission engine.

[0148] Step S705: If the verification result is inconsistent, enter the structure update mode and transmit the unique identifier list of the newly added path subtree and the deleted path to the central analysis platform through the difference transmission engine.

[0149] After dynamically adjusting the sampling rate, data sampling is performed based on the sampling rate, and the sampled data is transmitted. This embodiment exemplifies this data transmission process. In this embodiment, data transmission is executed through a differential transmission engine, which significantly reduces the transmission bandwidth requirements from the edge to the central platform through topology fingerprint comparison and incremental data packaging strategies, while ensuring timely synchronization of abnormal path information. Specifically, a service dependency matrix is ​​first generated for each time window. Perform upper triangularization and extract its upper triangular element matrix. And calculate the largest eigenvalue of the matrix. By splicing and Then, the first topological hash fingerprint is calculated using a cyclic redundancy check function. This fingerprint is used to quickly determine whether the link topology within the current window is consistent with the historical baseline recorded by the central platform. Specifically, it can be used to perform consistency verification based on the first topology hash fingerprint and the reference hash fingerprint corresponding to the historical baseline to obtain the verification result.

[0150] If the verification result is consistent (hash value matching), indicating that the current topology has not changed, then the system enters the incremental metric mode, transmitting only the metric difference for each path within the current time window via the difference transmission engine. ,in For key performance indicators such as current window response time or error rate, Use the baseline value to correspond to the indicator value, and avoid repeatedly transmitting static structure data.

[0151] If the verification result is inconsistent, i.e. the hash values ​​do not match, indicating that a new or deleted service path has appeared in the link topology, the structure update mode is entered. The newly added path subtree is transmitted using the encoding method generated by the previous prefix tree compression, and a unique identifier list of the deleted path is attached. This allows the central analysis platform to complete the accurate repair of the topology snapshot without receiving the full data.

[0152] It should be noted that all transmitted content is encapsulated using a Type-Length-Value (TLV) structure. The Type field identifies the data type, Length indicates the data length, and Value contains the specific content. Type labels include 0x01 for a full topology update packet, 0x02 for an incremental metric packet, and 0x03 for a path deletion notification packet. This mechanism, through a structure-aware and metric-sensitive collaborative transmission strategy, achieves lightweight data and enhanced anomaly focus in high-frequency link monitoring scenarios, providing timely and sparsity-balanced data support for subsequent full-link performance evolution modeling.

[0153] Optionally, the method described above further includes:

[0154] If a reverse retransmission request is received from the central analysis platform, a complete topology snapshot and compression operation log are reported for data repair. The reverse retransmission request is sent by the central analysis platform when the second topology hash fingerprint obtained by reconstructing the link topology structure based on the received incremental data of the nodes is inconsistent with the first topology hash fingerprint.

[0155] It should also be noted that after the differential transmission is completed, the central analysis platform receives incremental data from the edge AI computing nodes and reconstructs the link topology within the current time window. It then calculates the second topological hash fingerprint by reassembling the upper triangular elements of the service dependency matrix and the largest eigenvalue. ,in This is the upper triangular dependency matrix after incremental reconstruction. Its largest feature value is used to compare whether the topological state identified by the central platform and the edge AI computing nodes is consistent. If With the original record at the edge Inconsistencies indicate potential issues such as encoding errors, packet loss, or context mismatch during incremental transmission. In such cases, the central analysis platform immediately initiates a reverse retransmission request. Upon receiving this request, the edge AI computing nodes immediately report a complete snapshot of the original window topology and compression process logs for data repair. The topology snapshot includes the start and end timestamps of the current window, a list of service node names involved, and a sparse representation of the service dependency matrix, transmitted using Compressed Sparse Row (CSR) format to significantly compress the data volume of the asymmetric sparse matrix. Simultaneously, the edge AI computing nodes synchronously report all log records that trigger prefix tree subtree merging operations within the time window. Each log entry is represented by a structured tuple [timestamp, parent node ID, number of merged child nodes, number of retained samples], facilitating the central analysis platform's complete retrospective analysis of the data compression process, restoration of the semantic structure of the compression path, and comparison of the mapping relationship between the original and compressed paths. This mechanism achieves robust correction of inconsistent transmission results through error detection based on fingerprint matching, state completion based on sparse snapshots, and operation reproduction based on log trajectories, ensuring the integrity and auditability of the link topology evolution process at the data layer.

[0156] See Figure 8 Preferably, the method described above further includes:

[0157] Step S801: If it is detected that the current situation is under a preset extreme resource pressure scenario, the circuit breaker mechanism is activated. The circuit breaker mechanism includes emergency compression mode and / or lossy transmission mode.

[0158] Step S802: If it is detected that the preset extreme resource pressure scenario is exited, the circuit breaker mechanism is exited, and data repair is performed based on historical link observation data of the same time period, and a data repair report is generated.

[0159] In edge AI-driven real-time link anomaly detection, a circuit breaker mechanism is built-in to maintain basic operational capabilities and the continuity of core data pathways in response to extreme resource pressure scenarios. Preset extreme resource pressure scenarios include: memory usage exceeding a third threshold for a continuous preset duration, and network bandwidth usage exceeding a fourth threshold. Examples of one of these preset extreme resource pressure scenarios are provided below.

[0160] When the memory usage of an edge AI computing node exceeds 90% for three consecutive minutes, an emergency compression mode is triggered, forcibly reducing the sampling rate of all non-critical paths to a lower level. ,in, The lower bound strategy is the total number of paths within the current window. It adaptively adjusts the sampling density based on the path size to ensure that representative link samples can still be retained even when node memory is extremely tight. At the same time, it extends the current time window length by 50% based on the original value of the dynamic time window, reducing the trigger frequency of data aggregation and compression, thereby slowing down the growth rate of memory.

[0161] When network bandwidth usage exceeds 80%, a lossy transmission mode is triggered. To control data outbound traffic, the link metric difference is adjusted. The original floating-point type is converted into an 8-bit integer for compression encoding, introducing a numerical error of ±0.5ms within the allowable precision range. At the same time, the error code distribution feature data of all non-leaf nodes are deleted, while the information of the smallest diagnostic unit is retained to ensure the basic identification capability of abnormal paths.

[0162] The aforementioned circuit breaker state is a forced degradation protection phase. Once resource usage is detected to have recovered to below the corresponding normal threshold, the circuit breaker mode is automatically exited. Based on complete link observation data from the same historical time period, compensation and reconstruction operations are performed on the lost portion, filling in gaps in the abnormal trajectory and generating a data repair report. This ensures the recoverability of link evolution analysis and model training in terms of data continuity. Through the coordinated control of resource awareness, adaptive degradation, and abnormal data compensation, a dynamic closed loop is achieved, enabling the system to maintain a resilient response during resource depletion and maintain accuracy in subsequent stages.

[0163] The various methods of the embodiments of this application have been described above. Apparatus for implementing the above methods will now be provided.

[0164] Please refer to Figure 9 This application also provides an apparatus for real-time link anomaly detection and sampling based on edge AI, comprising:

[0165] The first processing module 901 is used to receive the raw link tracing data generated in real time by the service node, and then to segment and aggregate the raw link tracing data in each time window through a dynamic time window division mechanism to generate a statistical feature vector of the link topology in the time window. The raw link tracing data includes: link topology relationships, service call levels and performance indicators.

[0166] The second processing module 902 is used to input the statistical feature vector into the pre-trained anomaly detection model for analysis to obtain a path anomaly probability score.

[0167] The third processing module 903 is used to dynamically adjust the sampling strategy for the next time window based on the path anomaly probability score.

[0168] Preferably, in the apparatus described above, the first processing module includes:

[0169] The first vector acquisition unit is used to perform statistical analysis on the response time series of all Spans in the same service call path in the path time aggregation layer to obtain a first feature sub-vector. The first feature sub-vector includes multiple preset delay quantiles and the kurtosis statistics of the response time series.

[0170] The second vector acquisition unit is used to perform clustering statistical analysis on the response of the service node based on the HTTP status code at the error code distribution layer to obtain a second feature sub-vector. The second feature sub-vector includes the frequency distribution of at least two error status codes and the obtained error entropy value.

[0171] The third vector acquisition unit is used to acquire a third feature vector in the topology layer based on the service dependency matrix constructed based on the call frequency of each node within the time window. The third feature vector includes: the maximum eigenvalue, the maximum eigenvector, and the asymmetry of the service dependency matrix.

[0172] The vector merging unit is used to merge the first feature sub-vector, the second feature sub-vector, and the third feature sub-vector to obtain the statistical feature vector.

[0173] Specifically, in the apparatus described above, the first processing module further includes:

[0174] The first processing unit is configured to construct an initial time window based on a first duration, and adjacent time windows have a first overlap ratio.

[0175] The second processing unit obtains the current cumulative number of time windows and the traffic fluctuation coefficient;

[0176] The third processing unit, if the number of time windows and the traffic fluctuation coefficient meet the first preset condition, obtains the service traffic change rate based on the difference in the number of requests and the time interval between adjacent time windows.

[0177] The fourth processing unit dynamically adjusts the window duration of the next time window based on the flow rate change.

[0178] Preferably, in the apparatus described above, the fourth processing unit includes:

[0179] The first sub-processing unit is used to shorten the window duration of the next time window to a second duration if the service traffic change rate is greater than or equal to a first threshold, wherein the second duration is within a first preset duration range.

[0180] The second sub-processing unit is used to extend the window duration of the next dynamic time window to a third duration if the service traffic change rate is less than or equal to the second threshold. The third duration is within the range of the second preset duration, and the second threshold is less than the first threshold.

[0181] Preferably, in the apparatus described above, the second processing module includes:

[0182] The fifth processing unit is used to input the statistical feature vectors corresponding to the first preset number of consecutive time windows into the time series analysis channel of the anomaly detection model to obtain the response time surge index. The time series analysis channel is a three-layer long short-term memory network architecture.

[0183] The sixth processing unit is used to perform attention-weighted aggregation based on the edge features of each service node and its neighboring nodes in the topology analysis channel to obtain the topology anomaly score of each calling path. The topology analysis channel is a graph attention network, and the edge features include average response latency and error rate.

[0184] The seventh processing unit is used to perform concatenation and weighted fusion based on the response time surge index and the topology anomaly score to obtain the path anomaly probability score corresponding to each of the calling paths.

[0185] Optionally, the apparatus described above further includes:

[0186] The fourth processing module is used to periodically receive global model parameters issued by the central analysis platform according to the first preset period;

[0187] The fifth processing module is used to obtain the cosine similarity based on the global model parameters and the local model parameters obtained by the anomaly detection model through local update.

[0188] The sixth processing module is used to update the model parameters of the anomaly detection model based on federated learning when the cosine similarity is less than the first threshold.

[0189] Furthermore, the device described above also includes:

[0190] The seventh processing module is used to perform gradient descent model optimization on the anomaly detection model according to a second preset period, based on the first number of the most recent target call paths stored in the sliding window buffer and a preset learning rate, if there is a target call path whose path anomaly probability score is greater than the first score threshold.

[0191] Preferably, in the apparatus described above, the third processing module includes:

[0192] The eighth processing unit is used to perform high-precision sampling on the critical abnormal paths whose path anomaly probability scores are greater than or equal to the second scoring threshold.

[0193] The ninth processing unit is used to perform data aggregation based on the prefix tree compression algorithm for non-critical paths whose path anomaly probability score is less than the second score threshold, and adjust the sampling rate based on a pre-built control model of resource consumption and sampling accuracy.

[0194] Specifically, in the apparatus described above, the ninth processing unit includes:

[0195] The third sub-processing unit is used to convert the non-critical path into a dot-separated string representation.

[0196] The fourth sub-processing unit constructs a multi-branch tree structure based on the depth-first traversal strategy according to the string representation;

[0197] The fifth sub-processing unit performs statistical analysis on the frequency of occurrence of each subtree in the multi-branch tree structure, and determines the subtrees that meet the preset merging conditions as candidate subtrees;

[0198] The fifth sub-processing unit compresses the candidate subtree to obtain compressed aggregated data, wherein the aggregated data is encoded using structured markup.

[0199] Furthermore, in the apparatus described above, the sixth sub-processing unit specifically includes:

[0200] If there is a target Span data in the candidate subtree whose time consumption is greater than the preset quantile or contains a preset error status code, then the target Span data is forcibly retained according to the forced retention strategy.

[0201] Preferably, in the apparatus described above, the ninth processing unit specifically includes:

[0202] When the resource consumption meets the preset conditions, a negative feedback adjustment mechanism is activated to linearly decrease the sampling rate corresponding to the non-critical path, wherein the sampling rate is greater than or equal to the minimum sampling rate.

[0203] Preferably, the apparatus as described above further includes:

[0204] The eighth processing module is used to perform upper triangularization on the service dependency matrix within each time window to obtain an upper triangular element matrix, and to obtain the maximum eigenvalue of the upper triangular element matrix.

[0205] The ninth processing module is used to obtain the first topological hash fingerprint based on the upper triangular element matrix, the maximum eigenvalue, and the cyclic redundancy check function.

[0206] The tenth processing module is used to perform consistency verification based on the first topological hash fingerprint and the reference hash fingerprint corresponding to the historical baseline, and obtain the verification result;

[0207] The eleventh processing module is used to enter the indicator increment mode if the verification result is consistent, and transmit the indicator difference of each call path in the current time window to the central analysis platform through the difference transmission engine.

[0208] The twelfth processing module is used to enter the structure update mode if the verification result is inconsistent, and transmit the unique identifier list of the newly added path subtree and the deleted path to the central analysis platform through the difference transmission engine.

[0209] Optionally, the apparatus described above further includes:

[0210] The thirteenth processing module is used to report a complete topology snapshot and compression operation log for data repair if it receives a reverse retransmission request sent by the central analysis platform. The reverse retransmission request is sent by the central analysis platform when the second topology hash fingerprint obtained by reconstructing the link topology structure based on the received incremental data of the nodes is inconsistent with the first topology hash fingerprint.

[0211] Preferably, the apparatus as described above further includes:

[0212] The fourteenth processing module is used to activate the circuit breaker mechanism if a preset extreme resource pressure scenario is detected. The circuit breaker mechanism includes an emergency compression mode and / or a lossy transmission mode.

[0213] The fifteenth processing module is used to exit the circuit breaker mechanism if the preset extreme resource pressure scenario is detected, and to perform data repair based on historical link observation data of the same time period, and generate a data repair report.

[0214] It should be noted that the device in this embodiment corresponds to the aforementioned real-time link anomaly detection and sampling method based on edge AI. The implementation methods in the above embodiments are all applicable to the embodiments of this device and can achieve the same technical effect. The device provided in this application embodiment can implement all the method steps implemented in the above method embodiments and can achieve the same technical effect. Therefore, the parts that are the same as those in the method embodiments and the beneficial effects will not be described in detail here.

[0215] See Figure 10Another embodiment of this application provides an electronic device 100, including: a processor 1001, a memory 1002 and a program stored in the memory 1002 and executable on the processor 1001. When the program is executed by the processor 1001, it implements the steps of the real-time link anomaly detection and sampling method based on edge AI as described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0216] Another embodiment of this application provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the steps of the edge AI-driven real-time link anomaly detection and sampling method described above, achieving the same technical effect. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0217] Another embodiment of this application provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the edge AI-driven real-time link anomaly detection and sampling method described above, and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0218] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0219] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0220] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A real-time link anomaly detection and sampling method based on edge AI, characterized in that, include: After receiving the raw link tracing data generated in real time by the service node, the raw link tracing data in each time window is segmented and aggregated through a dynamic time window division mechanism to generate a statistical feature vector of the link topology within the time window. The raw link tracing data includes: link topology relationships, service call levels, and performance indicators. The statistical feature vector is input into the pre-trained anomaly detection model for analysis to obtain a path anomaly probability score. Based on the path anomaly probability score, the sampling strategy for the next time window is dynamically adjusted.

2. The method according to claim 1, characterized in that, The method of segmenting and aggregating the original link tracing data within each time window using a dynamic time window division mechanism to generate a statistical feature vector of the link topology within the time window includes: In the path time aggregation layer, the response time series of all Spans in the same service call path are statistically analyzed to obtain the first feature vector. The first feature vector includes multiple preset delay quantiles and the kurtosis statistics of the response time series. In the error code distribution layer, clustering statistical analysis is performed on the response of the service node based on the HTTP status code to obtain a second feature vector. The second feature vector includes the frequency distribution of at least two error status codes and the obtained error entropy value. In the topology layer, a third feature vector is obtained based on the service dependency matrix constructed based on the call frequency of each node within the time window. The third feature vector includes: the maximum eigenvalue, the maximum eigenvector, and the asymmetry of the service dependency matrix. The first feature vector, the second feature vector, and the third feature vector are merged to obtain the statistical feature vector.

3. The method according to claim 1 or 2, characterized in that, The dynamic time window division mechanism includes: An initial time window is constructed based on a first duration, and adjacent time windows have a first ratio of overlap. Get the current cumulative number of time windows and the traffic fluctuation coefficient; If the number of time windows and the traffic fluctuation coefficient meet the first preset condition, then the service traffic change rate is obtained based on the difference in the number of requests and the time interval between adjacent time windows. The window duration of the next time window is dynamically adjusted based on the flow rate change.

4. The method according to claim 3, characterized in that, The step of dynamically adjusting the window duration of the next time window based on the flow rate change includes: If the rate of change of the business traffic is greater than or equal to the first threshold, then the window duration of the next time window will be shortened to the second duration, which is within the range of the first preset duration. If the rate of change of the business traffic is less than or equal to the second threshold, the window duration of the next dynamic time window will be extended to a third duration, the third duration being within the range of the second preset duration, and the second threshold being less than the first threshold.

5. The method according to claim 1, characterized in that, The step of inputting the statistical feature vector into the pre-trained anomaly detection model for analysis to obtain a link anomaly probability score includes: The statistical feature vectors corresponding to the first preset number of consecutive time windows are input into the time series analysis channel of the anomaly detection model to obtain the response time surge index. The time series analysis channel is a three-layer long short-term memory network architecture. In the topology analysis channel, attention-weighted aggregation is performed based on the edge features of each service node and its neighboring nodes to obtain the topology anomaly score of each call path. The topology analysis channel is a graph attention network, and the edge features include average response latency and error rate. Based on the response time surge index and the topology anomaly score, the results are concatenated and weighted to obtain the path anomaly probability score corresponding to each of the calling paths.

6. The method according to claim 5, characterized in that, Also includes: According to the first preset cycle, it periodically receives global model parameters issued by the central analysis platform; The cosine similarity is obtained based on the global model parameters and the local model parameters obtained by the anomaly detection model through local updates. When the cosine similarity is less than the first threshold, the model parameters of the anomaly detection model are updated based on federated learning.

7. The method according to claim 6, characterized in that, Also includes: If there is a target call path whose path anomaly probability score is greater than the first score threshold, then according to the second preset period, the anomaly detection model is periodically optimized by gradient descent based on the most recent first number of target call paths stored in the sliding window buffer and the preset learning rate.

8. The method according to claim 1, characterized in that, The step of dynamically adjusting the sampling strategy for the next time window based on the path anomaly probability score includes: For critical abnormal paths whose path anomaly probability scores are greater than or equal to the second scoring threshold, high-precision sampling is performed; For non-critical paths whose path anomaly probability score is less than the second score threshold, data aggregation based on the prefix tree compression algorithm is performed, and the sampling rate is adjusted based on a pre-built control model of resource consumption and sampling accuracy.

9. The method according to claim 8, characterized in that, For non-critical paths with an anomaly probability score less than the second score threshold, data aggregation based on a prefix tree compression algorithm is performed, including: The non-critical paths are converted to a dot-separated string representation. Construct a multi-branch tree structure based on a depth-first traversal strategy according to the string representation; The frequency of occurrence of each subtree in the multi-branch tree structure is statistically analyzed, and the subtrees that meet the preset merging conditions are determined as candidate subtrees; The candidate subtrees are compressed to obtain compressed aggregated data, wherein the aggregated data is encoded using structured markup.

10. The method according to claim 9, characterized in that, The compression process of the candidate subtrees to obtain compressed aggregated data includes: If there is a target Span data in the candidate subtree whose time consumption is greater than the preset quantile or contains a preset error status code, then the target Span data is forcibly retained according to the forced retention strategy.

11. The method according to claim 8, characterized in that, The control model based on pre-built resource consumption and sampling accuracy adjusts the sampling rate, including: When the resource consumption meets the preset conditions, a negative feedback adjustment mechanism is activated to perform a linear decrease adjustment on the sampling rate corresponding to the non-critical path, wherein the sampling rate is greater than or equal to the minimum sampling rate.

12. The method according to claim 1, characterized in that, Also includes: The service dependency matrix within each time window is processed into an upper triangular matrix to obtain an upper triangular element matrix, and the maximum eigenvalue of the upper triangular element matrix is ​​obtained. The first topological hash fingerprint is obtained based on the upper triangular element matrix, the maximum eigenvalue, and the cyclic redundancy check function; A consistency check is performed based on the first topological hash fingerprint and the reference hash fingerprint corresponding to the historical baseline to obtain the check result; If the verification results are consistent, then the indicator increment mode is entered, and the indicator difference of each call path in the current time window is transmitted to the central analysis platform through the difference transmission engine. If the verification result is inconsistent, the system enters the structure update mode and transmits the unique identifier list of the newly added path subtree and the deleted path to the central analysis platform through the difference transmission engine.

13. The method according to claim 12, characterized in that, Also includes: If a reverse retransmission request is received from the central analysis platform, a complete topology snapshot and compression operation log are reported for data repair. The reverse retransmission request is sent by the central analysis platform when the second topology hash fingerprint obtained by reconstructing the link topology structure based on the received incremental data of the nodes is inconsistent with the first topology hash fingerprint.

14. The method according to claim 1, characterized in that, Also includes: If a preset extreme resource pressure scenario is detected, a circuit breaker mechanism is activated, which includes an emergency compression mode and / or a lossy transmission mode. If the preset extreme resource pressure scenario is detected to be exited, the circuit breaker mechanism will be exited, and data repair will be performed based on historical link observation data of the same time period, and a data repair report will be generated.

15. A device for real-time link anomaly detection and sampling based on edge AI, characterized in that, include: The first processing module is used to receive the raw link tracing data generated in real time by the service node, and then use a dynamic time window division mechanism to segment and aggregate the raw link tracing data in each time window to generate a statistical feature vector of the link topology in the time window. The raw link tracing data includes: link topology relationships, service call levels and performance indicators. The second processing module is used to input the statistical feature vector into the pre-trained anomaly detection model for analysis, and obtain a path anomaly probability score. The third processing module is used to dynamically adjust the sampling strategy for the next time window based on the path anomaly probability score.

16. A terminal, characterized in that, include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the edge AI-driven real-time link anomaly detection and sampling method as described in any one of claims 1 to 14.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the edge AI-driven real-time link anomaly detection and sampling method as described in any one of claims 1 to 14.

18. A computer program product, characterized in that, It includes computer instructions that, when executed by a processor, implement the steps of the edge AI-driven real-time link anomaly detection and sampling method as described in any one of claims 1 to 14.

Images