Edge computing network observability monitoring method

Abstract

The present invention discloses an edge computing network observable monitoring method, comprising: collecting the operating status of multiple eBPF probes on edge devices, analyzing their access to the same kernel mount point, identifying monitoring conflicts and generating a probe conflict mapping table; based on the conflict relationship, using an improved distributed mutual exclusion algorithm combined with a data fingerprint mechanism to coordinate sampling permissions between probes, avoid repeated collection, and generate a probe sampling scheduling sequence; predicting future key events based on historical event patterns, dynamically adjusting the timing interleaving strategy of each probe, and forming an optimized time slice allocation table; each probe performs collaborative sparse sampling based on the time slice allocation table and the data fingerprint library, and reconstructs complete monitoring information from the sparse data based on compressed sensing theory to achieve fault diagnosis. The present invention can significantly reduce the system overhead of multi-probe monitoring and achieve low-overhead, high-precision network observability.

Description

Technical Field

[0001] The present invention relates to network technology, and in particular to an observable monitoring method for edge computing networks based on collaborative conflict resolution. Background Art

[0002] With the rapid development of 5G, the Internet of Things (IoT), and artificial intelligence (AI) technologies, computing models are evolving from traditional centralized cloud computing to edge computing. Edge computing pushes computing and data storage to the edge of the network, providing services closer to the source of data storage. This offers key advantages such as low latency, high bandwidth, and data privacy protection. However, edge nodes often have limited resources, heterogeneous environments, and dynamically changing topologies, which poses significant challenges in ensuring the stability, performance, and security of the services running on them. Establishing an efficient, accurate, and low-overhead observability and monitoring system for edge computing networks is crucial for promptly detecting and diagnosing anomalies such as network congestion, equipment failures, and security attacks, and for ensuring the quality of service (QoS) of edge services. It is a core supporting technology for promoting the large-scale implementation and reliable application of edge computing technology, and holds significant research significance and application value.

[0003] Currently, various approaches to monitoring computing systems have been developed. Traditional network monitoring relies heavily on the Simple Network Management Protocol (SNMP) to poll basic device metrics, or employ technologies like NetFlow and sFlow to generate traffic statistics. These methods provide a high-level view of the network. At the system level, observability is typically achieved through the collection and analysis of system logs (Logging), aggregated metrics (Metrics), and distributed tracing. The recent maturity of eBPF (Extended Berkeley Packet Filter) technology, with its kernel-mode programmability and ability to securely and efficiently capture system events without modifying kernel source code, has made it an ideal tool for achieving fine-grained system observability. Existing eBPF monitoring solutions are typically designed for specific purposes. For example, by deploying eBPF probes on kprobes or tracepoint mount points, they monitor specific kernel function calls, network packet transmission and reception, disk I / O, and other activities, reporting the collected data to user-mode for processing and analysis. These technologies provide unprecedented insights into the internal operating state of a system.

[0004] However, existing monitoring technologies still face some problems when applied to edge computing scenarios with limited resources and complex environments. For example, they allow probe conflicts to worsen, or adopt simple static priority strategies, which cannot perform fair and efficient dynamic arbitration at runtime, and thus cannot fundamentally solve the performance and stability problems caused by conflicts. Summary of the Invention

[0005] The purpose of the invention is to provide an edge computing network observable monitoring method, in order to solve at least one problem existing in the prior art.

[0006] Technical solution: an edge computing network observable monitoring method, including:

[0007] Collect the running status data of multiple eBPF probes on edge devices, identify conflicts in monitoring the same kernel event, and generate a probe conflict mapping table;

[0008] Read real-time event requests, coordinate sampling permissions between probes based on the probe conflict mapping table, and generate probe sampling scheduling sequences and data fingerprint libraries;

[0009] Predict key event distribution based on historical event patterns and probe sampling scheduling sequences, adjust time allocation, and form a time slice allocation table;

[0010] Perform distributed sampling on the kernel event stream according to the time slice allocation table, query the data fingerprint library and adjust the sampling strategy accordingly to generate a sampling data set;

[0011] Generate complete monitoring data based on the sampled data set and perform pattern matching and fault location based on it.

[0012] According to one aspect of the present application, collecting running status data of multiple eBPF probes on an edge device, identifying monitoring conflicts for the same kernel event, and generating a probe conflict mapping table include:

[0013] Build a probe registry;

[0014] For each active probe in the probe registry, read the real-time performance indicators of each probe from the shared memory and analyze the monitoring status of multiple probes on the same kernel event;

[0015] When multiple probes are detected mounted on the same mount point, the resource contention level is calculated and a probe conflict mapping table including conflicting probe pairs, conflict event types, and contention intensity is generated.

[0016] Preferably, generating the probe conflict mapping table includes:

[0017] Construct a probe conflict matrix to record the historical conflict frequency and resource competition intensity of each pair of probes;

[0018] The conflict similarity between probes is calculated based on the probe conflict matrix, and probes with similar conflict patterns and low conflict frequencies are clustered into probe groups;

[0019] A group ID is assigned to each probe group, and a group probe conflict mapping table including the probe ID, the group ID to which it belongs, and a list of group members is generated.

[0020] Preferably, when updating the probe conflict mapping table, the future resource contention level is predicted, specifically:

[0021] Probe monitoring data of continuous sampling periods is extracted from the real-time performance indicators read from the shared memory to construct a multi-dimensional load feature vector sequence.

[0022] The first-order difference and second-order difference of the multi-dimensional load characteristic vector sequence are calculated to obtain the load change acceleration.

[0023] The current resource contention level, first-order difference, and load change acceleration of the probe are integrated to generate the predicted contention intensity.

[0024] The probe conflict map is updated using the predicted contention intensity.

[0025] Preferably, the process of generating the predicted competition intensity further includes introducing a pattern recognition mechanism, specifically:

[0026] Analyze the autocorrelation and variance of multi-dimensional load feature vector sequences to identify periodic or bursty load patterns.

[0027] Based on the identified load pattern, the corresponding prediction model is dynamically selected or the weight is adjusted.

[0028] Use the adapted prediction model or weights to generate predicted competition intensity.

[0029] Preferably, the process of generating the predicted competition intensity further includes:

[0030] The deviation between the predicted competition intensity and the subsequent actual observed load value is continuously monitored to form a prediction residual series.

[0031] The time series distribution characteristics of the prediction residual sequence are analyzed to distinguish isolated pulse deviations from continuous step deviations, and the load dynamic mode is determined accordingly.

[0032] According to the determined load dynamic mode, the process noise covariance matrix inside the predictor is dynamically adjusted.

[0033] When it is determined to be a step deviation, the value of the process noise covariance matrix is ​​increased.

[0034] According to one aspect of the present application, sampling authority coordination between probes is performed in conjunction with the probe conflict mapping table, including the determination of the competition relationship between probes, specifically:

[0035] Monitor the sampling requests in the real-time event request queue and determine whether the probe initiating the request has resource competition based on the probe conflict mapping table.

[0036] For competing probes, the hash value of the event to be collected is first calculated to check whether other probes have collected the same data. If so, the data is reused directly; otherwise, it is collected again.

[0037] Preferably, the sampling authority coordination between probes is performed in combination with the probe conflict mapping table, including:

[0038] According to the group identifier in the group probe conflict mapping table, local ticket competition is performed within the probe group to generate the best probe in the group as the group representative;

[0039] Representatives from each group calculate the overall priority based on the monitoring event type, waiting time, and historical contribution;

[0040] Inter-group arbitration is conducted by comparing the comprehensive priorities of representatives from each group, determining the final sampling executor and granting sampling authority;

[0041] After the sampling is completed, the permission holder queries the waiting queue and directly transfers the permission to the next highest priority waiter, updating the probe sampling scheduling sequence.

[0042] The calculation of comprehensive priority includes:

[0043] Extract the event type from the real-time event request and query the preset event importance score table to obtain the basic priority score;

[0044] Read the waiting time of the group representative. When it exceeds the dynamic threshold, its timeliness priority is increased according to the preset function.

[0045] The resource efficiency factor is calculated by comparing the historical effective data output and resource consumption of the statistical group representatives;

[0046] The basic priority score, timeliness priority, and resource efficiency factor are weighted and integrated to generate a comprehensive priority for inter-group arbitration.

[0047] Passing permissions directly to the next highest priority waiter includes:

[0048] The probe that has completed sampling traverses the waiting queue and selects the next executor based on the comprehensive priority;

[0049] Write the sampling permission token directly into the permission status bit of the selected probe, bypassing the re-contention process;

[0050] Send a wake-up signal to the selected probe through the event notification descriptor to activate it from the dormant state;

[0051] The awakened probe reads the permission status bit to confirm that it has obtained permission and then performs sampling immediately without waiting.

[0052] According to one aspect of the present application, the process of re-collecting and generating a probe sampling scheduling sequence and a data fingerprint library is specifically as follows:

[0053] Probes in the same conflict group obtain sampling permissions through a distributed mutual exclusion algorithm.

[0054] The probes that have obtained permission perform data collection and generate a probe sampling scheduling sequence based on the sampling order;

[0055] After the collection is completed, the hash value and timestamp of the event data are stored in the shared storage to form a data fingerprint library.

[0056] According to one aspect of the present application, obtaining sampling rights through a distributed mutual exclusion algorithm includes:

[0057] When the probe receives a sampling request, it sets its own initial selection array position;

[0058] Traverse the ticket number arrays of all other probes in the conflict group, obtain the current ticket number of each probe, find the maximum value and add 1 as your own ticket number, and update it to your own selection array position;

[0059] Two levels of checks are performed on each conflicting probe: spin-wait to ensure that other probes have completed ticket number selection, compare ticket numbers, and continue waiting if other probes have smaller ticket numbers or the ticket numbers are the same but the probe ID is smaller, until the highest priority sampling permission is obtained.

[0060] According to one aspect of the present application, the hash value and timestamp of the event data are stored in a shared storage, an expiration date is set, and the data fingerprint library is cleaned up regularly to form the data fingerprint library, including:

[0061] Perform a two-level hash calculation on the event data. Use the first hash algorithm to calculate a 64-bit hash value for the event type, timestamp, and key parameters. Combine this hash value with the probe identifier and use the second hash algorithm to calculate a 32-bit fingerprint to obtain the event data fingerprint.

[0062] The event data fingerprint is stored in a hash table structure to form a fingerprint storage index and is added, deleted, and maintained according to the activity to obtain a data fingerprint library.

[0063] Preferably, the process of forming the data fingerprint library also includes establishing a hierarchical adaptive verification mechanism, specifically:

[0064] Build a hierarchical verification cost model that includes various verification costs and depths.

[0065] Collect historically verified error rate data from the data fingerprint library.

[0066] Based on the error rate data and the hierarchical verification cost model, a dynamic verification strategy configuration is generated. This configuration determines the execution ratio of each verification level in subsequent verification operations.

[0067] Execute dynamic verification policy configuration within the preset total system resource budget.

[0068] Preferably, the layered adaptive verification mechanism also includes an error source location method, specifically:

[0069] The probe source distribution of error events in the statistical fingerprint library is collected, and probes with abnormal error rates are identified as high-risk probes.

[0070] Allocate differentiated focused verification resources to high-risk probes to adjust dynamic verification strategy configuration.

[0071] An error propagation model is constructed to temporarily enhance the verification strength of neighboring probe events when a probe fails within a specific time window.

[0072] According to one aspect of the present application, predicting the distribution of key events based on historical event patterns and probe sampling scheduling sequences, adjusting time allocation, and forming a time slice allocation table include:

[0073] Analyze the historical sampling patterns of each probe based on the probe sampling scheduling sequence, extract the event interval distribution from the historical event pattern data, and construct the event time series features;

[0074] The prediction model is trained using event time series features, and the state transition probability is updated in real time. When a burst pattern is detected, high-frequency events in the future time window are predicted to obtain a key event prediction set.

[0075] The monitoring window is divided into multiple time slices. Under normal circumstances, staggered time slices are allocated to probes. Time periods requiring intensive sampling are identified based on the key event prediction set. The time slices of related probes in this time period are extended to generate a time slice allocation table.

[0076] Preferably, the process of adjusting time allocation to form a time slice allocation table is furthered by a multi-resource coordinated adjustment mechanism, specifically:

[0077] Analyze historical resource usage data to build a resource dependency matrix across multiple system resources. Combine historical event patterns with probe sampling scheduling sequences to generate a multi-resource demand forecast vector covering multiple resource types.

[0078] By integrating the multi-resource demand forecast vectors and the resource dependency matrix, a collaborative adjustment execution plan is calculated and generated. This collaborative adjustment execution plan replaces the time slice allocation table and is used to guide subsequent collaborative sparse sampling.

[0079] Preferably, the multi-resource coordinated adjustment mechanism also includes an adjustment fallback function, specifically:

[0080] After executing the co-tuning execution plan, continuously monitor key performance indicators.

[0081] Establish and maintain a reconciliation decision snapshot cache to store the most recent reconciliation decisions.

[0082] When a key performance indicator deteriorates and exceeds a predetermined threshold, the system is restored to a previous stable state using the adjustment decision snapshot cache.

[0083] Preferably, the multi-resource collaborative adjustment mechanism also has the ability of strategy self-evolution, specifically:

[0084] Collect and evaluate performance feedback data from historical reconciliation operations to form a reconciliation risk assessment model.

[0085] Based on the regulation risk assessment model, the core parameters used to generate the collaborative regulation execution plan are continuously optimized.

[0086] Introducing strategy version management, versioning optimized core parameters, and supporting parallel testing and comparison of new and old strategies.

[0087] According to one aspect of the present application, obtaining a key event prediction set includes:

[0088] Read the event interval data in the event time series feature, map it to a predefined quantization interval through logarithmic transformation, and generate a quantized event interval sequence;

[0089] Based on the quantized event interval sequence, the state transition matrix of the Markov chain is updated and a new state transition matrix is ​​calculated;

[0090] Monitor the number of consecutive occurrences of the same interval in the quantified event interval sequence. When the same interval appears for a preset number of consecutive times, a burst detection is triggered, and the burst intensity is calculated and marked as a burst event.

[0091] Combining the new state transition matrix and emergency event markers, the probability distribution of each event type in the future time window is calculated to improve the prediction probability of emergencies and generate a key event prediction set.

[0092] According to one aspect of the present application, a time slice allocation table for each probe is calculated based on a key event prediction set, including:

[0093] Divide the monitoring window into a fixed number of basic time slices based on the number of active probes, and generate an initial bit mask with uniformly staggered time slices for each probe;

[0094] When the key event prediction set indicates that a high-priority event exists in a certain time window, the probes responsible for monitoring the event are identified, the mask bits of these probes within the prediction window are expanded, and whether there is a conflict is detected.

[0095] If there is a conflict, priority arbitration is performed based on the event prediction confidence level to ensure the reasonable allocation of each time slice and output the time slice allocation table.

[0096] According to one aspect of the present application, generating a sample data set includes:

[0097] Each probe activates monitoring of the kernel event stream within the specified time slice according to the time window allocated in the time slice allocation table, and captures the corresponding kernel event data;

[0098] When the predicted key event actually occurs, the relevant probe activates the compensation sampling mode, ignores the time slice limitation and performs dense sampling to capture the complete key event;

[0099] The kernel event data and compensation sampling data collected by each probe in the allocated time slice are summarized, and the ratio of the actual sampling points to the theoretical maximum sampling points is calculated to form a sampling data set.

[0100] According to one aspect of the present application, the process of coordinating sampling permissions through a distributed mutual exclusion algorithm is also regulated by an emergency status flag and includes:

[0101] When any eBPF probe matches a predefined high-risk event signature, the probe sets the emergency flag.

[0102] All other probes waiting for sampling permission will immediately give up their current permission request after detecting that the emergency flag is set;

[0103] The probe with the emergency flag set will bypass all scheduling and coordination and obtain immediate sampling authority;

[0104] After the probe completes sampling, it clears the emergency status flag to resume the normal scheduling coordination process.

[0105] The beneficial effect is that it can significantly reduce the system overhead of multi-probe monitoring and achieve low-overhead, high-precision network observability. BRIEF DESCRIPTION OF THE DRAWINGS

[0106] Figure 1 It is a flow chart of the present invention.

[0107] Figure 2 This is a flow chart of the present invention for generating a probe conflict mapping table.

[0108] Figure 3 This is a flow chart of the probe competition relationship judgment of the present invention.

[0109] Figure 4 This is a flow chart of the present invention for generating a probe sampling scheduling sequence and a data fingerprint library. DETAILED DESCRIPTION

[0110] like Figure 1 As shown, existing solutions generally lack effective mechanisms for managing runtime conflicts when multiple eBPF probes coexist. In real edge environments, multiple monitoring or security applications from different businesses and vendors may be deployed simultaneously. These applications may create multiple eBPF probes and attach them to the same critical kernel events (such as tcp_sendmsg and block_rq_issue). This uncoordinated concurrent monitoring can trigger intense CPU resource contention and redundant data collection, leading to a dramatic increase in system overhead, severe performance jitter, and even system instability in extreme cases. Existing technologies either ignore these probe conflicts and allow them to fester, or employ simple static priority policies that fail to provide fair and efficient dynamic arbitration at runtime, thus failing to fundamentally resolve the performance and stability issues caused by conflicts. Second, existing technologies struggle to strike an effective balance between comprehensive monitoring coverage and system overhead. To capture the sudden performance anomalies or security attacks common in edge networks, monitoring systems require high-resolution sampling capabilities. However, continuous high-frequency sampling can incur unacceptable performance overhead on resource-constrained edge nodes. Conversely, adopting a low-frequency static sampling strategy will almost inevitably miss a large number of fleeting critical events, resulting in the failure of fault diagnosis. Existing methods lack an intelligent mechanism that can predict system load and event patterns and adaptively adjust the sampling strategy accordingly.

[0111] According to one aspect of the present application, an edge computing network observable monitoring method based on collaborative conflict resolution is provided.

[0112] S1. Collect the running status data and kernel event flow information of multiple eBPF probe instances on the edge device in real time. Build a probe conflict mapping table by analyzing the access frequency and resource usage of each probe to the same kernel mount point. At the same time, calculate resource competition indicators such as CPU occupancy and memory usage to provide a decision-making basis for subsequent conflict resolution.

[0113] S11. Traverse all loaded eBPF probe instances in the edge device kernel, extract the mount point information, probe type and priority configuration of each probe by reading / sys / kernel / debug / tracing / kprobe_events and the file descriptor returned by the bpf system call, and build a probe registry.

[0114] S12. For each active probe in the probe registry, read its real-time performance indicators through the shared memory of the BPF_MAP_TYPE_PERCPU_ARRAY type, including the number of triggers per second, single execution time, CPU cycle consumption and other data to form a probe performance indicator set.

[0115] S13. Analyze the monitoring status of multiple probes on the same kernel event (such as block_rq_issue, tcp_sendmsg, etc.) in the probe performance indicator set. When two or more probes are detected mounted on the same tracepoint or kprobe, calculate the degree of resource competition and generate a probe conflict mapping table containing conflicting probe pairs, conflicting event types and competition intensity.

[0116] For example, if multiple probes monitor tcp_sendmsg events at the same time, it is considered that there is a monitoring conflict. In this case, it is necessary to calculate the degree of resource competition between them. For example, two probes P i and P j Competition intensity C on the same mount point ij It can be calculated by the following formula:

[0117] C ij =w t *{(N i +N j ) / N total}+w c *{T i *N i +T j *N j ) / ∑ n k=1 (T k *N k )};

[0118] C ij Probe P i and P j The intensity of competition between i , N j Probe P i , P j The number of triggers per unit time. N total It is the total number of times the mount point is triggered per unit time. i , T j Probe P i , P j The average execution time of a single execution. n is the total number of active probes in the system. t , w c It is a configurable weight factor, which represents the degree of attention paid to the two dimensions of trigger frequency and execution time. For example, both can be set to 0.5.

[0119] All identified conflict relationships and their quantified competition strengths are summarized to generate a probe conflict mapping table. The data structure of this table can be a set of {(ProbeID_i, ProbeID_j), EventType, CompetitionStrength}, which clearly records which probe pairs, on which types of events, and the degree of competition.

[0120] S14. Collect system-level resource usage data, including CPU usage in / proc / stat, memory usage in / proc / meminfo, and hardware performance counter data obtained through perf_event_open. Combined with the probe performance indicator set, calculate the resource usage ratio of each probe and output the resource contention indicator.

[0121] S2. Based on the conflict relationships identified in the probe conflict mapping table and the real-time event request queue, the improved distributed Bakery algorithm is used to coordinate the sampling permissions between probes. By introducing a data fingerprint mechanism, multiple probes are prevented from repeatedly collecting the same event. A probe sampling scheduling sequence is generated and a data fingerprint library is maintained to ensure that each probe works efficiently and collaboratively.

[0122] Through the distributed mutual exclusion and collaboration mechanism, it is ensured that at any time, for a conflict event, only one probe can obtain the final sampling permission, and redundant collection is avoided through the data multiplexing mechanism.

[0123] S21. Read the probe conflict mapping table to identify the probe groups currently competing for resources, initialize the distributed coordination data structure for each group of conflicting probes, including the atomic ticket number array, the selection flag array, and the data hash storage area, and establish a conflict coordination context.

[0124] S22. Listen to the sampling requests in the real-time event request queue. When the probe needs to collect data, it first calculates the hash value of the event to be collected, traverses the data fingerprint library to check whether other probes have collected the same data. If so, it reuses the data directly to avoid repeated collection and updates the data reuse statistics.

[0125] S23. For events that require new collection, the improved Bakery algorithm is executed to obtain sampling authority: the probe first sets the selection flag, reads the current ticket numbers of all conflicting probes, selects the maximum value plus 1 as its own ticket number, and then clears the selection flag; determines the sampling order by comparing the ticket number size and the probe ID, waits for sampling authority to be obtained, and then performs data collection to generate a probe sampling scheduling sequence.

[0126] S24. The probe that successfully collects data stores the hash value and timestamp of the event data in the shared data fingerprint library, sets the fingerprint validity period to 100 milliseconds, and automatically clears it after the timeout to ensure that the fingerprint library does not grow indefinitely and provide data reuse opportunities for other probes.

[0127] This process transforms the disordered resource competition when multiple eBPF probes concurrently access the same kernel mount point into an orderly, fair, and efficient cooperative scheduling process.

[0128] S3. Combining the probe sampling scheduling sequence and historical event pattern data, a lightweight Markov chain model is used to predict the distribution of key events in the future time window. Based on this, the probe timing interleaving strategy is dynamically adjusted to output an optimized time slice allocation table and key event prediction set to achieve a balance between sampling coverage and system overhead.

[0129] S31. Analyze the historical sampling pattern of each probe based on the probe sampling scheduling sequence, extract the event interval distribution within the last 1000 sampling cycles from the historical event pattern data, construct an event interval histogram with 16 logarithmic intervals as buckets, and form event timing features.

[0130] S32. Use event timing features to train a lightweight Markov chain model, use the exponential moving average algorithm to update the state transition probability in real time, trigger burst mode detection when three consecutive event intervals fall into the same bucket, predict high-frequency events that may occur in the next 10 milliseconds, and output a key event prediction set.

[0131] S33. Dynamically calculate the time slice allocation strategy for each probe based on the resource competition index and the key event prediction set: the basic allocation uses a 32-bit mask to represent the time slice within a 10-millisecond window (each bit represents approximately 0.3 milliseconds). Under normal circumstances, the probes obtain staggered time slices. When a key event is predicted, the time slices of the relevant probes are temporarily extended to generate an optimized time slice allocation table.

[0132] S34. Set the trigger threshold and duration of the compensation mechanism. When the event confidence in the key event prediction set exceeds 0.8, the relevant probes are allowed to break through the timing interleaving limit and perform full sampling for 100 microseconds, and then automatically return to the normal time slice allocation mode to ensure that the system will not be in a high-load state for a long time.

[0133] Based on these predictions, the system dynamically adjusts each probe's time-slice allocation table, intelligently shifting valuable sampling resources (i.e., CPU time slices) from periods of stable network behavior to peak periods of impending critical events (such as network congestion and application anomalies). Compensatory sampling mechanisms are also triggered to ensure data integrity. This minimizes system overhead for ongoing daily monitoring without sacrificing the ability to capture key fault characteristics.

[0134] S4. Perform distributed sampling on the kernel event stream based on the optimized time slice allocation table. Each probe performs collaborative data collection according to the allocated time slice and predicted key events, while referring to the data fingerprint library to avoid redundant sampling, and finally form a sparse sampling data set and its corresponding sampling coverage matrix.

[0135] S41. Each probe activates monitoring of the kernel event stream within the specified time slice according to the time window allocated in the optimized time slice allocation table, and captures the corresponding kernel event data, including timestamp, event type, key parameters and other information, through the eBPF BPF_PROG_TYPE_TRACEPOINT program.

[0136] S42. During the sampling process, the probe first queries the data fingerprint library to verify whether the event to be collected has been processed by other probes. If a matching fingerprint is found, the actual collection step is skipped, and the existing data is directly referenced and the reference count is updated, reducing the repeated kernel-mode-user mode data copy overhead.

[0137] S43. When a predicted event in the key event prediction set actually occurs, the relevant probe immediately activates the compensation sampling mode, temporarily ignores the time slice limit and performs intensive sampling to ensure the complete capture of the key event. The collected supplementary data is marked with a special flag to distinguish it from regular sampling data.

[0138] S44. Summarize the event data collected by each probe within the allocated time slice, calculate the ratio of the actual number of sampling points in each time window to the theoretical maximum number of sampling points, construct a sampling coverage matrix that reflects the data integrity, and integrate all the collected raw data into a sparse sampling data set.

[0139] S5. The sparse sampling dataset and sampling coverage matrix are input into the data reconstruction module based on compressed sensing theory. Combined with the pre-calculated sparse basis, the complete monitoring information is restored from the partial observations through the improved orthogonal matching pursuit algorithm to generate the reconstructed complete monitoring data and its reconstruction confidence score.

[0140] S51. Load precomputed sparse basis matrices (64×256 dimensions) and measurement matrices (32×256 dimensions) optimized for kernel event characteristics. These matrices are obtained through offline training based on a large amount of historical monitoring data and effectively represent the sparse characteristics of kernel events. For example, the K-SVD algorithm is trained on 100,000 pieces of collected normal and abnormal network event data to generate an overcomplete dictionary as the sparse basis matrix.

[0141] S52. Group the sparse sampling data set according to event type and time window, apply the measurement matrix to perform dimensionality reduction projection on each group of data, and mark the missing positions with reference to the sampling coverage matrix to generate a compressed measurement vector.

[0142] S53. Execute the improved orthogonal matching pursuit (OMP) algorithm for signal reconstruction: first calculate the inner product of the compressed measurement vector and each atom of the sparse basis to find the best matching basis vector, iteratively update the sparse representation coefficient and calculate the residual, and terminate when the residual norm is lower than the preset threshold or reaches the maximum number of iterations to obtain the sparse representation coefficient.

[0143] In this embodiment, after each iteration, the kurtosis value of the current residual signal is calculated. Kurtosis is a statistic that measures the impact of the signal. When the kurtosis value is much greater than 0, it indicates that the residual still contains significant, non-Gaussian sparse signal components, and the iteration should continue; when the kurtosis value approaches 0, it indicates that the residual is close to Gaussian white noise, the main signal components have been extracted, and the iteration can be terminated. This improvement enables the OMP algorithm to intelligently stop iteration based on the actual sparse characteristics of the signal, thereby achieving a better balance between reconstruction accuracy and computational efficiency, and solving the disadvantages of a fixed number of iterations.

[0144] S54. Use sparse representation coefficients and pre-calculated sparse basis to reconstruct the signal and restore the complete monitoring data time series. At the same time, calculate the credibility of each reconstructed data point based on the residual size and number of iterations in the reconstruction process, and output the reconstructed complete monitoring data and the corresponding reconstruction confidence score.

[0145] In the actual application of edge computing, sampling can be performed with less local CPU and memory resources, and monitoring data can be transmitted back to the central platform with smaller network bandwidth (due to the smaller amount of data transmitted), while still obtaining a monitoring view close to the full collection.

[0146] S6. Comprehensively analyze the reconstructed complete monitoring data and reconstruction confidence score, perform pattern matching with the preset fault feature library, identify abnormal network behavior and locate the source of the fault, and ultimately output a fault diagnosis report that includes the fault type, impact range, and repair suggestions, as well as in-depth root cause analysis results.

[0147] S61. Input the reconstructed complete monitoring data into the anomaly detection engine, and use a sliding window method to calculate the statistical characteristics of key indicators such as network delay, packet loss rate, and bandwidth utilization. When the indicator deviates from the baseline by more than 3 times the standard deviation, an abnormal alarm is triggered and an abnormal event sequence is generated.

[0148] S62. The confidence of the abnormal event sequence is weighted based on the reconstruction confidence score. High-confidence abnormal events are prioritized. By performing pattern matching with a predefined fault feature library, the fault type (such as network congestion, equipment failure, configuration error, etc.) is identified and a preliminary fault classification result is output.

[0149] S63. Based on the fault classification results and complete monitoring data, a fault propagation map is constructed to analyze the correlation between abnormal events in time and space dimensions. By tracing the fault source and impact path, the root cause and impact range of the fault are determined, and detailed root cause analysis results are generated.

[0150] S64. Integrate the fault classification results and root cause analysis results, combine them with the topology of the edge computing environment and business priorities, and generate a comprehensive fault diagnosis report that includes fault description, impact assessment, repair suggestions and preventive measures, providing actionable decision support for operation and maintenance personnel.

[0151] According to one aspect of the present application, S23, the improved Bakery algorithm, is specifically:

[0152] When a probe receives a sampling request, it first reads the shared data structure of the current probe group from the conflict coordination context, and sets its choosing array position to 1 through the atomic operation atomic_set, indicating that it has entered the ticket number selection phase. At the same time, it executes a memory barrier instruction to ensure that all CPU cores can see this state change and generate a selection state mark.

[0153] After the selection status mark takes effect, the probe traverses the ticket arrays of all other probes in the conflict group, reads the current ticket number of each probe through atomic_read, finds the maximum value max_ticket, writes (max_ticket+1) to its own ticket array position through atomic_set, then clears the choosing mark, completes the ticket number acquisition, and outputs the probe ticket number sequence.

[0154] The probe holding the probe ticket sequence enters the waiting phase and performs two layers of checks on each conflicting probe: first, it ensures that other probes have completed ticket selection (choosing[i]==0) by spin-waiting (cpu_relax), and then compares the ticket sizes. If the ticket numbers of other probes are smaller or the ticket numbers are the same but the probe ID is smaller, it continues to wait until it obtains the highest priority and forms a sampling permission token.

[0155] The probe that obtains the sampling permission token performs the actual data collection operation. After the collection is completed, the probe sets its own value in the ticket array to 0 through atomic_set, releases the sampling permission, and stores the collected data hash value in the shared buffer. Other waiting probes re-evaluate the priority after detecting that the ticket number is cleared, and finally output the updated probe sampling scheduling sequence.

[0156] In another embodiment of the present application, the priority is calculated as follows: Priority(i) = α·EventPriority(i) + β·ResourceEfficiency(i) + γ·HistoricalYield(i);

[0157] Where: EventPriority(i) is the real-time importance score of the event monitored by probe i; ResourceEfficiency(i) is the resource efficiency of probe i (collection value / resource consumption); HistoricalYield(i) is the historical effective data output rate of probe i; α is the event importance weight, β is the resource efficiency weight, and γ is the historical data output weight;

[0158] The ticket number generation strategy is Ticket(i) = BaseTicket + Priority(i) × TicketRange, where BaseTicket is the base ticket number and TicketRange is the coefficient used to amplify the influence of priority.

[0159] According to one aspect of the present application, S24, management of the data fingerprint library, specifically includes:

[0160] A two-level hash calculation is performed on the event data to be stored. First, the xxHash algorithm is used to calculate a 64-bit hash value for the event type, timestamp (accuracy down to microseconds) and the first 32 bytes of key parameters. This hash value is then combined with the probe ID, and the final 32-bit fingerprint is calculated through CRC32 to ensure that the same data from different probes produces the same fingerprint, and the event data fingerprint is output.

[0161] The event data fingerprint is stored in a hash table structure based on the open address method. The table size is 4096 slots. Each slot contains the fingerprint value, timestamp, reference count and next node (next) pointer. When a hash conflict occurs, linear probing is used to find the next empty slot. At the same time, a bidirectional linked list sorted by timestamp is maintained to facilitate timeout cleanup, forming a fingerprint storage index.

[0162] Start an independent cleaning thread to scan the fingerprint storage index every 10 milliseconds, starting from the head of the timestamp linked list, and mark the fingerprint items with a survival time of more than 100 milliseconds as expired. If the reference count is 0, it is immediately deleted and the slot is released. If the reference count is greater than 0, it is forcibly deleted after a delay of 50 milliseconds to ensure that the size of the fingerprint library is controllable and output the active fingerprint set.

[0163] Concurrent access to the active fingerprint collection is protected by read-write spin locks. Read operations (fingerprint queries) obtain read locks to allow multiple probes to query simultaneously. Write operations (insert / delete fingerprints) obtain write locks to ensure exclusive access. The try_lock mechanism is used to avoid long-term blocking. If the lock acquisition fails, the operation request is added to the pending queue, ultimately maintaining a consistent data fingerprint library.

[0164] According to another aspect of the present application, the processing is described by an example:

[0165] When a probe (such as P i ) When you need to collect a kernel event, execute the following process:

[0166] First, competition relationship judgment and data reuse check are performed. i According to the probe conflict mapping table generated in Example 1, it is determined whether there is a competition relationship between the events currently to be collected. If so, the probe P i First, calculate the data fingerprint of the event to be collected (the calculation method is detailed later) and query a globally shared data fingerprint library. If the fingerprint already exists, it means that other conflicting probes have collected the same or highly similar event data in the recent period (for example, within 100 milliseconds). At this time, P i The data will be reused directly, and the current collection request will be abandoned, thus avoiding repeated kernel operations and data transmission. If the data fingerprint does not exist, it indicates that a new data collection is required. At this time, a distributed mutual exclusion algorithm is activated to obtain a unique sampling permission.

[0167] In this embodiment, an improved distributed bakery algorithm is preferably used. The process is as follows:

[0168] Entering the selection phase: Probe P i In a globally shared Boolean choosing array of size n (the number of probes in the conflict group), choosing[i] is set to 1 through an atomic operation. This indicates that P i Selecting ticket number.

[0169] Get ticket number: P iIterates through the globally shared, n-sized integer ticket array, finds the maximum of all current ticket numbers, adds 1 to it, and writes it to ticket[i] atomically. Afterwards, it sets choosing[i] to 0, indicating that the ticket number has been selected.

[0170] Waiting for authorization: P i Enters the waiting loop. It traverses all other probes P in the conflict group j (where j=i), two levels of checks are performed:

[0171] First level check: wait for choosing[j] to become 0, ensure P j It has completed its ticket number selection.

[0172] The second level check: wait for the condition (ticket[j]==0) OR ((ticket[j], j)>(ticket[i], i)) to be met. Among them, (ticket[j], j)>(ticket[i], i) means to compare the ticket number and the probe ID in lexicographic order. That is, if the probe P j The ticket number is P i Small, or the ticket number is the same but P j ID than P i If P is small, i Must wait. ticket[j]==0 means probe P j Sampling is not currently requested.

[0173] Execute acquisition and release: When P i After passing the inspection of all other probes, it obtains the only sampling permission and can perform the actual data collection operation. i Set ticket[i] to 0 through atomic operation to release the permission.

[0174] Through the above distributed mutual exclusion algorithm, a fair and conflict-free probe sampling scheduling sequence will eventually be formed.

[0175] After collecting data, the probe that obtains sampling permission needs to store the fingerprint of the data in the shared data fingerprint library. The fingerprint generation and storage mechanism is as follows:

[0176] Fingerprint calculation: Two-level hash calculation is used to reduce the probability of collision.

[0177] It should be noted that in the previous embodiment, the hash value is combined with the probe ID, and the final 32-bit fingerprint is calculated through CRC32 to ensure that the same data of different probes produces different fingerprints.

[0178] If the fingerprint includes the probe ID, different probes will generate different fingerprints when collecting the same data, which will not achieve the purpose of data reuse. Therefore, the following method is used to solve the problem:

[0179] At the first level, a high-speed hash algorithm (such as xxHash) is used to calculate a 64-bit hash value H64 for the core event data: H64 = xxHash(EventType1 | Timestamp | mus | KeyParams1..32); where Timestamp is a timestamp with microsecond precision, KeyParams1..32 is the first 32 bytes of key parameters of the event, EventType1 is the event type field, and mus indicates a timestamp with microsecond precision when the event occurred.

[0180] In the second stage, a 32-bit fingerprint F32 is calculated for this 64-bit hash value, for example, using the CRC32 algorithm: F32=CRC32(H64). The F32 is the final event data fingerprint.

[0181] Storage and Management: The calculated event data fingerprint F32, the current timestamp, and a reference count are stored in a shared hash table implemented using the open addressing method. Each fingerprint entry is assigned an expiration time, such as 100 milliseconds. A separate cleanup thread periodically scans the hash table, removing expired fingerprint entries with zero reference counts. This ensures that the fingerprint database does not grow indefinitely and maintains its currency.

[0182] In another embodiment of the present application, when a probe needs to collect data, the data flow follows the following path:

[0183] A probe first checks whether it has a pre-assigned time slot. If so, it proceeds with sampling. Otherwise, it submits a sampling request to its group and competes for a local ticket within the group. The probe with the lowest ticket in the group becomes the temporary representative of the group.

[0184] Each group's representative participates in the second-level competition with their group's sampling requirements. The system selects the final sampling executor based on the overall priority. After the authorized probe completes data collection, it writes the results to the shared buffer, checks the waiting queue, and wakes up the next waiting probe.

[0185] Specifically, the system first dynamically divides probes into several groups based on their historical conflict patterns and resource usage characteristics. Each group contains 3 to 7 probes. The principle of grouping is to group probes with low conflict frequencies together, thereby reducing the intensity of competition within the group.

[0186] When a probe needs to sample, it first conducts local competition within its group. Each group maintains its own ticket number sequence, and the probe only needs to check the status of other members in the group. This reduces the number of probes that would normally need to be checked from n to around five, significantly reducing coordination overhead.

[0187] Each group selects a temporary representative through local competition. This representative carries the group's sampling requests and participates in the second-level inter-group competition. Inter-group competition uses a priority-based scheduling strategy. Instead of simply comparing ticket numbers, it calculates dynamic priorities based on event type, wait time, and historical contribution.

[0188] Balances fairness and efficiency: Relatively fair rotation is maintained within the group, while intelligent scheduling is performed between groups based on task importance.

[0189] In another embodiment of the present application, step S24 may also be:

[0190] Probes that successfully collect data must convert the event information into a probe-independent, efficient data fingerprint for deduplication and store it in a globally shared data fingerprint database. This process aims to provide reliable data reuse for other conflicting probes. Preferred implementations include:

[0191] Level 1 (Content Hashing): Using a high-speed hashing algorithm (e.g., xxHash), a 64-bit intermediate hash value is calculated for the core content of the event. This core content preferably includes the event type, a timestamp with microsecond precision, and key parameters that uniquely identify the event (e.g., the source / destination IP and port numbers of network packets, the block address for disk I / O, etc., with the first 32 bytes truncated). The microsecond timestamp precision ensures consistent hash values ​​even when captured nearly simultaneously by different probes on different CPU cores.

[0192] Second level (final fingerprint): In order to obtain a shorter fingerprint suitable for use as a key in a hash table, a second hash algorithm (such as CRC32) is further applied to the above 64-bit intermediate hash value to calculate the 32-bit final fingerprint.

[0193] The calculated 32-bit fingerprint, along with its corresponding precise timestamp, reference count (initially 1), and optional data storage location pointer, is stored in a globally shared hash table based on the open address method. The hash table can be pre-set in size (e.g., 4096 slots), and hash collisions are resolved using linear probing.

[0194] A separate, low-priority cleanup thread is launched, which scans the fingerprint database at a fixed interval (for example, every 10 milliseconds). Based on the timestamp stored in each fingerprint entry, it marks fingerprint entries that have survived beyond a preset expiration time (TTL, for example, 100 milliseconds) as expired. Expired entries with a reference count of zero are immediately removed from the hash table. For entries with a reference count greater than zero, a longer forced deletion time can be set to prevent invalidation of ongoing read operations. This mechanism ensures the timeliness of the fingerprint database and prevents it from growing indefinitely.

[0195] Preferably, xxHash can be replaced by a hash algorithm such as MurmurHash3. Regarding storage structure, in addition to the open address method, hash tables can also be implemented using a chaining method. In scenarios with extremely high collisions, a more complex lock-free hash table design can be optionally adopted to eliminate lock contention overhead. Regarding the cleanup mechanism, in addition to independent cleanup threads, a distributed cleanup method can also be used, where several potentially expired slots are randomly selected and cleaned during each write operation.

[0196] According to one aspect of the present application, the real-time update process of the Markov chain model is specifically as follows:

[0197] The raw event interval data (nanosecond level) in the event timing features is read and mapped into 16 predefined buckets through logarithmic transformation log2(interval), where bucket[0] represents [0, 1] microseconds and bucket

[15] represents [32768, 65536] microseconds. Values ​​out of range are truncated to generate a quantized event interval sequence.

[0198] The state transition matrix of the Markov chain is updated based on the quantized event interval sequence, and the exponential moving average (EMA) algorithm is used: the new transition probability = α × observation frequency + (1-α) × historical probability, where α = 0.125 ensures the smooth decay of historical information. After each state transition is updated, it is immediately normalized to ensure that the sum of the probabilities is 1, and the real-time state transition matrix is ​​output.

[0199] Monitor the number of consecutive occurrences of the same bucket in the quantitative event interval sequence. When the same bucket appears three or more times in a row, burst detection is triggered. The burst intensity score is calculated as the number of consecutive occurrences × the historical rarity of the bucket. When the score exceeds the threshold of 10, it is marked as a burst event. The start time and expected duration of the burst are recorded to generate a burst event marker.

[0200] Combining the real-time state transition matrix and emergency event labels, a forward algorithm is used to calculate the probability distribution of each bucket within the next 10 milliseconds. For event types marked as emergencies, the predicted probability is increased by an additional 50%. The final output is a key event prediction set that includes the event type, prediction time window, and confidence level (in the range of 0-1).

[0201] According to one aspect of the present application, S33, the dynamic time slice allocation process, is specifically as follows:

[0202] Based on the number of active probes N in the system, the 10-millisecond monitoring window is divided into 32 basic time slices, each approximately 0.3125 milliseconds. A round-robin algorithm is used to generate an initial 32-bit mask for each probe. For example, if there are three probes, an interleaved pattern of 0x92492492 (001001...), 0x49249249 (010010...), and 0x24924924 (100100...) is assigned to each probe to ensure uniform distribution. The basic time slice mask is then output.

[0203] When the key event prediction set indicates that a high-priority event exists in a certain time window, the probe set responsible for monitoring this event type is identified, and the mask bits of these probes in the predicted time window are expanded to all 1s through OR operation. At the same time, the time slices of other probes are proportionally compressed (up to 50% of the original) to generate an extended time slice mask.

[0204] Detect conflicts in the extended time slice mask. When the extension requests of multiple probes cause some time slices to be over-allocated (more than two probes occupying them at the same time), start priority arbitration: retain the extension rights of the probe with the highest event prediction confidence, and restore the remaining probes to the basic allocation. Through iterative adjustment, ensure that each time slice is shared by at most two probes, and finally output the coordinated and optimized time slice allocation table.

[0205] Learn the temporal patterns of event occurrence from historical data and use these patterns to predict the future, thereby proactively tilting limited sampling resources to time windows and probes where key events are more likely to occur.

[0206] Specifically, this step can be broken down into:

[0207] First, construct event timing features. Based on the probe sampling schedule generated in Example 2, analyze the timestamps of successfully sampled events recorded therein and calculate the intervals between consecutive events. To facilitate model processing, logarithmically transform and quantize the raw nanosecond interval data, mapping it into 16 predefined discrete buckets to generate a quantized event interval sequence. For example, bucket[k] can represent events with an interval in the range [2k, 2k+1) microseconds.

[0208] Secondly, a prediction model is used to generate a key event prediction set. In this embodiment, a lightweight Markov chain model is preferably used.

[0209] Using the above quantized event interval sequence as input, a 16×16 state transition matrix M is constructed. The elements in the matrix M ij represents the probability of an event interval shifting from bucket[i] to bucket[j]. This matrix is ​​updated in real time using the exponential moving average (EMA) algorithm to adapt to changes in event patterns: ij，new =α•O ij +(1-α)•M ij，old , where: M ij，new and M ij，old are the transition probabilities before and after updating, respectively. ij is an indication of whether the currently observed transition is i→j (1 if yes, 0 otherwise). α is a smoothing factor that can be set to 0.125 to balance the importance of historical data and current observations.

[0210] Monitor the sequence of quantized event intervals. When the same bucket (e.g., bucket[0] representing a very small time interval) appears a preset number of times (e.g., 3 times), a burst mode detection is triggered. A burst intensity score can be calculated. When the score exceeds a threshold, the event type is marked as a burst event.

[0211] Combining the updated state transition matrix M with the emergency event labels, a forward algorithm predicts the probability distribution of each bucket (i.e., the intervals between various events) within a short future time window (e.g., 10 milliseconds). For events marked as emergencies, their predicted probabilities are increased (e.g., by 50%). This ultimately outputs a key event prediction set containing the event type, predicted time window, and confidence score.

[0212] Then, a time slice allocation table is dynamically calculated based on the prediction results. A monitoring window (e.g., 10 milliseconds) is divided into a fixed number of basic time slices (e.g., 32), each approximately 0.3 milliseconds. The time slice allocation for each probe is represented by a 32-bit bitmask.

[0213] Under normal circumstances, a uniformly staggered initial bit mask is generated for the N active probes to ensure equal sampling opportunities. For example, for 3 probes, the mask can be the result of a cyclic shift to ensure that at most one probe is active per time slice.

[0214] When the critical event prediction set indicates a high probability of a high-priority event within a certain time window, the probe responsible for that event is identified. The mask bits for that probe within the corresponding time window are expanded to all 1s through an OR operation, granting it temporary intensive sampling privileges. Simultaneously, the time slices of other non-relevant probes can be proportionally compressed to balance the overall system load. If extension requests from multiple probes conflict, priority arbitration is performed based on the confidence of the event prediction, retaining the extension privileges for the probe with the highest confidence.

[0215] Through the above steps, an optimized time slice allocation table that changes dynamically over time is finally generated to guide the next stage of collaborative sampling.

[0216] According to one aspect of the present application, S53, the improved OMP algorithm, is specifically:

[0217] The inner product of the compressed measurement vector and each atom in the pre-computed sparse basis is calculated. The SIMD instruction set (such as AVX2) is used to parallelize the dot product operation of multiple atoms. The 64×256 sparse basis is divided into 8 sub-blocks for parallel calculation. The atom index with the largest absolute value of the inner product is found and recorded as the best matching atom and its correlation coefficient.

[0218] The residual vector is updated based on the best matching atoms and correlation coefficients, and a delayed update strategy is adopted: the contributions of multiple atoms are first accumulated into a temporary buffer, and the actual residual is updated every 4 atoms to reduce memory access overhead. At the same time, the orthogonal basis of the selected atom set is maintained. The Gram-Schmidt orthogonalization process is used to ensure that the newly added atoms are orthogonal to the selected atoms to generate an updated residual vector.

[0219] The sparsity parameter is dynamically adjusted according to the energy distribution of the updated residual vector. When the residual energy is concentrated in a few components, the sparsity is reduced to accelerate convergence. When the residual energy is dispersed, the sparsity is increased to improve the reconstruction accuracy. The kurtosis of the residual is calculated as the basis for adjustment and the adaptive sparsity parameter is output.

[0220] Combined with the L2 norm of the updated residual vector, the current number of iterations, and the adaptive sparsity parameter, a multi-condition convergence judgment is performed: if the residual norm is lower than 0.1% of the input signal energy, or the number of iterations exceeds twice the sparsity, or the improvement for three consecutive iterations is less than 1%, the iteration is terminated and the final sparse representation coefficient is output.

[0221] According to another aspect of the present application, in this embodiment, this step is the final execution link of all the aforementioned scheduling and prediction strategies. Each eBPF probe acts as an independent execution unit and works in coordination according to a globally consistent strategy.

[0222] Specifically, each probe performs the following operations:

[0223] First, distributed sampling based on time slices is performed. Each probe loads the optimized time slice allocation table generated in Example 3. During its allocated time slice, it activates monitoring of the kernel event stream and data capture. During unallocated time slices, the probe is in a dormant or low-power state, thereby reducing overall system overhead.

[0224] Next, compensated sampling is performed. When a key event marked by the key event prediction set actually occurs, the probe responsible for monitoring the event immediately activates compensated sampling mode. In this mode, the probe temporarily ignores the current time slice and performs intensive sampling for a short period of time (for example, 100 microseconds) to ensure that critical, high-value event data is fully captured. The collected compensated data is specially marked to distinguish it.

[0225] Finally, the data is aggregated and a sampling coverage matrix is ​​constructed. The regular event data collected by all probes within their respective time slices, as well as the key event data collected in compensation mode, are summarized. Furthermore, the ratio of the actual number of event points collected to the theoretical maximum number of event points that can be collected within each monitoring window is calculated to construct a sampling coverage matrix. This matrix, along with the aggregated sparsely sampled dataset, serves as input for subsequent steps, such as data reconstruction in S500.

[0226] Through the method provided in the embodiment of the present invention, by distributing and coordinating sampling conflicts between eBPF probes, dynamically adjusting the sampling strategy in combination with event prediction, and performing collaborative sparse sampling based on the generated strategy, the technical effect of significantly reducing the system performance overhead brought about by concurrent monitoring of multiple probes while ensuring the integrity of key events is achieved, thereby solving the contradiction between the eBPF monitoring solution in the existing technology that is difficult to strike a balance between comprehensive monitoring and low system overhead.

[0227] This embodiment aims to explain in detail how to specifically calculate the degree of resource contention between multiple eBPF probes when multiple eBPF probes are detected to be mounted on the same kernel mount point in step S13, thereby providing a quantifiable and accurate decision-making basis for generating a probe conflict mapping table.

[0228] In this embodiment, the probe P i and P j Resource contention level C on the same mount point ij It is defined as a comprehensive indicator that combines the consideration of two dimensions: CPU time consumption ratio and event triggering frequency conflict. Its specific calculation formula is as follows: C ij =w cpu •{(N i *T i +Nj *T j ) / ∑ k∈S (N k *T k )}+w freq {(N i +N j ) / N event};

[0229] The parameters in the formula are defined as follows: C ij Indicates probe P i and P j The resource contention degree between the two nodes is in the range of [0, 1]. The larger the value, the more intense the contention. S represents the set of all active probes on the current conflicting mount point. N i , N j , N k They represent the probe P in the unit statistical period (for example, 1 second). i , P j , P k The number of times it is triggered. i , T j , T k Represents probe P i , P j , P k The average CPU time spent on a single execution (e.g. in nanoseconds). event Indicates the total number of times the event corresponding to the kernel mount point occurs in the same unit statistical period. cpu and w freq : Represents the weight factors of CPU contention and frequency contention respectively, both of which are positive and sum to 1 (for example, w cpu =0.6, w freq = 0.4). This allows for tailoring the focus on different types of competition depending on the monitoring objective.

[0230] The first part of the formula measures the proportion of the total CPU time consumed by the two probes in the total consumption of all conflicting probes, reflecting the intensity of their direct competition for CPU resources. The second part measures the proportion of the sum of the trigger times of the two probes in the total number of events, reflecting the overlap or conflict frequency of their monitoring. ij The value will be compared with the conflict probe pair {P i , P j} and the conflict event type are written into the probe conflict mapping table.

[0231] The standard OMP algorithm typically uses a fixed sparsity K (or a fixed number of iterations) as the termination condition. In this embodiment, the termination condition is dynamically changed. After updating the residual vector rk in each round or every few rounds, a residual evaluation and parameter adjustment process is added.

[0232] After the kth iteration, calculate the kurtosis value Kurt(r k Kurtosis is a statistic that measures the degree of sharpness of a probability distribution. Its calculation formula is: Kurt(r k )={[∑ n i=1 (r k，i -r * k ) 4 ) / n] / ∑ n i=1 ((r k，i -r * k ) 2 ) 2 / n}-3; where n is the dimension of the residual vector, r k，i is the i-th component of the vector, r * k is the mean of the vector. Using the excess kurtosis, which is the form after minus 3, the kurtosis of the normal distribution is 0.

[0233] Adjust the iteration strategy of the OMP algorithm based on the kurtosis value. Gaussian trend judgment (Kurt (r k )≈0): When the kurtosis value is close to 0 (for example, -0.5 <Kurt(r k ) < 0.5), indicating that the residual signal is becoming more and more like Gaussian white noise, and the useful signal components have been basically extracted. At this point, the iteration should be terminated.

[0234] Pulse characteristic judgment (Kurt(r k )≫0): When the kurtosis value is much greater than 0 (for example, Kurt(r k )>2), indicating that there are still significant, pulse-like, unrecovered signal components in the residual signal. In this case, the iteration should be continued or even accelerated.

[0235] Dynamically adjust sparsity / iteration limit: set an initial maximum number of iterations K max (i.e., the initial sparsity). During the iteration process, an effective iteration upper limit Keff is dynamically adjusted according to the kurtosis value:

[0236] After the kth iteration, if Kurt(r k) first enters the interval close to 0, the effective iteration upper limit Keff is updated to k+Δk, where Δk is a small additional iteration number (such as 2 or 3) to allow the algorithm to perform final fine-tuning before termination.

[0237] If Kurt (r k ) continues to be large, then Keff remains K max , the algorithm continues to iterate normally.

[0238] The termination condition of OMP becomes: when the number of iterations k reaches Keff, or the residual norm is lower than the preset threshold, the algorithm terminates.

[0239] Through this method, the OMP algorithm can intelligently determine the convergence state of the reconstruction process and stop in time when the residual approaches noise, avoiding overfitting and unnecessary calculations; it also ensures the continuity of iteration when there is still a strong signal in the residual, thereby achieving a better balance between efficiency and accuracy.

[0240] According to one aspect of the present application, a specific method for constructing a lightweight Markov chain model is as follows:

[0241] In order to improve the prediction accuracy and domain relevance of the model, the model state (State) in this embodiment is defined as a two-tuple: State = (EventType, IntervalBucket).

[0242] State space definition: EventType(E): represents a specific kernel event type, such as tcp_sendmsg, block_rq_issue, etc. Assuming that the system focuses on M key event types, then E∈{E1, E2, ..., EM}.

[0243] IntervalBucket (I): represents the time interval between the current event and its previous event of the same type, and is mapped to one of the 16 discrete buckets after logarithmic transformation, i.e., I∈{I0, I1, ..., I15}.

[0244] Therefore, the size of the state space S of the entire model is M × 16. Each state si,j = (Ei, Ij) represents the occurrence of event Ei, and the time interval between it and the previous Ei event is within the range of bucket Ij.

[0245] State transfer matrix construction and initialization:

[0246] The state transition matrix P is a square matrix with dimensions of (M×16)×(M×16).

[0247] The matrix element Puv represents the probability that the system transitions from state u to state v. Specifically, P(Ei, Ij), (Ek, Il) represents the probability that after an event Ei is observed with an interval belonging to Ij, the next observed event is Ek with an interval belonging to Il.

[0248] Initialization: During a cold start of the system, the transfer matrix P can be initialized to a uniform distribution matrix, meaning that all transition probabilities are equal. Preferably, a large amount of historical monitoring data collected offline can be used to pre-train an initial transfer matrix that is more consistent with general principles.

[0249] Model usage and prediction:

[0250] Online Update: The kernel event stream is monitored in real time. Whenever a new event (Ek, Il) occurs after an event (Ei, Ij), a state transition observation is observed. At this point, the corresponding element P(Ei, Ij), (Ek, Il) and the corresponding row of the matrix P are updated using the exponential moving average (EMA) algorithm mentioned in the solution.

[0251] Prediction: When the system is currently in state scurr = (Ec, Ic), the row vector corresponding to scurr in the state transition matrix P, P[scurr,:], provides the complete probability distribution of the next state. By selecting the states with the highest probability values ​​in this row vector, a key event prediction set containing event type, expected time window, and confidence level (i.e., probability value) is generated, providing decision support for dynamic time slice allocation in step S33.

[0252] According to another aspect of the present application, this embodiment also provides an introduction of a complementary mechanism with an emergency preemption channel, which enables the system to respond instantly to the highest priority events while ensuring the efficiency of conventional scheduling.

[0253] During the system initialization phase, in addition to defining the corresponding intra-group ticket number array and intra-group selection flag array for each probe conflict group, and defining the inter-group ticket number array and inter-group selection flag array for each group representative, this embodiment additionally defines a key global shared data structure: Emergency Status Flag (EmergencyStatusFlag): defines a globally shared integer variable that can be read and written through atomic operations, such as global_emergency_flag. This flag is initialized to 0, indicating that the system is in the normal scheduling state. When its value is 1, it indicates that the system has entered an emergency state and the preemption channel needs to be activated.

[0254] Under normal circumstances (i.e., when the global_emergency_flag is 0), the system strictly follows a hierarchical bakery algorithm for scheduling. Probes first compete for group representative privileges within their group through a first-level bakery algorithm. The winner then competes for the right to handle kernel events through a second-level inter-group bakery algorithm. This process is designed to reduce scheduling latency during normal operation.

[0255] When any probe in the system (regardless of the group it belongs to or the current ticket size) identifies a predefined high-risk event feature through its own monitoring logic (for example, by matching it with the built-in attack signature library or critical kernel error features), it will immediately perform preemptive activation operations.

[0256] This probe performs an atomic write operation to set the global_emergency_flag value from 0 to 1. This operation ensures that all CPU cores can see the state change immediately.

[0257] The waiting logic of all non-urgent probes participating in the Bakery algorithm polling wait at each level is modified to place the check of the emergency status flag at the highest priority.

[0258] Modified wait logic: In the spin-wait logic of a probe, it first atomically reads the value of global_emergency_flag before checking the ticket number and selection flag of other probes.

[0259] If the value read is 1, the probe recognizes that the system has entered an emergency state. At this point, it must immediately perform a coordinated avoidance operation: using an atomic write operation, it resets its ticket number in the corresponding ticket number array to 0 and immediately exits the current waiting loop, effectively abandoning the queued request.

[0260] If the read value is 0, the original Bakery algorithm waiting logic continues to execute.

[0261] This mechanism ensures that once the emergency signal is issued, all queued probes will quickly clear the runway to make way for the execution of the emergency probe.

[0262] After successfully activating the preemption channel, a probe with the emergency flag set can bypass the queuing and waiting logic of all layered bakery algorithms, immediately obtain exclusive processing rights for kernel events, and execute its high-risk event handling procedures (for example, recording detailed scenes, blocking malicious connections, etc.).

[0263] After the emergency probe completes its critical task, it is responsible for restoring the system to its normal state by resetting the value of global_emergency_flag from 1 to 0 in an atomic write operation.

[0264] After the flag is cleared, probes that abandoned the queue due to avoidance will find that the system has returned to normal in their subsequent retry logic, and can restart a new round of hierarchical scheduling requests. At this point, the entire system seamlessly switches from an emergency state to an efficient regular scheduling state.

[0265] Through the above steps, this embodiment ultimately forms a fusion solution: under normal conditions, the system implements efficient, low-latency fair scheduling through the layered bakery algorithm; while in emergency conditions, the emergency preemption channel mechanism ensures instantaneous response to the highest priority events, solving the entire problem chain and achieving a dynamic balance between the system's efficiency and responsiveness.

[0266] According to one aspect of the present application, a method for proactively predicting the degree of resource contention between eBPF probes is described in detail. The method comprises the following steps:

[0267] Step S101 : extracting probe monitoring data of continuous sampling periods from the real-time performance indicators read from the shared memory, and constructing a multi-dimensional load feature vector sequence.

[0268] In this embodiment, the multi-dimensional load feature vector is a quantitative description of the resource consumption state of a single eBPF probe at a certain moment, which is used for subsequent dynamic analysis. The dimension of the vector can be configured according to the monitoring precision requirement.

[0269] Specifically, a preferred three-dimensional load feature vector L(t) includes: 1) normalized trigger frequency; 2) normalized CPU execution time; and 3) normalized memory access count. The raw monitoring data from N consecutive sampling periods (e.g., N = 16) is subjected to maximum-minimum normalization so that its values ​​fall within the interval [0, 1], forming the multi-dimensional load feature vector sequence {L(t-N+1), ..., L(t)}.

[0270] In step S102, to address the poor adaptability of traditional prediction models in dynamic edge computing load environments, a dual-modal residual-driven adaptive Kalman filter is used to generate the predicted contention intensity. The prediction residual refers to the difference between the load value predicted by the filter and the actual load value observed at the next moment.

[0271] In the Kalman filter, the process noise covariance matrix Q quantifies the uncertainty of the system model. The larger the Q value, the less trust the existing model has, and the more it will rely on new observations for corrections.

[0272] Initialize a Kalman filter with the state vector x(t) being [load value, speed, acceleration]. At each sampling period, the filter executes a prediction-update loop, compares the predicted value generated in the prediction step with the actually observed load value at the end of the period to obtain the prediction residual e(t), and forms the predicted residual sequence with consecutive residuals.

[0273] Perform real-time analysis on the predicted residual sequence to determine the load dynamic mode.

[0274] When it is detected that the signs of M consecutive (preferably M = 3) residual values are the same and the absolute value of their mean exceeds K times (preferably K = 2.5) the historical residual standard deviation, it is determined that the current load dynamic mode is the step mode.

[0275] If the step mode condition is not satisfied, but the absolute value of a single residual exceeds L times (preferably L = 4) the historical residual standard deviation, it is determined as the pulse mode.

[0276] According to the determined load dynamic mode, dynamically adjust the process noise covariance matrix Q inside the predictor. When it is determined as the step mode, perform a dynamic injection operation on the process noise covariance matrix Q. Specifically, instantaneously increase the diagonal element values by one order of magnitude (for example, multiply by 10). This operation will significantly increase the Kalman gain of the filter and accelerate the convergence of the predictor to the structural changes of the system. When it is determined as the pulse mode or the normal mode, keep the value of the Q matrix stable.

[0277] The filter after adaptive adjustment outputs the predicted load value for the next period, and this value is the predicted competition intensity.

[0278] The determination of pulse deviation is as follows: |residual(t)| > k1×σ and duration < T1; residual(t) is the predicted residual at time t; k1 is the pulse detection sensitivity coefficient; duration is the deviation duration; T1 is the upper limit of the pulse duration; σ is the standard deviation of the predicted residual.

[0279] The determination of step deviation is mean(residual[t - w:t]) > k2×σ and duration > T2;

[0280] residual[t - w:t] is the residual sequence within the time window [t - w, t]; w is the sliding window width; k2 is the step detection sensitivity coefficient; T2 is the lower limit of the step duration.

[0281] In the process noise covariance matrix, Q_new = Q_old × (1 + λ × deviation_score), where deviation_score is calculated based on the deviation type and strength. λ is a coefficient, Q_new is the updated covariance matrix, and Q_old is the old covariance matrix.

[0282] Step S103: updating the probe conflict mapping table using the predicted contention intensity to identify high-risk conflicts in advance.

[0283] In addition to recording the current conflict relationship, the probe conflict mapping table's data structure also includes a predicted risk level field. The predicted contention intensity generated in step S102 is used to update this field. For example, when the predicted contention intensity exceeds 150% of the current contention intensity, the risk level of the conflicting pair is marked as high, providing forward-looking input for subsequent scheduling decisions.

[0284] Through the above steps, the present invention can upgrade from simple current status monitoring to accurate prediction of future conflict risks, providing a solid guarantee for the stability and resource utilization of the entire system.

[0285] In another embodiment of the present application, steps S102 and S103 may also be:

[0286] The first-order difference and the second-order difference of the multi-dimensional load characteristic vector sequence are calculated to obtain the load change acceleration and generate the predicted competition intensity.

[0287] For the multi-dimensional load characteristic vector sequence L(t), calculate its first-order difference (velocity) and second-order difference (acceleration).

[0288] First-order difference Δfirst(t)=L(t)-L(t-1);

[0289] The second-order difference Δsecond(t)=Δfirst(t)-Δfirst(t-1); wherein the second-order difference Δsecond(t) is the load change acceleration.

[0290] A weighted fusion prediction model is used to fuse the current resource contention level, first-order difference, and load change acceleration of the probe to generate the predicted contention intensity Cpredicted(t+1) for the next cycle.

[0291] Preferably, the fusion formula can be: Cpredicted(t+1)=wc•Ccurrent(t)+wv•Δfirst(t)+wa•Δsecond(t). Ccurrent(t) is the current resource contention level of the probe at time t. wc, wv, and wa are the fusion weights of the current state, velocity, and acceleration, respectively. In a basic implementation, they can be set to wc=1.0, wv=1.0, and wa=0.5. The weights represent the degree to which the prediction of the future state depends on the current position, velocity, and acceleration.

[0292] This example assumes that the system's short-term behavior can be approximated by its current state and its changing trend (velocity and acceleration). This approach offers minimal computational overhead and rapid response, making it ideal for real-time, short-term trend prediction on resource-constrained edge nodes.

[0293] Preferably, the fusion weights wc, wv, wa are not fixed values, but can be dynamically adjusted according to the pattern recognition mechanism.

[0294] The system can analyze the autocorrelation and variance of the multi-dimensional load feature vector sequence in parallel. When the autocorrelation coefficient is greater than 0.8, it is identified as a periodic pattern; when the variance exceeds a preset threshold, it is identified as a burst pattern.

[0295] The weights are adjusted based on the identified load patterns. For example:

[0296] In periodic mode, load changes are highly predictable, and the acceleration weight wa can be appropriately reduced (for example, to 0.25) to avoid overfitting.

[0297] In emergencies, where system states change dramatically and require a quick response, the speed weight wv and acceleration weight wa can be appropriately increased (e.g., to 1.2 and 0.75). This allows the prediction model to adapt to varying load dynamics, further improving prediction accuracy.

[0298] The predicted contention intensity, Cpredicted, is used to update a dedicated field in the probe conflict mapping table, such as the predicted risk level. When Cpredicted exceeds a warning threshold (for example, exceeding 150% of the current contention intensity), the system marks the conflicting pair as high risk, triggering subsequent more proactive resource scheduling or isolation strategies.

[0299] According to one aspect of the present application, a method for introducing proactive and intelligent data quality assurance capabilities into a data fingerprint library is described, comprising:

[0300] Step S201: construct a hierarchical verification cost model including multiple verification costs and depths.

[0301] In this embodiment, data verification operations are divided into three levels, namely, quick verification, standard verification, and deep verification, which are:

[0302] Execute the xxHash64 algorithm on the event core data to generate a hash value for comparison. This method is the fastest and is used for initial deduplication.

[0303] Apply the CRC32 algorithm to the key fields of the event, providing higher reliability than the fast checksum.

[0304] Perform lightweight semantic checks, such as checking that the IP address is within a valid subnet or that the port number is a known service port. This method is the most expensive but can find logical errors.

[0305] By offline calibration of the CPU cycle consumption of each level of operation, a relative verification cost model is established, for example, the cost ratio is 1:4:16.

[0306] Step S202: Generate a dynamic verification strategy configuration based on historical verification error rate data and the hierarchical verification cost model, and continuously calculate the comprehensive error rate within a sliding time window (e.g., the most recent 1000 verifications).

[0307] When the combined error rate exceeds twice the preset target, a policy upgrade is triggered. Specifically, the execution ratio of deep verification is increased to twice the original level, while the proportion of fast verification is correspondingly reduced to ensure that the total verification cost is within the budget.

[0308] When the combined error rate falls below half of the preset target, a policy downgrade is triggered. Specifically, the execution ratio of fast verification is doubled, while the proportion of deep verification is reduced to conserve system resources. The output of this process is a dynamically adjusted configuration of the verification layer execution ratio [R_fast, R_standard, R_deep], where R_fast is the fast verification ratio, R_standard is the standard verification ratio, and R_deep is the deep verification ratio.

[0309] Step S203 : Count the probe source distribution of error events, identify high-risk probes, and build an error propagation model.

[0310] The system maintains a statistical table of error contributions for each probe. When the error rate of a probe exceeds three times the average error rate of all probes, the probe is marked as a high-risk probe.

[0311] When executing the strategy of step S202 , for events marked as high-risk probes, the probability of deep verification will be additionally multiplied by an amplification factor (eg, 4 times).

[0312] When an event from a probe is confirmed to be an error, the system will temporarily increase the verification strength of the events collected by other probes within 100 milliseconds before and after in the time dimension, for example, upgrading its standard verification to deep verification.

[0313] Through the method described in this embodiment, the data fingerprint library is no longer a passive deduplication tool, but an intelligent module with active quality monitoring and self-adaptation capabilities.

[0314] According to one aspect of the present application, an intelligent resource scheduling method is described in detail to respond to the prediction results of Example 1. It includes:

[0315] Step S301 : Analyze historical resource usage data and establish a resource dependency matrix among various system resources.

[0316] The system collects historical time series data for various resources, such as CPU utilization, memory bandwidth, and network I / O throughput. Using the Pearson correlation coefficient algorithm, the correlation between resource pairs within a sliding window (e.g., the most recent 1000 sampling points) is calculated to form the resource correlation matrix. For example, if the correlation coefficient between CPU and memory bandwidth is greater than 0.7, they are marked as a strongly correlated resource pair.

[0317] Step S302: The predicted competition intensity and the resource correlation matrix are integrated to calculate and generate a collaborative adjustment execution plan.

[0318] The execution plan is the final scheduling instruction. Its input is the predicted contention intensity, which the system converts into future demand for specific resources. Combined with the resource dependency matrix, the system performs coordinated adjustment planning. For example, if CPU demand is predicted to surge, and CPU and memory bandwidth are strongly correlated, the execution plan will not only include increasing CPU scheduling priority or binding large cores, but also reserving or increasing memory bus frequency. The plan specifies the adjustment order, adjustment amount, and delay time.

[0319] Step S303: After executing the collaborative adjustment execution plan, continuously monitor key performance indicators and implement an adjustment rollback function.

[0320] Before executing any adjustment operation, the system saves the current key configuration parameters in a adjustment decision snapshot cache of size 5. After execution, within a 100 millisecond monitoring window, it continuously collects metrics such as application throughput and request latency.

[0321] When throughput drops by more than 10% or latency increases by more than 20 milliseconds, the rollback mechanism is automatically triggered and the system restores the most recent stable configuration from the snapshot cache.

[0322] Step S304: collecting and evaluating performance feedback data of historical adjustment operations to achieve strategy self-evolution capability.

[0323] The system records the scenario characteristics and final effects of each adjustment operation (including success, failure, and rollback) for training an adjustment risk assessment model.

[0324] Preferably, a lightweight reinforcement learning model can be used. Here, the system state represents the quantified load pattern and predicted trend, the action represents the specific regulation strategy, and the reward represents the performance improvement achieved after the regulation. Through continuous learning, the model can continuously optimize its ability to select the optimal regulation strategy in specific scenarios, enabling the self-evolution of core parameters.

[0325] According to one aspect of the present application, the process of generating the predicted competitive intensity is replaced by a prediction method based on sub-scale analysis, specifically:

[0326] Discrete wavelet transform is applied to the multi-dimensional load feature vector sequence to decompose it into at least one low-frequency approximation component and one high-frequency detail component, thereby forming a plurality of sub-band signals of different scales.

[0327] An independent predictor is configured for each sub-band signal to perform parallel scaled prediction to generate a set of scaled prediction results, and the set of scaled prediction results are fused and reconstructed to generate the final predicted competition intensity.

[0328] The configuring an independent predictor for each of the sub-band signals specifically includes configuring a dual-modal residual driven adaptive filter as described in the first embodiment as an independent predictor for each of the sub-band signals.

[0329] The fusion and reconstruction of the sub-scaled prediction results specifically includes calculating the real-time energy of each sub-band signal and generating a set of normalized energy weights based on the energy weights. Before reconstruction, the set of sub-scaled prediction results are weighted using the energy weights. An inverse discrete wavelet transform is applied to the weighted sub-scaled prediction results to complete the reconstruction.

[0330] In some scenarios where prediction accuracy is extremely high, the prediction module in the above embodiment can be replaced by the higher-order scale prediction module described in this embodiment. This solves the problem that a single model cannot take into account multi-scale dynamics. Specifically, the method includes:

[0331] Step S401 : applying discrete wavelet transform (DWT) to the multi-dimensional load feature vector sequence to decompose it into a plurality of sub-band signals of different scales.

[0332] In this embodiment, the computationally efficient Daubechies4 (db4) wavelet basis is preferably used to perform a three-layer wavelet decomposition on the generated load feature vector sequence. The output of this operation is a set (here, four) of mutually orthogonal subband signals {A3, D3, D2, D1}, which represent different scale dynamics, from long-term trends to short-term mutations.

[0333] Step S402 : configuring an independent dual-mode adaptive filter as described in the first embodiment for each sub-band signal to perform parallelized scaled prediction.

[0334] This step is the core structure of this embodiment. The system creates a filter bank, instantiating an adaptive Kalman filter for each subband signal in {A3, D3, D2, D1}. Each filter independently predicts its input subband signal and can be configured for specialized purposes. For example, the step-mode decision threshold of the filter processing the low-frequency component A3 can be set to be more sensitive.

[0335] Step S403 : Calculate the real-time energy spectrum of each sub-band signal, generate weighting coefficients based on the spectrum, and perform weighted fusion and reconstruction on the prediction results of each sub-scale.

[0336] For each subband signal, calculate its energy E in the most recent time window i =∑|c j ∣ 2 , where c j is the coefficient of the subband signal.

[0337] Normalize the energy of each sub-band to obtain the energy weight of each scale Wi=E i / ∑E k .

[0338] The sub-scale prediction results output by each parallel filter are multiplied by the corresponding energy weight Wi. Subsequently, the weighted sub-scale prediction results are reconstructed using the inverse discrete wavelet transform (IDWT) to obtain the final predicted competition intensity that integrates the dynamics of all scales.

[0339] Through this embodiment, the system can dynamically focus its prediction attention on the load scale with the most severe current fluctuations, and the output prediction results can be directly used as high-quality input for the collaborative adjustment mechanism in the third embodiment.

[0340] In summary, to resolve the runtime conflicts and performance issues caused by uncoordinated concurrent monitoring of multiple eBPF probes in the prior art, this application first collects and analyzes the operating status of all probes in the system, and accurately constructs a probe conflict mapping table by quantifying the intensity of their resource competition for the same kernel event. On this basis, this application designs a two-layer collaborative conflict resolution mechanism. The first layer is a pre-filter based on the data fingerprint library. Before requesting sampling, the probe first checks whether the event to be collected has been recently processed by other conflicting probes. If it has been processed, the data is directly reused, eliminating a large amount of unnecessary repeated collection from the root, greatly reducing the probability of conflict. The second layer is to introduce a distributed bakery algorithm for fair arbitration of sampling rights for events that must be newly collected. At any time, only one probe in the conflicting group can obtain collection rights, transforming the original chaotic resource preemption mode into an orderly, deterministic queuing mode, thereby completely avoiding system performance jitter and data storms caused by probe fighting, and ensuring the stability and fairness of the monitoring system.

[0341] To address the existing trade-off between comprehensive monitoring coverage and system overhead, a lightweight Markov chain model is used to learn the time interval distribution of historical event streams, enabling forward-looking predictions of potential event bursts or critical anomalies within future time windows. Based on these predictions, the time slice resources allocated to each probe are dynamically adjusted. When the system is operating smoothly, the probes operate in a staggered mode with a lower frequency to save overhead. When a critical event is predicted to be imminent, the sampling time window for the relevant probes is temporarily expanded to perform intensive sampling to ensure that critical information is not lost. This intelligent scheduling overcomes the limitations of static sampling rates. Furthermore, to compensate for the information incompleteness caused by sparse sampling, a signal reconstruction technique based on compressed sensing is introduced. This technique can recover the complete monitoring data time series with a high probability from limited, sparsely sampled data points. This application can operate with extremely low system overhead most of the time, while increasing monitoring granularity on demand when necessary. This achieves the technical effect of obtaining high-fidelity panoramic data with low-cost sparse sampling, perfectly resolving the contradiction between coverage and system overhead.

[0342] The preferred embodiments of the present invention are described in detail above. However, the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all fall within the scope of protection of the present invention.

Claims

1. An edge computing network observable monitoring method, characterized in that: include: Collect the running status data of multiple eBPF probes on edge devices, identify conflicts in monitoring the same kernel event, and generate a probe conflict mapping table; Read real-time event requests, coordinate sampling permissions between probes based on the probe conflict mapping table, and generate probe sampling scheduling sequences and data fingerprint libraries; Predict key event distribution based on historical event patterns and probe sampling scheduling sequences, adjust time allocation, and form a time slice allocation table; Perform distributed sampling on the kernel event stream according to the time slice allocation table, query the data fingerprint library and adjust the sampling strategy accordingly to generate a sampling data set; Generate complete monitoring data based on the sampled data set and perform pattern matching and fault location based on it; The process of collecting the running status data of multiple eBPF probes on edge devices, identifying conflicts in monitoring the same kernel event, and generating a probe conflict mapping table includes: building a probe registry, reading the real-time performance indicators of each probe from shared memory for each active probe in the probe registry, and analyzing the monitoring of the same kernel event by multiple probes; when multiple probes are detected to be mounted on the same mount point, calculating the degree of resource competition and generating a probe conflict mapping table containing conflicting probe pairs, conflicting event types, and competition intensity; Among them, the sampling authority coordination between probes is carried out in combination with the probe conflict mapping table, including the judgment of probe competition relationship. Specifically, the sampling requests in the real-time event request queue are monitored, and the probe initiating the request is judged according to the probe conflict mapping table whether there is a resource competition relationship. For the probes in the competition relationship, the hash value of the event to be collected is first calculated to check whether other probes have collected the same data. If so, the data is directly reused; otherwise, the data is collected again. The process of re-collecting and generating a probe sampling scheduling sequence and a data fingerprint library is as follows: Probes in the same conflicting group obtain sampling permissions through a distributed mutual exclusion algorithm. The probes that obtain permissions perform data collection, and a probe sampling scheduling sequence is generated based on the sampling order. After the collection is completed, the hash value and timestamp of the event data are stored in a shared storage to form a data fingerprint library. Among them, the sampling authority is obtained through a distributed mutual exclusion algorithm, including: when a probe receives a sampling request, it sets its own initial selection array position; it traverses the ticket number arrays of all other probes in the conflict group, obtains the current ticket number of each probe, finds the maximum value and adds 1 as its own ticket number, and updates it to its own selection array position; performs two layers of checks on each conflicting probe: ensures that other probes complete ticket number selection through spin waiting, compares ticket number sizes, and continues to wait if other probes have smaller ticket numbers or the ticket numbers are the same but the probe identifier is smaller, until the highest priority sampling authority is obtained.

2. The method according to claim 1, characterized in that The hash value and timestamp of the event data are stored in shared storage, with an expiration date set and periodic cleanup. The data fingerprint library includes: Perform a two-level hash calculation on the event data. Use the first hash algorithm to calculate a 64-bit hash value for the event type, timestamp, and key parameters. Combine this hash value with the probe identifier and use the second hash algorithm to calculate a 32-bit fingerprint to obtain the event data fingerprint. The event data fingerprint is stored in a hash table structure to form a fingerprint storage index and is added, deleted, and maintained according to the activity to obtain a data fingerprint library.

3. The method according to claim 1, characterized in that Based on historical event patterns and probe sampling scheduling sequences, we predict the distribution of key events, adjust time allocation, and form a time slice allocation table, including: Analyze the historical sampling patterns of each probe based on the probe sampling scheduling sequence, extract the event interval distribution from the historical event pattern data, and construct the event time series features; The prediction model is trained using event time series features, and the state transition probability is updated in real time. When a burst pattern is detected, high-frequency events in the future time window are predicted to obtain a key event prediction set. The monitoring window is divided into multiple time slices. Under normal circumstances, staggered time slices are allocated to probes. Time periods requiring intensive sampling are identified based on the key event prediction set. The time slices of related probes in this time period are extended to generate a time slice allocation table.

4. The method according to claim 3, characterized in that Obtain key event prediction sets, including: Read the event interval data in the event time series feature, map it to a predefined quantization interval through logarithmic transformation, and generate a quantized event interval sequence; Based on the quantized event interval sequence, the state transition matrix of the Markov chain is updated and a new state transition matrix is ​​calculated; Monitor the number of consecutive occurrences of the same interval in the quantified event interval sequence. When the same interval appears for a preset number of consecutive times, a burst detection is triggered, and the burst intensity is calculated and marked as a burst event. Combining the new state transition matrix and emergency event markers, the probability distribution of each event type in the future time window is calculated to improve the prediction probability of emergencies and generate a key event prediction set.

5. The method according to claim 3, characterized in that Based on the key event prediction set, calculate the time slice allocation table for each probe, including: Divide the monitoring window into a fixed number of basic time slices based on the number of active probes, and generate an initial bit mask for each probe with evenly staggered time slices. When the key event prediction set indicates that a high-priority event exists in a certain time window, the probes responsible for monitoring the event are identified, the mask bits of these probes within the prediction window are expanded, and whether there is a conflict is detected. If there is a conflict, priority arbitration is performed based on the event prediction confidence level to ensure the reasonable allocation of each time slice and output the time slice allocation table.

Images