AI computing power resource monitoring and predicting system and method based on data driving

Abstract

The invention belongs to the technical field of artificial intelligence computing power resource management, and particularly relates to an AI computing power resource monitoring and predicting system and method based on data driving. The acquired computing power resources are sent to an observability analysis module to perform distributed tracking on various application services, and a generated performance flame map is sent to a data integration and output module; meanwhile, protocol analysis is carried out on the obtained computing power resources, full-stack performance indexes are collected to serve as historical computing power resource use data to be sent to the computing power demand prediction module for modeling and prediction, and an obtained computing power resource demand prediction result is sent to the data integration and output module; and the data integration and output module gathers all the data together and injects a uniform label to finally provide a basis for resource scheduling. According to the invention, zero-intruding panoramic monitoring is realized, the coverage range is wide, and the use condition of computing power resources can be mastered more comprehensively by various application services.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence computing power resource management, and specifically is a data-driven AI computing power resource monitoring and prediction system and method. Background Art

[0002] In the field of AI computing power resource monitoring and prediction, some existing technical solutions mainly include traditional monitoring tools and simple data analysis methods. Traditional monitoring tools only monitor specific types of services and cannot achieve panoramic monitoring of various application services, AI services, and infrastructure services. In terms of data analysis, a relatively single method is used for resource prediction, lacking in-depth modeling and multi-model fusion of time series data. In summary, existing AI computing power resource monitoring mainly relies on manual monitoring, log analysis, and some monitoring tools, and has the following shortcomings:

[0003] Monitoring blind spots: It is difficult to fully cover all application services, AI services, and infrastructure services. There are monitoring blind spots, making it difficult to discover hidden performance bottlenecks.

[0004] Intrusive monitoring: Traditional monitoring methods usually require modifying code or deploying agents, which has a certain impact on system performance and is complex to deploy.

[0005] Low prediction accuracy: The lack of an effective prediction model makes it difficult to accurately predict future computing resource requirements, resulting in waste or shortage of resources.

[0006] Insufficient observability: The lack of collection and correlation of full-stack performance indicators makes it difficult to quickly locate and solve performance problems.

[0007] Data silos: The data generated by different monitoring tools are isolated from each other, making it difficult to form a unified view, which affects the overall analysis efficiency.

[0008] Therefore, in response to the above problems, it is particularly important to propose an AI computing resource monitoring and prediction system that can realize panoramic monitoring of various application services, AI services and infrastructure services, as well as in-depth modeling of time series data and multi-model fusion. Summary of the invention

[0009] The purpose of the present invention is to provide a data-driven AI computing resource monitoring and prediction system to solve the problems of incomplete monitoring, inaccurate prediction, insufficient observability, and difficulty in integrating with mainstream technology stacks in the current AI computing resource management. By adopting eBPF technology to achieve zero-intrusive panoramic monitoring, combined with time series analysis methods such as ARIMA and LSTM, the present invention can collect and analyze computing resource usage data in real time, quickly locate performance bottlenecks, and provide a scientific basis for resource scheduling, thereby improving resource utilization efficiency.

[0010] The technical solution adopted by the present invention to achieve the above-mentioned purpose is: a data-driven AI computing resource monitoring and prediction system, including: a data acquisition and preprocessing module, a computing demand prediction module, an observability analysis module, and a data integration and output module;

[0011] The data collection and preprocessing module is used to monitor various application services, AI services, and infrastructure services in real time. The acquired computing resources are sent to the observability analysis module. At the same time, it performs protocol analysis on the acquired computing resources, collects full-stack performance indicators, and sends them as historical computing resource usage data to the computing demand prediction module.

[0012] The computing power demand prediction module is used to model and predict the received historical computing power resource usage data through time series analysis methods, and send the obtained computing power resource demand prediction results to the data integration and output module;

[0013] The observability analysis module is used to perform distributed tracing of various application services, AI services, and infrastructure services, and send the generated performance flame graph to the data integration and output module;

[0014] The data integration and output module is used to collect the full-stack performance indicators in the data collection and preprocessing module, as well as the received computing resource demand prediction results and performance flame charts, and ultimately provide a basis for resource scheduling by providing multiple data interfaces and injecting unified labels.

[0015] The data acquisition and preprocessing module includes: a real-time monitoring unit, a protocol parsing unit, and a performance indicator acquisition unit;

[0016] The real-time monitoring unit uses eBPF to implement zero-intrusion panoramic monitoring of various application services, AI services, and infrastructure services, and is used to obtain computing resources for various services; the captured computing resource data is stored in the eBPF mapping table, the mapping table shares data between the user space and the kernel space, and the data is regularly read from the eBPF mapping table through the user space function, and the data is formatted for further call analysis and visualization of the protocol parsing unit and the observability analysis module;

[0017] The protocol parsing unit is used to capture network data packets containing computing resource data through the eBPF program, and identify the type of protocol used based on the characteristics of the data packets; for standard protocols, predefined parsing rules are used; for private protocols, custom parsers are used to parse data according to protocol specifications; performance-related indicators, i.e. full-stack performance indicators, are extracted from the parsed protocol data;

[0018] The performance indicator acquisition unit is used to obtain the full-stack performance indicators extracted by the protocol parsing unit and send them to the computing power demand prediction module as historical computing power resource usage data.

[0019] The computing power demand prediction module includes: an ARIMA model for predicting stable time series data and an LSTM model for processing time series data with long-term dependencies.

[0020] The ARIMA model performs the following steps:

[0021] 1-1) Data preprocessing: Ensure that the time series data of historical computing resource usage data is regular and has no missing values; perform differential analysis on the historical computing resource usage data to make it stable;

[0022] 1-2) Model identification: Use the autocorrelation function and partial autocorrelation function to determine the parameters of the ARIMA model, that is, determine the number of autoregressive terms p, the difference order d, and the number of moving average terms q;

[0023] 1-3) Model estimation: The maximum likelihood estimation method is used to estimate the parameters of the ARIMA model;

[0024] Use the established ARIMA model to predict data and determine whether the error between the original data and the predicted data is less than the preset range. If the error distribution is greater than the preset range, the model structure identification and parameter estimation need to be re-performed;

[0025] 1-4) Model diagnosis: By making the autocorrelation function of the residual sequence, check whether the residual of the model is a white noise sequence. If the residual meets the requirements of the white noise sequence, the model is used as the final prediction model;

[0026] 1-5) Future value prediction: Input historical computing resource usage data to the ARIMA model that completes the estimated parameters to predict future values.

[0027] The LSTM model performs the following steps:

[0028] 2-1) Data preprocessing: normalize or standardize the time series data of historical computing resource usage data; create data windows and convert the time series into a supervised learning problem;

[0029] 2-2) Model construction: Define the LSTM network structure, including the number of layers, number of neurons, and activation function; determine the number of LSTM network layers based on the complexity of the supervised learning problem and the characteristics of the data; start with the number of neurons from common empirical values ​​and then adjust based on the training verification results;

[0030] 2-3) Model training: Use historical computing resource usage data to train the LSTM model, and apply the specified optimizer and loss function;

[0031] 2-4) Model validation and prediction: Evaluate model performance through the validation set, determine the final model, and predict future values ​​through the final trained model.

[0032] The observability analysis module includes: a distributed tracing unit, a performance analysis and flame graph generation unit, and a performance analysis unit;

[0033] Distributed tracing unit, which is used to write eBPF programs to attach to kernel functions or tracepoints to capture performance metric data including function calls, network events, and file I / O operations;

[0034] The eBPF program is loaded into the kernel by using the eBPF loader. The eBPF program runs in the kernel, captures events of interest, and stores the data in the eBPF map. The captured data is associated with the span, which is achieved through traceID and spanID.

[0035] Performance profiling and flame graph generation unit, used to analyze the performance data captured by the eBPF probe and generate performance flame graphs;

[0036] The performance analysis unit aggregates and counts the function call stack information collected by the eBPF probe, and generates a visual flame graph including the sampling frequency, statistical window size, and flame graph generation algorithm parameters.

[0037] The data integration and output module includes: an observability platform, a seamlessly integrated mainstream observability technology stack, and a data conversion and label injection unit;

[0038] The observability platform receives monitoring data including full-stack performance indicators, computing resource demand forecast results, and performance flame graphs from various sources. The monitoring data formats vary. The data is parsed according to the data source and format, converted into a unified internal data model, and then stored in the database.

[0039] Seamless integration of mainstream observability technology stacks: As the storage backend of the observability platform, it provides data interfaces including SQL interface, PeomQL interface, and OpenTelemetry interface;

[0040] The data conversion and label injection unit is used to receive data containing a variety of different labels and dimensional information to eliminate data islands; that is: through the label management module, the data is standardized and a unified label is injected; for each data item, the corresponding label value must be added according to the unified label rule; it also automatically identifies and extracts key labels according to predefined rules and machine learning models, and the labels will be added to the metadata of the data to facilitate subsequent query and analysis; and through hash mapping, custom labels are provided for label key-value pairs, data source identifiers, predefined label rules, and label mapping table parameters, allowing users to customize label rules.

[0041] The observability platform is: a Prometheus platform that integrates support for PromQL queries and an OpenTelemetry platform that supports SQL queries;

[0042] For the Prometheus platform of PromQL query, PromQL query is directly processed by Prometheus query engine;

[0043] For the OpenTelemetry platform for SQL queries, the internal data model is converted into a relational database model and then processed using the SQL engine.

[0044] A method for a data-driven AI computing resource monitoring and prediction system includes the following steps:

[0045] 1) The data collection and preprocessing module monitors various application services, AI services, and infrastructure services in real time, and sends the acquired computing resources to the observability analysis module. At the same time, it performs protocol analysis on the acquired computing resources, collects full-stack performance indicators, and sends them as historical computing resource usage data to the computing demand prediction module;

[0046] 2) The computing power demand prediction module models and predicts the received historical computing power resource usage data through time series analysis methods, and sends the obtained computing power resource demand prediction results to the data integration and output module;

[0047] 3) The observability analysis module performs distributed tracing of various application services, AI services, and infrastructure services, and sends the generated performance flame graph to the data integration and output module;

[0048] 4) Data integration and output module, which is used to collect the full-stack performance indicators in the data collection and preprocessing module, as well as the received computing resource demand prediction results and performance flame charts, and finally provide a basis for resource scheduling by providing multiple data interfaces and injecting unified labels.

[0049] The present invention has the following beneficial effects and advantages:

[0050] 1. Comprehensive monitoring: The present invention uses eBPF technology to achieve zero-intrusion panoramic monitoring with a wide coverage range, including various application services, AI services, and infrastructure services, and can more comprehensively grasp the use of computing resources.

[0051] 2. Comprehensive application of multiple advanced technologies: This invention integrates eBPF technology, ARIMA and LSTM time series analysis methods, distributed tracing, and mainstream observability technology stacks to achieve comprehensive monitoring, accurate prediction, and efficient management of AI computing resources. This method of comprehensive application of multiple technologies is relatively rare in existing technologies and is innovative.

[0052] 3. Strong protocol parsing capability: The present invention automatically parses standard protocols and private protocols, collects full-stack performance indicators, helps to quickly locate performance bottlenecks, and improves the efficiency of troubleshooting.

[0053] 4. Accurate prediction: This invention adopts time series analysis methods such as ARIMA and LSTM to model and predict historical computing resource usage data, combining the advantages of different models. This multi-model fusion method gives full play to the advantages of different models and adapts to different types of time series data to provide more reliable real-time data support for resource scheduling.

[0054] 5. Powerful observability: The present invention implements zero-intrusive distributed tracing based on eBPF, collects full-stack performance indicators and automatically associates them with distributed tracing data. It can also provide continuous performance analysis and generate function-level performance flame graphs, which helps to quickly locate performance bottlenecks.

[0055] 6. Seamless integration: The present invention can seamlessly integrate with mainstream observability technology stacks, serve as the storage backend of Prometheus, OpenTelemetry, etc., provide multiple data interfaces and inject unified labels, eliminate data silos, and facilitate integration with existing systems.

[0056] 7. Non-intrusive monitoring and distributed tracing: The combination of the non-intrusive nature of the eBPF technology of the present invention and distributed tracing enables the present invention to provide detailed system performance information without affecting system performance. This is very important for AI application scenarios with high performance requirements.

[0057] 8. Unified labels: The present invention automatically injects unified labels into the observation data, eliminating data silos and improving the availability and value of data. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 System framework diagram of the data-driven AI computing resource monitoring and prediction system of the present invention;

[0059] Figure 2 A timing diagram of the data-driven AI computing resource monitoring and prediction method of the present invention;

[0060] Figure 3 Schematic diagram of the principles of the data-driven AI computing resource monitoring and prediction system of the present invention. DETAILED DESCRIPTION

[0061] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0062] In the production, sales, and management of the tobacco industry, more and more AI technologies are being applied, such as quality inspection, smart warehousing, market forecasting, etc. Effective management of AI computing resources is crucial.

[0063] The technical solutions adopted by the present invention mainly involve the following sub-fields:

[0064] 1. System monitoring technology: Use eBPF technology to achieve panoramic monitoring of various application services, AI services, and infrastructure services, collect full-stack performance indicators, and automatically parse protocols to locate performance bottlenecks.

[0065] Real-time monitoring: Through eBPF technology, AI services and related infrastructure services involved in various equipment on the tobacco production line (such as automated packaging machines, quality inspection equipment, etc.) are monitored in a panoramic manner. The communication protocols between devices are automatically parsed, and full-stack performance indicators including equipment operating status, network traffic, and computing resource usage are collected to promptly identify possible performance bottlenecks and ensure stable production.

[0066] 2. Data prediction technology: ARIMA and LSTM in time series analysis methods are used to model and predict historical computing resource usage data to provide support for resource scheduling.

[0067] Demand forecasting: ARIMA and LSTM are used to model and forecast historical computing resource usage data. For example, based on the demand for AI computing power in different production stages in the past, the computing power resources required for peak season production are forecasted, and resources are allocated in advance to avoid production progress being affected by insufficient computing power.

[0068] 3. Observability technology: Based on eBPF, it implements zero-intrusion distributed tracing, collects full-stack performance indicators and automatically associates them with distributed tracing data, provides continuous performance analysis and generates performance flame graphs, and seamlessly integrates the mainstream observability technology stack, provides multiple data interfaces, and injects unified tags to eliminate data silos.

[0069] Observability and performance analysis: Distributed tracing is implemented based on eBPF, covering various applications from tobacco planting management systems to sales data analysis platforms. Full-stack performance indicators are collected and automatically associated with distributed tracing data, and function-level performance flame graphs are generated to quickly locate possible performance problems, such as performance bottlenecks when a data analysis algorithm processes large amounts of data. At the same time, continuous performance analysis helps optimize algorithms and system configurations and improve overall efficiency. Seamless integration: Seamlessly integrates with existing monitoring systems in the tobacco industry. As the storage backend of Prometheus, OpenTelemetry, etc., it provides a variety of data interfaces to facilitate the integration and analysis of AI computing resource monitoring data and other production data. Automatically inject unified labels into observation data to eliminate data silos and achieve comprehensive monitoring and optimized management of the entire tobacco production supply chain.

[0070] like Figure 1 and Figure 3 As shown, it is a system framework diagram of the present invention. The present invention is a data-driven AI computing resource monitoring and prediction system, including: a data acquisition and preprocessing module, a computing demand prediction module, an observability analysis module, and a data integration and output module;

[0071] (1) Data collection and preprocessing module, which is used to monitor various application services, AI services, and infrastructure services in real time. The acquired computing resources are sent to the observability analysis module. At the same time, it performs protocol analysis on the acquired computing resources, collects full-stack performance indicators, and sends them as historical computing resource usage data to the computing demand prediction module;

[0072] Data acquisition and preprocessing module, including: real-time monitoring unit, protocol analysis unit, performance index acquisition unit;

[0073] Real-time monitoring of AI computing resource usage: eBPF is used to achieve zero-intrusion panoramic monitoring of various application services, AI services, and infrastructure services. eBPF technology allows user-defined programs to run in the operating system kernel, and can monitor the system without modifying the kernel code, thereby avoiding interference with the monitored object. Automatically parse standard protocols and private protocols, and collect full-stack performance indicators. By parsing different protocols, more comprehensive system performance information can be obtained, which helps to quickly locate performance bottlenecks. For example, for deep learning frameworks in AI services, specific private protocols are used for communication, and these protocols are parsed to understand the progress of model training and resource usage.

[0074] eBPF Programs and Data Collection: Contains probes for capturing performance metrics, kprobes (kernel functions), uprobes (user space functions), tracepoints (trace points). Load the program into the kernel. Ensure that the eBPF program passes the security check of the Verifier. The eBPF program runs in the kernel and captures performance data such as CPU cycles, memory usage, network packets, file I / O operations, etc. The captured data is stored in eBPF maps, which share data between user space and kernel space. User space programs periodically read data from the eBPF maps. Format the data for further analysis and visualization.

[0075] Protocol parsing: Use eBPF program to capture network packets. Identify the protocol used based on the characteristics of the packet (such as port number, protocol header, etc.). For standard protocols (such as HTTP, TCP, etc.), use predefined parsing rules. For private protocols, use a custom parser to parse the data according to the protocol specification. Extract performance-related indicators from the parsed protocol data, such as response time, transmission size, error rate, etc. Implement the protocol identification logic in the eBPF program. Implement the corresponding parsing function according to the protocol type. The parser requires the following parameters: Port number is used to identify a specific service. Protocol specification is used to guide the parsing of private protocols. Performance indicator definition, clearly specify which performance indicators need to be extracted.

[0076] eBPF Example:

[0077] / / Capture TCP packets

[0078] SEC("socket_filter")

[0079] int bpf_prog(struct__sk_buff skb){

[0080] / / ...parse the data in skb and extract performance indicators...

[0081] return 0; / / Continue processing data packets

[0082] }

[0083] User space programs

[0084] from bcc import BPF

[0085] bpf=BPF(src_file="bpf_prog.c")

[0086] ...load the eBPF program...

[0087] Read data from the mapping table

[0088] def print_event(cpu,data,size):

[0089] event=bpf["events"].event(data)

[0090] print(event)

[0091] bpf["events"].open_perf_buffer(print_event)

[0092] while 1:

[0093] bpf.kprobe_poll()

[0094] (2) A computing power demand prediction module, which is used to model and predict the received historical computing power resource usage data through a time series analysis method, and send the obtained computing power resource demand prediction results to the data integration and output module;

[0095] Demand forecasting: ARIMA (autoregressive integrated moving average model) and LSTM (long short-term memory network) in time series analysis methods are used to model and forecast historical computing resource usage data. ARIMA is used to forecast stable time series data, and LSTM processes time series data with long-term dependencies. These two methods will be combined to improve the accuracy of forecasts. Provide real-time data support for resource scheduling and improve resource utilization efficiency. By accurately predicting future computing resource needs, resources can be allocated in advance to avoid resource waste and performance bottlenecks. Specific implementation details:

[0096] (2-1) ARIMA algorithm process: Data preprocessing to ensure that the time series data is regular and has no missing values. Differentiate the data to make it stable.

[0097] Model Identification: ACF (autocorrelation function) and PACF (partial autocorrelation function) are used to determine the parameters p (number of autoregressive terms), d (order of difference), and q (number of moving average terms) of the ARIMA model.

[0098] Model estimation: Estimate the parameters of the ARIMA model using maximum likelihood estimation or other optimization methods.

[0099] Use the established ARIMA model to predict data and determine whether the error between the original data and the predicted data is less than the preset range. If the error distribution is greater than the preset range, the model structure identification and parameter estimation need to be re-performed;

[0100] Model diagnosis: Check whether the model residual is white noise to ensure the effectiveness of the model.

[0101] Forecasting: Use the estimated model to predict future values.

[0102] ARIMA Implementation:

[0103] from statsmodels.tsa.arima.model import ARIMA

[0104] Assume data is a Pandas Series object containing time series data

[0105] model=ARIMA(data,order=(p,d,q))

[0106] model_fit = model.fit()

[0107] prediction = model_fit.forecast (steps = 5) predicts the value of the next 5 time points

[0108] Parameters: p: number of autoregressive terms, d: order of difference, q: number of moving average terms, steps: number of future time points to predict.

[0109] (2-2) LSTM algorithm flow: data preprocessing, normalizing or standardizing time series data. Creating a data window and converting the time series into a supervised learning problem.

[0110] Model construction: Define the LSTM network structure, including the number of layers, number of neurons, and activation function. The number of LSTM network layers is determined based on the complexity of the problem and the characteristics of the data. For simple problems, one or two layers are sufficient, and for complex problems, the number of layers can be increased appropriately. For example, when predicting computing resource usage data with obvious seasonality and trend changes; if the number of neurons is too small, it will be underfitting. Start with common empirical values, such as 32, 64, 128, and then adjust according to the training verification results.

[0111] Model training: Train the LSTM model using historical data. Apply appropriate optimizers and loss functions.

[0112] Model Validation: Evaluate model performance using a validation set.

[0113] Prediction: Use the trained model to predict future values.

[0114] LSTM Implementation:

[0115] from keras.models import Sequential

[0116] from keras.layers import LSTM,Dense

[0117] Assume X_train and y_train are preprocessed data

[0118] model=Sequential()

[0119] model.add(LSTM(units=50,return_sequences=True,input_shape=(X_train.shape[1],1)))

[0120] model.add(LSTM(units=50))

[0121] model.add(Dense(1))

[0122] model.compile(optimizer='adam',loss='mean_squared_error')

[0123] model.fit(X_train,y_train,epochs=100,batch_size=32)

[0124] Predicting future values

[0125] predictions=model.predict(X_test)

[0126] Parameters: units: the number of neurons in the LSTM layer, return_sequences: whether to return all outputs in the sequence, or only the last output, input_shape: the shape of the input data, epochs: the number of iterations to train the model, batch_size: the number of samples for each training.

[0127] By combining ARIMA and LSTM, a hybrid model is constructed, using ARIMA to handle the seasonality and trend of the data, and the residual is input into the LSTM model to capture the nonlinear characteristics of the time series data. Combining the advantages of both, the accuracy and robustness of the forecast are improved.

[0128] (3) Observability analysis module, which is used to perform distributed tracing of various application services, AI services, and infrastructure services, and send the generated performance flame graph to the data integration and output module;

[0129] Observability: Implement non-intrusive distributed tracing based on eBPF. Distributed tracing helps understand the interactions between components in the system and locate performance issues. The non-intrusive nature of eBPF technology ensures that the tracing process does not have a negative impact on system performance. Full-stack performance indicators such as network and file I / O are collected and automatically associated with distributed tracing data. Comprehensive performance indicator collection provides a more detailed view of system performance, and automatic association with distributed tracing data helps to quickly locate problems.

[0130] Provides continuous performance analysis and generates function-level performance flame graphs to locate performance bottlenecks. Performance analysis provides an in-depth understanding of the performance characteristics of the system, and the flame graph displays the function call relationship and resource consumption in an intuitive way, helping developers quickly locate performance bottlenecks. In this way, eBPF technology provides an efficient and extremely low-intrusive method to monitor and analyze system performance, especially when in-depth understanding of application and infrastructure performance is required.

[0131] eBPF is used for distributed tracing and performance profiling: eBPF programs are written to attach to kernel functions or tracepoints to capture trace data such as function calls, network events, file I / O operations, etc.

[0132] Loading eBPF programs: Use an eBPF loader (e.g., bpftrace, BCC) to load eBPF programs into the kernel. The eBPF program runs in the kernel, captures events of interest, and stores the data in the eBPF map. Associate the captured data with spans in the distributed tracing system, usually through traceID and spanID. Performance profiling: Analyze the captured data and generate performance profiling reports, such as flame graphs. Visualization: Visualize the profiling results so that developers can quickly identify performance bottlenecks.

[0133] LSTM Implementation:

[0134] from keras.models import Sequential

[0135] from keras.layers import LSTM,Dense

[0136] Assume X_train and y_train are preprocessed data

[0137] model=Sequential()

[0138] model.add(LSTM(units=50,return_sequences=True,input_shape=(X_train.shape[1],1)))

[0139] model.add(LSTM(units=50))

[0140] model.add(Dense(1))

[0141] model.compile(optimizer='adam',loss='mean_squared_error')

[0142] model.fit(X_train, y_train, epochs=100, batch_size=32)

[0143] Predicting future values

[0144] predictions=model.predict(X_test)

[0145] Parameters: units: the number of neurons in the LSTM layer, return_sequences: whether to return all outputs in the sequence, or just the last output. input_shape: the shape of the input data, epochs: the number of iterations to train the model, batch_size: the number of samples for each training.

[0146] Processing steps: Select the corresponding eBPF program according to the event type to be monitored. Load eBPF: Use the BCC tool to load the program into the kernel. The eBPF program runs in the kernel and captures event data. Associate the captured data with the span of the distributed tracing system. Analyze the data and generate a performance analysis report. Use tools such as FlameGraph to convert the analysis results into a flame graph.

[0147] (4) Data integration and output module, which is used to collect the full-stack performance indicators in the data collection and preprocessing module, as well as the received computing resource demand prediction results and performance flame charts, and ultimately provide a basis for resource scheduling by providing multiple data interfaces and injecting unified labels.

[0148] Data integration and output module, including: observability platform, seamless integration of mainstream observability technology stack, and data conversion and label injection unit;

[0149] (4-1) The observability platform receives monitoring data including full-stack performance indicators, computing resource demand forecast results, and performance flame graphs from various sources. The monitoring data formats vary. The platform parses the data according to the data source and format, converts it into a unified internal data model, and then stores the data in the database.

[0150] (4-2) Seamless integration of mainstream observability technology stacks: As the storage backend of Prometheus, OpenTelemetry, etc., it provides SQL, PromQL, and OTLP data interfaces. This allows the present invention to be integrated with existing observability tools, making it easier for users to query and analyze data. Unified labels are automatically injected into observed data to eliminate data silos. Unified labels make it easier to associate and analyze data from different sources, improving the availability and value of data.

[0151] Data query and interface: Provides multiple data query interfaces to support the needs of different observability tools. For Prometheus, supports PromQL query; for OpenTelemetry, supports OTLP protocol; for general data analysis, supports SQL query. For example, PromQL query directly uses Prometheus query engine for processing; SQL query converts the internal data model into the relational database model, and then uses the SQL engine for processing.

[0152] Data reception and storage: Receive monitoring data from multiple sources such as eBPF probes, Prometheus exporter, and OpenTelemetryagent. These data formats vary, and the data reception and conversion module parses them according to the data source and format, and converts them into a unified internal data model. After that, the data is stored in the database. Parameters include database connection (address, port, user name, etc.), data reception buffer size, data conversion rules, etc.

[0153] (4-3) Data conversion and label injection unit: The received data contains a variety of different labels and dimensional information. In order to eliminate data silos, the label management module is used to standardize the data and inject unified labels. For each data item, the corresponding label value must be added according to the unified label rules. If the data source is an AI service, fill in the name of the AI ​​service in the "Data Source" label; if it is network performance indicator data, fill in "Network Performance Indicator" in the "Data Type" label. This module automatically identifies and extracts key labels, such as service name, instance ID, environment type, etc., based on predefined rules and machine learning models. Labels will be added to the metadata of the data to facilitate subsequent query and analysis. Custom labels are provided to allow users to customize label rules. Hash mapping is used to map label key-value pairs, data source identifiers, predefined label rules, and label mapping table parameters.

[0154] like Figure 2 As shown, it is a timing diagram of the data-driven AI computing resource monitoring and prediction method of the present invention, and the present invention includes the following steps:

[0155] 1) The data collection and preprocessing module monitors various application services, AI services, and infrastructure services in real time, and sends the acquired computing resources to the observability analysis module. At the same time, it performs protocol analysis on the acquired computing resources, collects full-stack performance indicators, and sends them as historical computing resource usage data to the computing demand prediction module;

[0156] 2) The computing power demand prediction module models and predicts the received historical computing power resource usage data through time series analysis methods, and sends the obtained computing power resource demand prediction results to the data integration and output module;

[0157] 3) The observability analysis module performs distributed tracing of various application services, AI services, and infrastructure services, and sends the generated performance flame graph to the data integration and output module;

[0158] 4) Data integration and output module, which is used to collect the full-stack performance indicators in the data collection and preprocessing module, as well as the received computing resource demand prediction results and performance flame charts, and finally provide a basis for resource scheduling by providing multiple data interfaces and injecting unified labels.

[0159] In summary, the present invention overcomes these shortcomings in the following ways:

[0160] Zero-intrusive panoramic monitoring: The eBPF technology of the present invention realizes zero-intrusive real-time monitoring, comprehensively covers all services, and eliminates monitoring blind spots.

[0161] Automatic protocol analysis: Automatically analyze standard and private protocols and collect full-stack performance indicators without manual intervention.

[0162] Accurate prediction model: ARIMA and LSTM models improve the accuracy of computing resource demand prediction and reduce resource waste.

[0163] Enhanced observability: eBPF-driven distributed tracing and continuous performance profiling provide function-level performance flame graphs to quickly locate performance bottlenecks.

[0164] Seamless integration: Seamlessly integrates with mainstream observability technology stacks such as Prometheus and OpenTelemetry to eliminate data silos and provide a unified view.

[0165] Application examples and effects in the government field: A government department deployed a large-scale AI system for public services, such as intelligent traffic management and urban security monitoring.

[0166] Before application: Due to the lack of effective monitoring and prediction, computing resource utilization is low and there is resource waste. Performance problems are difficult to locate: When performance problems occur, it is difficult to quickly locate the cause, resulting in service interruption or slow response. High operation and maintenance costs: A large amount of manpower is required for monitoring and maintenance, resulting in high operation and maintenance costs.

[0167] After application: Real-time monitoring and accurate prediction can optimize resource scheduling, improve resource utilization, reduce operating costs, and increase resource utilization by 10% to 30% or even higher. Full-stack performance indicators and distributed tracing can quickly locate performance bottlenecks and shorten fault recovery time, from days to hours or even higher. Automated monitoring and analysis reduce the workload of operation and maintenance personnel and improve operation and maintenance efficiency. Data-driven decision-making: Based on monitoring and prediction data, government departments make more scientific resource planning and decisions.

Claims

1. A data-driven AI computing resource monitoring and prediction system, characterized in that: include: Data collection and preprocessing module, computing power demand prediction module, observability analysis module, and data integration and output module; The data collection and preprocessing module is used to monitor various application services, AI services, and infrastructure services in real time. The acquired computing resources are sent to the observability analysis module. At the same time, it performs protocol analysis on the acquired computing resources, collects full-stack performance indicators, and sends them as historical computing resource usage data to the computing demand prediction module. The computing power demand prediction module is used to model and predict the received historical computing power resource usage data through time series analysis methods, and send the obtained computing power resource demand prediction results to the data integration and output module; The observability analysis module is used to perform distributed tracing of various application services, AI services, and infrastructure services, and send the generated performance flame graph to the data integration and output module; The data integration and output module is used to collect the full-stack performance indicators in the data collection and preprocessing module, as well as the received computing resource demand prediction results and performance flame charts, and ultimately provide a basis for resource scheduling by providing multiple data interfaces and injecting unified labels.

2. According to the data-driven AI computing resource monitoring and prediction system of claim 1, it is characterized in that: The data acquisition and preprocessing module includes: a real-time monitoring unit, a protocol parsing unit, and a performance indicator acquisition unit; The real-time monitoring unit uses eBPF to implement zero-intrusion panoramic monitoring of various application services, AI services, and infrastructure services, and is used to obtain computing resources for various services; the captured computing resource data is stored in the eBPF mapping table, the mapping table shares data between the user space and the kernel space, and the data is regularly read from the eBPF mapping table through the user space function, and the data is formatted for further call analysis and visualization of the protocol parsing unit and the observability analysis module; The protocol parsing unit is used to capture network data packets containing computing resource data through the eBPF program, and identify the type of protocol used based on the characteristics of the data packets; for standard protocols, predefined parsing rules are used; for private protocols, custom parsers are used to parse data according to protocol specifications; performance-related indicators, i.e. full-stack performance indicators, are extracted from the parsed protocol data; The performance indicator acquisition unit is used to obtain the full-stack performance indicators extracted by the protocol parsing unit and send them to the computing power demand prediction module as historical computing power resource usage data.

3. According to the data-driven AI computing resource monitoring and prediction system of claim 1, it is characterized in that: The computing power demand prediction module includes: an ARIMA model for predicting stable time series data and an LSTM model for processing time series data with long-term dependencies.

4. According to claim 3, a data-driven AI computing resource monitoring and prediction system is characterized in that: The ARIMA model performs the following steps: 1-1) Data preprocessing: Ensure that the time series data of historical computing resource usage data is regular and has no missing values; perform differential analysis on the historical computing resource usage data to make it stable; 1-2) Model identification: Use the autocorrelation function and partial autocorrelation function to determine the parameters of the ARIMA model, that is, determine the number of autoregressive terms p, the difference order d, and the number of moving average terms q; 1-3) Model estimation: The maximum likelihood estimation method is used to estimate the parameters of the ARIMA model; Use the established ARIMA model to predict data and determine whether the error between the original data and the predicted data is less than the preset range. If the error distribution is greater than the preset range, the model structure identification and parameter estimation need to be re-performed; 1-4) Model diagnosis: By making the autocorrelation function of the residual sequence, check whether the residual of the model is a white noise sequence. If the residual meets the requirements of the white noise sequence, the model is used as the final prediction model; 1-5) Future value prediction: Input historical computing resource usage data to the ARIMA model that completes the estimated parameters to predict future values.

5. According to claim 3, a data-driven AI computing resource monitoring and prediction system is characterized in that: The LSTM model performs the following steps: 2-1) Data preprocessing: normalize or standardize the time series data of historical computing resource usage data; create data windows and convert the time series into a supervised learning problem; 2-2) Model construction: Define the LSTM network structure, including the number of layers, number of neurons, and activation function; determine the number of LSTM network layers based on the complexity of the supervised learning problem and the characteristics of the data; start with the number of neurons from common empirical values ​​and then adjust based on the training verification results; 2-3) Model training: Use historical computing resource usage data to train the LSTM model, and apply the specified optimizer and loss function; 2-4) Model validation and prediction: Evaluate model performance through the validation set, determine the final model, and predict future values ​​through the final trained model.

6. The data-driven AI computing resource monitoring and prediction system according to claim 1 is characterized in that: The observability analysis module includes: a distributed tracing unit, a performance analysis and flame graph generation unit, and a performance analysis unit; Distributed tracing unit, which is used to write eBPF programs to attach to kernel functions or tracepoints to capture performance metric data including function calls, network events, and file I / O operations; The eBPF program is loaded into the kernel by using the eBPF loader. The eBPF program runs in the kernel, captures events of interest, and stores the data in the eBPF map. The captured data is associated with the span, which is achieved through traceID and spanID. Performance profiling and flame graph generation unit, used to analyze the performance data captured by the eBPF probe and generate performance flame graphs; The performance analysis unit aggregates and counts the function call stack information collected by the eBPF probe, and generates a visual flame graph including the sampling frequency, statistical window size, and flame graph generation algorithm parameters.

7. The data-driven AI computing resource monitoring and prediction system according to claim 1 is characterized in that: The data integration and output module includes: an observability platform, a seamlessly integrated mainstream observability technology stack, and a data conversion and label injection unit; The observability platform receives monitoring data including full-stack performance indicators, computing resource demand forecast results, and performance flame graphs from various sources. The monitoring data formats vary. The data is parsed according to the data source and format, converted into a unified internal data model, and then stored in the database. Seamless integration of mainstream observability technology stacks: As the storage backend of the observability platform, it provides data interfaces including SQL interface, PeomQL interface, and OpenTelemetry interface; The data conversion and label injection unit is used to receive data containing a variety of different labels and dimensional information to eliminate data islands; that is: through the label management module, the data is standardized and a unified label is injected; for each data item, the corresponding label value must be added according to the unified label rule; it also automatically identifies and extracts key labels according to predefined rules and machine learning models, and the labels will be added to the metadata of the data to facilitate subsequent query and analysis; and through hash mapping, custom labels are provided for label key-value pairs, data source identifiers, predefined label rules, and label mapping table parameters, allowing users to customize label rules.

8. The data-driven AI computing resource monitoring and prediction system according to claim 7 is characterized in that: The observability platform is: a Prometheus platform that integrates support for PromQL queries and an OpenTelemetry platform that supports SQL queries; For the Prometheus platform of PromQL query, PromQL query is directly processed by Prometheus query engine; For the OpenTelemetry platform for SQL queries, the internal data model is converted into a relational database model and then processed using the SQL engine.

9. The method of a data-driven AI computing resource monitoring and prediction system according to claim 1, characterized in that: The following steps are involved: 1) The data collection and preprocessing module monitors various application services, AI services, and infrastructure services in real time, and sends the acquired computing resources to the observability analysis module. At the same time, it performs protocol analysis on the acquired computing resources, collects full-stack performance indicators, and sends them as historical computing resource usage data to the computing demand prediction module; 2) The computing power demand prediction module models and predicts the received historical computing power resource usage data through time series analysis methods, and sends the obtained computing power resource demand prediction results to the data integration and output module; 3) The observability analysis module performs distributed tracing of various application services, AI services, and infrastructure services, and sends the generated performance flame graph to the data integration and output module; 4) Data integration and output module, which is used to collect the full-stack performance indicators in the data collection and preprocessing module, as well as the received computing resource demand prediction results and performance flame charts, and finally provide a basis for resource scheduling by providing multiple data interfaces and injecting unified labels.

Images