Internet application performance monitoring and optimizing system based on artificial intelligence

Abstract

The invention discloses an Internet application performance monitoring and optimizing system based on artificial intelligence, and the system is characterized in that a data collection module is used for collecting data streams of a plurality of data sources, determining the priority of each data source based on a preset rule, and dynamically configuring the collection frequency of the data streams based on the priority of each data source; collecting the data flow of each data source according to the collection frequency; the exception monitoring module is connected with the data preprocessing module and used for monitoring the preprocessed data flow according to a machine learning model to obtain an exception mode; the implementation plan generation module is connected with the abnormity monitoring module and is used for generating an implementation plan based on the abnormal mode obtained through monitoring. The dynamically configured acquisition strategy effectively solves the problem of resource waste or information omission caused by the traditional static acquisition frequency: the acquisition frequency is improved in a high fluctuation period to enhance the sensitivity of anomaly detection, the frequency is reduced in a stationary period to save bandwidth, storage and computing resources, and refined and intelligent data acquisition management is realized.

Description

Technical Field

[0001] This invention relates to the field of monitoring and optimization technology, and in particular to an artificial intelligence-based Internet application performance monitoring and optimization system. Background Technology

[0002] As internet applications become increasingly complex and large-scale, the need for system performance monitoring and optimization has become more urgent. Traditional performance monitoring methods typically rely on static rules and fixed-frequency data collection mechanisms, which are ill-suited to dynamically changing traffic patterns and sudden anomalies. This static strategy often leads to wasted resources (such as over-collection during periods of low volatility) or detection delays (such as insufficient collection during periods of high volatility), thereby affecting the system's real-time responsiveness and stability.

[0003] In recent years, artificial intelligence (AI) technologies, especially machine learning models, have demonstrated tremendous potential in anomaly detection, predictive analytics, and automated decision-making. AI-based monitoring systems can significantly improve the accuracy and response speed of anomaly identification through adaptive data acquisition, feature engineering, and dynamic model training. However, existing solutions still have limitations in efficiently incorporating historical data fluctuation characteristics, dynamically adjusting acquisition frequency, and generating executable optimization plans. For example, traditional systems typically set the acquisition frequency to a fixed value, failing to dynamically adjust it based on actual traffic fluctuations. This can lead to insufficient data support at critical points in time and waste of unnecessary computational resources during stable periods. Furthermore, existing anomaly detection models are often trained solely on raw data streams, failing to fully utilize higher-order statistical features that reflect the intensity and persistence of data fluctuations. These features reduce the model's ability to understand anomaly patterns, lower prediction confidence, and increase false positives and false negatives.

[0004] Therefore, there is an urgent need for a technical solution for monitoring and optimizing the performance of Internet applications based on artificial intelligence. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides an artificial intelligence-based internet application performance monitoring and optimization system, specifically comprising: Data Acquisition Module: Collects data streams from multiple data sources, determines the priority of each data source based on preset rules, dynamically configures the data stream acquisition frequency based on the data source priority, and collects data streams from each data source according to the acquisition frequency. Data Preprocessing Module: Connected to the data acquisition module, it performs preprocessing operations on the acquired data streams. Anomaly Monitoring Module: Connected to the data preprocessing module, it monitors the preprocessed data streams using a machine learning model to identify anomaly patterns. Implementation Plan Generation Module: Connected to the anomaly monitoring module, it generates an implementation plan based on the detected anomaly patterns. Plan Optimization Module: Connected to the implementation plan generation module, it optimizes the implementation plan to obtain an optimized plan. Implementation Plan Execution Submodule: Executes the implementation plan. Key Performance Indicator (KPI) Acquisition Submodule: Collects key performance indicators (KPIs) before and after the implementation plan execution. Key Performance Indicator Difference Acquisition Submodule: Obtains the KPI difference based on the collected KPIs before and after the implementation plan execution. The comprehensive performance improvement index calculation submodule calculates the mean and standard deviation of key performance indicator differences, and then uses these to calculate the comprehensive performance improvement index. The difference level determination submodule presets a target judgment threshold for the comprehensive performance improvement index and compares the index with this threshold to determine the difference level. The plan optimization submodule optimizes the implementation plan based on the difference level to obtain an optimized plan.

[0006] Furthermore, the data acquisition module includes the following sub-modules: a data stream collection sub-module for collecting data streams from multiple data sources; a data source priority determination sub-module for determining the priority of each data source based on preset rules; a collection frequency configuration sub-module for dynamically configuring the collection frequency of the data streams according to the priority of each data source; and a data stream acquisition sub-module for acquiring data streams from each data source according to the collection frequency.

[0007] Furthermore, the frequency acquisition configuration submodule includes the following units: a historical traffic fluctuation data acquisition unit, used to determine historical traffic fluctuation data based on the data stream of each data source; a segmentation and statistics unit, used to define a first time window and a second time window, and to divide the historical traffic fluctuation data into several statistical data sets based on the first time window and the second time window; the first time window is divided into several time intervals through the second time window, and the variance of the statistical data sets is obtained; a fluctuation energy density index definition unit, used to define a fluctuation energy density index based on the statistical data sets in each time interval; and a fluctuation persistence index definition unit, used to preset an initial threshold for the variance, extract the number of consecutive times the variance is greater than the initial threshold in the time intervals within the first time window, and if the number of consecutive times is greater than or equal to 2 / 3 of the number of time intervals, then a fluctuation persistence index is defined based on the statistical data sets in the time intervals. If the number of consecutive occurrences is less than 2 / 3 of the number of time intervals, the current acquisition frequency is maintained. The acquisition frequency judgment unit is used to preset a first judgment threshold for the volatility persistence index and define a second judgment threshold for the volatility energy density index. It judges whether the volatility persistence index is greater than the first judgment threshold and the volatility energy density index is greater than the second judgment threshold. If so, it enters high-frequency acquisition mode; otherwise, it maintains the current acquisition frequency. The acquisition frequency adjustment unit is used to obtain the volatility energy density growth rate within adjacent time intervals in the first time window, preset a third judgment threshold for the volatility energy density growth rate, and judge whether the volatility energy density growth rate is greater than the third judgment threshold. If so, it configures the first acquisition frequency based on the volatility energy density growth rate; otherwise, it configures the second acquisition frequency based on the statistical data set of the n time intervals preceding the current time interval. The first acquisition frequency is greater than the second acquisition frequency.

[0008] Furthermore, the anomaly monitoring module includes the following sub-modules: A training set acquisition sub-module: used to use data streams from various data sources, historical traffic fluctuation data, fluctuation persistence indicators, and fluctuation persistence metrics as the training set, and to label the data streams from each data source as normal / abnormal modes. A machine learning model: used to input the training set into the machine learning model for training, obtaining the trained machine learning model. A monitoring sub-module: used to collect data streams from various data sources according to the collection frequency, determine the real-time fluctuation persistence indicators and real-time fluctuation persistence metrics of each data source, and input the data streams from each data source, the real-time fluctuation persistence indicators, and real-time fluctuation persistence metrics of each data source into the trained machine learning model to obtain the normal / abnormal mode.

[0009] Furthermore, an implementation plan is generated based on the abnormal patterns detected, which mainly includes: based on the abnormal patterns detected, calling a predefined problem strategy mapping table, matching historical problem cases, and extracting the repair strategies and implementation paths of historical problem cases; combining the system architecture topology diagram, automatically generating an implementation plan, including service restart sequence, configuration file modification instructions and dependency check logic; and encapsulating the generated operation script into an executable package, and marking the execution risk level and rollback contingency plan.

[0010] Furthermore, the configuration process for the first sampling frequency is as follows: , The first sampling frequency, This is the gain coefficient. This represents the growth rate of fluctuating energy density.

[0011] Furthermore, the third judgment threshold is divided into a first sub-threshold and a second sub-threshold. If the fluctuation energy density growth rate is less than the first sub-threshold, the gain coefficient is assigned the first coefficient p1. If the fluctuation energy density growth rate is greater than or equal to the first sub-threshold and less than the second sub-threshold, the gain coefficient is assigned the second coefficient p2. If the fluctuation energy density growth rate is greater than the second sub-threshold, the gain coefficient is assigned the third coefficient p3. <p2<p3。

[0012] Furthermore, the historical traffic data within the first time interval of the first time window is used as the initial data to calculate the growth rate of the historical traffic data within the time interval, and the second collection frequency is obtained based on the growth rate and the initial data.

[0013] The embodiments of the present invention have the following technical effects: This application proposes an Internet application performance monitoring and optimization system based on artificial intelligence. By introducing the fluctuation energy density index and the fluctuation persistence index, a dynamic acquisition frequency configuration mechanism for different data sources is realized. Combined with machine learning models, abnormal pattern recognition, implementation plan generation, and execution effect evaluation and optimization are performed, thereby significantly improving the system's adaptive capability, resource utilization efficiency, and operation and maintenance decision quality.

[0014] First, a fluctuation energy density index is defined based on historical traffic fluctuation data to quantify the intensity of data fluctuations per unit time. Then, a fluctuation persistence index is extracted by determining whether the variance consistently exceeds a set threshold, reflecting the stability of the fluctuations. Based on this, a multi-level logical judgment process is constructed by combining multiple judgment thresholds to dynamically decide whether to enter a high-frequency acquisition mode or maintain the current acquisition frequency. This allows for the configuration of an acquisition frequency for each data source that matches its actual changing characteristics. This differentiated acquisition strategy effectively solves the resource waste or information omission problems caused by traditional static acquisition frequencies: increasing the acquisition frequency during periods of high fluctuation enhances the sensitivity of anomaly detection, while reducing the frequency during stable periods saves bandwidth, storage, and computing resources, achieving refined and intelligent data acquisition management.

[0015] Furthermore, the system uses the collected data stream, fluctuation energy density index, and fluctuation persistence index as training sets to train the machine learning model. This enables the model to learn higher-order statistical information reflecting the dynamic characteristics of the data, thereby significantly improving the accuracy and robustness of anomaly identification. Compared to traditional methods that rely solely on raw data streams, the features introduced in this approach enhance the model's understanding of abnormal behavior, reduce false positive and false negative rates, and improve the prediction confidence of the overall monitoring system. Attached Figure Description

[0016] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is a structural diagram of an artificial intelligence-based Internet application performance monitoring and optimization system provided in an embodiment of the present invention; Figure 2 This is a flowchart of the acquisition frequency configuration submodule of an artificial intelligence-based Internet application performance monitoring and optimization system provided in this embodiment of the invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0019] Example 1: As Figure 1 As shown, this invention provides an artificial intelligence-based internet application performance monitoring and optimization system, comprising the following modules: Data acquisition module: Used to collect data streams from multiple data sources, determine the priority of each data source based on preset rules, dynamically configure the data stream acquisition frequency based on the priority of each data source, and collect data streams from each data source according to the acquisition frequency.

[0020] The data acquisition module collects data streams from multiple data sources, which are historical data streams. It dynamically configures the data stream acquisition frequency based on the historical data streams and acquires real-time data streams based on the acquisition frequency.

[0021] Data Source Priority Determination Submodule: Used to determine the priority of each data source based on preset rules.

[0022] Prioritizing data sources is a crucial step in the entire data acquisition process. This step directly impacts which data will be processed first and how resources are allocated. To this end, the system has set preset rules to dynamically assess the importance of each data source and assign it a corresponding priority. These preset rules are primarily based on the following considerations: Business Impact: Historical data analysis shows that certain types of data have a more significant impact on the overall business process. For example, API call records from core transaction systems typically have a high business impact because they directly relate to user experience and business revenue. Failure Probability: By learning from past failure patterns, data sources more prone to anomalies are identified. For example, web servers frequently subjected to DDoS attacks often require higher attention to their data streams. Emergency Response Needs: Some data sources may be involved in security compliance checks; this type of data usually requires immediate attention and rapid action. For example, log files related to personal privacy protection fall into this category.

[0023] Data update frequency: Considering the differences in update speed among different data sources, data sources that change drastically and are highly time-sensitive should be given higher priority to ensure that any potential problems are captured promptly. By comprehensively considering the above factors, the system can score each data source and generate a final priority list based on the scores. This mechanism not only helps improve resource utilization efficiency but also ensures that the most important information is always under the highest level of monitoring. The above priority determination process is a routine operation and will not be described in detail in this embodiment.

[0024] The data acquisition module includes the following sub-modules: Data Stream Collection Sub-module: This module collects data streams from multiple data sources. It is responsible for acquiring data streams from various sources and ensuring that this data can be subsequently processed and analyzed. To achieve this goal, the system employs multiple technical means to ensure effective data collection. For example, in cloud deployment scenarios, it directly connects to various services (such as databases, web servers, message queues, etc.) via API interfaces to capture the required data. Furthermore, for some traditional systems or third-party services that do not support direct API access, data can be indirectly obtained through log file parsing or network traffic monitoring. Specifically, the data sources supported by this system include, but are not limited to: Server Logs: Records various operational information during system operation, including error logs and access logs. API Call Records: Records detailed usage of API interfaces, such as the number of requests, response time, and error rate. User Behavior Data: Covers user interaction behaviors, such as clicks, browsing paths, and dwell time. Network Transmission Rate: Reflects the quality of network connections, including upload and download speeds and packet loss rates. Sensor Data: Applicable to Internet of Things (IoT) application scenarios, including changes in physical quantities such as temperature, humidity, and location. Each type of data provides a unique perspective on the health of the system, so collecting this data comprehensively and meticulously is the foundation for ensuring the accuracy of subsequent analysis.

[0025] In this embodiment, a data stream refers to a specific sequence of numerical values ​​generated from a particular data source within a unit of time. For example, for a web server, its data stream includes the number of HTTP requests received per minute, average response time, and the number of error codes. Therefore, the data stream for each data source is not a single parameter, but rather consists of multiple key performance indicators reflecting the state of that data source. Specifically, for different types of internet applications, these data streams include, but are not limited to: API services: number of requests, response time, error rate, CPU utilization; Database operations: number of queries, execution time, number of failed transactions; User interaction behavior: page views, number of clicks, session duration.

[0026] The data acquisition frequency configuration submodule is used to dynamically configure the data acquisition frequency based on the priority of each data source.

[0027] The sampling frequency configuration submodule includes the following units, such as... Figure 2 As shown: Historical traffic fluctuation data acquisition unit: used to determine historical traffic fluctuation data based on the data stream of each data source.

[0028] Historical traffic fluctuation data specifically refers to the statistical results of data volume collected within fixed time intervals based on the aforementioned data stream. In this embodiment, the data volume refers to the specific numerical value of the data stream per unit time, such as the number of requests per minute, response time, error rate, number of queries, execution time, number of failed transactions, page views, number of clicks, session duration, etc.

[0029] Divide the statistical units: This is used to define the first time window and the second time window. Based on the first time window and the second time window, the historical traffic fluctuation data is divided into several statistical data sets. The first time window is divided into several time intervals through the second time window, and the variance of the statistical data sets is obtained.

[0030] In this embodiment, the core function of dividing statistical units is to extract statistically significant time intervals and corresponding fluctuation characteristics by performing structured processing on historical traffic fluctuation data along the time dimension. To this end, this embodiment introduces two nested time window concepts: a first time window and a second time window. The first time window is a relatively long time period, such as 1 day, 7 days, or 30 days, used to cover a sufficiently long historical data range to capture the long-term trend of the overall traffic pattern. The second time window is a short period unit within the first time window, such as 5 minutes, 15 minutes, or 1 hour, used to subdivide the long time period into multiple consecutive smaller time periods for more refined analysis.

[0031] By dividing the first time window into several second time windows of equal length, this embodiment can classify the original historical traffic fluctuation data according to these time intervals, forming multiple statistical data sets. Each statistical data set corresponds to a set of historical traffic fluctuation data values ​​collected within a second time window, such as key indicators like the number of requests per minute and response time.

[0032] For example, taking the API call monitoring of an e-commerce platform as an example, the first time window is set to 7 days (168 hours), and the second time window is 1 hour. The entire first time window will be divided into 168 second time windows, that is, each hour is a sub-interval. For each second time window, the system will collect all the data stream sample points collected within that hour (e.g., the number of requests recorded once per minute) and combine them into a data set.

[0033] For example, in the first second time window (between 00:00 and 01:00 on the first day), the following request count data per minute was collected: [24,27,29,31,30,28,26,25,23,22,20,19,21,23,25,27,28,30,32,31,29,28,27,26]. This array constitutes the statistical data set for that time interval, representing the change in API request volume per minute within that hour. Repeating this process yields the statistical data sets for all second time windows under the first time window, thus providing the basic data for subsequent variance calculations.

[0034] After the data is partitioned, this embodiment calculates the variance of the statistical data set within each second time window. Variance is an important statistical indicator that measures the degree to which a set of values ​​deviates from its mean.

[0035] Fluctuation energy density index definition unit: used to define a fluctuation energy density index based on the statistical data set within each time interval.

[0036] A fluctuation energy density index is calculated for each second time window. The index is mainly calculated by using the historical flow fluctuation data per unit time within each second time window to calculate the average value of historical flow fluctuation data. Based on the historical flow fluctuation data per unit time and the average value of historical flow fluctuation data, the fluctuation energy density index is calculated to measure the fluctuation intensity of the data flow within the current second time window.

[0037] E d For fluctuation energy density index, x is the length of the second time window. i This refers to the historical traffic fluctuation data for the i-th unit within the second time window. is the average value of historical traffic fluctuation data, and n is the total number of historical traffic fluctuation data within the second time window.

[0038] The volatility energy density index is calculated based on variance theory. It primarily works by summing the squared differences between each data point and the mean within each unit of time, then dividing the result by the time window length. This not only accurately reflects the dispersion within the dataset but also considers the distribution of these discrete values ​​along the time axis, providing a deep insight into the density and cumulative effect of data volatility. Specifically, by subtracting the mean of all data points within its time interval from each data point and then squaring the result, data points far from the mean—i.e., abnormal volatility—are highlighted, amplifying the impact of extreme values ​​and making significant fluctuations easier to identify. Summing the squared differences of all these variations yields a result that measures the overall level of data volatility over the entire time period, and normalization ensures standardized comparisons between time windows of different lengths. This step ensures that the volatility energy density index accurately reflects the "density" of volatility per unit of time.

[0039] Furthermore, this indicator emphasizes the time-series characteristics of data fluctuations, focusing on how data changes over time and the magnitude of these changes. Introducing the concept of a time window not only tracks short-term trends but also reveals the cumulative effect of data fluctuations over a period of time. For example, if data is relatively stable for the first 30 minutes of an hour, but then experiences sharp fluctuations in the following 30 minutes, this change will not only affect the final fluctuation energy density value but also show the timing of the fluctuations and their scope of impact.

[0040] The volatility persistence index definition unit is used to preset an initial threshold for variance, extract the number of consecutive times the variance is greater than the initial threshold within the time interval of the first time window. If the number of consecutive times is greater than or equal to 2 / 3 of the number of time intervals, a volatility persistence index is defined based on the statistical data set within the time interval; if the number of consecutive times is less than 2 / 3 of the number of time intervals, the current collection frequency is maintained.

[0041] First, the initial threshold was obtained through analysis of historical data. Historical data streams under normal system operation were selected as a reference, for example, the low-load period from 2:00 AM to 4:00 AM each day over the past week. Similarly, based on the first and second time windows, several historical time intervals were obtained. The variance of the historical data streams for each historical time interval within the selected period was calculated, and the average variance of all reference time intervals plus a standard deviation was taken as the initial threshold.

[0042] The number of consecutive times the variance within the time intervals of the first time window is greater than the initial threshold refers to the number of times the variance of at least two time intervals within the first time window is greater than the initial threshold. If the number of consecutive times is greater than or equal to 2 / 3 of the number of time intervals, a volatility persistence index is defined based on the statistical data set within the time intervals of the first time window; if the number of consecutive times is less than 2 / 3 of the number of time intervals, no volatility persistence index is defined.

[0043] In this embodiment, the fluctuation intensity within each time interval is first calculated based on the fluctuation energy density index, thus obtaining the fluctuation distribution within the entire first time window. Then, by comparing the variance of each time interval with a preset initial threshold, the number of consecutive occurrences of variance exceeding the initial threshold is extracted to determine whether the data exhibits persistent fluctuation characteristics, and a fluctuation persistence index is defined accordingly. Specifically, fluctuations are defined as having significant persistence only when the variance of more than 2 / 3 of the time intervals within a certain time window is higher than the initial threshold, triggering further adjustments to the monitoring strategy, such as increasing the sampling frequency to capture more details. Conversely, if the number of consecutive occurrences is lower than this standard, a fluctuation persistence index is not defined, and the existing sampling frequency remains unchanged. The rationale for this design is that for cases where the persistent fluctuation standard is not met, the system may have only experienced brief or sporadic fluctuations, rather than entering a new unstable state. In this case, not adjusting the sampling frequency helps avoid resource waste caused by over-response, while also ensuring the stability and efficiency of the monitoring system. Furthermore, by maintaining the existing sampling frequency, the system can still continuously monitor subsequent data changes, and once a potential trend is detected, timely action can be taken, achieving the goal of flexibly responding to various fluctuation situations while ensuring system performance. This method not only improves the accuracy and response speed of monitoring, but also effectively balances the relationship between resource consumption and monitoring needs.

[0044] The process for calculating the volatility persistence index is as follows: First, obtain the current time interval. If the current time interval is not yet complete, use the previous time interval as the current interval. Determine a timestamp for each time interval, using the end time of the time interval as the timestamp. Finally, calculate the volatility persistence index based on the variance of the statistical data set for each time interval, the initial threshold, and the timestamps for each time interval.

[0045] For example, assuming the current time is 8:30 and each time interval is one hour long, the current time interval should be from 8:00 to 9:00. Since this interval has not yet ended, this embodiment uses 7:00 to 8:00 as the current time interval for calculation. The purpose of this is to ensure that all data involved in the calculation is complete, avoiding data incompleteness issues caused by incomplete intervals. In this way, it can be guaranteed that each calculation is based on stable and reliable historical data, thereby improving the accuracy and consistency of the volatility persistence indicator.

[0046] Furthermore, the volatility persistence index is constructed based on statistical principles, time series analysis theory, and the practical needs of system stability monitoring. The calculation formula is as follows:

[0047] , As an indicator of volatility persistence, Let be the variance of the statistical data set within the j-th time interval. Let t be the initial threshold, e be the exponential function, and t be the value of t. m t represents the timestamp corresponding to the current time interval. j Let m be the timestamp corresponding to the j-th time interval, and m be the total number of time intervals within the first time window.

[0048] The volatility persistence index is designed to provide a scientific basis for system stability monitoring by quantifying the volatility patterns and intensity of data over time. First, the formula considers the significance of dividing the variance of each time interval by an initial threshold to standardize the volatility intensity. This operation not only eliminates dimensional differences in data within time intervals, making volatility intensity comparable, but also quantifies the degree of deviation of volatility from the "normal level" within each interval. For example, in an API response time monitoring case, if the variance of a certain time interval is significantly higher than the preset initial threshold, it indicates that the volatility within that time period exceeds the expected range, suggesting a potential problem. This design is based on statistical process control theory, identifying whether the process is out of control by setting upper and lower control limits. The anti-tampering measures and the initial threshold are essentially quantifications of the degree of deviation from the control limits.

[0049] Furthermore, another crucial component of the CPI formula is the consideration of the difference between the timestamp of the current time interval and other timestamps. Specifically, the time difference between the end of the current time interval and the end of the j-th time interval is weighted using an exponential function. The significance of this design lies in introducing a time decay mechanism, emphasizing the importance of recent fluctuations in the overall assessment. As time progresses, the influence of past data on current judgments gradually weakens, allowing for a more sensitive response to recent trend changes. This approach borrows from the idea of ​​the exponentially weighted moving average model, widely used in fields such as network monitoring, which better captures trend changes by assigning higher weights to recent observations.

[0050] Therefore, when the CPI is below the standard defining the persistence of volatility, it means that the volatility does not show significant persistence. In this case, it is reasonable not to adjust the collection frequency, as this avoids overreaction due to short-term fluctuations and ensures system stability and efficient resource utilization. This design provides stable, accurate, and interpretable monitoring results in practical applications and is suitable for various system scenarios that require dynamic perception of data fluctuations.

[0051] Acquisition frequency judgment unit: It is used to preset a first judgment threshold for the fluctuation persistence index and define a second judgment threshold for the fluctuation energy density index. It judges whether the fluctuation persistence index is greater than the first judgment threshold and the fluctuation energy density index is greater than the second judgment threshold. If so, it enters the high-frequency acquisition mode; otherwise, it maintains the current acquisition frequency.

[0052] Historical data streams from when the system is operating normally are selected as a reference, for example, the period from 2:00 AM to 4:00 AM each day over the past week is considered a low-load period. Similarly, based on the first and second time windows, several historical time intervals are obtained. For each historical time interval within the selected period, fluctuation energy density and fluctuation persistence indices are calculated. The average of the fluctuation persistence indices for all reference periods plus a standard deviation of the fluctuation persistence index is taken as the first judgment threshold, and the average of the fluctuation energy density indices for all reference periods plus a standard deviation of the fluctuation energy density index is taken as the second judgment threshold.

[0053] When both the volatility persistence index and the volatility energy density index exceed the first judgment threshold, the system will automatically switch to high-frequency acquisition mode. The principle behind this strategy is to ensure, through a dual verification mechanism, that further action is only triggered when the volatility is both strong and persistent. First, a volatility energy density exceeding the second judgment threshold indicates a high volatility intensity, suggesting a system anomaly. Second, a volatility persistence index exceeding the first judgment threshold indicates that this high-intensity volatility is not accidental but rather exhibits a certain degree of persistence and regularity. This design not only improves the system's response speed, enabling the detection of potential risks and rapid reaction, but also reduces the false alarm rate. Conversely, if neither of these conditions is simultaneously met, the existing acquisition frequency remains unchanged. This saves resources and avoids unnecessary computational burden, storage space, and network bandwidth consumption caused by frequent adjustments to the acquisition frequency. Furthermore, for some non-critical volatility, even if it occasionally exceeds the normal range but does not form a persistent trend, no immediate action should be taken to reduce interference and ensure system stability and reliability.

[0054] Acquisition frequency adjustment unit: used to obtain the fluctuation energy density growth rate within adjacent time intervals within the first time window, preset a third judgment threshold for the fluctuation energy density growth rate, and determine whether the fluctuation energy density growth rate is greater than the third judgment threshold. If it is, the first acquisition frequency is configured according to the fluctuation energy density growth rate. If not, the second acquisition frequency is configured according to the statistical data set of the L time intervals before the current time interval. The first acquisition frequency is greater than the second acquisition frequency.

[0055] Calculate the fluctuation energy density growth rate based on the fluctuation energy density growth density of the current time interval and the previous time interval within the first time window: , E represents the fluctuation energy density growth rate. d (j) represents the fluctuation energy density in the j-th time interval, E d (j-1) represents the fluctuation energy density of the (j-1)th time interval.

[0056] Historical data streams from the system are selected as a reference, such as the data streams from 2:00 AM to 4:00 AM each day over the past week, representing periods of low load, high load, and abnormal activity. Similarly, based on the first and second time windows, several historical time intervals are obtained. The fluctuation energy density growth rate is calculated for the historical data streams of each selected time interval. The average fluctuation energy density growth rate of all reference periods, plus a standard deviation of the fluctuation energy density growth rate, is taken as the third judgment threshold.

[0057] The design logic of the sampling frequency adjustment unit aims to identify the direction and speed of fluctuation trends by analyzing the fluctuation energy density growth rate. When the fluctuation energy density growth rate exceeds a preset third judgment threshold, it indicates that the fluctuation is rapidly increasing and the system is about to enter an unstable state. At this time, using a higher first sampling frequency can capture more detailed data, enabling proactive sampling frequency adjustment and ensuring timely detection and response to potential problems. Conversely, if the fluctuation energy density growth rate does not exceed the threshold, it indicates that the fluctuation trend is relatively stable or in a downward state. There is no need to immediately switch to a high-frequency sampling mode. Instead, a lower second sampling frequency is configured based on the statistical data set from the L time intervals preceding the current time interval. This avoids frequent adjustments to the sampling frequency due to occasional fluctuations, reduces resource waste, and maintains a basic level of monitoring of the system's operating status. This design draws on theories such as adaptive sampling and sliding window time series forecasting, achieving the goal of optimizing resource utilization efficiency while ensuring anomaly response capabilities.

[0058] The configuration process for the first sampling frequency is as follows: , The first sampling frequency is in Hertz. is the gain coefficient, is the growth rate of the fluctuating energy density.

[0059] The third judgment threshold is divided into a first sub-threshold and a second sub-threshold. If the growth rate of the fluctuating energy density is less than the first sub-threshold, the first coefficient p1 is assigned to the gain coefficient. If the growth rate of the fluctuating energy density is greater than or equal to the first sub-threshold and less than the second sub-threshold, the second coefficient p2 is assigned to the gain coefficient. If the growth rate of the fluctuating energy density is greater than the second sub-threshold, the third coefficient p3 is assigned to the gain coefficient, where p1 < p2 < p3.

[0060] Preferably, the first sub-threshold is obtained by adding 0.7 standard deviations of the growth rate of the fluctuating energy density to the average value of the growth rate of the fluctuating energy density over all reference periods, and the second sub-threshold is obtained by adding 2 standard deviations of the growth rate of the fluctuating energy density to the average value of the growth rate of the fluctuating energy density over all reference periods. Therefore, the first sub-threshold < the third judgment threshold < the second sub-threshold, thus forming a clear classification system for the fluctuating intensity. When the growth rate of the fluctuating energy density is lower than the first sub-threshold, it indicates that the system is in a relatively stable state. At this time, a smaller gain coefficient p1 is adopted to maintain a lower acquisition frequency to save resources. When the fluctuation is between the first sub-threshold and the third judgment threshold, it means that the system has mild fluctuations but is still within the controllable range. At this time, a medium-sized gain coefficient p2 is selected to moderately increase the acquisition frequency to detect potential problems in a timely manner. When the fluctuation exceeds the third judgment threshold but does not reach the second sub-threshold, a larger gain coefficient p3 is applied to further increase the acquisition frequency to cope with significant fluctuations. Finally, if the growth rate of the fluctuating energy density exceeds the second sub-threshold, it means that the system has entered a highly unstable state, and the maximum gain coefficient p4 needs to be enabled to switch to the high-frequency acquisition mode to ensure that it can respond quickly and take necessary measures. This hierarchical gain coefficient setting method can not only accurately distinguish different levels of fluctuating intensity, but also optimize the response ability and resource utilization efficiency of the monitoring system, achieving the maximum reduction of resource consumption while ensuring the system performance.

[0061] It should be noted that although specific numerical values of p1, p2, p3, and p4 cannot be given in this embodiment because these gain coefficients need to be adjusted through experiments according to the actual situation, the final values should satisfy the relationship of p1 < p2 < p3 < p4 to ensure that the system can achieve the optimal monitoring effect and resource utilization efficiency under different fluctuating intensities. This hierarchical gain coefficient setting method can accurately distinguish different levels of fluctuating intensity.

[0062] If the fluctuation energy density growth rate is less than or equal to the third judgment threshold, a second collection frequency is configured based on the statistical data set from the L time intervals preceding the current time interval. Preferably, L = 0.67m. Selecting data from the previous L time intervals for calculation aims to retain sufficient historical information to reflect the overall trend of the system without overreacting to short-term fluctuations. Setting L = 0.67m (i.e., 67% of the total number of time intervals within the first time window) is a compromise strategy that balances data representativeness and real-time response. This ensures that the collection frequency calculation is based on relatively sufficient historical data, effectively smoothing out interference from accidental fluctuations and improving the stability of the evaluation results. Secondly, compared to a higher proportion (such as using 3 / 4 or all m intervals), 0.67m retains a certain degree of "timeliness," enabling the system to maintain a certain sensitivity to newer trend changes and avoiding decision-making lag due to the use of outdated historical data. Finally, this proportion has been widely verified in practical engineering as a robust empirical value, achieving a good balance between resource utilization and monitoring accuracy in most non-extreme fluctuation scenarios. Therefore, setting L to 0.67m helps to build a dynamic acquisition mechanism that is both stable and adaptable.

[0063] Using the historical traffic data within the first time interval of the first time window as the initial data, calculate the growth of historical traffic data in adjacent time intervals within the L time intervals before the current time interval, and obtain the second collection frequency based on the growth and the initial data.

[0064] The increase is obtained by subtracting historical traffic data within adjacent time intervals.

[0065] , This is the second sampling frequency, measured in Hertz, where V0 represents the initial data. This represents the growth rate of historical traffic data for the j-th time interval and the (j-1)-th time interval.

[0066] This method considers the growth rate of traffic data within adjacent time intervals, i.e., the magnitude of traffic change in a given time interval compared to the previous one. This growth rate reflects the trend of traffic change over a period of time. Simultaneously, the method uses historical traffic data from the first time interval within the first time window as a baseline value to standardize the calculation of the growth rate. Dividing the growth rate of each time interval by this initial baseline value and summing the results yields a cumulative growth rate reflecting the overall trend. Based on this cumulative growth rate, a second sampling frequency is determined, enabling the system to respond more accurately to traffic changes: when the cumulative growth rate indicates a significant increase in traffic, the system increases the sampling frequency accordingly to better capture potential fluctuations; conversely, if the cumulative growth rate is low, indicating minimal traffic change, the system can appropriately reduce the sampling frequency to conserve resources. Therefore, by analyzing the growth characteristics of historical traffic data, this method achieves the goal of intelligently sensing the system's status and optimizing response strategies, ensuring the efficiency and accuracy of the monitoring system.

[0067] Data stream acquisition submodule: Used to acquire data streams from each data source according to the acquisition frequency. It acquires real-time data streams from each data source based on the acquisition frequency calculated above.

[0068] Data collection frequency refers to the number of times or time intervals during which data is collected from a specific data source per unit of time. For example, if the collection frequency is set to once every 10 seconds, it means that the system will retrieve the latest data from the specified data source every 10 seconds. It can be expressed as "number of times / time unit", such as 5 times per second (5Hz).

[0069] Based on the calculated acquisition frequency, the data stream acquisition submodule can customize the acquisition frequency for each data source, significantly improving the overall system performance and resource utilization efficiency. By setting appropriate acquisition frequencies for data sources with different characteristics and needs, the system can more effectively respond to the demands of various application scenarios. For data sources with extremely high real-time and accuracy requirements, a higher acquisition frequency ensures that the system can promptly capture any subtle changes, supporting rapid decision-making and immediate response. For systems with slower or more stable changes, a lower acquisition frequency is sufficient to meet monitoring needs while reducing unnecessary data transmission and storage overhead. This differentiated acquisition strategy not only improves the speed and accuracy of acquiring key information but also optimizes the allocation of system resources, avoiding the burden of over-acquisition. Therefore, by setting personalized acquisition frequencies for each data source, the system can flexibly respond to diverse needs while ensuring efficient operation, achieving the best balance between performance and resources. In this way, whether dealing with rapidly changing market dynamics or tracking long-term stable development trends, the system can provide accurate and reliable support.

[0070] The data preprocessing module, connected to the data acquisition module, performs preprocessing operations on the acquired data stream. After data is acquired, it typically contains noise, redundant information, or inconsistent formats. Therefore, the data preprocessing module performs a series of operations to ensure data quality and consistency: Cleaning: Removes duplicate records and corrects erroneous values. Standardization / Normalization: Due to potential format differences in data from different data sources, numerical values ​​need to be standardized or normalized. Anomaly Monitoring Module: Connects to the data preprocessing module and uses a machine learning model to monitor the preprocessed data stream for anomaly patterns.

[0071] The anomaly monitoring module includes the following sub-modules: Training set acquisition sub-module: used to use the data streams, historical traffic fluctuation data, fluctuation persistence indicators, and fluctuation persistence indicators of each data source as training sets, and to mark the data streams of each data source as normal mode / abnormal mode.

[0072] The data streams from each data source are the historical data streams from the data acquisition module. These include API call logs, server response times, database query latency, and user request frequency. The above modules calculate the fluctuation persistence index and fluctuation persistence metric. Finally, the data streams from each data source, historical traffic fluctuation data, fluctuation persistence index, and fluctuation persistence metric are used as the training set for the machine learning model. Alarm events / abnormal events based on the historical data streams are labeled as normal mode / abnormal mode, with different labels assigned to normal mode and abnormal mode: 0 for normal mode and 1 for abnormal mode.

[0073] For example, the training set consists of multiple feature dimensions: the number of requests in the current time period (numerical); the average number of requests in the previous z time windows (numerical); the fluctuation energy density (numerical); the fluctuation persistence index (numerical); the data source type identifier (categorical variable); and the pattern label (integer encoding).

[0074] Machine learning model: Used to input the training set into the machine learning model for training, resulting in a trained machine learning model. This invention preferably uses a Support Vector Machine (SVM) as the anomaly detection model. This model has good few-shot learning and generalization capabilities, and is particularly suitable for scenarios where the data dimension is relatively limited but there are complex nonlinear relationships between features, as in this application. All numerical features in the training set are Z-score standardized, that is, each feature value is subtracted from its mean and then divided by its standard deviation, thereby eliminating the influence of differences in the units of measurement between different features, making the model training more stable and effective. The standardized training set is used as the training set for the support vector machine.

[0075] The training set was divided into training samples, validation samples, and test samples in an 8:1:1 ratio. The training samples were used to learn the classification boundary; the validation samples were used for hyperparameter tuning; and the test samples were used for final performance evaluation.

[0076] This invention preferably uses radial basis functions (RBFs) as the kernel function. The RBF kernel form is:

[0077] , , indicating sample points and The similarity or inner product between two points. In Support Vector Machines, the RBF kernel is used to calculate the distance between two sample points in a high-dimensional feature space. Representing the One sample point, Represents the Y-th sample point, Representing the The Euclidean distance between the Y-th sample point and the Y-th sample point This is a hyperparameter that determines the width, or radius of influence, of the RBF kernel. Specifically, it controls how quickly the kernel function value decreases as the distance between samples increases. A smaller... A larger value implies a wider range of influence, meaning more samples will be considered "similar"; a larger value indicates a wider range of influence. The value makes the model pay more attention to local details, and only very similar samples will be assigned a high similarity weight. Representing the The inner product estimate after mapping the Y-th sample point to the high-dimensional space is transformed so that data that was originally linearly inseparable in the low-dimensional space can become linearly separable in the high-dimensional space, thereby enhancing the classifier's ability.

[0078] To optimize model performance, this embodiment uses a grid search combined with K-fold cross-validation to tune two key hyperparameters: Penalty coefficient C: controls the classifier's tolerance for misclassified samples. A larger C value will make the model more inclined to strictly distinguish between the two classes, but may cause overfitting. Kernel function parameters : Controls the "width" of the RBF core, smaller A value implies a wider range of influence, a larger value. The value makes the model pay more attention to local details.

[0079] In the specific optimization process, C and The range of candidate values, for example , Multiple rounds of combined training and validation are performed on the training set, and the parameter combination with the highest average accuracy on the validation set is selected as the optimal hyperparameter.

[0080] Once the optimal hyperparameters are determined, the model is trained using the entire training sample. This process essentially involves finding a way to separate normal and abnormal patterns, known as the decision boundary, which best separates samples from different classes. Support vectors (SVMs) are the sample points located on the class boundaries and are crucial for defining the separating surface. In other words, SVMs are the samples closest to the opposite class. The standardized training samples are then input into the SVM model, and optimization algorithms (such as the SMO algorithm) are used to solve for the support vectors and their corresponding hyperparameters. This process yields the decision boundary and the set of support vectors, thus determining the decision boundary. After completing this step, a trained SVM model is obtained.

[0081] After training, the model is not deployed immediately. Instead, it is fed into a test sample set for performance evaluation. Evaluation metrics include accuracy, recall, F1 score, and AUC. The model is considered successfully trained and ready for deployment only when all metrics on the test samples meet preset thresholds (accuracy ≥ 90%, F1 score ≥ 0.85, AUC ≥ 0.90). Otherwise, the model is deemed unqualified, triggering hyperparameter tuning or data augmentation mechanisms to improve model performance.

[0082] The successfully trained model will be serialized and saved, and deployed to the monitoring submodule for real-time reception of feature vectors during system operation and output of normal / abnormal judgment results. Furthermore, to adapt to the changing trends of system behavior over time, this invention also includes a periodic retraining mechanism and a model rollback mechanism to ensure that the model is always in an effective state.

[0083] Monitoring submodule: Used to collect data streams from each data source according to the collection frequency, determine the real-time fluctuation persistence index and real-time fluctuation persistence index of each data source, and input the data streams, real-time fluctuation persistence index and real-time fluctuation persistence index of each data source into the trained machine learning model to obtain the normal / abnormal mode.

[0084] The real-time data streams from each data source are collected at a specific frequency. If the time interval for which the real-time data stream from each data source is acquired is not yet complete, the data from the previous time interval is used to calculate the real-time volatility persistence index.

[0085] This embodiment significantly improves model performance by introducing fluctuation energy density and fluctuation persistence index as higher-order statistical features. This not only enhances the model's ability to understand complex data patterns but also improves its generalization ability and confidence across different scenarios.

[0086] Fluctuation energy density is a metric that measures the intensity of data fluctuations and can sensitively reflect sudden performance degradation or jitter. When a system experiences a sudden performance drop or anomaly, the value of fluctuation energy density rises rapidly, helping the model to promptly capture such short-term, drastic fluctuations. For example, in API call logs, even if the response time increases from 130ms to 190ms, if it is accompanied by a high fluctuation energy density, the model can immediately identify this as an anomaly rather than a normal load change.

[0087] CPI is used to assess the persistence and periodicity of fluctuations. It helps the model understand whether the current fluctuation is a one-off event or a long-term problem. A high CPI value indicates that the fluctuation is persistent and may be a sign of system failure or other long-term problems. By combining fluctuation energy density and CPI, the model can more accurately distinguish between short-term disturbances and long-term trends, avoiding misclassifying normal fluctuations as anomalies and thus reducing the false positive rate.

[0088] Using data streams from various data sources as basic features, while providing some information, lacks a description of the overall trend or fluctuation characteristics of the data. This can lead to false positives or false negatives in models when facing complex scenarios. Fluctuation energy density and CPI, as higher-order statistical features, can capture complex patterns in the data, especially anomalies that are not easily identified by a single dimension. For example, in sample B, although the response time and error rate are the same as in sample A, the higher values ​​of fluctuation energy density and CPI allow the model to be more certain that it is an anomaly, thus giving a higher confidence level.

[0089] By incorporating fluctuation energy density and CPI, the model gains more contextual information, thereby enhancing its generalization ability. For example, fluctuation energy density helps the model identify sudden performance degradation, while CPI helps distinguish between short-term disturbances and long-term trends. This additional information enables the model to make accurate judgments in a wider range of scenarios, thus improving confidence levels. Therefore, even when faced with new types of data or unseen anomalous patterns, the model exhibits better adaptability and accuracy.

[0090] For example, this embodiment uses a comparison table of model predictions based on the introduction of higher-order statistical features and existing basic features:

[0091] Implementation plan generation module: Connects to the anomaly monitoring module and is used to generate implementation plans based on the anomaly patterns detected by monitoring.

[0092] The implementation plan is generated based on the abnormal patterns detected. This mainly includes: calling a predefined problem strategy mapping table based on the abnormal patterns detected, matching historical problem cases, and extracting the repair strategies and implementation paths of historical problem cases; automatically generating an implementation plan based on the system architecture topology diagram, including service restart sequence, configuration file modification instructions and dependency check logic; and packaging the generated operation scripts into an executable package and marking the execution risk level and rollback contingency plan.

[0093] The problem strategy mapping table is a knowledge base in existing technologies. Its construction typically involves the structured organization of historical operational data, troubleshooting records, and expert experience. For example, feature tags are extracted from historical work orders and log analysis reports, and a mapping relationship of "anomaly feature combination → repair strategy" is established. When matching historical problem cases, the system uses similarity calculation technology to compare the fluctuation pattern of the current anomaly's indicators, related components, and scope of impact with historical cases in the mapping table, selecting the most relevant historical problems as references.

[0094] Based on the system architecture topology diagram of the target environment, the dependencies, communication links, and deployment locations between various service components are described. A graph traversal algorithm is used to automatically deduce a reasonable service restart order, ensuring that dependent services are stopped first and core services are started later. At the same time, corresponding configuration file modification instructions (such as YAML / JSON patches) are generated and dependency check logic (such as health check scripts and port connectivity verification) is embedded to avoid accidental operations. Finally, all generated operation scripts are encapsulated into a standardized executable package, and the execution risk level (such as low, medium, and high) is marked according to factors such as the scope of the operation, whether stateful services are involved, and whether there is a risk of data loss. At the same time, corresponding rollback plans (such as configuration backup, transaction rollback commands, and service degradation scripts) are automatically generated, thereby realizing a closed-loop response mechanism from anomaly identification to automated handling.

[0095] Plan Optimization Module: Connects to the implementation plan generation module and is used to optimize the implementation plan to obtain an optimized plan.

[0096] The plan optimization module includes the following sub-modules: Implementation Plan Execution Sub-module: Used to execute the implementation plan. Key Performance Indicator (KPI) Collection Sub-module: Used to collect key performance indicators before and after the implementation plan execution.

[0097] Response time, CPU utilization, and memory usage are used as key performance indicators. However, due to the dynamic and uncertain nature of the system state, a single sample is easily affected by noise and cannot accurately reflect the true trend. Therefore, this embodiment optimizes the implementation plan by using multiple sampling methods.

[0098] Preferably, in this embodiment, after collecting data streams from various data sources at a sampling frequency and inputting them into a machine learning model to identify abnormal patterns, an implementation plan is generated and executed, which serves as one sampling. Key performance indicators before and after each sampling are recorded to obtain a sequence A{a1, a2, ..., a...} c} and B{b1, b2, ..., b c Let A be the set of key performance indicators (KPIs) before the implementation plan is executed, and B be the set of KPIs after the implementation plan is executed. c For the c-th key performance indicator value before the implementation plan, b c Let c be the value of the key performance indicator after the implementation plan is executed, where c = 1, 2, ..., N, and N is the total number of key performance indicators.

[0099] Key Performance Indicator Difference Acquisition Submodule: This module is used to obtain the key performance indicator difference based on the key performance indicators collected before and after the implementation plan is executed.

[0100] , This represents the difference between the key performance indicators (KPIs) before and after the implementation of the plan for the c-th key performance indicator.

[0101] The comprehensive performance improvement index calculation submodule is used to calculate the mean and standard deviation of the difference based on the difference of key performance indicators, and to calculate the comprehensive performance improvement index based on the mean and standard deviation of the difference. , , ; The mean of the differences The standard deviation of the difference. is the overall performance improvement index, and e is the exponential function.

[0102] The mean difference reflects the average effect of the implementation plan across all sampling points, showcasing the overall trend and serving as a core indicator of plan effectiveness. A large mean value suggests that the overall implementation direction is correct. The standard deviation of the difference measures the stability of the plan's performance. Even with a high mean, a large standard deviation indicates significant fluctuations in optimization effectiveness, suggesting uncertainty and hindering stable operational decisions. However, The emphasis is on how the confidence level of the implementation plan's effectiveness changes with the increase of the number of samples; as N increases... As the overall performance improvement index gradually decreases, it "amplifies" the importance of results based on a large number of sampling points and suppresses the risk of overestimation when there are insufficient early samples.

[0103] The difference level determination submodule is used to preset the target judgment threshold for the comprehensive performance improvement index and compare the comprehensive performance improvement index with the target judgment threshold to determine the difference level.

[0104] This embodiment uses historical problem cases from before the implementation plan was generated, and their repair strategies and implementation paths as historical implementation plans. The historical comprehensive performance improvement index of these historical implementation plans is calculated and sorted from smallest to largest. The 25th percentile (y1), 50th percentile (y2), and 75th percentile (y3) are used as target judgment thresholds. When the comprehensive performance improvement index is less than or equal to y1, the difference level is defined as inefficient, indicating poor implementation plan performance, requiring re-evaluation of the optimization strategy. When the comprehensive performance improvement index is greater than y1 and less than or equal to y2, the difference level is defined as good, indicating that the implementation plan achieved the expected results and optimization was relatively successful. When the comprehensive performance improvement index is greater than y2, the difference level is defined as excellent, indicating that the implementation plan's performance significantly surpasses historical performance and is worth promoting.

[0105] The plan optimization submodule is used to optimize the implementation plan based on the difference level to obtain an optimized plan.

[0106] Implementation plans with a difference level of good and / or excellent are considered as optimization plans, while implementation plans with a difference level of inefficient need further optimization.

[0107] Optimizing implementation plans with an inefficient level primarily aims to identify shortcomings in the current implementation plan and take targeted measures to improve these shortcomings. The optimization process for inefficient plans is as follows: Problem Diagnosis: First, it is necessary to conduct an in-depth analysis of the specific reasons why the implementation plan was rated as "inefficient." These reasons may include inappropriate technology selection, unreasonable resource allocation, and low execution efficiency.

[0108] Data Collection and Analysis: Collect all relevant data regarding this implementation plan, including but not limited to performance metrics, resource usage, and user feedback. Identify bottlenecks through data analysis.

[0109] Root cause analysis: Based on the above data, use fishbone diagrams or other root cause analysis tools to identify the root causes of inefficiency.

[0110] Develop optimization strategies: If the problem stems from inappropriate technology selection, consider switching to a more suitable technology stack or framework. If it's due to unreasonable resource allocation, adjust server configurations, database parameters, etc.

[0111] For issues of low execution efficiency, it may be necessary to refactor the code, optimize the algorithm, or introduce a caching mechanism. For poor user experience, improvements can be made to the interface design and loading time can be reduced.

[0112] Implement the optimization plan: Implement optimization measures step by step according to the established strategy, and ensure that each step is recorded in detail for subsequent tracking of the effect.

[0113] Monitoring and Evaluation: After implementing optimization measures, continuously monitor system performance, compare key performance indicators before and after optimization, and evaluate whether the optimization effect has achieved the expected results. Iterative Optimization: If the initial optimization fails to significantly improve performance, further adjustments to the strategy are necessary, iterating until satisfactory results are achieved.

[0114] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.

Claims

1. An artificial intelligence-based internet application performance monitoring and optimization system, characterized in that, Includes the following modules: Data acquisition module: used to collect data streams from multiple data sources, determine the priority of each data source based on preset rules, dynamically configure the data stream acquisition frequency based on the priority of each data source, and collect data streams from each data source according to the acquisition frequency; The data preprocessing module, connected to the data acquisition module, is used to preprocess the acquired data stream. Anomaly monitoring module: Connects to the data preprocessing module and is used to monitor the preprocessed data stream to obtain anomaly patterns based on the machine learning model; Implementation plan generation module: Connects to the anomaly monitoring module and is used to generate implementation plans based on the anomaly patterns detected by monitoring; The plan optimization module connects to the implementation plan generation module and is used to optimize the implementation plan to obtain an optimized plan. The plan optimization module includes the following sub-modules: Implementation Plan Execution Submodule: Used to execute the implementation plan; Key Performance Indicator (KPI) Collection Submodule: Used to collect key performance indicators before and after the implementation plan is executed; Key Performance Indicator Difference Acquisition Submodule: Used to obtain the key performance indicator difference based on the key performance indicators collected before and after the implementation plan is executed; The comprehensive performance improvement index calculation submodule is used to calculate the mean and standard deviation of the difference based on the difference of key performance indicators, and to calculate the comprehensive performance improvement index based on the mean and standard deviation of the difference. Difference Level Determination Submodule: Used to preset a target judgment threshold for the comprehensive performance improvement index, and compare the comprehensive performance improvement index with the target judgment threshold to determine the difference level; The plan optimization submodule is used to optimize the implementation plan based on the difference level to obtain an optimized plan.

2. The Internet application performance monitoring and optimization system based on artificial intelligence according to claim 1, characterized in that, The data acquisition module includes the following sub-modules: Data stream collection submodule: used to collect data streams from multiple data sources; Data Source Priority Determination Submodule: Used to determine the priority of each data source based on preset rules; The data acquisition frequency configuration submodule is used to dynamically configure the data acquisition frequency based on the priority of each data source. Data stream acquisition submodule: Used to acquire data streams from various data sources according to the acquisition frequency.

3. The Internet application performance monitoring and optimization system based on artificial intelligence according to claim 2, characterized in that, The sampling frequency configuration submodule includes the following units: Historical traffic fluctuation data acquisition unit: used to determine historical traffic fluctuation data based on the data stream of each data source; Dividing statistical units: This is used to define the first time window and the second time window, and to divide the historical traffic fluctuation data into several statistical data sets based on the first time window and the second time window; the first time window is further divided into several time intervals through the second time window, and the variance of the statistical data sets is obtained; Fluctuation energy density index definition unit: used to define a fluctuation energy density index based on the statistical data set within each time interval; The volatility persistence index definition unit is used to preset an initial threshold for variance, extract the number of consecutive times the variance is greater than the initial threshold within the time interval of the first time window. If the number of consecutive times is greater than or equal to 2 / 3 of the number of time intervals, a volatility persistence index is defined based on the statistical data set within the time interval; if the number of consecutive times is less than 2 / 3 of the number of time intervals, the current collection frequency is maintained. Acquisition frequency judgment unit: used to preset a first judgment threshold for the fluctuation persistence index and define a second judgment threshold for the fluctuation energy density index. It judges whether the fluctuation persistence index is greater than the first judgment threshold and the fluctuation energy density index is greater than the second judgment threshold. If so, it enters the high-frequency acquisition mode; otherwise, it maintains the current acquisition frequency. Acquisition frequency adjustment unit: used to obtain the fluctuation energy density growth rate of adjacent time intervals within the first time window, preset a third judgment threshold for the fluctuation energy density growth rate, and determine whether the fluctuation energy density growth rate is greater than the third judgment threshold. If it is, the first acquisition frequency is configured according to the fluctuation energy density growth rate. If not, the second acquisition frequency is configured according to the statistical data set of n time intervals before the current time interval. The first acquisition frequency is greater than the second acquisition frequency.

4. The Internet application performance monitoring and optimization system based on artificial intelligence according to claim 3, characterized in that, The anomaly monitoring module includes the following sub-modules: Training set acquisition submodule: used to use the data streams, historical traffic fluctuation data, fluctuation persistence indicators, and fluctuation persistence indicators from various data sources as training sets, and to label the data streams from each data source as normal mode / abnormal mode; Machine learning model: Used to input the training set into the machine learning model for training, and obtain the trained machine learning model; Monitoring submodule: Used to collect data streams from each data source according to the collection frequency, determine the real-time fluctuation persistence index and real-time fluctuation persistence index of each data source, and input the data streams, real-time fluctuation persistence index and real-time fluctuation persistence index of each data source into the trained machine learning model to obtain the normal / abnormal mode.

5. The Internet application performance monitoring and optimization system based on artificial intelligence according to claim 4, characterized in that, The implementation plan is generated based on the abnormal patterns detected. This mainly includes: calling a predefined problem strategy mapping table based on the abnormal patterns detected, matching historical problem cases, and extracting the repair strategies and implementation paths of historical problem cases; automatically generating an implementation plan based on the system architecture topology diagram, including service restart sequence, configuration file modification instructions and dependency check logic; and packaging the generated operation scripts into an executable package and marking the execution risk level and rollback contingency plan.

6. The Internet application performance monitoring and optimization system based on artificial intelligence according to claim 3, characterized in that, The configuration process for the first sampling frequency is as follows: , The first sampling frequency, This is the gain coefficient. This represents the growth rate of fluctuating energy density.

7. The Internet application performance monitoring and optimization system based on artificial intelligence according to claim 6, characterized in that, The third judgment threshold is divided into a first sub-threshold and a second sub-threshold. If the fluctuation energy density growth rate is less than the first sub-threshold, the gain coefficient is assigned the first coefficient p1. If the fluctuation energy density growth rate is greater than or equal to the first sub-threshold and less than the second sub-threshold, the gain coefficient is assigned the second coefficient p2. If the fluctuation energy density growth rate is greater than the second sub-threshold, the gain coefficient is assigned the third coefficient p3. <p2<p3。 8. The Internet application performance monitoring and optimization system based on artificial intelligence according to claim 3, characterized in that, Using historical traffic data within the first time interval of the first time window as initial data, calculate the growth rate of historical traffic data within the time interval, and obtain the second collection frequency based on the growth rate and the initial data.

Images