Software application performance optimization method based on big data analysis

Abstract

The invention discloses a software application performance optimization method based on big data analysis, and relates to the technical field of data analysis and mining, and the method comprises the following steps: tracking software application service call link data through a distributed tracking technology, collecting log data during software application operation through a log aggregation system, and monitoring the performance of software application through a performance monitoring tool. The method comprises the steps of collecting performance index data of a software application program, establishing a Prophet time sequence prediction model, predicting the performance trend of the software application, monitoring the performance data of the software application through an isolated forest algorithm, implementing real-time anomaly detection, and presenting an analysis result to a software application user through a visual tool. By means of the big data analysis technology, accurate prediction and real-time monitoring of software application performance are achieved, the efficiency and accuracy of performance optimization are effectively improved, meanwhile, the analysis result is visually presented, a user can visually understand the software performance condition, and stable operation and optimization upgrading of software application are promoted.

Description

Technical Field

[0001] The present invention relates to the technical field of data analysis and mining, and in particular to a software application performance optimization method based on big data analysis. Background Art

[0002] Software application performance refers to the comprehensive capabilities of software in terms of efficiency, response speed, stability and resource utilization during operation. It is one of the important indicators for measuring software quality and is directly related to user experience and system performance. First, efficiency refers to the speed and ability of software to process tasks, including the degree of algorithm optimization and the speed of data processing. Efficient software can complete more tasks in a shorter time, thereby improving user efficiency. Second, response speed refers to the reaction time of software to user operations. Quick response can improve user satisfaction and make the software smoother and easier to use. Stability refers to the ability of software to not fail or crash during long-term operation. Stable software can continue to provide services and ensure business continuity. Unstable software may cause data loss, service interruption and other problems, causing losses to users. Finally, resource utilization refers to the occupation of system resources by software during operation, including CPU, memory, disk, etc. Efficient resource utilization can reduce operating costs and improve the overall performance of the system. In summary, software application performance is an important manifestation of software quality, which is related to multiple aspects such as user experience, system performance and operating costs. Therefore, in the process of software development and maintenance, it is necessary to pay attention to the optimization and improvement of software application performance.

[0003] In order to solve the problems of software performance optimization in performance bottleneck identification and anomaly detection, predictive maintenance and trend prediction, the existing technology uses traditional monitoring tools and methods, such as log analysis, performance counter monitoring, etc. However, these methods can only provide limited information, making it difficult to fully and accurately identify performance bottlenecks and anomalies, and lack the ability to predict future performance trends. As a result, during the operation of the software, problems such as performance degradation and frequent failures may occur, seriously affecting user experience and business continuity. Therefore, a software application performance optimization method based on big data analysis is proposed. Summary of the invention

[0004] The purpose of the present invention is to provide a software application performance optimization method based on big data analysis to solve the problems raised in the above background technology.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is: a method for optimizing software application performance based on big data analysis, comprising the following steps: S1. Use distributed tracing technology to track software application service call link data, use the log aggregation system to collect log data during software application runtime, and use performance monitoring tools to collect performance indicator data of software applications; S2. Establish the Prophet time series prediction model to predict the performance trend of software applications, monitor the performance data of software applications through the isolation forest algorithm, and implement real-time anomaly detection; S3. Present the analysis results to software application users through visualization tools.

[0006] A further improvement of the technical solution of the present invention is that in S1, the process of tracking the software application service call link data by distributed tracing technology includes: Deploy distributed tracing tools on each service node of the software application to record request and response information between services. When each service instance starts, the tracing component is automatically loaded and initialized. When a user initiates a request and reaches the software application entrance, the distributed tracing tool generates a unique tracing identifier, attaches the tracing identifier to the request, and passes it between different services. Each service receives the request with the tracing identifier, records the log information processed locally, and continues to carry the same tracing identifier for downstream requests, forming a call chain graph. Preprocessing the collected original service call link data, wherein the preprocessing operation includes filtering out incomplete tracking records and repeated requests, storing them in a unified format, and correcting timestamps; Key features that assist in identifying performance bottlenecks and service dependencies are extracted from the preprocessed service call link data, wherein the key features include request paths, response times, and error codes.

[0007] A further improvement of the technical solution of the present invention is that in S2, the process of collecting log data when the software application is running through the log aggregation system includes: Deploy a log aggregation system on each service node of the software application. The log aggregation system collects log information from different services. When each service instance starts, the log aggregation component is automatically loaded and initialized. When a user initiates a request and reaches the software application entrance, the log data generated by each service in the process of processing the request is recorded in real time. The log aggregation system collects the log data containing timestamps, service names and operation details through the network and file system, and stores them centrally; Preprocessing the collected log data, wherein the preprocessing operation includes deleting meaningless and repeated log records, converting the logs into a standardized format, and correcting the timestamp; Classify the information, warnings, and errors in the log to distinguish events of different severity, identify and record the specific operations and method names described in the log, parse the start and end time of each operation from the log, calculate the operation duration, and when the operation fails, extract the error code and additional diagnostic information, and record the user ID and session ID related to the operation.

[0008] A further improvement of the technical solution of the present invention is that in S3, the process of collecting the performance indicator data of the software application through the performance monitoring tool includes: Deploy a performance monitoring tool on each service node of the software application. The performance monitoring tool collects performance indicator data during service operation. When each service instance is started, the performance monitoring component is automatically loaded and initialized. When a user initiates a request and reaches the software application entrance, the performance indicator data generated by each service in the process of processing the request is captured in real time. The performance monitoring tool obtains the performance indicator data with timestamps and service identifiers through system calls and interfaces. The performance indicator data includes but is not limited to CPU usage, memory usage, disk read and write speed, and network transmission rate; Preprocessing the collected performance indicator data includes identifying and removing out-of-range values ​​and erroneous format data, detecting and deleting identical performance indicator records by comparing timestamps, service identifiers, and service names, defining standard field names and types for each performance indicator, converting the performance indicator data into a structured table format so that each row represents a complete performance snapshot at a point in time, each column corresponds to a performance indicator, correcting timestamps, and aggregating periodic data by minutes; Extract key features from the preprocessed performance indicator data to assist in identifying performance bottlenecks and service dependencies. The key features include calculating the average, peak, and fluctuation of CPU usage, monitoring total memory usage, available memory, and garbage collection frequency, counting the speed and number of disk I / O operations, identifying whether there are bottlenecks, measuring inbound and outbound traffic, and analyzing network latency and packet loss rates.

[0009] A further improvement of the technical solution of the present invention is that in S2, the process of initializing the Prophet time series prediction model and predicting the software application performance trend includes: Select Prophet as the forecasting tool, set model parameters, define the linear growth rate, set the corresponding frequency and mode based on the daily cycle factors in the historical data, and add holidays and dates that affect performance and the degree of impact to the Prophet model.

[0010] A further improvement of the technical solution of the present invention is that in S2, the process of training the Prophet time series prediction model, generating performance indicator values, and predicting software application performance trends includes: The collected historical performance indicator data are divided into training set and test set in chronological order, where the early performance indicator data are divided into the training set and the recent performance indicator data are divided into the test set. The trend function g(t) is used to fit the trend part of the historical data to determine the overall rising and falling direction. The seasonal function s(t) is used to capture the changes in software application performance in the same time period every day. The holiday effect function h(t) is used to quantify the impact of special events on performance. The performance of the test model in different time periods is tested by cross-validation, and the performance indicator value The calculation process is as follows: ; in, Represents random noise and unpredictable factors; Generate future time points, use the trained Prophet model to predict future time points, output performance indicator values ​​and their confidence intervals, combine the prediction results with the original performance indicator data, form a complete performance trend chart, and display the future performance trend of the software application.

[0011] A further improvement of the technical solution of the present invention is that in S2, the process of evaluating and updating the Prophet time series prediction model includes: We selected mean square error, mean absolute error, and mean absolute percentage error as evaluation indicators, used incremental learning technology to dynamically adjust model parameters, periodically retrained the Prophet model, established a feedback loop from prediction to problem solving, and collected feedback from the operation and maintenance team.

[0012] A further improvement of the technical solution of the present invention is that in S7, the process of training the isolation forest model and monitoring the performance data of the software application includes: Construct an isolation forest model, set the isolation forest model parameters, define the number of trees in the forest, extract data points from the training set to build each tree, use the normal data points in the training set to train the isolation forest model, so that the isolation forest model learns the distribution pattern of normal data. The isolation forest model isolates data points by recursively selecting features and random cutting points of the features until the data points are completely isolated, and calculates the anomaly score of each data point. The calculation process is as follows: ; in, represents the expected value of the path length required for the data point x to be isolated, It represents the path length of the data point x isolated in a single tree, and the 99% of the anomaly score distribution is set as the threshold to distinguish normal points from abnormal points.

[0013] A further improvement of the technical solution of the present invention is that in S8, the process of implementing real-time anomaly detection by the isolation forest model includes: Use the trained isolation forest model to make real-time predictions on the new data collected, calculate the anomaly score of each new data point, and if the anomaly score of a new data point exceeds the threshold, mark it as an anomaly point and trigger the alarm mechanism; if the anomaly score of a new data point is lower than the threshold, mark it as a normal point, combine the prediction results with the original performance indicator data, and form a time series graph containing anomaly markers to show the moment when the abnormal behavior occurs; The confusion matrix, precision, and recall are used as evaluation metrics to measure the anomaly detection effect of the model. The isolation forest model is retrained regularly to perform anomaly detection based on the latest data, thus establishing a feedback loop from anomaly detection to problem solving.

[0014] A further improvement of the technical solution of the present invention is that: in S3, the process of presenting the analysis results to the software application user through the visualization tool Tableau includes: Create key charts, including performance trend charts that show actual and predicted performance changes, anomaly detection charts that mark abnormal time points, and resource usage charts that reflect changes in CPU usage and memory usage over time. Provide interactive exploration functions, allowing users to click on specific time periods and anomaly points to view detailed information, set data to be filtered by service name and time range, provide an instant alarm mechanism, trigger an alarm to remind users when an anomaly is detected, generate periodic reports to summarize performance and abnormal events, conduct comprehensive analysis, associate data from different sources to reveal service dependencies and service bottlenecks, provide preliminary diagnostic suggestions based on anomaly detection results and performance trends, guide optimization measures, and establish a feedback loop from visualization to problem solving.

[0015] Due to the adoption of the above technical solution, the present invention has the following technical advances compared with the prior art: 1. The present invention provides a software application performance optimization method based on big data analysis. Through multi-dimensional data collection methods such as distributed tracing, log aggregation and performance monitoring, the performance data of the software application during runtime can be comprehensively and accurately obtained, providing a solid foundation for performance optimization.

[0016] 2. The present invention provides a software application performance optimization method based on big data analysis. It uses the Prophet time series prediction model and the isolation forest algorithm to achieve accurate prediction of software application performance and real-time anomaly detection, effectively improving the efficiency and accuracy of performance optimization and reducing the risk of business interruption caused by performance problems.

[0017] 3. The present invention provides a software application performance optimization method based on big data analysis, which presents the analysis results through visualization tools, so that software application users can understand the software performance status intuitively and conveniently, promotes communication and collaboration between users and developers, and helps to jointly promote the continuous optimization and improvement of software application performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0019] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] Examples, such as Figure 1 As shown, the present invention provides a software application performance optimization method based on big data analysis, comprising the following steps: S1. Track the call link data of software application services through distributed tracing technology. Collect the log data of software application runtime through log aggregation system. Collect the performance indicator data of software application through performance monitoring tool. Deploy distributed tracing tools at each service node of software application to record the request and response information between services. When each service instance starts, automatically load and initialize the tracing component. When the user initiates a request to the software application entrance, the distributed tracing tool generates a unique tracing identifier, attaches this tracing identifier to the request, and passes it between different services. Each service receives the request with the tracing identifier, records the log information processed locally, and continues to carry the same tracing identifier for downstream requests to form a call link graph. Preprocess the collected original service call link data. The preprocessing operation includes filtering out incomplete tracing records and repeated requests, storing them in a unified format, correcting timestamps, and extracting key features from the preprocessed service call link data to assist in identifying performance bottlenecks and service dependencies. The key features include request path, response time and error code.A log aggregation system is deployed on each service node of the software application. The log aggregation system collects log information from different services. When each service instance is started, the log aggregation component is automatically loaded and initialized. When a user initiates a request and arrives at the software application entrance, the log data generated by each service in the process of processing the request is recorded in real time. The log aggregation system collects log data containing timestamps, service names and operation details through the network and file system, and stores them centrally. The collected log data is preprocessed. The preprocessing operation includes deleting meaningless and repeated log records, converting logs into standardized formats, correcting timestamps, classifying according to log information, warnings and errors to distinguish events of different severity, identifying and recording the specific operations and method names described in the log, parsing the start time and end time of each operation from the log, calculating the operation time, extracting error codes and additional diagnostic information when the operation fails, and recording user identifiers and session identifiers related to the operation. A performance monitoring tool is deployed on each service node of the software application. The performance monitoring tool collects performance indicator data during the service operation. When each service instance is started, the performance monitoring component is automatically loaded and initialized. When a user initiates a request and arrives at the software application entrance, the performance monitoring tool ... When the service is opened, the performance indicator data generated by each service in the process of processing the request is captured in real time. The performance monitoring tool obtains the performance indicator data with timestamps and service identifiers through system calls and interfaces. The performance indicator data includes but is not limited to the CPU usage, memory usage, disk read and write speed and network transmission rate. The collected performance indicator data is preprocessed. The preprocessing operation includes identifying and removing out-of-range values ​​and data with incorrect format. By comparing the timestamp, service identifier and service name, the completely identical performance indicator records are detected and deleted. Each performance indicator defines a standard field name and type. The performance indicator data is converted into a structured table form so that each row represents a complete performance snapshot at a point in time and each column corresponds to a performance indicator. The timestamp is corrected and the periodic data is summarized by minutes. The key features that assist in identifying performance bottlenecks and service dependencies are extracted from the preprocessed performance indicator data. The key features include calculating the average, peak and fluctuation of the CPU usage, monitoring the total memory usage, available amount and garbage collection frequency, counting the speed and number of disk I / O operations, identifying whether there is a bottleneck, measuring inbound and outbound traffic, and analyzing network latency and packet loss rate. S2. Establish the Prophet time series prediction model to predict the performance trend of software applications. Use the isolation forest algorithm to monitor the performance data of software applications and implement real-time anomaly detection. Select Prophet as the prediction tool, set model parameters, define the linear growth rate, set the corresponding frequency and mode according to the daily cycle factors in the historical data, and add holidays and the dates and degree of impact on performance to the Prophet model. Divide the collected historical performance indicator data into training sets and test sets in chronological order, of which the early performance indicator data is divided into training sets and the recent performance indicator data is divided into test sets. Fit the trend part of the historical data through the trend function g(t) to determine the overall rising and falling direction. Use the seasonal function s(t) to capture the changes in software application performance in the same time period every day. Use the holiday effect function h(t) to quantify the impact of special events on performance. Test the performance of the model in different time periods by cross-validation. The performance indicator value The calculation process is as follows: ; in, Represent random noise and unpredictable factors, generate future time points, use the trained Prophet model to predict future time points, output performance indicator values ​​and their confidence intervals, combine the prediction results with the original performance indicator data to form a complete performance trend chart, and display the future performance trend of the software application. Select mean square error, mean absolute error, and mean absolute percentage error as evaluation indicators, use incremental learning technology to dynamically adjust model parameters, periodically retrain the Prophet model, establish a feedback loop from prediction to problem solving, and collect feedback from the operation and maintenance team, build an isolation forest model, set isolation forest model parameters, define the number of trees in the forest, extract data points from the training set to build each tree, use normal data points in the training set to train the isolation forest model, so that the isolation forest model learns the distribution pattern of normal data, and isolates data points by recursively selecting features and random cut points of the features until the data points are completely isolated. Calculate the anomaly score of each data point. The calculation process is as follows: ; in, represents the expected value of the path length required for the data point x to be isolated, Indicates the path length of the data point x isolated in a single tree. Set 99% of the anomaly score distribution as the threshold to distinguish normal points from abnormal points. Use the trained isolation forest model to make real-time predictions on the collected new data. Calculate the anomaly score of each new data point. If the anomaly score of the new data point exceeds the threshold, it is marked as an anomaly and the alarm mechanism is triggered. If the anomaly score of the new data point is lower than the threshold, it is marked as a normal point. Combine the prediction results with the original performance indicator data to form a time series graph containing anomaly identification to show the moment when abnormal behavior occurs. Use confusion matrix, accuracy and recall as evaluation indicators to measure the anomaly detection effect of the model. Regularly retrain the isolation forest model to perform anomaly detection based on the latest data, and establish a feedback loop from anomaly detection to problem solving. S3. Present the analysis results to software application users through visualization tools and create key charts, including performance trend charts showing actual and predicted performance changes, anomaly detection charts with annotated abnormal time points, and resource usage charts reflecting changes in CPU usage and memory usage over time. Provide interactive exploration functions, allowing users to click on specific time periods and abnormal points to view detailed information, set data filtering by service name and time range, provide an instant alarm mechanism, trigger an alarm to remind users when an anomaly is detected, generate periodic reports to summarize performance and abnormal events, conduct comprehensive analysis, associate data from different sources to reveal service dependencies and service bottlenecks, provide preliminary diagnostic suggestions based on anomaly detection results and performance trends, guide optimization measures, and establish a feedback loop from visualization to problem solving.

[0022] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A software application performance optimization method based on big data analysis, characterized in that: The following steps are involved: S1. Use distributed tracing technology to track software application service call link data, use the log aggregation system to collect log data during software application runtime, and use performance monitoring tools to collect performance indicator data of software applications; S2. Establish the Prophet time series prediction model to predict the performance trend of software applications, monitor the performance data of software applications through the isolation forest algorithm, and implement real-time anomaly detection; S3. Present the analysis results to software application users through visualization tools.

2. The method for optimizing software application performance based on big data analysis according to claim 1, characterized in that: In S1, the process of tracking the software application service call link data through distributed tracing technology includes: Deploy distributed tracing tools on each service node of the software application to record request and response information between services. When each service instance starts, the tracing component is automatically loaded and initialized. When a user initiates a request and reaches the software application entrance, the distributed tracing tool generates a unique tracing identifier, attaches the tracing identifier to the request, and passes it between different services. Each service receives the request with the tracing identifier, records the log information processed locally, and continues to carry the same tracing identifier for downstream requests, forming a call chain graph. Preprocessing the collected original service call link data, wherein the preprocessing operation includes filtering out incomplete tracking records and repeated requests, storing them in a unified format, and correcting timestamps; Key features that assist in identifying performance bottlenecks and service dependencies are extracted from the preprocessed service call link data, wherein the key features include request paths, response times, and error codes.

3. The method for optimizing software application performance based on big data analysis according to claim 2, characterized in that: In S1, the process of collecting log data of the software application during operation through the log aggregation system includes: Deploy a log aggregation system on each service node of the software application. The log aggregation system collects log information from different services. When each service instance starts, the log aggregation component is automatically loaded and initialized. When a user initiates a request and reaches the software application entrance, the log data generated by each service in the process of processing the request is recorded in real time. The log aggregation system collects the log data containing timestamps, service names and operation details through the network and file system, and stores them centrally; Preprocessing the collected log data, wherein the preprocessing operation includes deleting meaningless and repeated log records, converting the logs into a standardized format, and correcting the timestamp; Classify the information, warnings, and errors in the log to distinguish events of different severity, identify and record the specific operations and method names described in the log, parse the start and end time of each operation from the log, calculate the operation duration, and when the operation fails, extract the error code and additional diagnostic information, and record the user ID and session ID related to the operation.

4. The method for optimizing software application performance based on big data analysis according to claim 3, characterized in that: In S1, the process of collecting the performance indicator data of the software application through the performance monitoring tool includes: Deploy a performance monitoring tool on each service node of the software application. The performance monitoring tool collects performance indicator data during service operation. When each service instance is started, the performance monitoring component is automatically loaded and initialized. When a user initiates a request and reaches the software application entrance, the performance indicator data generated by each service in the process of processing the request is captured in real time. The performance monitoring tool obtains the performance indicator data with timestamps and service identifiers through system calls and interfaces. The performance indicator data includes CPU usage, memory usage, disk read and write speed, and network transmission rate; Preprocessing the collected performance indicator data includes identifying and removing out-of-range values ​​and erroneous format data, detecting and deleting identical performance indicator records by comparing timestamps, service identifiers, and service names, defining standard field names and types for each performance indicator, converting the performance indicator data into a structured table format so that each row represents a complete performance snapshot at a point in time, each column corresponds to a performance indicator, correcting timestamps, and aggregating periodic data by minutes; Extract key features from the preprocessed performance indicator data to assist in identifying performance bottlenecks and service dependencies. The key features include calculating the average, peak, and fluctuation of CPU usage, monitoring total memory usage, available memory, and garbage collection frequency, counting the speed and number of disk I / O operations, identifying whether there are bottlenecks, measuring inbound and outbound traffic, and analyzing network latency and packet loss rates.

5. The method for optimizing software application performance based on big data analysis according to claim 4, characterized in that: In S2, the process of initializing the Prophet time series prediction model and predicting the software application performance trend includes: Select Prophet as the forecasting tool, set model parameters, define the linear growth rate, set the corresponding frequency and mode based on the daily cycle factors in the historical data, and add holidays and dates that affect performance and the degree of impact to the Prophet model.

6. The method for optimizing software application performance based on big data analysis according to claim 5, characterized in that: In S2, the process of training the Prophet time series prediction model, generating performance indicator values, and predicting software application performance trends includes: The collected historical performance indicator data are divided into training set and test set in chronological order, where the early performance indicator data are divided into the training set and the recent performance indicator data are divided into the test set. The trend function g(t) is used to fit the trend part of the historical data to determine the overall rising and falling direction. The seasonal function s(t) is used to capture the changes in software application performance in the same time period every day. The holiday effect function h(t) is used to quantify the impact of special events on performance. The performance of the test model in different time periods is tested by cross-validation, and the performance indicator value The calculation process is as follows: ； in, Represents random noise and unpredictable factors; Generate future time points, use the trained Prophet model to predict future time points, output performance indicator values ​​and their confidence intervals, combine the prediction results with the original performance indicator data, form a complete performance trend chart, and display the future performance trend of the software application.

7. The method for optimizing software application performance based on big data analysis according to claim 6, characterized in that: In S2, the process of evaluating and updating the Prophet time series forecasting model includes: We selected mean square error, mean absolute error, and mean absolute percentage error as evaluation indicators, used incremental learning technology to dynamically adjust model parameters, periodically retrained the Prophet model, established a feedback loop from prediction to problem solving, and collected feedback from the operation and maintenance team.

8. The method for optimizing software application performance based on big data analysis according to claim 7, characterized in that: In S2, the process of training the isolation forest model and monitoring the performance data of the software application includes: Construct an isolation forest model, set the isolation forest model parameters, define the number of trees in the forest, extract data points from the training set to build each tree, use the normal data points in the training set to train the isolation forest model, so that the isolation forest model learns the distribution pattern of normal data. The isolation forest model isolates data points by recursively selecting features and random cutting points of the features until the data points are completely isolated, and calculates the anomaly score of each data point. The calculation process is as follows: ； in, represents the expected value of the path length required for the data point x to be isolated, It represents the path length of the data point x isolated in a single tree, and the 99% of the anomaly score distribution is set as the threshold to distinguish normal points from abnormal points.

9. The method for optimizing software application performance based on big data analysis according to claim 8, characterized in that: In S2, the process of implementing real-time anomaly detection using the isolation forest model includes: Use the trained isolation forest model to make real-time predictions on the new data collected, calculate the anomaly score of each new data point, and if the anomaly score of a new data point exceeds the threshold, mark it as an anomaly point and trigger the alarm mechanism; if the anomaly score of a new data point is lower than the threshold, mark it as a normal point, combine the prediction results with the original performance indicator data, and form a time series graph containing anomaly markers to show the moment when the abnormal behavior occurs; The confusion matrix, precision, and recall are used as evaluation metrics to measure the anomaly detection effect of the model. The isolation forest model is retrained regularly to perform anomaly detection based on the latest data, thus establishing a feedback loop from anomaly detection to problem solving.

10. The method for optimizing software application performance based on big data analysis according to claim 9, characterized in that: In the S3, the process of presenting the analysis results to the software application user through the visualization tool Tableau includes: Create key charts, including performance trend charts that show actual and predicted performance changes, anomaly detection charts that mark abnormal time points, and resource usage charts that reflect changes in CPU usage and memory usage over time. Provide interactive exploration functions, allowing users to click on specific time periods and anomaly points to view detailed information, set data to be filtered by service name and time range, provide an instant alarm mechanism, trigger an alarm to remind users when an anomaly is detected, generate periodic reports to summarize performance and abnormal events, conduct comprehensive analysis, associate data from different sources to reveal service dependencies and service bottlenecks, provide preliminary diagnostic suggestions based on anomaly detection results and performance trends, guide optimization measures, and establish a feedback loop from visualization to problem solving.

Images