Method and device for determining performance bottleneck index of system and electronic equipment

Abstract

The invention discloses a method and device for determining performance bottleneck indexes of a system and electronic equipment. The method comprises the following steps: acquiring a first time delay data set and a first index data set for processing each request in a first request set by a production system; obtaining a second request set corresponding to the first request set from the target system, and obtaining a second time delay data set and a second index data set for processing each request in the second request set by the target system; training a preset neural network model according to the time delay data set and the index data set to obtain a first prediction model and a second prediction model; and determining a performance bottleneck index of the target system according to the first prediction model and the second prediction model. According to the method and the device, the problem of relatively low efficiency of manually determining the performance bottleneck index of the system in related technologies is solved.

Description

Technical Field

[0001] This application relates to the field of big data, and in particular, to a method, apparatus, and electronic device for determining performance bottleneck indicators of a system. Background Art

[0002] In the emerging field, as the degree of localization of each part of the system gradually increases, it is necessary to evaluate the key performance indicators of the system to determine whether the localized system can operate normally.

[0003] With the rapid development of information technology, the scale and complexity of Internet applications are constantly increasing, and the performance of application systems in processing HTTP requests has become an important standard for evaluating system operation efficiency. The performance of application systems is usually evaluated through multiple key performance indicators, including CPU occupancy, memory usage, number of waiting connections, switch utilization, etc. In traditional performance testing, benchmark testing and stress testing methods are often used, and the performance level of the system is judged by setting thresholds for key performance indicators.

[0004] However, although the above testing methods can help developers understand the performance of the system, they cannot further locate the specific performance bottleneck in case of performance anomalies.

[0005] Currently, the troubleshooting and location of performance bottlenecks mainly rely on the experience and manual analysis of developers. This manual method is inefficient and consumes a large amount of time and human resources. Especially when facing complex and multi-dimensional application system performance data, it is difficult to reveal deep-seated performance problems simply by relying on threshold judgment.

[0006] In view of the problem of low efficiency in manually determining the performance bottleneck indicators of the system in the related art, no effective solution has been proposed yet. Summary of the Invention

[0007] This application provides a method, apparatus, and electronic device for determining performance bottleneck indicators of a system to solve the problem of low efficiency in manually determining the performance bottleneck indicators of the system in the related art.

[0008] According to one aspect of the present application, a method for determining performance bottleneck metrics of a system is provided. The method includes: obtaining a first latency data set and a first metric data set for the production system to process each request in the first request set; obtaining a second request set corresponding to the first request set from the target system, and obtaining a second latency data set and a second metric data set for the target system to process each request in the second request set, where the requests included in the first request set are the same as the requests included in the second request set; training a preset neural network model according to the first latency data set and the first metric data set to obtain a first prediction model, and training the preset neural network model according to the second latency data set and the second metric data set to obtain a second prediction model; determining the performance bottleneck metrics of the target system based on the first prediction model and the second prediction model.

[0009] Optionally, determining the performance bottleneck metrics of the target system based on the first prediction model and the second prediction model includes: respectively obtaining a first metric importance sequence of the production system and a second metric importance sequence of the target system from the first prediction model and the second prediction model; determining the performance bottleneck metrics of the target system according to the first metric importance sequence and the second metric importance sequence.

[0010] Optionally, determining the performance bottleneck metrics of the target system according to the first metric importance sequence and the second metric importance sequence includes: determining whether the first metric importance sequence and the second metric importance sequence are the same; in the case where the first metric importance sequence and the second metric importance sequence are the same, determining the performance bottleneck metric of the target system as the first performance metric in the second metric importance sequence; in the case where the first metric importance sequence and the second metric importance sequence are different, obtaining the performance metrics whose serial numbers in the first metric importance sequence are greater than those in the second metric importance sequence, to obtain at least one performance bottleneck metric.

[0011] Optionally, the first request set includes M first requests, the first latency data set includes M first latency data, and the first metric data set includes M groups of first metric data. Obtaining the first latency data set and the first metric data set for the production system to process each request in the first request set includes: for any one of the first requests, obtaining the processed data obtained by the production system processing the first request according to the attribute information of the first request, and obtaining the first latency data from the processed data; according to the timestamp in the processed data, obtaining the performance metric data corresponding to the processed data in the host database of the production system, to obtain the first metric data.

[0012] Optionally, according to the timestamps in the processed data, obtain performance metric data corresponding to the processed data in the host database of the production system. The obtained first metric data includes: obtaining the collection frequency of the performance metric data in the production system; in the case where the collection frequency is greater than or equal to the preset frequency, determining the performance metric data in the production system that is the same as the time of the timestamp as the first metric data of the first latency data; in the case where the collection frequency is less than the preset frequency, determining the time interval according to the collection frequency and the timestamp, and determining the performance metric data within the time interval as the first metric data of the first latency data.

[0013] Optionally, the second request set includes M second requests, the second latency data set includes M second latency data, and the second metric data set includes M groups of second metric data. Obtaining the second request set corresponding to the first request set from the target system includes: for any one first request, obtaining the processed requests with the same attribute information in the database of the target system according to the attribute information of the first request, obtaining a plurality of processed requests; sequentially determining the time difference between each processed request and the first request, and obtaining the processed requests with a time difference less than the preset time difference, obtaining the second requests corresponding to the first request; combining the second requests corresponding to each first request in the database into a second request set.

[0014] Optionally, before training the preset neural network model according to the first latency data set and the first metric data set to obtain the first prediction model, the method further includes: performing a preprocessing operation on each group of first metric data in the first metric data set to obtain a third metric data set, where the preprocessing operation includes at least one of the following: feature scaling, standardization, normalization; determining whether there are outliers in the third metric data set; in the case where there are outliers in the third metric data set, performing smoothing processing on the outliers to obtain a fourth metric data set; determining the metric data correlated with the latency data in the fourth metric data set by the Pearson correlation coefficient method to obtain a fifth metric data set; using the fifth metric data set to replace the first metric data set, and performing the operation of training the preset neural network model according to the first latency data set and the first metric data set to obtain the first prediction model.

[0015] According to another aspect of the present application, a device for determining performance bottleneck metrics of a system is provided. The device includes: a first acquisition unit configured to acquire a first latency data set and a first metric data set for the production system to process each request in the first request set; a second acquisition unit configured to acquire a second request set corresponding to the first request set from the target system, and acquire a second latency data set and a second metric data set for the target system to process each request in the second request set, wherein the requests included in the first request set are the same as the requests included in the second request set; a training unit configured to train a preset neural network model according to the first latency data set and the first metric data set to obtain a first prediction model, and train the preset neural network model according to the second latency data set and the second metric data set to obtain a second prediction model; and a first determination unit configured to determine the performance bottleneck metrics of the target system based on the first prediction model and the second prediction model.

[0016] According to another aspect of the present invention, a computer program product is further provided, including a computer program, which when executed by a processor, implements a method for determining performance bottleneck metrics of a system provided in the foregoing embodiments of the present application.

[0017] According to another aspect of the present invention, an electronic device is further provided, including one or more processors and a memory; the memory stores computer-readable instructions, and the processor is configured to run the computer-readable instructions, wherein the computer-readable instructions, when running, execute the method for determining performance bottleneck metrics of a system provided in the foregoing embodiments.

[0018] Through this application, the following steps are adopted: obtaining a first delay data set and a first metric data set of the production system for processing each request in the first request set; obtaining a second request set corresponding to the first request set from the target system, and obtaining a second delay data set and a second metric data set of the target system for processing each request in the second request set, wherein the requests included in the first request set are the same as those included in the second request set; training a preset neural network model according to the first delay data set and the first metric data set to obtain a first prediction model, and training the preset neural network model according to the second delay data set and the second metric data set to obtain a second prediction model; determining the performance bottleneck metrics of the target system based on the first prediction model and the second prediction model. This solves the problem of low efficiency in manually determining the performance bottleneck metrics of the system in the related art. By using the processing results of processing requests between the production system and the target system as the training set to train the model, on the basis of obtaining the prediction model, the key metric sequences generated by the prediction model during the learning of the training set can be used to determine which metrics are the bottleneck metrics affecting the delay data, so as to quickly and accurately obtain the bottleneck metrics of the target system, and thus achieve the technical effect of improving the determination efficiency of the bottleneck metrics. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings constituting a part of this application are used to provide a further understanding of this application. The schematic embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation to this application. In the accompanying drawings:

[0020] Figure 1 is a flowchart of a method for determining the performance bottleneck metrics of a system according to an embodiment of this application;

[0021] Figure 2 is a flowchart of a request alignment method according to an embodiment of this application;

[0022] Figure 3 is a flowchart of an optional performance metric data processing flow of a system according to an embodiment of this application;

[0023] Figure 4 is a flowchart of an optional model training flow of a system according to an embodiment of this application;

[0024] Figure 5 is a schematic diagram of a device for determining the performance bottleneck metrics of a system according to an embodiment of this application;

[0025] Figure 6 is a schematic diagram of an electronic device according to an embodiment of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments may be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0027] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the protection scope of the present application.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so as to implement the embodiments of the present application described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily limit to those clearly listed steps or units, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0029] It should be noted that the method, device, and electronic device for determining the performance bottleneck index of the system determined in the present disclosure can be used in the field of big data, and can also be used in any field other than the field of big data. The application field of the method, device, and electronic device for determining the performance bottleneck index of the system determined in the present disclosure is not limited.

[0030] It should be noted that the information collected, user information (including but not limited to user device information, user personal information, etc.), and data (including but not limited to data for analysis, stored data, displayed data, etc.) used in the present application are all information and data authorized by the user or fully authorized by all parties. And the processing of relevant data, such as collection, storage, use, processing, transmission, provision, disclosure, and application, all comply with the relevant laws, regulations, and standards in the relevant regions, take necessary confidentiality measures, do not violate public order and good customs, and provide corresponding operation entrances for users to choose to authorize or refuse to use. If the user chooses to refuse, the expert decision-making process will be entered. For example, there is an interface between the present system and relevant users or institutions. Before obtaining relevant information, a request for acquisition needs to be sent to the aforementioned users or institutions through the interface, and after receiving the consent information feedback from the aforementioned users or institutions, the relevant information can be obtained.

[0031] The embodiments or examples of the present disclosure are not exhaustive. They are only schematic representations of some embodiments or examples and do not constitute specific limitations on the protection scope of the present disclosure. Without contradiction, each step in a certain embodiment or example can be implemented as an independent example, and the steps can be combined arbitrarily. For example, a solution obtained by removing some steps in a certain embodiment or example can also be implemented as an independent example, and the order of the steps in a certain embodiment or example can be exchanged arbitrarily. Additionally, the optional methods or optional examples in a certain embodiment or example can be combined arbitrarily; moreover, the embodiments or examples can be combined arbitrarily. For example, some or all of the steps of different embodiments or examples can be combined arbitrarily, and a certain embodiment or example can be combined arbitrarily with the optional methods or optional examples of other embodiments or examples.

[0032] For ease of description, some nouns or terms related to the embodiments of the present application are described below:

[0033] Random forest model analysis method: It is an algorithm based on ensemble learning and is widely used in classification, regression, and feature importance analysis. By constructing multiple decision trees, it can effectively capture the complex interaction relationships between multiple features, especially showing high accuracy and robustness when facing high-dimensional data. Due to the ability of the random forest to automatically evaluate feature importance and handle non-linear problems, it can help identify the factors that have the greatest impact on the processing delay of the application system from a large number of performance indicators.

[0034] HTTP request: An HTTP request is a request message sent by a client to a server to obtain resources or perform operations. HTTP stands for Hypertext Transfer Protocol, which is the basic protocol for transmitting data on the Web.

[0035] PCAP packet: "Packet Capture", is a file format for capturing network packets, usually used for network analysis and troubleshooting.

[0036] According to an embodiment of the present application, a method for determining the performance bottleneck indicators of a system is provided.

[0037] Figure 1 It is a flowchart of the method for determining the performance bottleneck indicators of the system provided according to the embodiments of the present application. As Figure 1 shown, the method includes the following steps:

[0038] Step S101, obtain a first delay data set and a first metric data set for the production system to process each request in the first request set.

[0039] It should be noted that the production system can be a system environment that is operating normally and is the system that actually provides services. For example, the production system can be a server host that processes HTTP requests. Each module in the production system, such as the CPU, database, etc., can be the production system that was operating normally before localization. The first request set can be a set of HTTP requests collected in the production system, including various types of requests such as GET, POST, etc., and the relevant data of these requests is recorded in the system for subsequent analysis. The purpose of collecting the request set is to have a performance baseline in the production system so that it can be compared and analyzed with the target system. The first latency data set can be the actual response time data when the above first request set is processed in the production system. Latency refers to the entire time difference from the start of the request to the complete processing of the request and the return of the response. The purpose of recording the latency data is to evaluate the performance of the production system when processing requests, that is, the system response speed. The first metric data set can be the system performance metric data when the production system processes each HTTP request, such as CPU occupancy rate, memory usage rate, disk I / O read / write rate, network bandwidth utilization rate, etc. These data reflect the resource consumption situation of the system when processing requests and are used for subsequent analysis of which resource usage situations affect the processing latency of the requests.

[0040] Specifically, when determining the performance bottleneck metrics of the system, first, it is necessary to determine the impact of each performance metric on the latency data under normal operation of the system. At this time, it is necessary to obtain the latency data and the corresponding metric data for each request processed by the production system, so that the correlation between each metric data and the latency data can be determined according to the metric data and the latency data, and then the impact of each performance metric on the latency data can be determined.

[0041] For example, the target system received 1000 user HTTP requests within a certain period of time, and these requests constituted the "first request set". Subsequently, the processing latency (i.e., response time) of each login request can be captured through an analysis tool, and a series of data such as 0.71 seconds, 51.4 milliseconds, 25.1 milliseconds, etc. are obtained, which constitute the "first latency data set". At the same time, the CPU occupancy rate, memory usage rate and other performance metrics when processing these requests can be recorded through a system monitoring tool to obtain the "first metric data set", so as to obtain various data of the target system when processing requests.

[0042] Further, after obtaining various data of the system processing request, for example, the metric data are: the CPU occupancy rate is A, the memory usage rate is B, the disk I / O read / write rate is C, and the network bandwidth utilization rate is D. In this case, the latency data is X. At this time, based on multiple pairs of metric data and latency data, the performance metric that has the greatest impact on the latency data can be determined, and the degree of influence of each performance metric on the latency data can be sorted to obtain the influence of each performance metric in the target system on the latency data.

[0043] It should be noted that the information of the request latency data can be obtained from the PCAP data packet. Among them, the PCAP data packet contains the original network data frames passing through the network interface, and these data frames contain the header information of the data packet (such as source address, destination address, protocol type, etc.) and the payload data (i.e., the information actually transmitted in the data packet). By parsing these data packets, analysts can understand the details of network communication in detail, including but not limited to: timestamp, source and destination addresses, protocol information, payload data, etc.

[0044] In performance testing and bottleneck analysis, since the PCAP data packet records the original details of network communication, it can be used to analyze the time of requests and responses, check the data transmission efficiency, and identify any network problems that may affect performance. For example, capturing and analyzing PCAP files can deeply understand the whole process of HTTP request processing, including request sending, receiving responses, data transmission rate, etc., thereby helping to locate and solve performance bottlenecks.

[0045] Step S102, obtain a second request set corresponding to the first request set from the target system, and obtain a second latency data set and a second metric data set for the target system to process each request in the second request set, where the requests included in the first request set are the same as those included in the second request set.

[0046] Specifically, the target system can be a system running in a test environment. The target system can be a system obtained by localizing some modules on the basis of the production system. Since after localizing some modules in the production system, it needs to be tested before being put into production, it is necessary to use the same second request set as the first request set in the production system to test the target system, and then determine the influence of each performance metric in the target system on the latency data according to the second latency data set and the second metric data set obtained by the target system processing the second request set, so as to determine whether the performance of the target system is the same as that of the production system, ensuring that after the target system is put into production, it can still run normally when processing requests normally.

[0047] For example, after the target system processes the second request set, the metric that has the greatest impact on the latency data detected is the CPU occupancy rate. After the production system processes the first request set with the same content as the second request set, the metric that has the greatest impact on the latency data detected is the network bandwidth utilization rate. This indicates that after the target system is localized, the CPU occupancy rate is abnormal, which means the CPU needs to be adjusted, thereby completing the operation of determining the performance bottleneck metric.

[0048] Step S103: Train a preset neural network model based on the first latency data set and the first metric data set to obtain a first prediction model, and train the preset neural network model based on the second latency data set and the second metric data set to obtain a second prediction model.

[0049] Specifically, after obtaining the first latency data set and the first metric data set, the model can be trained using the first latency data set and the first metric data set to obtain a first prediction model. Among them, the first prediction model can predict the latency data when the production system processes requests based on the metric data. Similarly, in the case of obtaining the second latency data set and the second metric data set, the preset neural network model can be trained using the second latency data set and the second metric data set to obtain a second prediction model. Among them, the second prediction model can predict the latency data when the target system processes requests based on the metric data.

[0050] It should be noted that the first prediction model and the second prediction model can be random forest models.

[0051] Step S104: Determine the performance bottleneck metric of the target system based on the first prediction model and the second prediction model.

[0052] Specifically, in the case of obtaining the first prediction model and the second prediction model, the random forest model can be used to identify the performance metric that has the greatest impact on the processing latency through feature importance analysis. In the random forest model, each decision tree evaluates the contribution of each performance metric feature during the splitting process. By statistically aggregating the cumulative contribution values of a certain feature across all decision trees, the random forest can output a feature list sorted by importance. By analyzing the feature importance ranking, it is possible to intuitively see which performance metrics have the greatest impact on the processing latency. For example, the model may identify that CPU utilization, memory usage, or disk I / O latency are the main bottlenecks causing the processing latency of HTTP requests. These key performance metrics provide a clear direction for system performance optimization, and targeted in-depth analysis and optimization can be carried out on these performance metrics.

[0053] Further, when determining the performance bottleneck metrics, it is possible to compare the feature list of the target system with the feature list of the production system to determine which serial numbers of the metric features have changed, and then determine the performance bottleneck metrics through the method of feature list comparison.

[0054] The method for determining the performance bottleneck metrics of the system provided by the embodiments of the present application includes: obtaining a first latency data set and a first metric data set for the production system to process each request in the first request set; obtaining a second request set corresponding to the first request set from the target system, and obtaining a second latency data set and a second metric data set for the target system to process each request in the second request set, where the requests included in the first request set are the same as the requests included in the second request set; training a preset neural network model according to the first latency data set and the first metric data set to obtain a first prediction model, and training the preset neural network model according to the second latency data set and the second metric data set to obtain a second prediction model; determining the performance bottleneck metrics of the target system based on the first prediction model and the second prediction model. This solves the problem of low efficiency in manually determining the performance bottleneck metrics of the system in the related art. By using the processing results of processing requests between the production system and the target system as the training set to train the model, based on the obtained prediction model, it is possible to determine which metrics are the bottleneck metrics affecting the latency data through the key metric sequence generated by the prediction model during the process of learning the training set, so as to quickly and accurately obtain the bottleneck metrics of the target system, and thus achieve the technical effect of improving the determination efficiency of the bottleneck metrics.

[0055] Optionally, in the method for determining the performance bottleneck metrics of the system provided by the embodiments of the present application, determining the performance bottleneck metrics of the target system based on the first prediction model and the second prediction model includes: respectively obtaining a first metric importance sequence of the production system and a second metric importance sequence of the target system from the first prediction model and the second prediction model; determining the performance bottleneck metrics of the target system according to the first metric importance sequence and the second metric importance sequence.

[0056] Specifically, when determining the performance bottleneck metrics, since the HTTP requests processed by the production system and the target system are exactly the same, it is possible to compare the processing latencies and performance metrics of the two systems to identify the performance bottleneck points in the target system.

[0057] First, use the random forest model to train in the production system to obtain the first prediction model, and identify the sequence of performance metric characteristics that affect the latency of the application system under the production system, that is, the first importance sequence of metrics. Then, use the same model structure to train and predict in the target system, and identify the sequence of performance metric characteristics of the target system, that is, the second importance sequence of metrics. In the random forest model, the importance of features (i.e., performance metrics) is calculated during the construction of each tree, and this importance reflects the contribution degree of the feature to the prediction result. After the model training is completed, by integrating the feature importances of all trees, a total ranking of the importance of metrics can be obtained, that is, the first importance sequence of metrics and the second importance sequence of metrics. Since the target system usually simulates the load of the production system, the performance bottlenecks in the target system compared to the production system can be located by comparing the importance sequences of metrics of the two systems.

[0058] For example, in the production system, the first prediction model may identify that the memory usage rate, network bandwidth utilization rate, disk I / O latency, etc. are the top three performance metrics that have the greatest impact on latency, forming the first importance sequence of metrics. In the target system, the second prediction model may list the CPU occupancy rate, memory usage rate, and network bandwidth utilization rate as the top three that have the greatest impact on latency, forming the second importance sequence of metrics.

[0059] By comparing the importance rankings of features of the two systems, it can be clearly seen which performance metrics have anomalies in the target system compared to the production system. Compare the ranking changes of each performance metric in the two sequences, and focus on identifying the metrics whose rankings rise in the second importance sequence of metrics, that is, those performance metrics that have a greater impact on latency in the target system than in the production system. Furthermore, the performance metrics with anomalies can be determined as performance bottleneck metrics.

[0060] For example, when comparing the first importance sequence of metrics and the second importance sequence of metrics, it is found that the ranking of the CPU occupancy rate in the second prediction model jumps from the 5th place in the production system to the 1st place, indicating that in the target system, the CPU occupancy rate suddenly becomes the main factor affecting latency. Therefore, the CPU occupancy rate is determined as the performance bottleneck metric, which means that there may be a bottleneck in the CPU resource management of the target system and further optimization is needed. At this time, the target system can be optimized for the CPU occupancy rate to ensure the normal operation of the target system and the normal processing of requests.

[0061] The above implementation steps of this embodiment can not only automatically discover performance bottlenecks, but also accurately locate the main reasons for the performance degradation in the target system by comparing the importance sequences of metrics under different systems, thus greatly improving the efficiency and accuracy of performance analysis and providing a clear direction for performance optimization.

[0062] Optionally, in the method for determining the performance bottleneck metrics of the system provided in the embodiments of the present application, determining the performance bottleneck metrics of the target system according to the first metric importance sequence and the second metric importance sequence includes: determining whether the first metric importance sequence and the second metric importance sequence are the same; in the case where the first metric importance sequence and the second metric importance sequence are the same, determining the performance bottleneck metric of the target system as the first performance metric in the second metric importance sequence; in the case where the first metric importance sequence and the second metric importance sequence are different, obtaining the performance metrics whose sequence numbers in the first metric importance sequence are greater than those in the second metric importance sequence, to obtain at least one performance bottleneck metric.

[0063] Specifically, in the case of determining the performance bottleneck metrics, it is first necessary to compare the first metric importance sequence and the second metric importance sequence to check whether the arrangement order of the metrics in the two sequences is consistent. By comparing the metrics in the sequence and their corresponding sequence numbers, it can be determined whether there are differences in the sorting of the performance metrics of the target system and the production system in terms of the impact on performance.

[0064] If the sorting of the metrics in the first metric importance sequence and the second metric importance sequence is exactly the same, the first performance metric in the second sequence (target system) can be directly determined as the performance bottleneck metric. That is, in the target system, the impact degree of the first metric on the latency increases significantly compared to other metrics, which may be due to the configuration or load conditions of the system. If there are differences between the first metric importance sequence and the second metric importance sequence, attention can be paid to the performance metrics whose sequence numbers (i.e., the sorting of the impact degree) are advanced in the second sequence (target system). The importance of these metrics increases significantly in the target system, so they may constitute performance bottlenecks.

[0065] For example, assume that in the production system, the sorting of the impact degrees of the performance metrics determined by the first sequence is: memory usage rate (sequence number 1), CPU occupancy rate (sequence number 2), disk I / O latency (sequence number 3); while in the target system, the sorting of the second sequence is: CPU occupancy rate (sequence number 1), memory usage rate (sequence number 2), disk I / O latency (sequence number 4), network bandwidth utilization rate (sequence number 3). It can be found that the two sequences are not exactly the same. Especially, the impact of the network bandwidth utilization rate on the latency suddenly increases in the target system, while the impact of the disk I / O latency decreases. At this time, since the sequence number of the network bandwidth utilization rate in the second sequence is 3, while in the first sequence (production system) it is 4, this means that the impact degree of the network bandwidth utilization rate has increased in the target system. Therefore, the network bandwidth utilization rate can be used as one of the at least one performance bottleneck metrics.

[0066] Further, when the index sorting in the two sequences is exactly the same and the first index is the CPU occupancy rate, it indicates that in the target system, the influence degree of the CPU occupancy rate on the latency under the same conditions becomes more significant, that is, the primary performance bottleneck index.

[0067] In this embodiment, by comparing the index importance sequences of the two systems, the performance bottleneck index of the target system is identified. It can not only quickly locate the first index that significantly affects performance, but also find out those performances in the target system whose influence degree is significantly improved compared with the production system from the sequence differences, so as to analyze and locate the performance bottleneck from multiple angles and comprehensively.

[0068] Optionally, in the method for determining the performance bottleneck index of the system provided in the embodiments of the present application, the first request set includes M first requests, the first latency data set includes M first latency data, and the first index data set includes M groups of first index data. Obtaining the first latency data set and the first index data set for the production system to process each request in the first request set includes: for any one of the first requests, obtaining the processing data obtained by the production system processing the first request according to the attribute information of the first request, and obtaining the first latency data from the processing data; according to the timestamp in the processing data, obtaining the performance index data corresponding to the processing data in the host database of the production system to obtain the first index data.

[0069] Specifically, when obtaining the latency data, the latency data for processing the request can be calculated from the attribute information of the request according to the reception time of the request by the system and the completion time of the request being processed by the system.

[0070] It should be noted that when obtaining the latency data and the performance index data, in order to ensure the accurate correspondence between the latency data and the index data, it is first necessary to align each latency data and index data through the timestamp, so as to ensure the correlation between each first latency data and the corresponding first index data when the first latency data and the first index data are obtained.

[0071] That is, the HTTP request data and performance data of the production system and the target system are synchronized and aligned according to timestamps to ensure the accuracy of analysis. By parsing the captured PCAP data packets, the time difference from the issuance to the end of the response of each HTTP request (i.e., the processing delay) is extracted. This delay data can directly reflect the response performance of the system at a specific time point and serve as a key indicator of system performance. The order and quantity of requests in the two systems are basically the same, so the HTTP request data can be aligned through timestamps. During the synchronization process, it is necessary to ensure that each request matches in the two systems, that is, requests at the same time point or similar time points should correspond to the same request event for subsequent analysis and comparison. Since the performance data of the system is collected at a certain sampling interval (such as every minute or every second), it is necessary to perform fine-grained time alignment on the performance data and the HTTP request delay. Further, during synchronization, not only the timestamps need to be aligned, but also the order of HTTP request processing in the production system and the target system needs to be ensured to be the same. If there are disordered request orders or missing data, compensation or adjustment is required to maintain the consistency of synchronization.

[0072] For example, the response time of the first user login request at 3 pm is 0.71 seconds. Subsequently, according to the timestamp of this request, in the host database of the production system, within a certain time (such as ±5 seconds) before and after 3 pm, index data such as CPU occupancy rate and memory usage rate are retrieved. These data together constitute the first index data and are then added to the first index data set.

[0073] Through the above process, this embodiment not only clarifies how to accurately obtain the processing delay data of HTTP requests from the production system, but also emphasizes how to accurately obtain the performance index data at the same time as or within the relevant time window of request processing through timestamp alignment, ensuring the accuracy and reliability of subsequent analysis.

[0074] Optionally, in the method for determining the performance bottleneck index of the system provided in the embodiment of the present application, according to the timestamp in the processed data, retrieving the performance index data corresponding to the processed data in the host database of the production system, and obtaining the first index data includes: obtaining the collection frequency of the performance index data in the production system; when the collection frequency is greater than or equal to the preset frequency, determining the performance index data in the production system that is the same as the timestamp moment as the first index data of the first delay data; when the collection frequency is less than the preset frequency, determining the time interval according to the collection frequency and the timestamp, and determining the performance index data within the time interval as the first index data of the first delay data.

[0075] Specifically, when aligning the latency data and the performance metric data, it is necessary to access the host database of the production system to determine the collection frequency of the performance metric data, and then determine how to perform the alignment operation based on the collection frequency.

[0076] When the collection frequency of the production system performance metrics is greater than or equal to the preset frequency, it means that the performance data is collected frequently enough to be directly aligned one-to-one with the timestamps of request processing. In this case, it is only necessary to directly locate the performance metric data at the corresponding moment of the timestamp. Through the timestamp, the performance metric data that exactly matches it can be directly found in the host database as the first metric data associated with the first latency data.

[0077] When the collection frequency of the production system performance metrics is lower than the preset frequency, it indicates that the collection interval of the performance data is relatively long and cannot achieve a direct one-to-one match with the timestamps of request processing. At this time, it is necessary to determine a reasonable time interval according to the collection frequency and the timestamps of request processing, so as to use the performance metric data within this interval as the first metric data associated with the first latency data. That is, according to the collection frequency, calculate the previous and subsequent time points adjacent to the timestamp to form a time window or interval, and then select the performance metric data from this interval for alignment. For example, if the data collection frequency is once a minute and the preset frequency is once every 5 seconds, the average value of the performance data within a few minutes before and after the timestamp or the data at a specific time point can be used as the first metric data.

[0078] For example, if the collection interval of the metric data is short (such as once per second), the HTTP request latency within the same second can be directly aligned with the metric data of that second. If the collection interval of the metric data is long (such as once per minute), a time window needs to be introduced. For example, set a 5-second time window, and regard the HTTP request latency data within 5 seconds before and after the sampling time point as associated with the metric data at that moment. In this way, even if there is a small time deviation, the correct mapping between the HTTP request data and the metric data can be ensured.

[0079] Through the above process, this embodiment ensures that the timestamps and the performance metric data in the processed data can be accurately aligned at different collection frequencies to obtain the first metric data, thus ensuring an accurate corresponding relationship between the latency data and the metric data.

[0080] It should be noted that the alignment process between the second latency data and the second metric data can be the same as the alignment process between the first latency data and the first metric data.

[0081] Optionally, in the method for determining the performance bottleneck metrics of the system provided in the embodiments of the present application, the second request set includes M second requests, the second delay data set includes M second delay data, and the second metric data set includes M groups of second metric data. Obtaining the second request set corresponding to the first request set from the target system includes: for any first request, obtaining the processed requests with the same attribute information in the database of the target system according to the attribute information of the first request, and obtaining multiple processed requests; sequentially determining the time difference between each processed request and the first request, and obtaining the processed requests with a time difference less than the preset time difference, to obtain the second request corresponding to the first request; combining the second requests corresponding to each first request in the database into a second request set.

[0082] Specifically, in the case of obtaining the second request set, in order to ensure that the requests in the first request set and the second request set are exactly the same, therefore, it is necessary to align the requests in the target system and the production system, so as to ensure that the obtained requests are the same requests.

[0083] For each first request in the first request set, first query the processed requests with exactly the same attributes in the database of the target system according to the attribute information (such as request path, query parameters, source port, etc.). Through this step of attribute comparison, it can be ensured that the requests found in the target system are consistent with the request types and contents in the production system. In the target system database, an SQL query statement or an API call can be used to retrieve all processed requests based on the attribute information of the first request (such as path, query, src_port, etc.) and find the request set that exactly matches the attribute information of the first request.

[0084] Furthermore, the time difference between each processed request in the target system and the corresponding first request in the production system can be calculated, and the processed requests with a time difference less than the preset time difference can be determined as the second requests corresponding to the first request, thereby excluding requests with time mismatches and improving the accuracy and timeliness of the data. For example, by comparing the timestamps of each request with the timestamp of the first request, calculating the time difference, and using the preset time difference as the screening condition, the requests with a time difference less than the threshold are retained to obtain the second requests matching the first request.

[0085] Finally, the second requests corresponding to each first request screened out in the above steps are combined to form a complete second request set. In this way, it is ensured that all requests processed by the target system have undergone double comparison of attributes and time, and are highly consistent with the request set of the production system, providing a consistent data set for subsequent data comparison and performance analysis.

[0086] For example, in the production usage environment and the playback test environment, the processing data of the application system when processing the same HTTP request and the performance data of the application system host are collected respectively. Specific performance data may include CPU occupancy rate, memory usage rate, I / O waiting time, number of waiting connections, switch utilization rate, etc. By analyzing the pcap data packets captured in the two environments respectively, key information such as the processing delay of each HTTP application request, the request path URL, source and destination IPs, source and destination ports in the data packets can be extracted to ensure that the same request data in multiple environments can be accurately aligned.

[0087] Match each pair of adjacent requests in the production environment with the adjacent requests in the playback environment. The matching basis is attribute information such as the path (Path), query parameters (Query), source port (Src_Port) of the application request. At the same time, considering the time difference between adjacent requests, the time difference between adjacent requests in the production environment can be utilized to set a matching constraint threshold. In the playback environment, search for application requests that not only meet the matching basis but also fall within the constraint threshold, and ensure the accuracy of the matching through a dynamic time threshold. Figure 2 It is a flowchart of the request alignment method provided by an embodiment of the present application. As Figure 2 shown, the specific implementation steps for aligning requests can be as follows:

[0088] Step 1: Read and parse data: Read the request data from the storage files in the production environment and the playback environment, including key information such as path (Path), query parameters (Query), source port (Src_Port), timestamp, etc. Parse the timestamp through the parse_time() function to uniformly convert timestamps in different formats into datetime objects for subsequent calculation of time differences and complete time parsing.

[0089] Step 2: Traverse adjacent requests in the production environment: Traverse each pair of adjacent requests in the production environment, calculate the time difference time_diff_production between them, and set a time threshold time_threshold to constrain the request matching in the playback environment. This threshold can be set as Min{m * time_diff_production, n}, that is, m times the time difference time_diff_production between adjacent requests, and the maximum does not exceed n seconds. (m, n are adjustable).

[0090] Step 3: Match playback requests:

[0091] First match: Select the request with the earliest time in the playback environment for matching.

[0092] Subsequent matching: Based on the time of the matched requests, the playback request with the smallest time difference that is within the threshold range is selected.

[0093] Step 4: Update matching results:

[0094] Match successful and within the threshold range: Indicates a successful match. The first request in the production system and the second request in the target system are matched and recorded.

[0095] Matching failure or matching success but outside the threshold range: indicates a matching failure, and the request is skipped.

[0096] If the match is successful, the time list of the most recently matched playback request is updated to ensure that the playback request will not be matched repeatedly.

[0097] Step 5: Data preprocessing: By aligning timestamps, we match performance data within the same time period with the corresponding HTTP application request processing latency. We clean and standardize the collected data, removing outliers to ensure data accuracy and consistency. This yields a second set of requests in the target system that corresponds to the first set of requests.

[0098] Through the above process, this embodiment ensures the consistency of the second request set obtained in the target system and the first request set in the production system in terms of content and time, providing an accurate data basis for performance comparison and analysis.

[0099] Optionally, in the method for determining the performance bottleneck indicator of the system provided in an embodiment of the present application, before training the preset neural network model based on the first delay data set and the first indicator data set to obtain the first prediction model, the method also includes: performing a preprocessing operation on each group of first indicator data in the first indicator data set to obtain a third indicator data set, wherein the preprocessing operation includes at least one of the following: feature scaling, standardization, and normalization; determining whether there are outliers in the third indicator data set; if there are outliers in the third indicator data set, smoothing the outliers to obtain a fourth indicator data set; determining the indicator data in the fourth indicator data set that is correlated with the delay data by the Pearson correlation coefficient method to obtain a fifth indicator data set; using the fifth indicator data set to replace the first indicator data set, and executing the operation of training the preset neural network model based on the first delay data set and the first indicator data set to obtain the first prediction model.

[0100] It should be noted that the third indicator data set is an upgraded version of the first indicator data set obtained after preprocessing (such as feature scaling, standardization, normalization). The fourth indicator data set is a purified version of the third indicator data set after detecting and handling outliers. The fifth indicator data set is an indicator data set that has been screened out through Pearson correlation coefficient analysis and has a significant correlation with the processing delay data for subsequent model training. Preprocessing operations: involve data preprocessing means such as feature scaling, standardization, and normalization, aiming to improve data quality and meet the requirements of machine learning models.

[0101] Before model training, it is necessary to preprocess the collected data to ensure that all indicator data is numerically comparable and avoid the problem that the model may overweight certain indicators due to differences in data magnitude. Further, it is necessary to determine whether there are outliers in the preprocessed third indicator data set, and in the case of having outliers, smooth them to generate the fourth indicator data set to improve the quality of the model training data.

[0102] After handling the outliers, the Pearson correlation coefficient method can be used to analyze the fourth indicator data set to determine the indicator data with a strong correlation with the HTTP request processing delay, forming the fifth indicator data set. Finally, use the fifth indicator data set to replace the first indicator data set as the input data for model training, so as to obtain high-quality data after multiple rounds of preprocessing and screening for training the first prediction model.

[0103] For example, after data synchronization, the obtained HTTP request delay and performance indicators need to be further processed to ensure that these data can be used as effective input features for subsequent machine learning models. The processing process can include feature scaling, outlier handling, etc.

[0104] First, perform feature scaling. Since performance data usually has different dimensions (for example, CPU utilization is expressed as a percentage, while network bandwidth is expressed in bytes), in order to avoid the adverse effects of these dimensional differences on the training of machine learning models, it is necessary to standardize or normalize these data:

[0105] First, adopt the Z-score standardization method to standardize the data of each performance indicator according to its mean, so that the data is converted into a normal distribution data with a mean of 0 and a standard deviation of 1. The standardization formula is:

[0106]

[0107] where X is the original data, μ is the mean, and σ is the standard deviation.

[0108] Furthermore, perform a normalization operation to scale the performance metric data to the range of [0, 1], so that all features are in the same numerical range, preventing features with larger dimensions from having too much impact on the training of the model. The formula for normalization is:

[0109]

[0110] where Xmax and Xmin are the minimum and maximum values of the performance metric feature values respectively.

[0111] Finally, perform an outlier handling operation. In actual data, there may be some abnormal performance metric values or HTTP request delays, such as extreme data points caused by network failures or system anomalies. These outliers may have an adverse impact on subsequent model training, so outlier detection and handling are required: detect outliers through Z-score, and select to remove outliers or smooth them according to business requirements to ensure the quality of the input feature data, thereby improving the robustness and prediction accuracy of the model.

[0112] After performing the above performance metric data processing operations, it is also necessary to screen a large amount of performance metric data to reduce the interference of redundant features on the model, while improving the performance and computational efficiency of the model.

[0113] To screen out the performance metric features closely related to the HTTP request processing delay, the Pearson correlation coefficient can be used to analyze the linear relationship between each performance metric feature and the request delay. The Pearson correlation coefficient measures the linear correlation between two variables, and its value ranges from -1 to 1:

[0114] When the correlation coefficient is close to 1, it indicates that the two variables are positively correlated, and the performance metric feature and the HTTP delay show the same change trend.

[0115] When the correlation coefficient is close to -1, it indicates that the two variables are negatively correlated, and the performance metric feature and the HTTP delay show the opposite change trend.

[0116] When the correlation coefficient is close to 0, it indicates that there is no significant linear relationship between the two variables.

[0117] Through correlation analysis, the performance metric features that have an important impact on the HTTP request delay can be screened out, and those features with less or no impact on the delay can be excluded.

[0118] Furthermore, a feature redundancy elimination operation can also be performed, that is, through preliminary correlation analysis and other statistical methods (such as principal component analysis PCA, etc.), redundant performance metric features can be removed, thereby reducing the computational complexity of the model, avoiding the overfitting problem caused by too many features, and improving the generalization ability of the model.

[0119] Furthermore, in addition to performing feature selection based on statistical methods such as correlation, it is also possible to select certain metric data that may be of high importance to the system performance in combination with business requirements, even though its correlation with latency may not be high. These metrics should also be used as alternative options for model features to ensure the comprehensiveness and business interpretability of the model.

[0120] Through the above process, this embodiment performs a series of professional preprocessing and screening on the original first metric data set, thereby obtaining a more refined and accurate metric data set. It not only eliminates the interference of possible outliers and dimensional differences in the data on model training, but also focuses on the performance metrics that are crucial for latency through correlation analysis, significantly improving the efficiency of model training and the prediction accuracy of the final model.

[0121] Figure 3 is a flowchart of an optional performance metric data processing process of the system provided by an embodiment of the present application. As Figure 3 shown, first, a data collection operation is performed, that is, PCAP packets in the production system and the target system are captured to obtain the processing latency data of each HTTP request, and the performance metric data of each system (such as CPU occupancy rate, memory usage rate, I / O waiting time) is collected. Then, the latency data and the performance metric data are aligned according to the timestamp, and further feature processing is performed, that is, standardization and normalization. Z-score standardization can be used to convert each performance metric data into the same numerical range, and outliers are processed. Missing values, outliers, etc. in the performance metric data are processed. Finally, a feature selection operation is performed. By calculating the Pearson correlation coefficient, performance features having a certain linear relationship with the HTTP request processing latency are selected, and features that are irrelevant or have low contribution to latency are removed, thereby obtaining the processed performance metric data.

[0122] Figure 4 is a flowchart of an optional model training process of the system provided by an embodiment of the present application. As Figure 4 shown, during the model training process, the processing latency of the HTTP request is used as the target value (i.e., the variable to be predicted), and the performance metrics at each moment are used as input features.

[0123] To ensure that the model has good generalization ability, the entire data set is divided into a training set and a test set. The training set is used for model training, and the test set is used for model evaluation and verification. The specific division method adopts the 80 / 20 splitting principle, that is, 80% of the data is used for training and 20% of the data is used for testing.

[0124] Furthermore, the random forest makes predictions by constructing multiple decision trees. In each decision tree, the KPI performance data is split multiple times, and a prediction model is constructed through different combined paths. Each decision tree generates independent prediction values based on different combinations of KPI features. Finally, by integrating the prediction results of each decision tree, the final prediction value of the processing delay is output.

[0125] After the model training is completed, it needs to be evaluated to ensure its prediction accuracy and robustness. In the present invention, the following two main evaluation metrics are adopted:

[0126] Mean Squared Error (MSE): MSE is used to measure the error magnitude between the model prediction value and the true value. The calculation formula is:

[0127]

[0128] where y i is the true value, is the model prediction value, and n is the total number of samples. The smaller the MSE, the smaller the prediction error of the model.

[0129] R 2 value: The R 2 value measures the explanatory ability of the model for the target variable (HTTP request processing delay). The calculation formula is:

[0130]

[0131] where y i is the true value, is the model prediction value, is the mean of the true values. The closer the R 2 value is to 1, the stronger the explanatory ability of the model for the delay.

[0132] By using different data sets in the production environment and the playback environment, two independent models are trained respectively, and they are evaluated separately. By comparing the evaluation results of these two models, the differences and performance bottlenecks in the two environments can be further analyzed.

[0133] To ensure the optimal performance of the random forest model, the hyperparameters of the model are tuned. Hyperparameters (such as the number of trees, maximum depth, etc.) directly affect the accuracy and computational efficiency of the model. Therefore, the methods of grid search and cross-validation can be adopted for optimization.

[0134] Finally, based on the output of the random forest model, specific performance optimization suggestions can be provided. For example, if the model identifies that CPU utilization is the main bottleneck affecting latency, optimization can be considered by increasing CPU resources, adjusting the load balancing strategy, etc. Similarly, if network performance is the bottleneck, latency can be reduced by adjusting the network topology or increasing bandwidth. In this way, the present invention can not only automatically identify performance bottlenecks but also provide effective decision-making support for performance optimization.

[0135] It should be noted that in machine learning, especially when using tree-based models such as random forests, the model is actually learning the relationship between features (here referring to performance data) and target variables (HTTP request processing latency) in the data during the training process. Although in the preparation stage of training samples, performance data is simply used as input features and latency as the output target, the random forest will automatically calculate the importance of features during the construction process.

[0136] Specifically, the random forest model consists of many decision trees. When constructing each tree, a part of the features is randomly selected to find the best split point to maximize the prediction accuracy. In this process, the model will evaluate the contribution degree of each feature to the prediction result. The greater the contribution, the more important this feature is in predicting the target variable (i.e., HTTP request processing latency). This is because when constructing each decision tree, the model will calculate the purity gain of each feature, that is, the degree of improvement in prediction accuracy that can be brought by using a certain feature for splitting at the node of the tree. The more times a feature is selected for splitting and the greater the purity gain brought by each split, the higher the total importance score of this feature.

[0137] When all the trees are constructed, the random forest will summarize the feature importance scores in all the trees to generate a global feature importance ranking. This ranking can indicate which performance metric data plays the most crucial role in predicting the HTTP request processing latency, that is, the performance bottleneck metrics.

[0138] Therefore, even if the feature importance ranking is not directly mentioned in the training samples, the random forest model will automatically generate such a ranking during the training process, which benefits from its internal feature selection and purity gain evaluation mechanism. Through this ranking, it is possible to focus on the KPI metrics that have the greatest impact on system performance for targeted performance optimization and problem troubleshooting.

[0139] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0140] The embodiment of the present application also provides an apparatus for determining the performance bottleneck index of a system. It should be noted that the apparatus for determining the performance bottleneck index of the system in the embodiment of the present application can be used to execute the method for determining the performance bottleneck index of the system provided in the embodiment of the present application. The following introduces the apparatus for determining the performance bottleneck index of the system provided in the embodiment of the present application.

[0141] Figure 5 It is a schematic diagram of the apparatus for determining the performance bottleneck index of the system provided in the embodiment of the present application. As Figure 5 shown, the apparatus includes: a first acquisition unit 51, a second acquisition unit 52, a training unit 53, and a first determination unit 54.

[0142] The first acquisition unit 51 is configured to acquire a first delay data set and a first metric data set for the production system to process each request in the first request set.

[0143] The second acquisition unit 52 is configured to acquire a second request set corresponding to the first request set from the target system, and acquire a second delay data set and a second metric data set for the target system to process each request in the second request set, where the requests included in the first request set are the same as the requests included in the second request set.

[0144] The training unit 53 is configured to train a preset neural network model according to the first delay data set and the first metric data set to obtain a first prediction model, and train the preset neural network model according to the second delay data set and the second metric data set to obtain a second prediction model.

[0145] The first determination unit 54 is configured to determine the performance bottleneck index of the target system based on the first prediction model and the second prediction model.

[0146] The device for determining the performance bottleneck indicators of the system provided by the embodiment of the present application obtains a first delay data set and a first indicator data set of the production system for processing each request in the first request set through the first acquisition unit 51; the second acquisition unit 52 obtains a second request set corresponding to the first request set from the target system, and obtains a second delay data set and a second indicator data set of the target system for processing each request in the second request set, where the requests included in the first request set are the same as the requests included in the second request set; the training unit 53 trains a preset neural network model according to the first delay data set and the first indicator data set to obtain a first prediction model, and trains the preset neural network model according to the second delay data set and the second indicator data set to obtain a second prediction model; the first determination unit 54 determines the performance bottleneck indicators of the target system based on the first prediction model and the second prediction model, solving the problem of low efficiency in manually determining the performance bottleneck indicators of the system in the related art. By using the processing results of processing requests between the production system and the target system as the training set to train the model, on the basis of obtaining the prediction model, the key indicator sequence generated by the prediction model during the learning of the training set can be used to determine which indicators are the bottleneck indicators affecting the delay data, so as to quickly and accurately obtain the bottleneck indicators of the target system, and thus achieve the technical effect of improving the determination efficiency of the bottleneck indicators.

[0147] Optionally, in the device for determining the performance bottleneck indicators of the system provided by the embodiment of the present application, the first determination unit 54 includes: a first acquisition module, configured to respectively obtain a first indicator importance sequence of the production system and a second indicator importance sequence of the target system from the first prediction model and the second prediction model; a first determination module, configured to determine the performance bottleneck indicators of the target system according to the first indicator importance sequence and the second indicator importance sequence.

[0148] Optionally, in the device for determining the performance bottleneck indicators of the system provided by the embodiment of the present application, the first determination module includes: a judgment sub-module, configured to judge whether the first indicator importance sequence and the second indicator importance sequence are the same; a first determination sub-module, configured to, when the first indicator importance sequence and the second indicator importance sequence are the same, determine that the performance bottleneck indicator of the target system is the first performance indicator in the second indicator importance sequence; a first acquisition sub-module, configured to, when the first indicator importance sequence and the second indicator importance sequence are different, acquire the performance indicators whose serial numbers in the first indicator importance sequence are greater than the serial numbers in the second indicator importance sequence, to obtain at least one performance bottleneck indicator.

[0149] Optionally, in the apparatus for determining the performance bottleneck metrics of the system provided in the embodiments of the present application, the first request set includes M first requests, the first delay data set includes M first delay data, the first metric data set includes M groups of first metric data, and the first obtaining unit 51 includes: a second obtaining module, configured to, for any one of the first requests, obtain the processing data obtained by the production system processing the first request according to the attribute information of the first request, and obtain the first delay data from the processing data; a third obtaining module, configured to obtain the performance metric data corresponding to the processing data in the host database of the production system according to the timestamp in the processing data, so as to obtain the first metric data.

[0150] Optionally, in the apparatus for determining the performance bottleneck metrics of the system provided in the embodiments of the present application, the third obtaining module includes: a second obtaining sub-module, configured to obtain the collection frequency of the performance metric data in the production system; a second determining sub-module, configured to, when the collection frequency is greater than or equal to the preset frequency, determine the performance metric data in the production system that is the same as the moment of the timestamp as the first metric data of the first delay data; a third determining sub-module, configured to, when the collection frequency is less than the preset frequency, determine the time interval according to the collection frequency and the timestamp, and determine the performance metric data located in the time interval as the first metric data of the first delay data.

[0151] Optionally, in the apparatus for determining the performance bottleneck metrics of the system provided in the embodiments of the present application, the second request set includes M second requests, the second delay data set includes M second delay data, the second metric data set includes M groups of second metric data, and the second obtaining unit 52 includes: a fourth obtaining module, configured to, for any one of the first requests, obtain the processed requests with the same attribute information in the database of the target system according to the attribute information of the first request, so as to obtain a plurality of processed requests; a second determining module, configured to sequentially determine the time difference between each processed request and the first request, and obtain the processed requests with the time difference less than the preset time difference, so as to obtain the second requests corresponding to the first request; a combining module, configured to combine the second requests corresponding to each first request in the database into a second request set.

[0152] Optionally, in the apparatus for determining the performance bottleneck metrics of the system provided in the embodiments of the present application, before training a preset neural network model according to the first latency data set and the first metric data set to obtain a first prediction model, the apparatus further includes: a preprocessing unit, configured to perform preprocessing operations on each group of first metric data in the first metric data set to obtain a third metric data set, where the preprocessing operations include at least one of the following: feature scaling, standardization, and normalization; a judgment unit, configured to judge whether there are outliers in the third metric data set; a processing unit, configured to smooth the outliers in the case where there are outliers in the third metric data set to obtain a fourth metric data set; a second determination unit, configured to determine the metric data correlated with the latency data in the fourth metric data set by the Pearson correlation coefficient method to obtain a fifth metric data set; and an execution unit, configured to replace the first metric data set with the fifth metric data set and execute the operation of training the preset neural network model according to the first latency data set and the first metric data set to obtain a first prediction model.

[0153] The apparatus for determining the performance bottleneck metrics of the above system includes a processor and a memory. The above first acquisition unit 51, second acquisition unit 52, training unit 53, first determination unit 54, etc. are all stored in the memory as program units, and the processor executes the above program units stored in the memory to implement corresponding functions.

[0154] The processor contains a kernel, and the kernel retrieves the corresponding program units from the memory. One or more kernels can be set, and by adjusting the kernel parameters, the problem of low efficiency in manually determining the performance bottleneck metrics of the system in the related art is solved.

[0155] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of, for example, read-only memory (ROM) or flash memory (flash RAM), and the memory includes at least one memory chip.

[0156] The embodiments of the present invention provide a computer-readable storage medium, on which a program is stored, and when the program is executed by a processor, the method for determining the performance bottleneck metrics of the system is implemented.

[0157] The embodiments of the present invention provide a processor, and the processor is used to run a program, where when the program runs, the method for determining the performance bottleneck metrics of the system is executed.

[0158] Figure 6 is a schematic diagram of an electronic device provided according to an embodiment of the present application, such as Figure 6As shown, an embodiment of the present invention provides an electronic device. The electronic device 60 includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps of the method for determining the performance bottleneck metrics of the above system are implemented. The devices herein can be servers, PCs, PADs, mobile phones, etc.

[0159] The present application also provides a computer program product which, when executed on a data processing device, is adapted to execute a program that initializes the steps of the method for determining the performance bottleneck metrics of the above system.

[0160] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0161] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0162] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that implement the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0163] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide means for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1A step that specifies a function in one or more boxes.

[0164] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0165] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0166] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0167] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0168] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A method for determining the performance bottleneck metrics of a system, characterized in that, Including: Obtain a first latency data set and a first metric data set for the production system to process each request in the first request set; Obtain a second request set corresponding to the first request set from the target system, and obtain a second latency data set and a second metric data set for the target system to process each request in the second request set, where the requests included in the first request set are the same as the requests included in the second request set; Train a preset neural network model according to the first latency data set and the first metric data set to obtain a first prediction model, and train the preset neural network model according to the second latency data set and the second metric data set to obtain a second prediction model; Determine the performance bottleneck metrics of the target system based on the first prediction model and the second prediction model.

2. The method according to claim 1, wherein Determining the performance bottleneck metrics of the target system based on the first prediction model and the second prediction model includes: Obtain a first metric importance sequence of the production system and a second metric importance sequence of the target system from the first prediction model and the second prediction model respectively; Determine the performance bottleneck metrics of the target system according to the first metric importance sequence and the second metric importance sequence.

3. The method according to claim 2, wherein Determining the performance bottleneck metrics of the target system according to the first metric importance sequence and the second metric importance sequence includes: Judge whether the first metric importance sequence and the second metric importance sequence are the same; When the first metric importance sequence and the second metric importance sequence are the same, determine the performance bottleneck metric of the target system as the first performance metric in the second metric importance sequence; When the first metric importance sequence and the second metric importance sequence are different, obtain the performance metrics whose serial numbers in the first metric importance sequence are greater than the serial numbers in the second metric importance sequence to obtain at least one performance bottleneck metric.

4. The method according to claim 1, characterized in that, The first request set includes M first requests, the first latency data set includes M first latency data, and the first metric data set includes M groups of first metric data. Obtaining the first latency data set and the first metric data set for the production system to process each request in the first request set includes: For any one of the first requests, obtain the processing data obtained by the production system processing the first request according to the attribute information of the first request, and obtain the first latency data from the processing data; According to the timestamp in the processing data, obtain the performance metric data corresponding to the processing data in the host database of the production system to obtain the first metric data.

5. The method according to claim 4, characterized in that According to the timestamp in the processing data, obtaining the performance metric data corresponding to the processing data in the host database of the production system to obtain the first metric data includes: Obtain the acquisition frequency of the performance metric data in the production system; When the acquisition frequency is greater than or equal to the preset frequency, determine the performance index data in the production system that is the same as the moment of the time stamp as the first index data of the first delay data; When the acquisition frequency is less than the preset frequency, determine the time interval according to the acquisition frequency and the time stamp, and determine the performance index data within the time interval as the first index data of the first delay data.

6. The method according to claim 1, characterized in that The second request set includes M second requests, the second delay data set includes M second delay data, and the second index data set includes M groups of second index data. Obtaining the second request set corresponding to the first request set from the target system includes: For any one first request, obtain the processed requests with the same attribute information as the attribute information in the database of the target system according to the attribute information of the first request, and obtain a plurality of processed requests; Determine the time difference between each processed request and the first request in sequence, and obtain the processed requests with the time difference less than the preset time difference to obtain the second request corresponding to the first request; Combine the second requests corresponding to each first request in the database into the second request set.

7. The method according to claim 1, wherein Before training the preset neural network model according to the first delay data set and the first index data set to obtain the first prediction model, the method further includes: Perform a preprocessing operation on each group of first index data in the first index data set to obtain a third index data set, where the preprocessing operation includes at least one of the following: feature scaling, standardization, normalization; Judge whether there are outliers in the third index data set; When there are outliers in the third index data set, perform smoothing processing on the outliers to obtain a fourth index data set; Determine the index data correlated with the delay data in the fourth index data set by the Pearson correlation coefficient method to obtain a fifth index data set; Replace the first index data set with the fifth index data set, and perform the operation of training the preset neural network model according to the first delay data set and the first index data set to obtain the first prediction model.

8. An apparatus for determining a performance bottleneck metric of a system, characterized in that, Including: A first acquisition unit, configured to acquire a first delay data set and a first index data set for the production system to process each request in the first request set; A second acquisition unit, configured to acquire a second request set corresponding to the first request set from the target system, and acquire a second delay data set and a second index data set for the target system to process each request in the second request set, where the requests included in the first request set are the same as the requests included in the second request set; A training unit, configured to train a preset neural network model according to the first delay data set and the first index data set to obtain a first prediction model, and train the preset neural network model according to the second delay data set and the second index data set to obtain a second prediction model; A first determination unit, configured to determine a performance bottleneck metric of the target system according to the first prediction model and the second prediction model.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein when the executable program runs, it controls the device where the computer-readable storage medium is located to execute the method for determining the performance bottleneck metric of the system according to any one of claims 1 to 7.

10. An electronic device, characterized in that, Comprising: A memory storing an executable program; A processor for running the program, wherein when the program runs, it executes the method for determining the performance bottleneck metric of the system according to any one of claims 1 to 7.

Images