A method and device for analyzing performance test results

Abstract

The present invention discloses a method and apparatus for analyzing performance test results, relating to the field of computer technology. A specific embodiment of the method includes: obtaining response data of a server within a set time period, and determining the response time of the server and the number of times the response time occurs according to the response data; determining expected normal distribution data of the response time and actual distribution data of the response time according to the response time of the server and the number of times the response time occurs; determining the fitting similarity between the expected normal distribution data and the actual distribution data; if the fitting similarity is greater than a set threshold, the performance test result is valid; otherwise, the performance test result is invalid. This embodiment adopts the technical means of calculating the fitting degree between the actual probability distribution of the response time and the theoretical normal distribution probability, thus overcoming the technical problem of relying on manual experience to analyze performance test results, and further achieving the technical effect of improving the accuracy and efficiency of analyzing performance test results.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and particularly to a method for analyzing performance test results. Background Art

[0002] In the field of performance testing, response time is one of the important performance metrics. Directly using the response time result data of performance monitoring tools or performance testing tools, setting the threshold of the response time data, when the threshold is not met, a warning message is given, or data is manually queried to determine potential problems. In the existing performance testing process online, the response time varies greatly under different software and hardware testing environments, and it is necessary to manually evaluate the effectiveness and accuracy of the performance test results. The performance graph analysis in the prior art generally uses time series as the X-axis and the average response time in the neighborhood of the current time as the Y-axis. At the same time, other performance metrics can be used as the information of the Y-axis for performance analysis in the form of a broken line.

[0003] In the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art:

[0004] 1. The analysis of the result data is relatively one-sided. For the process and overall situation of the frequency change of the response time, the monitoring means are relatively single, and the analysis process based on alarms relies more on manual experience.

[0005] 2. The performance graph analysis in the prior art uses the time series and the average response time in the neighborhood of the time series to depict the performance analysis results. However, it is difficult to collect time series data in a large range, and it is also difficult to select the corresponding neighborhood range of the time series, which is not conducive to the analysis of historical data, and it is also inconvenient to analyze the situation of multiple applications and multiple instances.

[0006] 3. Due to the time series data, it is difficult to correlate the response time for situations such as inconsistent time and asynchronous processing. Summary of the Invention

[0007] In view of this, an embodiment of the present invention provides a method for analyzing performance test results, which can solve the technical problem of analyzing performance test results relying on manual experience according to performance graphs in the prior art.

[0008] To achieve the above object, according to one aspect of the embodiments of the present invention, a method for analyzing performance test results is provided, including: obtaining response data of a server within a set time period, and determining the response time of the server and the number of times the response time occurs according to the response data; determining the expected normal distribution data of the response time and the actual distribution data of the response time according to the response time of the server and the number of times the response time occurs; determining the fitting similarity between the expected normal distribution data and the actual distribution data based on the expected normal distribution data of the response time and the actual distribution data of the response time; if the fitting similarity is greater than a set threshold, the performance test result is valid; otherwise, the performance test result is invalid.

[0009] Optionally, determining the response time of the server and the number of times the response time occurs according to the response data includes: generating data in a multimap structure with the response time as the key according to the response data, and the value corresponding to the key is 1; generating data in a map structure with the response time as the key and the number of times the response time occurs as the value according to the data in the multimap structure.

[0010] Optionally, determining the expected normal distribution data of the response time and the actual distribution data of the response time according to the response time of the server and the number of times the response time occurs includes: performing a complement processing on the data in the map structure to obtain the complemented map data; determining the total amount, average value, and standard deviation of the response time according to the data in the map structure; determining the normal distribution value of the value corresponding to the key in the complemented map data based on the complemented map data, in combination with the average value and standard deviation of the response time; generating a first map data according to the key in the complemented map data and the normal distribution value of the value corresponding to the key, and using the first map data as the expected normal distribution data of the response time; based on the complemented map data, using the value corresponding to the key in the complemented map data and the total amount of the response time as the frequency value of the value corresponding to the key; generating a second map data according to the key in the complemented map data and the frequency value of the value corresponding to the key, and using the second map data as the actual distribution data of the response time.

[0011] Optionally, determining the fitting similarity between the expected normal distribution data and the actual distribution data based on the expected normal distribution data of the response time and the actual distribution data of the response time includes: converting the value corresponding to the key in the first map data into a first multi-dimensional vector, and converting the value corresponding to the key in the second map data into a second multi-dimensional vector; using the cosine absolute value of the first multi-dimensional vector and the second multi-dimensional vector as the fitting similarity between the expected normal distribution data and the actual distribution data.

[0012] Optionally, after determining the response time of the server and the number of occurrences of the response time according to the response data, the method further includes: generating a two-dimensional image with the response time of the server as the X-axis and the number of occurrences of the response time as the Y-axis.

[0013] According to another aspect of the embodiments of the present invention, there is provided a performance test result analysis device, including: a data summary module, configured to: obtain the response data of the server within a set time period, and determine the response time of the server and the number of occurrences of the response time according to the response data; a data processing module, configured to: determine the expected normal distribution data of the response time and the actual distribution data of the response time according to the response time of the server and the number of occurrences of the response time; a data fitting module, configured to: determine the fitting similarity between the expected normal distribution data and the actual distribution data based on the expected normal distribution data of the response time and the actual distribution data of the response time; a data analysis module, configured to: if the fitting similarity is greater than a set threshold, the performance test result is valid; otherwise, the performance test result is invalid.

[0014] Optionally, the data summary module is further configured to: generate data in a multimap structure with the response time as the key according to the response data, and the value corresponding to the key is 1; generate data in a map structure with the response time as the key and the number of occurrences of the response time as the value according to the data in the multimap structure.

[0015] Optionally, the data processing module is further configured to: perform a complement processing on the data in the map structure to obtain the complemented map data; determine the total amount, average value, and standard deviation of the response time according to the data in the map structure; based on the complemented map data, combine the average value and standard deviation of the response time to determine the normal distribution value of the value corresponding to the key in the complemented map data; generate a first map data according to the key in the complemented map data and the normal distribution value of the value corresponding to the key, and use the first map data as the expected normal distribution data of the response time; based on the complemented map data, use the value corresponding to the key in the complemented map data and the total amount of the response time as the frequency value of the value corresponding to the key; generate a second map data according to the key in the complemented map data and the frequency value of the value corresponding to the key, and use the second map data as the actual distribution data of the response time.

[0016] Optionally, the data fitting module is further configured to: convert the value corresponding to the key in the first map data into a first multi-dimensional vector, and convert the value corresponding to the key in the second map data into a second multi-dimensional vector; use the cosine absolute value of the first multi-dimensional vector and the second multi-dimensional vector as the fitting similarity between the expected normal distribution data and the actual distribution data.

[0017] Optionally, the device further includes an image generation module, configured to: generate a two-dimensional image with the response time of the server as the X-axis and the number of occurrences of the response time as the Y-axis.

[0018] According to another aspect of the embodiments of the present invention, there is provided an electronic device, including: one or more processors; a storage device for storing one or more programs, which when executed by the one or more processors, cause the one or more processors to implement the performance test result analysis method provided in the first aspect embodiment of the present invention.

[0019] According to another aspect of the embodiments of the present invention, there is provided a computer-readable medium, having a computer program stored thereon, which when executed by a processor, implements the performance test result analysis method provided in the first aspect embodiment of the present invention.

[0020] One embodiment of the above invention has the following advantages or beneficial effects: By using the technical means of calculating the fitting degree between the actual probability distribution of the response time and the theoretical normal distribution probability, the technical problem of relying on manual experience to analyze the performance test results is overcome, and thus the technical effects of improving the accuracy and efficiency of the analysis of the performance test results are achieved.

[0021] The further effects of the above non-conventional optional manner will be described in conjunction with the specific embodiments below. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings are used to better understand the present invention and do not constitute an improper limitation to the present invention. Among them:

[0023] Figure 1 is a schematic diagram of the basic process of the performance test result analysis method according to the embodiments of the present invention.

[0024] Figure 2 is a schematic diagram of the multimap data structure and the map data structure according to the embodiments of the present invention.

[0025] Figure 3 is a schematic diagram of the process of complementing the data of the map structure according to the embodiments of the present invention.

[0026] Figure 4 is a schematic diagram of the expected normal distribution data of the response time and the actual distribution data of the response time according to the embodiments of the present invention.

[0027] Figure 5 is a schematic diagram of performance graph analysis in the prior art.

[0028] Figure 6It is a schematic diagram of the basic module of the performance test result analysis device according to an embodiment of the present invention.

[0029] Figure 7 It is an exemplary system architecture diagram to which an embodiment of the present invention can be applied.

[0030] Figure 8 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. Detailed implementation manners

[0031] The following describes exemplary embodiments of the present invention with reference to the accompanying drawings. Various details of the embodiments of the present invention are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted below.

[0032] Figure 1 It is a schematic diagram of the basic process of the performance test result analysis method according to an embodiment of the present invention. As Figure 1 shown, an embodiment of the present invention provides a performance test result analysis method, including:

[0033] Step S101. Obtain the response data of the server within a set time period, and determine the response time of the server and the number of times the response time occurs according to the response data;

[0034] Step S102. Determine the expected normal distribution data of the response time and the actual distribution data of the response time according to the response time of the server and the number of times the response time occurs;

[0035] Step S103. Based on the expected normal distribution data of the response time and the actual distribution data of the response time, determine the fitting similarity between the expected normal distribution data and the actual distribution data;

[0036] Step S104. If the fitting similarity is greater than a set threshold, the performance test result is valid; otherwise, the performance test result is invalid.

[0037] The embodiment of the present invention adopts the technical means of calculating the fitting degree between the actual probability distribution of the response time and the theoretical normal distribution probability, so it overcomes the technical problem of relying on manual experience to analyze the performance test results, and further achieves the technical effect of improving the accuracy and efficiency of the analysis of the performance test results.

[0038] In step S101 of the embodiment of the present invention, determining the response time of the server and the number of times the response time is generated according to the response data includes: generating data of a multimap structure with the response time as the key according to the response data, and the value corresponding to the key is 1; generating data of a map structure with the response time as the key and the number of times the response time is generated as the value according to the data of the multimap structure. Wherein, the map data is a data of a key-value pair structure, and has the function of automatically sorting by key; and the data of the multimap structure is a special map data, that is, a data of a map structure that allows key duplication.

[0039] Response time data generally exists in multiple servers. This type of data usually has multiple sources, such as monitoring systems, database creation and modification time, and application logs. The acquisition methods, content, and efficiency of different sources vary greatly. Therefore, this solution uses multi-threaded reading. For response time with monitoring data, it can be directly obtained from the monitoring data. For applications without monitoring data information, it is necessary to filter out the completion time corresponding to the request operation to be analyzed from the business log and system database. You can also obtain the start request time and service response time from the application that initiates the request, and calculate the time difference to get the response time. The response time usually includes the communication time of the network. Generally, the local response time of the called server is used, but it will still meet the similarity of the normal distribution of the response time, because the data transmission between networks in the intranet takes less time, and the total response time is also shorter, which will not change this feature.

[0040] The data on each server is read using multiple threads to obtain all the response time data within the same time period, such as the complete response time of a natural day on a server instance.

[0041] The response time is in milliseconds. For example, if the response time is 100ms, the key is 100 and the value of the key is 1. Multimap has a very high operating efficiency and can handle a large amount of response time data.

[0042] After the response time is obtained, it is inserted into an independent multimap data structure. The number of elements in the multimap is the total number on the server. The element refers to the stored key-value pair, the key name is the response time, and the key value is 1 time.

[0043] The result data of each server is aggregated into a map. Multiple threads traverse the data in each server. Each thread processes each key in each multimap one by one, obtaining the total number of times the key appears on each server (since the value corresponding to each key is 1). Due to the automatic sorting function, the elements with the same key in the multimap of each server will exist continuously. Therefore, the number of occurrences of the same key can be obtained with relatively high efficiency, that is, the total number of response times under this time length. This value is used as the new value and inserted into the map with a thread lock until all elements in all multimaps are processed.

[0044] Figure 2 It is a schematic diagram of the multimap data structure and the map data structure according to an embodiment of the present invention. As Figure 2 shown, a multimap data structure is created in each thread in the figure. The key in this structure is the response time, and the value is 1 indicating one time, sorted in ascending order. The keys with the same value will be aggregated in one interval. The final data link structure in the figure is used to represent the map data structure. The key in this structure is the response time, and the value is the total number of times the response time is generated, sorted in ascending order.

[0045] In step S102 of the embodiment of the present invention, according to the response time of the server and the number of times the response time is generated, determining the expected normal distribution data of the response time and the actual distribution data of the response time includes: performing a complement processing on the data of the map structure to obtain the complemented map data; determining the total amount, average value, and standard deviation of the response time according to the data of the map structure; based on the complemented map data, combining the average value and standard deviation of the response time to determine the normal distribution value of the value corresponding to the key in the complemented map data; generating the first map data according to the key in the complemented map data and the normal distribution value of the value corresponding to the key, and using the first map data as the expected normal distribution data of the response time; based on the complemented map data, using the value corresponding to the key in the complemented map data and the total amount of the response time as the frequency value of the value corresponding to the key; generating the second map data according to the key in the complemented map data and the frequency value of the value corresponding to the key, and using the second map data as the actual distribution data of the response time.

[0046] In an environment with limited memory, the process of generating multimap data can also be ignored, and the number of times can be directly summed up one by one in the thread. However, the performance will be relatively slow, and each time it is necessary to locate the key, determine whether the key exists, and take the corresponding value for summation.

[0047] Figure 3 It is a schematic diagram of completing the data of the map structure according to an embodiment of the present invention. As Figure 3 shown, loop through each time value from the minimum time to the maximum time, that is, the response time with a time interval of 1 ms. If the key of this time value does not exist in the map for this response time, insert this key into the map and initialize the value to 0.

[0048] Traverse the completed map and calculate the valid data, that is, the node data with a value not equal to 0, the total number, average value, and standard deviation of the response time (calculated here based on the uncompleted data).

[0049] Traverse the data of the completed map and calculate the result of the normal distribution corresponding to each value value,

[0050] f(x) = 1 / (√2π × standard deviation) × e^(-((x - average value)x^2) / (2 × standard deviation x^2));

[0051] x represents the key value, and f(x) represents the normal distribution value corresponding to the value value. Put these two values into the first map.

[0052] Traverse the map and calculate the frequency corresponding to each value value, that is, the value divided by the total number. The result is stored in the second map. The order of the elements in this map is the same as the order of the elements in the map of the response time calculated before, and the total number of its elements is the same, that is, all response times are sorted in ascending order from low to high.

[0053] In step S103 of the embodiment of the present invention, based on the expected normal distribution data of the response time and the actual distribution data of the response time, determining the fitting similarity between the expected normal distribution data and the actual distribution data includes: converting the value corresponding to the key in the first map data into a first multi-dimensional vector, and converting the value corresponding to the key in the second map data into a second multi-dimensional vector; using the cosine absolute value of the first multi-dimensional vector and the second multi-dimensional vector as the fitting similarity between the expected normal distribution data and the actual distribution data.

[0054] Figure 4 It is a schematic diagram of the expected normal distribution data of the response time and the actual distribution data of the response time according to an embodiment of the present invention. As Figure 4 shown, the X-axis is the response time value, and the Y-axis is the number of times this response time occurs. Calculate the fitting similarity, that is, the similarity between the two. Considering the dimension of the data, that is, there is no data for the response time of a certain theoretical value in the response time of the actual value, and also considering the number of times of the response time, the similarity between the two can be well measured.

[0055] The key orders in the two map data structures are the same as the key values in the same order. Therefore, only the value cases are considered. Convert the values in the two maps into multi-dimensional vectors, and convert the sorted Map data structure into a linear list. The dimensions of the two vectors are the same. For example, a map {10ms -> 20 times, 11ms -> 30 times, 12ms -> 50 times,...} is converted into a multi-dimensional vector [20 times, 30 times, 50 times]. After conversion into a vector, there is no longer the original key name key data, but both are sorted in ascending order. Since the data was complemented earlier, the dimensions of the vectors are also the same.

[0056] In the embodiments of the present invention, the cosine similarity of two multi-dimensional vectors can be calculated as the fitting similarity. The result of the cosine calculation is between -1 and 1, and the absolute value of the cosine is taken as the cosine similarity. If the fitting similarity between the two is greater than a predetermined threshold, it indicates that the response time in the performance test results satisfies the normal distribution and the performance test results are valid; otherwise, it does not satisfy the normal distribution and the performance test results are invalid.

[0057] The fitting similarity of the embodiments of the present invention can also be applied to:

[0058] Compare whether the data fitting of the response time of multiple groups of performance data in historical data is consistent with the response time in the current performance data. If the number of historical data groups with a fitting similarity less than the predetermined threshold is greater than half of the total number of groups, that is, if the fitting is successful with most historical data, it indicates that the response time data is normal; otherwise, it indicates that there may be an abnormality in performance monitoring and further problem location and analysis are required.

[0059] It is also possible to compare the fitting situations of the response times in the performance monitoring in two different environments. If the data fitting similarity is greater than the predetermined threshold, it indicates that the performance of the new environment is within an acceptable range; otherwise, the performance is unacceptable and the environment needs to be optimized and the performance evaluation needs to be carried out again.

[0060] For data in the two vectors with a difference greater than another predetermined difference, the system will screen and feedback it to relevant personnel. For data with a difference greater than the predetermined difference, it is usually caused by unstable factors in the system, which may be due to factors such as input data and environment, and further problem location and analysis are also required. For the response time of the fitting similarity, there may also be a situation where the difference is greater than the predetermined difference threshold, and further location and analysis are still required. For example, the two vectors are the two vectors involved in the previous cosine similarity formula. It can be a comparison between the theoretical normal distribution and the actual distribution at the same time and on the same server instance, or a comparison between two different instances or two different time periods on the same instance or system in practice.

[0061] It is also applicable to some performance test scenarios that do not satisfy the normal distribution, such as stress test scenarios. If it meets the corresponding data distribution, the performance test can be considered valid, otherwise it is considered invalid.

[0062] In this solution, the distribution fitting calculation can also be performed on the results of each server to find out potential environmental problems, such as load balancing problems. Usually, for the same performance monitoring scenario, the response time distribution of multiple groups of server instances running in parallel should meet similar fitting. For example, this performance test involves n server instances, and each server instance can get a multidimensional vector. These multidimensional vectors perform cosine similarity calculations between every two vectors. If the cosine similarity results between m vectors are greater than the set similarity threshold, and the size of m is much smaller than n / 2, it means that there is a load balancing problem in the system. If the size of m is much larger than n / 2, it means that there is a contention lock in the system, and code-level positioning is required. Contention locks will cause large differences in response time. The instance that gets the lock will be faster, and the instance that does not get the lock needs to wait, resulting in a longer response time. Contention locks also extend to different processes on the same instance or different threads in the same process.

[0063] The embodiments of the present invention can be combined with other existing performance analysis technical solutions to further improve the effectiveness of performance monitoring and result analysis.

[0064] Figure 5 It is a schematic diagram of performance graphic analysis in the prior art. Figure 5 As shown in the figure, the performance graphical analysis generally uses the time series as the X-axis, that is, the first minute, the second minute, the third minute, etc. of the performance monitoring, and the Y-axis is the average response time in the time neighborhood at that time. At the same time, other performance indicators can be used as a broken line as the Y-axis information for performance analysis. The traditional method of judging whether the data meets the normal distribution is generally to draw a histogram and then make a manual judgment, which is not accurate enough. It takes a lot of time to judge the normal distribution of a large amount of data. There are a lot of different performance data in performance monitoring. If the histogram is used for manual judgment, it will take a lot of time and the efficiency is relatively low.

[0065] In order to solve the above problem, after determining the response time of the server and the number of times the response time is generated according to the response data in step S101 of the embodiment of the present invention, the method also includes: generating a two-dimensional image with the response time of the server as the X-axis and the number of times the response time is generated as the Y-axis to reduce the storage space occupied by the response data.

[0066] Use the response time as the X-axis, and the Y-axis represents the quantity at different response time values. For example, during the entire performance monitoring process, there are 300 response times at the 100 ms time point. Here, 100 ms is the X-axis and 300 times is the Y-axis. Response time is a performance test metric that is easy to obtain and has a relatively high accuracy. For the actual response time, its unit is ms, its minimum value tends to 0, and its maximum value is equal to the maximum timeout time set by the application program. The points on this X-axis are finite, and the Y-axis represents the total quantity of this response time. The storage space occupied by the overall response time data is extremely small. The probability distribution of the response time frequency in normal performance testing is theoretically a normal distribution. In this solution, by comparing the difference between the actual probability distribution and the theoretical normal distribution, if the difference is large, there may be problems with the performance test, such as too little test data, a large number of empty data returned, or the tested program timing out. In addition to comparing the actual probability distribution with the theoretical probability distribution, it is also possible to compare the actual probability distributions on different server instances during actual performance testing or monitoring, or the actual probability distributions at different times for the same server instance. For the response time of a normal service, the results of the above probability distribution comparisons are similar. If the results are significantly different, there may be problems with the service performance, and other factors can be combined to further locate the cause of the problem, such as uneven service load, inconsistent server hardware specifications, or different parameters in the configuration files of service instances. There is a certain pattern in the response time in the performance results, that is, the data for the fastest and slowest response times is the least, while the quantity of most response times is the most, which conforms to the standard of normal distribution.

[0067] Figure 6 It is a schematic diagram of the basic module of the performance test result analysis device according to an embodiment of the present invention. As Figure 6 shown, an embodiment of the present invention provides a performance test result analysis device 600, including: a data summary module 601, configured to: obtain the response data of the server within a set time period, and determine the response time of the server and the number of times the response time occurs according to the response data; a data processing module 602, configured to: determine the expected normal distribution data of the response time and the actual distribution data of the response time according to the response time of the server and the number of times the response time occurs; a data fitting module 603, configured to: determine the fitting similarity between the expected normal distribution data and the actual distribution data based on the expected normal distribution data of the response time and the actual distribution data of the response time; a data analysis module 604, configured to: if the fitting similarity is greater than a set threshold, the performance test result is valid; otherwise, the performance test result is invalid.

[0068] The data summarization module in the embodiment of the present invention is further configured to: generate data in a multimap structure with the response time as the key, where the value corresponding to the key is 1, based on the response data; generate data in a map structure with the response time as the key and the number of times the response time occurs as the value, based on the data in the multimap structure.

[0069] The data processing module in the embodiment of the present invention is further configured to: perform a completion process on the data in the map structure to obtain the completed map data; determine the total amount, average value, and standard deviation of the response time based on the data in the map structure; determine the normal distribution value of the value corresponding to the key in the completed map data, in combination with the average value and standard deviation of the response time, based on the completed map data; generate a first map data based on the key and the normal distribution value of the value corresponding to the key in the completed map data, and use the first map data as the expected normal distribution data of the response time; use the value corresponding to the key in the completed map data and the total amount of the response time as the frequency value of the value corresponding to the key, based on the completed map data; generate a second map data based on the key and the frequency value of the value corresponding to the key in the completed map data, and use the second map data as the actual distribution data of the response time.

[0070] The data fitting module in the embodiment of the present invention is further configured to: convert the value corresponding to the key in the first map data into a first multi-dimensional vector, and convert the value corresponding to the key in the second map data into a second multi-dimensional vector; use the cosine absolute value of the first multi-dimensional vector and the second multi-dimensional vector as the fitting similarity between the expected normal distribution data and the actual distribution data.

[0071] The embodiment of the present invention is characterized in that the device further includes an image generation module, configured to: generate a two-dimensional image with the response time of the server as the X-axis and the number of times the response time occurs as the Y-axis, so as to reduce the storage space occupied by the response data.

[0072] Figure 7 Exemplary system architecture 700 is shown in which the performance test result analysis method or performance test result analysis device according to the embodiment of the present invention can be applied.

[0073] As Figure 7 shown, the system architecture 700 may include terminal devices 701, 702, 703, a network 704, and a server 705. The network 704 is used to provide a medium for a communication link between the terminal devices 701, 702, 703 and the server 705. The network 704 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0074] Users can use terminal devices 701, 702, and 703 to interact with server 705 via network 704 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 701, 702, and 703, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0075] Terminal devices 701, 702, and 703 can be various electronic devices with a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop computers, desktop computers, and so on.

[0076] Server 705 can be a server that provides various services. For example, it is a background management server that supports the shopping websites browsed by users using terminal devices 701, 702, and 703. The background management server can analyze and process data such as product information query requests received, and feedback the processing results, such as target push information, to the terminal device.

[0077] It should be noted that the performance test result analysis method provided by the embodiments of the present invention is generally executed by server 705. Correspondingly, the performance test result analysis device is generally set in server 705.

[0078] It should be understood that Figure 7 the numbers of terminal devices, networks, and servers in

[0079] are merely illustrative. According to the implementation requirements, there can be any number of terminal devices, networks, and servers.

[0080] The electronic device of the present invention includes: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the performance test result analysis method provided by the embodiments of the first aspect of the present invention.

[0081] The computer-readable medium of the present invention stores a computer program thereon, and when the program is executed by a processor, it implements the performance test result analysis method provided by the embodiments of the first aspect of the present invention.

[0082] Next, refer to Figure 8 which shows a schematic structural diagram of a computer system 800 of a terminal device suitable for implementing the embodiments of the present invention. Figure 8 The shown terminal device is merely an example and should not bring any limitation to the functions and usage scope of the embodiments of the present invention.

[0083] As Figure 8As shown, computer system 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 802 or programs loaded from a storage section 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the system 800 are also stored. The CPU 801, ROM 802, and RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0084] The following components are connected to the I / O interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed so that a computer program read from it can be installed into the storage section 808 as needed.

[0085] Specifically, according to the embodiments disclosed in the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present invention include a computer program product that includes a computer program carried on a computer-readable medium, and the computer program contains program codes for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 809, and / or installed from the removable medium 811. When the computer program is executed by the central processing unit (CPU) 801, the above-described functions defined in the system of the present invention are executed.

[0086] It should be noted that the computer-readable medium shown in the present invention can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device. In the present invention, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and this computer-readable medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0087] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code, and the above module, program segment, or part of code contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0088] The modules involved in the embodiments of the present invention can be implemented in software or in hardware. The described modules can also be provided in a processor. For example, it can be described as a processor including a data summarization module, a data processing module, a data fitting module, and a data analysis module. Among them, the names of these modules do not constitute a limitation to the module itself in some cases. For example, the data summarization module can also be described as "a module for obtaining and summarizing the response data of the server within a set time period".

[0089] As another aspect, the present invention also provides a computer-readable medium, which can be included in the device described in the above embodiments; or can exist alone without being assembled into the device. The above computer-readable medium carries one or more programs. When the above one or more programs are executed by the device, the device includes: obtaining the response data of the server within a set time period, determining the response time of the server and the number of times the response time is generated according to the response data; determining the expected normal distribution data of the response time and the actual distribution data of the response time according to the response time of the server and the number of times the response time is generated; determining the fitting similarity between the expected normal distribution data and the actual distribution data based on the expected normal distribution data of the response time and the actual distribution data of the response time; if the fitting similarity is greater than a set threshold, the performance test result is valid; otherwise, the performance test result is invalid.

[0090] The embodiments of the present invention adopt the technical means of calculating the fitting degree between the actual probability distribution of the response time and the theoretical normal distribution probability, so as to overcome the technical problem of relying on manual experience to analyze the performance test results, and further achieve the technical effect of improving the accuracy and efficiency of the analysis of the performance test results.

[0091] The above specific embodiments do not constitute a limitation to the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for analyzing performance test results, characterized in that, Including: Obtain the response data of the server within a set time period, and determine the response time of the server and the number of times the response time is generated according to the response data; According to the response time of the server and the number of times the response time is generated, determine the expected normal distribution data and the actual distribution data of the response time, including: Generate data in a map structure with the response time as the key and the number of times the response time is generated as the value; perform a complement processing on the data in the map structure to obtain the complemented map data; According to the data in the map structure, determine the total amount, average value, and standard deviation of the response time; based on the complemented map data, combine the average value and standard deviation of the response time to determine the normal distribution value of the value corresponding to the key in the complemented map data; According to the key in the complemented map data and the normal distribution value of the value corresponding to the key, generate a first map data, and use the first map data as the expected normal distribution data of the response time; Based on the complemented map data, use the value corresponding to the key in the complemented map data and the total amount of the response time as the frequency value of the value corresponding to the key; According to the key in the complemented map data and the frequency value of the value corresponding to the key, generate a second map data, and use the second map data as the actual distribution data of the response time; Based on the expected normal distribution data and the actual distribution data of the response time, determine the fitting similarity between the expected normal distribution data and the actual distribution data; If the fitting similarity is greater than the set threshold, the performance test result is valid; otherwise, the performance test result is invalid.

2. The method according to claim 1, characterized in that, According to the response data to determine the response time of the server and the number of times the response time is generated, including: According to the response data, generate data in a multimap structure with the response time as the key, and the value corresponding to the key is 1; According to the data in the multimap structure, generate data in a map structure with the response time as the key and the number of times the response time is generated as the value.

3. The method according to claim 1, characterized in that, Based on the expected normal distribution data and the actual distribution data of the response time, determine the fitting similarity between the expected normal distribution data and the actual distribution data, including: Convert the value corresponding to the key in the first map data into a first multi-dimensional vector, and convert the value corresponding to the key in the second map data into a second multi-dimensional vector; Use the cosine absolute value of the first multi-dimensional vector and the second multi-dimensional vector as the fitting similarity between the expected normal distribution data and the actual distribution data.

4. The method according to claim 1, characterized in that, After determining the response time of the server and the number of times the response time is generated according to the response data, the method further includes: Generate a two-dimensional image with the response time of the server as the X-axis and the number of times the response time is generated as the Y-axis.

5. A device for analyzing performance test results, characterized in that, Including: A data summary module for: obtaining the response data of the server within a set time period, and determining the response time of the server and the number of times the response time is generated according to the response data; A data processing module, configured to: determine the expected normal distribution data and the actual distribution data of the response time according to the response time of the server and the number of times the response time occurs, including: Generate data in a map structure with the response time as the key and the number of times the response time occurs as the value; perform a complement processing on the data in the map structure to obtain the complemented map data; Determine the total amount, average value, and standard deviation of the response time according to the data in the map structure; based on the complemented map data, in combination with the average value and standard deviation of the response time, determine the normal distribution value of the value corresponding to the key in the complemented map data; Generate a first map data according to the key in the complemented map data and the normal distribution value of the value corresponding to the key, and use the first map data as the expected normal distribution data of the response time; Based on the complemented map data, use the value corresponding to the key in the complemented map data and the total amount of the response time as the frequency value of the value corresponding to the key; Generate a second map data according to the key in the complemented map data and the frequency value of the value corresponding to the key, and use the second map data as the actual distribution data of the response time; A data fitting module, configured to: determine the fitting similarity between the expected normal distribution data and the actual distribution data based on the expected normal distribution data and the actual distribution data of the response time; A data analysis module, configured to: if the fitting similarity is greater than a set threshold, the performance test result is valid; otherwise, the performance test result is invalid.

6. The device according to claim 5, characterized in that, The data summarization module is further configured to: Generate data in a multimap structure with the response time as the key, and the value corresponding to the key is 1 according to the response data; Generate data in a map structure with the response time as the key and the number of times the response time occurs as the value according to the data in the multimap structure.

7. The device according to claim 5, characterized in that, The data fitting module is further configured to: Convert the value corresponding to the key in the first map data into a first multi-dimensional vector, and convert the value corresponding to the key in the second map data into a second multi-dimensional vector; Use the cosine absolute value of the first multi-dimensional vector and the second multi-dimensional vector as the fitting similarity between the expected normal distribution data and the actual distribution data.

8. The device according to claim 5, wherein The device further includes an image generation module, configured to: Generate a two-dimensional image with the response time of the server as the X-axis and the number of times the response time occurs as the Y-axis.

9. An electronic device, wherein Including: One or more processors; A storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1-4.

10. A computer-readable medium having a computer program stored thereon, wherein When the program is executed by the processor, it implements the method according to any one of claims 1-4.

Images