System testing method, device, equipment and product for adaptively adjusting load

Abstract

The present invention discloses a system testing method, device, equipment and product for adaptively adjusting load, comprising establishing a data set consisting of historical load characteristic data and historical performance characteristic data; using the data set to perform model training to obtain a prediction model; using the prediction model to predict predicted performance characteristic data corresponding to the instantaneous load characteristic data; and adjusting the test data input to the system according to a preset adaptive adjustment strategy based on the predicted performance characteristic data. The present invention discloses a system testing method, device, equipment and product for adaptively adjusting load, which can adaptively adjust the load to test the system, fully simulate the load scenario of actual business, improve the test effect on the system, and protect the stability of the system during the test.

Description

Technical Field

[0001] The present invention belongs to the field of system testing, and in particular relates to a system testing method, device, equipment and product for adaptively adjusting load. Background Art

[0002] As the complexity and scale of various software systems increase, the testing and evaluation of system performance becomes particularly important. Under existing technical conditions, traditional load testing methods often have difficulty in accurately simulating complex and changing load scenarios, and cannot be dynamically adjusted according to the real-time status of system resources. The effect of system testing is limited, and it is easy to encounter problems such as excessive system test load, performance degradation, and system failure during system testing. Summary of the invention

[0003] In view of this, the present invention aims to overcome the defects in the prior art and proposes a system testing method, device, equipment and product that are adaptable to adjusting load.

[0004] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0005] In a first aspect, the present invention discloses a system testing method for adaptively adjusting load, comprising:

[0006] Establishing a data set consisting of historical load characteristic data and historical performance characteristic data, wherein the load characteristic data is used to characterize the load state of the tested system, and the performance characteristic data is used to characterize the operating state of the tested system under the load characteristic data;

[0007] The data set is used to train the model to obtain a prediction model, which is used to predict performance characteristic data based on the load characteristic data;

[0008] Acquire instantaneous load characteristic data corresponding to the system in the test state, and use the prediction model to predict predicted performance characteristic data corresponding to the instantaneous load characteristic data;

[0009] Based on the predicted performance characteristic data, the test data input to the system is adjusted according to the preset adaptive adjustment strategy.

[0010] In one embodiment of the present invention, based on the predicted performance characteristic data and according to a preset adaptive adjustment strategy, adjusting the test data input to the system includes: calculating the average value of the historical performance characteristic data as the baseline performance characteristic data, establishing a threshold range centered on the baseline performance characteristic data, and adjusting the test data input to the system based on the comparison result between the predicted performance characteristic data and the threshold range.

[0011] In one embodiment of the present invention, the system testing method for adaptively adjusting load also includes: obtaining actual performance characteristic data of the system, and adjusting parameters of the prediction model according to the difference between the actual performance characteristic data and the predicted performance characteristic data corresponding to the same load characteristic data.

[0012] In one embodiment of the present invention, the system testing method for adaptively adjusting load further includes: generating a test report based on the instant load characteristic data and the corresponding actual performance characteristic data of the system, and the test report is used to optimize and upgrade the system.

[0013] In one embodiment of the present invention, adjusting the test data input by the system is adjusting the test data input by the system through an automated script.

[0014] In one embodiment of the present invention, the test report includes: actual performance characteristic data corresponding to evaluation into several performance levels.

[0015] In one embodiment of the present invention, before establishing a data set consisting of historical load characteristic data and historical performance characteristic data, it also includes: obtaining pre-processed historical load characteristic data and historical performance characteristic data.

[0016] In a second aspect, the present invention discloses a system testing device for adaptively adjusting load, the device comprising:

[0017] Establishing a data set module, used to establish a data set consisting of historical load characteristic data and historical performance characteristic data, wherein the load characteristic data is used to characterize the load state of the tested system, and the performance characteristic data is used to characterize the operating state of the tested system under the load characteristic data;

[0018] Establish a model module for using the data set to perform model training to obtain a prediction model, and the prediction model is used to predict performance characteristic data based on the load characteristic data;

[0019] A prediction module, used to obtain instantaneous load characteristic data corresponding to the system in the test state, and use a prediction model to predict predicted performance characteristic data corresponding to the instantaneous load characteristic data;

[0020] The adaptive adjustment module is used to adjust the test data input to the system based on the predicted performance characteristic data and according to a preset adaptive adjustment strategy.

[0021] In a third aspect, the present invention discloses an electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above method.

[0022] According to a fourth aspect, a computer program product comprises a computer program, wherein the computer program implements the above method when executed by a processor.

[0023] Compared with the prior art, the present invention has the following advantages:

[0024] The present invention discloses a system testing method, device, equipment and product for adaptively adjusting load, comprising establishing a data set consisting of historical load characteristic data and historical performance characteristic data; using the data set to perform model training to obtain a prediction model; using the prediction model to predict predicted performance characteristic data corresponding to the instantaneous load characteristic data; and adjusting the test data input to the system according to a preset adaptive adjustment strategy based on the predicted performance characteristic data. The present invention discloses a system testing method, device, equipment and product for adaptively adjusting load, which can adaptively adjust the load to test the system, fully simulate the load scenario of actual business, improve the test effect on the system, and protect the stability of the system during the test. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0026] In the attached picture:

[0027] Figure 1 A schematic diagram of an application scenario of a system testing method for adaptively adjusting load according to an embodiment of the present invention;

[0028] Figure 2 A schematic diagram of a system testing method for adaptively adjusting load according to an embodiment of the present invention;

[0029] Figure 3 A schematic diagram of a system testing device for adaptively adjusting load according to an embodiment of the present invention;

[0030] Figure 4 A schematic diagram of a system testing electronic device for adaptively adjusting load according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0032] In the description of the present invention, it should be further explained that the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "plurality" means two or more.

[0033] Under existing technical conditions, traditional load testing methods often find it difficult to accurately simulate complex and changeable load scenarios, and cannot be dynamically adjusted according to the real-time status of system resources. The effect on system testing is limited, and it is also easy for the system test load to be too high, performance to degrade, and system failures to occur during the system testing process.

[0034] The present invention discloses a system testing method, device, equipment and product for adaptively adjusting load. Figure 1 It is a schematic diagram of an application scenario. The present invention conducts model training by establishing a data set consisting of historical load characteristic data and historical performance characteristic data to obtain a prediction model. The prediction model is used to predict predicted performance characteristic data corresponding to the instantaneous load characteristic data. Finally, based on the predicted performance characteristic data and according to a preset adaptive adjustment strategy, the test data input by the system is adjusted. The load can be adaptively adjusted to test the system, fully simulating the load scenario of actual business, improving the test effect of the system, and protecting the stability of the system during the test.

[0035] The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0036] In one embodiment of the present invention, Figure 2 As shown, a system testing method for adaptively adjusting load includes:

[0037] Step S201, establishing a data set consisting of historical load characteristic data and historical performance characteristic data, wherein the load characteristic data is used to characterize the load state of the tested system, and the performance characteristic data is used to characterize the operating state of the tested system under the load characteristic data;

[0038] In this embodiment, the load characteristics may be, for example, CPU usage, memory occupancy, network card throughput, and I / O read / write speed; the performance characteristics may be system response time or throughput;

[0039] Before establishing a data set consisting of historical load characteristic data and historical performance characteristic data, it also includes: obtaining preprocessed historical load characteristic data and historical performance characteristic data. Preprocessing is to perform preprocessing operations such as cleaning, deduplication, and outlier processing on the acquired historical load characteristic data and historical performance characteristic data to ensure the accuracy and effectiveness of the data, and further perform standardization or normalization processing to facilitate subsequent model training and model parameter adjustment.

[0040] Step S202, using the data set to perform model training to obtain a prediction model, the prediction model is used to predict performance characteristic data based on the load characteristic data;

[0041] In this embodiment, 80% of the data in the data set will be used as a training set, and 20% of the data will be used as a test set.

[0042] In this embodiment, the model for model training can be a linear regression model, and the specific training process is as follows:

[0043] The linear regression model is expressed as follows:

[0044] Y = β 0 + β 1 X 1 + β 2 X 2 + ... + β n X n + ε , where Y is the dependent variable, X 1 , X 2 , ..., X n is the independent variable, β 0 is the intercept term, β 1 , β 2 , ..., β n is the regression coefficient, ε is a random error term, the dependent variable is the system response time or throughput, and the independent variables are CPU usage, memory usage, network card throughput, and I / O read and write speed;

[0045] By using the least squares method, the regression coefficients and intercept terms in the model are calculated as follows:

[0046] Suppose there are m observation data points, each of which contains n independent variables ( , , ) and a dependent variable Y, representing these data in matrix form, where: X is a The design matrix contains the independent variables and a column of all ones (corresponding to the intercept term ), the matrix is ​​as follows:

[0047]

[0048] Y is a 1, containing all dependent variable observations, the matrix is ​​as follows:

[0049]

[0050] β is a (n+1)×1 vector containing the intercept term and the regression coefficients of all independent variables ( , , ), the matrix is ​​as follows:

[0051]

[0052] The linear regression model is expressed as , is the error term, and the goal of the least squares method is to find β , so that Minimum, that is, the sum of squares of errors is minimum, which is equivalent to solving the normal equation: ,in yes The transposed matrix of β The value of .

[0053] Step S203, obtaining instantaneous load characteristic data corresponding to the system in the test state, and using the prediction model to predict the predicted performance characteristic data corresponding to the instantaneous load characteristic data;

[0054] Step S204: Based on the predicted performance characteristic data, the test data input to the system is adjusted according to a preset adaptive adjustment strategy.

[0055] Adjusting the test data input to the system is to adjust the test data input to the system through an automated script. Exemplarily, the automated script is used to increase the number of concurrent users, increase the request frequency, or increase the request complexity to simulate the performance of the system under high load.

[0056] The test method disclosed in the present invention can adjust the test data input by the system according to a preset adaptive adjustment strategy, can adaptively adjust the load to test the system, fully simulate the load scenario of the actual business, improve the test effect of the system, and protect the stability of the system during the test.

[0057] In another embodiment of the present invention, based on the predicted performance characteristic data and according to a preset adaptive adjustment strategy, adjusting the test data input to the system includes: calculating the average value of the historical performance characteristic data as the baseline performance characteristic data, establishing a threshold range centered on the baseline performance characteristic data, and adjusting the test data input to the system based on the comparison result between the predicted performance characteristic data and the threshold range.

[0058] Exemplarily, the benchmark performance characteristic data is the average value of the historical system response time or throughput. Combined with actual business needs, a reasonable percentage is set, such as ±10%, ±20%, etc. as the threshold range. When the predicted system performance characteristic data, such as the response time, is lower than the set lower threshold, or the throughput is higher than the set upper threshold, that is, the current system load is low, an automated script is used to increase the number of concurrent users, increase the request frequency, or increase the request complexity to simulate the performance of the system under high load. When the predicted system performance characteristic data, such as the response time, is higher than the set upper threshold, or the throughput is lower than the set lower threshold, that is, the current system load is too high, there may be a risk of system failure. At this time, an automated script is used to reduce the number of concurrent users, reduce the request frequency, or simplify the request complexity to reduce the load and prevent system overload, protect the stability of the system during testing, and reduce the test operation and maintenance costs caused by system crashes or performance degradation.

[0059] In this embodiment, by determining the benchmark performance characteristic data and establishing the threshold range, it is possible to adaptively adjust the test data input into the system, achieve a more comprehensive load state test effect on the system, and ensure the stability of the system during the test process.

[0060] In another embodiment of the present invention, the method further includes: acquiring actual performance characteristic data of the system, and adjusting parameters of the prediction model according to the difference between the actual performance characteristic data and the predicted performance characteristic data corresponding to the same load characteristic data.

[0061] In this embodiment, the parameters of the prediction model are adjusted to establish a feedback mechanism as follows:

[0062] The difference between the actual performance characteristic data and the predicted performance characteristic data corresponding to the same load characteristic data of the monitoring system is exemplarily plotted with the difference between the two as the y-axis and the independent variable data as the x-axis to draw a residual graph, and further analyze potential problems in the model, such as nonlinear relationships, heteroscedasticity, or a significant impact of an independent variable on the difference, and optimize the prediction model accordingly. If the analysis results are consistent with the expected linear relationship, more historical data can be obtained to tune the intercept term of the prediction model and the regression coefficients of all independent variables to reduce the difference. If the analysis results show that there are omitted variables or some variables do not contribute significantly to the prediction model, these variables can be added or deleted accordingly. If the analysis results show obvious nonlinear relationships or heteroscedasticity, nonlinear transformation of the independent or dependent variables or replacement of a more complex prediction model can be considered to improve the adaptability and accuracy of the model.

[0063] In this embodiment, the feedback mechanism can continuously optimize the prediction model and improve the accuracy of the prediction results of the prediction model.

[0064] In another embodiment of the present invention, the system testing method for adaptively adjusting load further includes: generating a test report based on the instant load characteristic data and the corresponding actual performance characteristic data of the system, and the test report is used to optimize and upgrade the system.

[0065] In this embodiment, the test report includes: actual performance characteristic data corresponding to the evaluation into several performance levels.

[0066] In this embodiment, through the analysis of the test report results, strong data support is provided for system performance optimization and architecture upgrade.

[0067] The present invention discloses a system test method for adaptively adjusting loads, which can be applied in industries such as finance and e-commerce, to predict the impact of load pressure that the system may face during business peaks on system performance, to warn of possible emergencies in advance, and to help optimize the system so that the system can ensure stable operation at critical moments. For different industries and different systems, the performance feature selection can be adaptively adjusted. For example, when the performance feature selection is the response time or throughput of the system, the actual performance feature data is evaluated as excellent, good, medium, and poor as follows:

[0068] Table 1: Response time

[0069]

[0070] Table 2: Throughput

[0071]

[0072] like Figure 3As shown, the present invention also discloses a system testing device for adaptively adjusting load, comprising:

[0073] A data set establishment module 301 is used to establish a data set consisting of historical load characteristic data and historical performance characteristic data, wherein the load characteristic data is used to characterize the load state of the tested system, and the performance characteristic data is used to characterize the operating state of the tested system under the load characteristic data;

[0074] A model building module 302 is used to perform model training using the data set to obtain a prediction model, where the prediction model is used to predict performance characteristic data based on the load characteristic data;

[0075] The prediction module 303 is used to obtain the instantaneous load characteristic data corresponding to the system in the test state, and use the prediction model to predict the predicted performance characteristic data corresponding to the instantaneous load characteristic data;

[0076] The adaptive adjustment module 304 is used to adjust the test data input by the system based on the predicted performance characteristic data and according to a preset adaptive adjustment strategy.

[0077] The present invention also discloses an electronic device, such as Figure 4 As shown, an embodiment is disclosed, which is a block diagram of an electronic device suitable for testing the system with the above-mentioned adaptive load adjustment.

[0078] The electronic device 40 of this embodiment includes a processor 401, which can perform various appropriate actions and processes according to the program stored in the ROM 402 or the program loaded from the storage part 408 into the RAM 403. The processor 401 may include, for example, a general-purpose microprocessor, an instruction set processor and / or a related chipset and / or a dedicated microprocessor, etc. The processor 401 may also include an onboard memory for caching purposes. The processor 401 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0079] In RAM403, various programs and data required for the operation of electronic device 40 are stored. Processor 401, ROM402 and RAM403 are connected to each other through bus 404, and processor 401 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM402 and / or RAM403. It should be noted that the program can also be stored in one or more memories other than ROM402 and RAM403, and processor 401 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in one or more memories.

[0080] According to an embodiment of the present invention, the electronic device 40 may further include an I / O interface 405, which is also connected to the bus 404. The electronic device 40 may further include one or more of the following components connected to the I / O interface 405: an input portion 406 including a keyboard, a mouse, etc.; an output portion 407 including a cathode ray tube, a liquid crystal display, and a speaker; a storage portion 408 including a hard disk, etc.; and a communication portion 409 including a network interface card such as a LAN card, a modem, etc. The communication portion 409 performs communication processing via a network such as the Internet. A drive 4010 is also connected to the I / O interface 405 as needed. A removable medium 4011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 4010 as needed, so that a computer program read therefrom is installed into the storage portion 408 as needed.

[0081] Embodiments of the present invention also include a computer program product.

[0082] The computer program product includes a computer program, which contains program codes for executing the method provided by the embodiment of the present invention. When the computer program product runs on an electronic device, the program codes are used to enable the electronic device to implement the method provided by the embodiment of the present invention.

[0083] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium. The program code included in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0084] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written by any combination of one or more programming languages, and specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages. Programming languages ​​include but are not limited to programming languages ​​such as Java, C++, python, C language or similar. The program code can be executed completely on the user computing device, partially on the user device, partially on the remote computing device, or completely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network or a wide area network, or can be connected to an external computing device.

[0085] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments and / or claims of the present invention can be combined and / or combined in various ways, even if such a combination or combination is not explicitly recorded in the present invention. In particular, without departing from the spirit and teaching of the present invention, the features described in the various embodiments and / or claims of the present invention may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present invention.

[0086] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination. The scope of the present invention is limited by the attached claims and their equivalents. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A system testing method for adaptively adjusting load, characterized in that: include: Establishing a data set consisting of historical load characteristic data and historical performance characteristic data, wherein the load characteristic data is used to characterize the load state of the tested system, and the performance characteristic data is used to characterize the operating state of the tested system under the load characteristic data; Using the data set to perform model training to obtain a prediction model, wherein the prediction model is used to predict the performance characteristic data based on the load characteristic data; Acquire instantaneous load characteristic data corresponding to the system in the test state, and use the prediction model to predict predicted performance characteristic data corresponding to the instantaneous load characteristic data; Based on the predicted performance characteristic data, adjusting the test data input by the system according to a preset adaptive adjustment strategy; The adjusting the test data input to the system based on the predicted performance characteristic data according to a preset adaptive adjustment strategy includes: calculating an average value of the historical performance characteristic data as the benchmark performance characteristic data, establishing a threshold range with the benchmark performance characteristic data as the center, and adjusting the test data input to the system according to a comparison result between the predicted performance characteristic data and the threshold range; The step of adjusting the test data input to the system according to the comparison result between the predicted performance characteristic data and the threshold range includes: If the system response time is lower than the set lower threshold, or the throughput is higher than the set upper threshold, the request frequency or request complexity is increased; If the system response time is higher than the set upper threshold, or the throughput is lower than the set lower threshold, the request frequency is reduced or the request complexity is simplified.

2. A system testing method for adaptively adjusting load according to claim 1, characterized in that: The method further includes: acquiring actual performance characteristic data of the system, and adjusting parameters of the prediction model according to a difference between the actual performance characteristic data and the predicted performance characteristic data corresponding to the same load characteristic data.

3. The system testing method for adaptively adjusting load according to claim 1, characterized in that: The method further includes: generating a test report according to the instantaneous load characteristic data and corresponding actual performance characteristic data of the system, wherein the test report is used to optimize and upgrade the system.

4. The system testing method for adaptively adjusting load according to claim 1, characterized in that: The adjusting the test data input by the system is adjusting the test data input by the system through an automated script.

5. The system testing method for adaptively adjusting load according to claim 3, characterized in that: The test report includes: the actual performance characteristic data is evaluated into several performance levels.

6. A system test device for adaptively adjusting load, characterized in that: The device comprises: Establishing a data set module, used to establish a data set consisting of historical load characteristic data and historical performance characteristic data, wherein the load characteristic data is used to characterize the load state of the tested system, and the performance characteristic data is used to characterize the operating state of the tested system under the load characteristic data; Establishing a model module for performing model training using the data set to obtain a prediction model, wherein the prediction model is used to predict the performance characteristic data based on the load characteristic data; A prediction module, used to obtain instantaneous load characteristic data corresponding to the system in the test state, and use the prediction model to predict predicted performance characteristic data corresponding to the instantaneous load characteristic data; An adaptive adjustment module is used to adjust the test data input to the system based on the predicted performance characteristic data and according to a preset adaptive adjustment strategy, calculate the average value of the historical performance characteristic data as the benchmark performance characteristic data, establish a threshold range with the benchmark performance characteristic data as the center, and adjust the test data input to the system according to the comparison result between the predicted performance characteristic data and the threshold range; if the system response time is lower than the set lower threshold, or the throughput is higher than the set upper threshold, the request frequency is increased or the request complexity is increased; if the system response time is higher than the set upper threshold, or the throughput is lower than the set lower threshold, the request frequency is reduced or the request complexity is simplified.

7. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are caused to perform the method according to any one of claims 1 to 5.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images