Pressure measurement simulation degree evaluation method and device, server, medium and program product

Abstract

The embodiment of the invention relates to a pressure measurement simulation degree evaluation method and device, a server, a medium and a program product. The method comprises the following steps: acquiring pressure measurement flow corresponding to a target service system, wherein the pressure measurement flow is obtained by performing full-link pressure measurement on the target service system; obtaining each simulation degree evaluation dimension, and obtaining a pressure measurement simulation degree evaluation value corresponding to the simulation degree evaluation dimension according to the pressure measurement flow; and performing statistical processing on the pressure measurement simulation degree evaluation values corresponding to the simulation degree evaluation dimensions to obtain a pressure measurement simulation degree evaluation result corresponding to the target business system. By adopting the method, the accuracy measurement and evaluation of the voltage measurement simulation degree can be realized.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of Internet technology, and in particular, to a stress testing simulation evaluation method, device, server, medium, and program product. Background Art

[0002] As online business scenarios become more complex, full-link stress testing technology has emerged. Full-link stress testing is a core means to deal with the many uncertainties faced by business systems in the current complex distributed environment that may lead to system availability issues. Full-link stress testing refers to the use of technical means to simulate the real requests of massive users and test and verify the entire business system in various scenarios.

[0003] Take the online car-hailing business as an example. The online car-hailing business has tidal characteristics and sudden traffic shocks during holidays. The stability guarantee of the online car-hailing business system is a huge challenge, so it is necessary to perform full-link stress testing on its business system. For the full-link stress testing of the online car-hailing business system, the biggest challenge is that there are a large number of uncertain factors when simulating real scenarios, such as the uncertainty of the time when passengers place orders, the uncertainty of road environment factors, etc. In the actual application process, it is necessary to ensure that the full-link stress testing is sufficiently simulated to help continuously discover potential stability risks of the business system.

[0004] In view of this, how to accurately measure and evaluate the stress testing simulation degree to ensure the full-link stress testing effect has become a problem that needs to be solved urgently. Summary of the invention

[0005] The embodiments of the present disclosure provide a stress test simulation degree evaluation method, device, server, medium and program product, which can achieve accurate measurement and evaluation of the stress test simulation degree.

[0006] In a first aspect, an embodiment of the present disclosure provides a stress test simulation evaluation method, the method comprising:

[0007] Obtain the stress test traffic corresponding to the target business system, where the stress test traffic is obtained by performing full-link stress testing on the target business system;

[0008] Obtain each simulation evaluation dimension, and obtain the stress testing simulation evaluation value corresponding to the simulation evaluation dimension according to the stress testing flow;

[0009] The stress testing simulation degree evaluation values ​​corresponding to each of the simulation degree evaluation dimensions are statistically processed to obtain the stress testing simulation degree evaluation results corresponding to the target business system.

[0010] In one of the embodiments, the simulation degree evaluation dimension includes an interface coverage evaluation dimension, a module coverage evaluation dimension, a link coverage evaluation dimension, a category coverage evaluation dimension, and at least two of the category coverage evaluation dimensions.

[0011] In one embodiment, the simulation evaluation dimension includes the interface coverage evaluation dimension and the module coverage evaluation dimension, and obtaining the stress testing simulation evaluation value corresponding to the simulation evaluation dimension according to the stress testing flow includes:

[0012] Obtaining the predicted traffic corresponding to the target business system, where the predicted traffic is obtained by predicting the real online traffic of the target business system within a preset time period;

[0013] According to the predicted flow and the stress test flow, a stress test simulation degree evaluation value corresponding to the simulation degree evaluation dimension is obtained.

[0014] In one embodiment, the simulation evaluation dimension includes the interface coverage evaluation dimension, and obtaining the stress test simulation evaluation value corresponding to the simulation evaluation dimension according to the predicted traffic and the stress test traffic includes:

[0015] According to the predicted traffic and the stress test traffic, obtaining a simulation degree evaluation value of the interface traffic of the target business system, and according to the stress test traffic, obtaining an interface coverage evaluation value of the target business system;

[0016] The interface traffic achievement simulation degree evaluation value and the interface coverage evaluation value are subjected to mathematical operation to obtain a stress test simulation degree evaluation value corresponding to the interface coverage evaluation dimension.

[0017] In one embodiment, obtaining the interface traffic of the target business system according to the predicted traffic and the stress test traffic to achieve a simulation degree evaluation value includes:

[0018] For each target interface in the target business system, based on the predicted traffic and the stress test traffic, a ratio of the first interface predicted traffic of the target interface to the first interface stress test traffic is substituted as an independent variable into the interface traffic achievement calculation formula to obtain a traffic achievement simulation degree evaluation value of the target interface;

[0019] According to the interface weight of each of the target interfaces, the traffic simulation degree evaluation value of each of the target interfaces is statistically processed to obtain the interface traffic simulation degree evaluation value.

[0020] In one embodiment, based on the predicted flow and the stress test flow, the ratio of the first interface predicted flow of the target interface to the first interface stress test flow is substituted as an independent variable into the interface flow achievement calculation formula to obtain the flow achievement simulation evaluation value of the target interface, including:

[0021] Determine, according to the predicted flow and the stress test flow, a ratio of the predicted flow of the first interface to the stress test flow of the first interface;

[0022] If the ratio is within the preset ratio interval, the reciprocal of the ratio is substituted as the independent variable into the interface flow achievement calculation formula to obtain the flow achievement simulation degree evaluation value of the target interface;

[0023] If the ratio is not within the preset ratio interval, the ratio is substituted as the independent variable into the interface flow achievement calculation formula to obtain the flow achievement simulation degree evaluation value of the target interface.

[0024] In one embodiment, obtaining the interface coverage evaluation value of the target business system according to the stress testing flow includes:

[0025] For each target interface in the target business system, a target interface coverage value of the target interface is obtained according to the stress testing traffic, wherein if the stress testing traffic indicates that the target interface is covered by traffic during the full-link stress testing process, the target interface coverage value is a first value; if the stress testing traffic indicates that the target interface is not covered by traffic during the full-link stress testing process, the target interface coverage value is a second value, and the first value and the second value are different;

[0026] According to the interface weight of each of the target interfaces, the target interface coverage value of each of the target interfaces is statistically processed to obtain the interface coverage evaluation value.

[0027] In one embodiment, the method further comprises:

[0028] Display the first display page;

[0029] Among them, the first display page includes the first evaluation entry of each target interface, and the first evaluation entry includes the interface identifier of the target interface, the first interface stress test flow of the target interface, the first interface predicted flow of the target interface, the interface weight of the target interface, the flow achievement simulation degree evaluation value of the target interface and the target interface coverage value of the target interface.

[0030] In one embodiment, the simulation evaluation dimension includes a module coverage evaluation dimension, and obtaining a stress test simulation evaluation value corresponding to the simulation evaluation dimension according to the predicted traffic and the stress test traffic includes:

[0031] For each business module in the target business system, obtaining module fitness data corresponding to the business module according to the predicted traffic and the stress test traffic;

[0032] According to the module weight of each of the business modules, statistical processing is performed on the fitting data of each of the modules to obtain a stress testing simulation evaluation value corresponding to the module coverage evaluation dimension.

[0033] In one embodiment, the obtaining module fitness data corresponding to the business module according to the predicted traffic and the stress test traffic includes:

[0034] According to the predicted flow and the stress test flow, obtaining an interface fitness value corresponding to each business interface in the business module and a resource consumption fitness value corresponding to the business module;

[0035] According to the interface weight of each of the service interfaces, statistical processing is performed on the interface fitness values ​​corresponding to each of the service interfaces to obtain the flow fitness values ​​corresponding to the service modules;

[0036] The resource consumption fitness value and the traffic fitness value are subjected to mathematical operation processing to obtain the module fitness data corresponding to the business module.

[0037] In one embodiment, obtaining the interface fitness value corresponding to each business interface in the business module and the resource consumption fitness value corresponding to the business module according to the predicted traffic and the stress test traffic includes:

[0038] Determine the predicted resource consumption quantity of the business module from the predicted traffic, determine the resource consumption stress test quantity of the business module from the stress test traffic, and obtain the resource consumption fitting degree value corresponding to the business module according to the predicted resource consumption quantity and the resource consumption stress test quantity;

[0039] For each of the business interfaces in the business module, the second interface predicted traffic of the business interface is determined from the predicted traffic, the second interface stress test traffic of the business interface is determined from the stress test traffic, and the interface fitness value corresponding to the business interface is obtained based on the second interface predicted traffic and the second interface stress test traffic.

[0040] In one embodiment, obtaining the interface fitness value corresponding to the business interface according to the second interface predicted traffic and the second interface stress test traffic includes:

[0041] Calculate a first fitting coefficient of the service interface according to the predicted traffic of the second interface and the stress test traffic of the second interface, and compare the first fitting coefficient with a preset coefficient threshold;

[0042] If the first fitting coefficient is greater than the coefficient threshold, the interface fitting value corresponding to the business interface is calculated based on the second interface predicted traffic, the second interface stress test traffic and the first fitting formula;

[0043] If the first fitting coefficient is less than or equal to the coefficient threshold, the interface fitting value corresponding to the business interface is calculated based on the predicted traffic of the second interface, the stress test traffic of the second interface and the second fitting formula, and the first fitting formula is different from the second fitting formula.

[0044] In one embodiment, obtaining the resource consumption fitting value corresponding to the business module according to the resource consumption prediction quantity and the resource consumption stress test quantity includes:

[0045] Calculate a second fitting coefficient of the business module according to the predicted resource consumption quantity and the resource consumption stress test quantity, and compare the second fitting coefficient with a preset coefficient threshold;

[0046] If the second fitting coefficient is greater than the coefficient threshold, the resource consumption fitting value corresponding to the business module is calculated based on the predicted resource consumption quantity, the resource consumption stress test quantity and the first fitting formula;

[0047] If the second fitting coefficient is less than or equal to the coefficient threshold, the resource consumption fitting value corresponding to the business module is calculated based on the predicted resource consumption quantity, the resource consumption stress test quantity and the second fitting formula, and the first fitting formula is different from the second fitting formula.

[0048] In one embodiment, the method further comprises:

[0049] Display the second display page;

[0050] Among them, the second display page includes second evaluation items of each of the business modules, and the second evaluation items include the module identification of the business module, the traffic fit value of the business module, the resource consumption fit value of the business module, and the stress testing simulation evaluation result of the business module. The stress testing simulation evaluation result of the business module is calculated based on the traffic fit value of the business module and the resource consumption fit value of the business module.

[0051] In one embodiment, the simulation degree evaluation dimension includes a link coverage evaluation dimension, and obtaining a stress test simulation degree evaluation value corresponding to the simulation degree evaluation dimension according to the stress test traffic includes:

[0052] According to the stress test flow, obtaining a link coverage evaluation value of each service link in the target service system;

[0053] According to the link weight of each of the service links, the link coverage evaluation value of each of the service links is statistically processed to obtain a stress test simulation evaluation value of the link coverage evaluation dimension.

[0054] In one of the embodiments, obtaining the link coverage evaluation value of each service link in the target service system according to the stress test flow includes:

[0055] For each of the service links, obtaining a link interface coverage value of each link interface in the service link according to the stress test flow;

[0056] According to the interface weight of each link interface, the link interface coverage value of each link interface is statistically processed to obtain the link coverage evaluation value of the service link.

[0057] In one embodiment, the method further comprises:

[0058] Display the third display page;

[0059] The third display page includes a third evaluation entry of each link interface in each service link, and the third evaluation entry includes at least an interface identifier of the link interface and a link interface coverage value of the link interface.

[0060] In one embodiment, the simulation evaluation dimension includes a scene coverage evaluation dimension, and obtaining the stress testing simulation evaluation value corresponding to the simulation evaluation dimension according to the stress testing flow includes:

[0061] Determine the scenario stress testing flow of each business scenario corresponding to the target business system according to the stress testing flow;

[0062] According to the scenario weight of each of the business scenarios, the stress testing traffic of each scenario is statistically processed to obtain a stress testing simulation degree evaluation value of the scenario coverage evaluation dimension, wherein the scenario weight is the ratio of the scenario stress testing traffic to the stress testing traffic.

[0063] In one embodiment, the method further comprises:

[0064] Display the fourth display page;

[0065] Among them, the fourth display page includes fourth evaluation items for each of the business scenarios, and the fourth evaluation items at least include the scenario identifier of the business scenario and the stress testing simulation evaluation result of the business scenario, and the stress testing simulation evaluation result of the business scenario is determined based on the scenario stress testing traffic of the business scenario.

[0066] In one embodiment, the simulation evaluation dimension includes a category coverage evaluation dimension, and obtaining the stress testing simulation evaluation value corresponding to the simulation evaluation dimension according to the stress testing flow includes:

[0067] Determine the category stress testing flow of each business category corresponding to the target business system according to the stress testing flow;

[0068] According to the category weight of each of the business categories, statistical processing is performed on the stress testing traffic of each category to obtain a stress testing simulation evaluation value of the category coverage evaluation dimension, wherein the category weight is the ratio of the category stress testing traffic to the stress testing traffic.

[0069] In one embodiment, the method further comprises:

[0070] Display the fifth display page;

[0071] Among them, the fifth display page includes the fifth evaluation item of each of the business categories, and the fifth evaluation item at least includes the category identification of the business category and the stress testing simulation evaluation result of the business category, and the stress testing simulation evaluation result of the business category is determined based on the category stress testing traffic of the business category.

[0072] In a second aspect, an embodiment of the present disclosure provides a stress test evaluation device, the device comprising:

[0073] An acquisition module is used to acquire the stress test traffic corresponding to the target business system, where the stress test traffic is obtained by performing a full-link stress test on the target business system;

[0074] A determination module is used to obtain each simulation evaluation dimension, and obtain a stress testing simulation evaluation value corresponding to the simulation evaluation dimension according to the stress testing flow;

[0075] The evaluation module is used to perform statistical processing on the stress testing simulation evaluation values ​​corresponding to each simulation evaluation dimension to obtain the stress testing simulation evaluation results corresponding to the target business system.

[0076] In a third aspect, an embodiment of the present disclosure provides a server, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the computer program.

[0077] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in the first aspect above.

[0078] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which implements the method described in the first aspect when executed by a processor.

[0079] The stress test simulation evaluation method, device, server, medium and program product provided by the embodiments of the present disclosure obtain the stress test traffic corresponding to the target business system, where the stress test traffic is obtained by performing full-link stress test on the target business system, and then obtain each simulation evaluation dimension, and obtain the stress test simulation evaluation value corresponding to the simulation evaluation dimension according to the stress test traffic, and then perform statistical processing on the stress test simulation evaluation value corresponding to each simulation evaluation dimension to obtain the stress test simulation evaluation result corresponding to the target business system. In this way, the embodiments of the present disclosure set multiple simulation evaluation dimensions, for example, starting from the dimensions of interface coverage, link coverage, scenario coverage, category coverage, etc., the stress test traffic obtained by performing full-link stress test on the target business system is used to calculate the stress test simulation evaluation value of each simulation evaluation dimension, and each stress test simulation evaluation value can accurately measure and evaluate the stress test simulation degree of the full-link stress test from different simulation evaluation dimensions, and then perform statistical processing on each stress test simulation evaluation value to obtain the overall stress test simulation evaluation result. The embodiments of the present disclosure achieve accurate measurement and evaluation of stress test simulation degree. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 This is a diagram of an application environment of a stress test simulation evaluation method in an embodiment;

[0081] Figure 2 Schematic diagram of a flow chart of a stress test simulation evaluation method in one embodiment;

[0082] Figure 3 A schematic diagram of a process in which a server obtains a stress test simulation degree evaluation value corresponding to a simulation degree evaluation dimension according to a stress test flow rate in another embodiment when the simulation degree evaluation dimension includes an interface coverage evaluation dimension and a module coverage evaluation dimension;

[0083] Figure 4 is a flow chart of step 302 in another embodiment;

[0084] Figure 5 It is a schematic diagram of the distribution of simulation degree evaluation values ​​achieved by the flows of each target interface when the absolute value of the difference between the predicted flow of the first interface of each target interface and the stress test flow of the first interface is used as an independent variable in another embodiment;

[0085] Figure 6 It is a schematic diagram of the distribution of the simulation degree evaluation value achieved by the flow of each target interface when the ratio of the first interface predicted flow of each target interface to the first interface stress test flow is used as an independent variable in another embodiment;

[0086] Figure 7 A schematic diagram of the distribution of the number of target interfaces corresponding to each stress test simulation degree evaluation value before traffic filtering in another embodiment;

[0087] Figure 8 A schematic diagram of the distribution of stress test simulation evaluation values ​​of each target interface before traffic filtering in another embodiment;

[0088] Fig. 9 A schematic diagram of the distribution of the number of target interfaces corresponding to each stress test simulation degree evaluation value after traffic filtering in another embodiment;

[0089] Fig.10 A schematic diagram of the distribution of stress test simulation evaluation values ​​of each target interface after traffic filtering in another embodiment;

[0090] Fig.11 is a schematic diagram of an exemplary first display page in another embodiment;

[0091] Fig.12 A curve diagram of pressure measurement flow and predicted flow in another embodiment;

[0092] Fig.13 is a flow chart of step 302 in another embodiment;

[0093] Fig.14 is a schematic diagram of an exemplary second display page in another embodiment;

[0094] Fig.15 A curve chart of resource consumption prediction quantity and resource consumption stress test quantity in another embodiment;

[0095] Fig.16 A schematic diagram of a process in which, in another embodiment, the simulation degree evaluation dimension includes a link coverage evaluation dimension, and the server obtains a stress test simulation degree evaluation value corresponding to the simulation degree evaluation dimension according to the stress test traffic;

[0096] Fig.17 It is a schematic diagram of link coverage evaluation values ​​of each service link when no filtering rule is adopted in another embodiment;

[0097] Fig.18 It is a schematic diagram of link coverage evaluation values ​​of each service link after traffic is filtered using filtering rules in another embodiment;

[0098] Fig.19 is a schematic diagram of an exemplary third display page in another embodiment;

[0099] Fig. 20 A schematic diagram of a process in which a server obtains a stress test simulation degree evaluation value corresponding to the simulation degree evaluation dimension according to a stress test traffic in another embodiment when the simulation degree evaluation dimension includes a scene coverage evaluation dimension;

[0100] Fig.21 A schematic diagram of a process in which a server obtains a stress test simulation degree evaluation value corresponding to the simulation degree evaluation dimension according to stress test traffic in another embodiment when the simulation degree evaluation dimension includes a category coverage evaluation dimension;

[0101] Fig. 22 is a schematic diagram of a fifth display interface of an exemplary fourth display page in another embodiment;

[0102] Fig.23 It is a structural block diagram of a stress test simulation evaluation device in another embodiment;

[0103] Fig.24 FIG. 4 is a diagram showing the internal structure of a server in another embodiment. DETAILED DESCRIPTION

[0104] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure more clear, the embodiments of the present disclosure are further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the embodiments of the present disclosure and are not used to limit the embodiments of the present disclosure.

[0105] With the increasing complexity of online business scenarios, full-link stress testing technology has emerged. Full-link stress testing is a core means to deal with the many uncertainties faced by business systems in the current complex distributed environment that may lead to system availability issues. Full-link stress testing refers to the use of technical means to simulate the real requests of massive users and test and verify the entire business system in various scenarios. Using full-link stress testing technology to test and verify the entire business system in various scenarios can continuously discover potential stability risks of the business system, assist in verifying whether the business system can withstand the traffic estimates of major activities and events that have been formulated, and assist various service providers in conducting risk investigations and problem solving.

[0106] Take the online car-hailing business as an example. The online car-hailing business has tidal characteristics and sudden traffic shocks during holidays. The stability guarantee of the online car-hailing business system is a huge challenge, so it is necessary to perform full-link stress testing on its business system. The biggest challenge for the full-link stress testing of the online car-hailing business system is that there are a lot of uncertain factors when simulating real scenarios, such as:

[0107] 1) There is spatial uncertainty in the time when passengers place their orders and the locations of drivers and passengers. After placing an order, passengers are required to receive the order within a fixed time. The matching conditions between drivers and passengers depend on the distance between them, but the locations where drivers are deployed are scattered according to the locations where online drivers leave.

[0108] 2) Uncertainty in order distance and driver pick-up and drop-off time. During the full-link stress testing process, the stress test driver pick-up and drop-off scenario cannot completely replicate the real online road scenario, such as traffic jams. If the stress test driver picks up and drops off too fast, it will lead to excessive capacity, and if it is too slow, it will lead to insufficient capacity, resulting in a deviation between the stress test scenario and the real online scenario.

[0109] 3) Uncertainty in order completion. Drivers’ acceptance of orders is completely dependent on the order matching strategy. The same driver at the same location may receive different orders. There are also factors such as scene uncertainty, category uncertainty, and the resource grabbing of public modules in the business system by traffic from other business lines, etc.

[0110] With so many uncertainties, what is the stress test simulation degree (or stress test effect, stress test accuracy or stress test credibility) of the full-link stress test? We need to ensure that the full-link stress test is sufficiently simulated to help continuously discover potential stability risks of the business system and achieve the goal of "withstanding any stress" for the business system.

[0111] In view of this, how to accurately measure and evaluate the stress testing simulation degree to ensure the full-link stress testing effect has become a problem that needs to be solved urgently.

[0112] In the related art, there is also a method of manually evaluating the stress test simulation degree. However, the evaluation criteria rely on manual experience, the evaluation dimension is single, and there is a lack of quantitative means, which leads to low reliability of stress test simulation degree evaluation.

[0113] In addition, it should be noted that the applicant has put in a lot of creative work in determining the measurement and evaluation of the stress test simulation degree required and the technical solutions introduced in the following embodiments.

[0114] The technical solutions involved in the embodiments of the present disclosure are introduced below in combination with the scenarios to which the embodiments of the present disclosure are applied.

[0115] The stress test simulation evaluation method provided in the embodiment of the present disclosure can be applied to Figure 1 In the application environment shown, the terminal 102 communicates with the server 104 through the network. Taking the online car-hailing service as an example, the terminal 102 can be a terminal held by the driver (ie, the driver's terminal) or a terminal held by the passenger (ie, the passenger's terminal).

[0116] The server 104 can obtain the stress testing traffic obtained by performing full-link stress testing on the target business system. For example, the full-link stress testing of the target business system can be implemented through the interaction between the server 104 and the terminal 102 to obtain the stress testing traffic. The server 104 obtains each simulation evaluation dimension, and obtains the stress testing simulation evaluation value corresponding to the simulation evaluation dimension based on the stress testing traffic. The server 104 statistically processes the stress testing simulation evaluation value corresponding to each simulation evaluation dimension to obtain the stress testing simulation evaluation result corresponding to the target business system.

[0117] The terminal 102 may be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, and portable wearable devices, and the server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0118] In one embodiment, Figure 2 As shown, a stress test simulation evaluation method is provided, which is applied to Figure 1 The server in the example is used to illustrate the following steps:

[0119] In step 201, the server obtains the stress test traffic corresponding to the target business system. The stress test traffic is obtained by performing a full-link stress test on the target business system.

[0120] The target business system can be a business system that can provide online services, such as an online car-hailing business system. When full-link stress testing is required, full-link stress testing technology is used to perform full-link stress testing on the target business system. After the stress testing is completed, the above-mentioned stress testing traffic is obtained. The stress testing traffic may include simulated user requests, traffic data generated during the full-link processing of the target business system for the user request, and so on.

[0121] Taking the online car-hailing business system as an example, before major holidays and traffic peaks, it is necessary to conduct full-link stress testing on the online car-hailing business system in advance to ensure the stability of the online car-hailing business system.

[0122] In the embodiment of the present disclosure, after the full-link stress test of the target business system is completed, the server obtains the stress test traffic corresponding to the target business system.

[0123] In a possible implementation, the server performs a full-link stress test on the target business system by itself. After the full-link stress test is completed, the server can obtain the above stress test traffic.

[0124] In other possible implementations, the full-link stress testing process for the target business system may also be performed by other devices, which obtain the above-mentioned stress testing traffic after the full-link stress testing is completed, and then the server obtains the above-mentioned stress testing traffic from the other devices.

[0125] In step 202, the server obtains each simulation evaluation dimension, and obtains the stress test simulation evaluation value corresponding to the simulation evaluation dimension according to the stress test flow.

[0126] The simulation evaluation dimensions can be determined by referring to the business type of the target business system. For example, taking the online car-hailing business as an example, there are a large number of interface calls in the course of the online car-hailing business, involving a variety of online car-hailing scenarios (such as airport taxi scenarios, peak-hour taxi scenarios, etc.), involving a variety of categories (such as express categories, private car categories), etc. Therefore, the simulation evaluation dimensions can be set as interface coverage evaluation dimensions, scenario coverage evaluation dimensions, category coverage evaluation dimensions, etc.

[0127] In the embodiments of the present disclosure, exemplarily, the above-mentioned simulation evaluation dimensions can be determined by a technician in this field based on relevant experience in stress testing simulation evaluation, and each simulation evaluation dimension can be manually input through an input interface (such as a screen included in the terminal). The server can obtain each simulation evaluation dimension by interacting with the terminal; of course, in other possible implementations, the server can also analyze the system characteristics of the target business system by itself, and determine each simulation evaluation dimension in combination with relevant algorithms or models, and no specific limitation is made here.

[0128] As an implementation manner, in the embodiments of the present disclosure, the simulation degree evaluation dimension may include at least two of the interface coverage evaluation dimension, the module coverage evaluation dimension, the link coverage evaluation dimension, the scene coverage evaluation dimension and the category coverage evaluation dimension. For example, the simulation degree evaluation dimension may include the above five evaluation dimensions.

[0129] Among them, the interface coverage evaluation dimension is to evaluate the stress testing simulation degree of the target business system by taking each interface in the target business system as the evaluation object; the module coverage evaluation dimension is to evaluate the stress testing simulation degree of the target business system by taking each business module in the target business system as the evaluation object; the link coverage evaluation dimension is to evaluate the stress testing simulation degree of the target business system by taking each business link (including main link and sub-link) in the target business system as the evaluation object; the scenario coverage evaluation dimension is to evaluate the stress testing simulation degree of the target business system by taking each business scenario corresponding to the target business system as the evaluation object; the category coverage evaluation dimension is to evaluate the stress testing simulation degree of the target business system by taking each business category in the target business system as the evaluation object.

[0130] After the server obtains each simulation degree evaluation dimension, for each simulation degree evaluation dimension, the server obtains a stress test simulation degree evaluation value corresponding to the simulation degree evaluation dimension according to the stress test traffic.

[0131] Exemplarily, the server can compare the stress testing traffic with the actual online traffic of the target business system (the online actual traffic can be predicted, so it can be called predicted traffic) to obtain the stress testing simulation evaluation value. For example, for the interface coverage evaluation dimension, the server can compare the difference between the stress testing traffic and the predicted traffic of each interface to evaluate the achievement of interface traffic during the full-link stress testing process. The server can also check the coverage of each interface during the full-link stress testing process based on the stress testing traffic, and so on. By quantifying the achievement of these interface traffic and interface coverage, the stress testing simulation evaluation value of the corresponding interface coverage evaluation dimension can be obtained.

[0132] The process of the server obtaining the stress test simulation evaluation value corresponding to the simulation evaluation dimension according to the stress test traffic will be described by distinguishing the simulation evaluation dimensions in the following embodiment.

[0133] It is understandable that full-link stress testing is usually carried out in advance. For example, before major holidays, the online car-hailing business system is subjected to full-link stress testing in advance to ensure the stability of the online car-hailing business system. Therefore, the stress test traffic is obtained by simulating the online business development process in a future time period (such as a major holiday mentioned above). Therefore, correspondingly, the online real traffic can also be the predicted traffic obtained by predicting the online real traffic of the target business system in that future time period.

[0134] In step 203, the server performs statistical processing on the stress test simulation evaluation values ​​corresponding to each simulation evaluation dimension to obtain a stress test simulation evaluation result corresponding to the target business system.

[0135] For different simulation evaluation dimensions, corresponding dimension weights can be set according to their importance. Then, based on the weights of each dimension, the server performs weighted summation processing on the stress test simulation evaluation values ​​corresponding to each simulation evaluation dimension, and obtains the final stress test simulation evaluation result corresponding to the target business system.

[0136] Furthermore, the server can also visualize the stress test simulation evaluation value corresponding to each simulation evaluation dimension and the stress test simulation evaluation result corresponding to the target business system.

[0137] The above-mentioned embodiment obtains the stress test traffic corresponding to the target business system, and the stress test traffic is obtained by performing full-link stress testing on the target business system. Then, each simulation evaluation dimension is obtained, and the stress test simulation evaluation value corresponding to the simulation evaluation dimension is obtained according to the stress test traffic. Then, the stress test simulation evaluation value corresponding to each simulation evaluation dimension is statistically processed to obtain the stress test simulation evaluation result corresponding to the target business system. In this way, the embodiment of the present disclosure sets multiple simulation evaluation dimensions. For example, starting from the dimensions of interface coverage, link coverage, scenario coverage, category coverage, etc., the stress test traffic obtained by performing full-link stress testing on the target business system is used to calculate the stress test simulation evaluation value of each simulation evaluation dimension. Each stress test simulation evaluation value can accurately measure and evaluate the stress test simulation degree of the full-link stress test from different simulation evaluation dimensions, and then the overall stress test simulation evaluation result can be obtained by statistically processing each stress test simulation evaluation value. The embodiment of the present disclosure realizes accurate measurement and evaluation of the stress test simulation degree.

[0138] In one embodiment, based on Figure 2 The embodiment shown, see Figure 3 This embodiment relates to the process of how the server obtains the stress test simulation evaluation value corresponding to the simulation evaluation dimension according to the stress test traffic when the simulation evaluation dimension includes the interface coverage evaluation dimension and the module coverage evaluation dimension. Figure 3 As shown, the server can execute Figure 3 Steps 301 and 302 shown in the figure implement the process of obtaining the stress test simulation evaluation value corresponding to the simulation evaluation dimension according to the stress test flow:

[0139] Step 301: The server obtains the predicted traffic corresponding to the target business system. The predicted traffic is obtained by predicting the real online traffic of the target business system within a preset time period.

[0140] The preset time period may be a time period in the future, such as a certain day in the future, an evening rush hour time period in a certain day in the future, an morning rush hour time period in a certain day in the future, and so on.

[0141] As mentioned above, the stress test traffic is obtained by simulating the online business development process in a future time period (such as a major holiday mentioned above). Therefore, the real online traffic can also be the predicted traffic obtained by predicting the real online traffic of the target business system in the future time period (i.e., the preset time period in step 301).

[0142] Continuing with the example of the online car-hailing business, suppose a full-link stress test is performed on the nearest major holiday to obtain the stress test traffic. When predicting the predicted traffic for this major holiday, the historical online real traffic of the same type of historical holidays (for example, the above-mentioned major holiday is National Day, and the historical holiday of the same type can be last year's National Day) can be combined for prediction. For example, the historical online real traffic of the historical holiday can be increased by a certain proportion, and combined with the historical traffic growth, factors such as weather and solar terms can be introduced to predict the predicted traffic for this major holiday. This prediction process can be achieved through the corresponding prediction algorithm.

[0143] In the actual traffic forecasting process, the actual online situation is very complicated. Any online changes or online anomalies will affect the traffic forecasting effect. For example, excessive traffic during holiday peak periods and business execution degradation will lead to large changes in the online traffic model. In the order matching business, if the underlying service is stuck / dead / avalanche, the upper link will be hit by a much larger impact than the estimated traffic.

[0144] Therefore, if the target business system's accurate predicted traffic can be predicted, the simulation accuracy can be greatly improved. Therefore, before traffic prediction, the prediction algorithm can be verified multiple times on actual holidays to ensure the accuracy of its predictions. For example, the prediction error of the prediction algorithm for core business indicators is controlled below 5%, thereby improving the accuracy of the prediction.

[0145] In step 302, the server obtains a stress test simulation evaluation value corresponding to the simulation evaluation dimension according to the predicted traffic and the stress test traffic.

[0146] After the server obtains the predicted traffic corresponding to the target business system, the server can obtain the stress test simulation evaluation value corresponding to the interface coverage evaluation dimension and the stress test simulation evaluation value corresponding to the module coverage evaluation dimension based on the predicted traffic and the stress test traffic.

[0147] In a possible implementation, before step 302, a black and white list may be set, the black and white list respectively including the interface identifiers (such as interface names) of the corresponding interfaces. Some interfaces that cannot obtain stress testing traffic and / or predicted traffic (such as interfaces involved in some offline task calls in the target business system) are added to the blacklist. In this way, the traffic corresponding to the interfaces in the blacklist can be filtered out from the stress testing traffic and predicted traffic; and for some core interfaces with lower traffic, the interface identifiers of such core interfaces can be added to the whitelist to avoid them being identified as low-frequency traffic and filtered out.

[0148] The following describes the process of obtaining a stress test simulation evaluation value corresponding to the simulation evaluation dimension based on the predicted traffic and stress test traffic by the server, with respect to the interface coverage evaluation dimension and the module coverage evaluation dimension.

[0149] 1) The simulation evaluation dimension includes the interface coverage evaluation dimension:

[0150] See also Figure 4 In the case where the simulation evaluation dimension includes the interface coverage evaluation dimension, step 302 may include Figure 4 Step 401 and step 402 are shown:

[0151] In step 401, the server obtains a simulation degree evaluation value of the interface traffic of the target business system based on the predicted traffic and the stress test traffic, and obtains an interface coverage evaluation value of the target business system based on the stress test traffic.

[0152] The interface traffic simulation evaluation value represents whether the stress test traffic of each interface in the target business system (hereinafter, the interface in the target business system is referred to as the target interface in the interface coverage evaluation dimension for distinction) reaches the expected target (predicted traffic), which can reflect whether the stress test means or stress test scenario configuration is reasonable; the interface coverage evaluation value represents whether each target interface is covered during the full-link stress test, that is, whether each target interface is stress tested.

[0153] The following introduces the process of the server obtaining the interface traffic of the target business system based on the predicted traffic and the stress test traffic to achieve the simulation evaluation value, and obtaining the interface coverage evaluation value of the target business system based on the stress test traffic.

[0154] On the one hand, illustratively, the server may perform the following steps A1 and A2 to achieve a process of obtaining a simulation degree evaluation value of the interface traffic of the target business system according to the predicted traffic and the stress test traffic:

[0155] Step A1, for each target interface in the target business system, the server substitutes the ratio of the first interface predicted traffic of the target interface to the first interface stress test traffic as the independent variable into the interface traffic achievement calculation formula based on the predicted traffic and stress test traffic, and obtains the traffic achievement simulation evaluation value of the target interface.

[0156] The calculation formula for the interface traffic volume can be shown as Formula 1:

[0157]

[0158] Before calculating the traffic of the target interface to achieve the simulation evaluation value, data normalization is first performed. The purpose of data normalization is to achieve forward processing (or consistency processing), dimensionless processing (that is, different indicators correspond to different units) and normalization processing of the data. Consistency processing refers to unifying the types of evaluation indicators so that the property value of the scheme with better performance becomes larger after the change.

[0159] In the embodiment of the present disclosure, in the selection process of the normalization method, it is considered that the expected value is preferably an appropriate intermediate value, such as the closer the expected stress test flow is to the predicted flow, the better, rather than the larger the stress test flow is, the better or the smaller the stress test flow is, the better. Therefore, the intermediate attribute transformation formula is selected as the interface flow calculation formula.

[0160] In formula 1, x i The i in the table represents the i-th target interface, and x i is the ratio of the first interface predicted flow of the target interface to the first interface stress test flow, k is x i The optimal expectation of x can be set to 1 (i.e., the predicted flow of the first interface of the target interface is equal to the stress test flow of the first interface). i The worst threshold is set to 7, for example.

[0161] It should be noted that the traffic simulation degree evaluation value is expected to be a fixed score value, and the previous score will not be affected by the addition of new traffic samples. i The threshold is set, if x i When it is greater than 7, the traffic simulation evaluation value is 0.

[0162] In this way, for each target interface, the server can use formula 1 to obtain the traffic simulation degree evaluation value of the target interface.

[0163] In a possible implementation of step A1, step A1 may also include: the server determines the ratio of the predicted traffic of the first interface to the stress test traffic of the first interface based on the predicted traffic and the stress test traffic; if the ratio is within a preset ratio range, the server substitutes the reciprocal of the ratio as an independent variable into the interface traffic achievement calculation formula to obtain the traffic achievement simulation degree evaluation value of the target interface; if the ratio is not within the preset ratio range, the server substitutes the ratio as an independent variable into the interface traffic achievement calculation formula to obtain the traffic achievement simulation degree evaluation value of the target interface.

[0164] That is, for each target interface, the server first determines the first interface predicted traffic corresponding to the target interface in the predicted traffic, and determines the first interface stress test traffic corresponding to the target interface in the stress test traffic, and then calculates the ratio of the first interface predicted traffic to the first interface stress test traffic.

[0165] The server compares the ratio with a preset ratio interval. The preset ratio interval may be an interval not greater than the worst threshold. For example, the preset ratio interval may be (0, 2).

[0166] During the specific implementation process, the inventors of the present application discovered through a large number of experimental simulations that when the ratio of the first interface predicted traffic to the first interface stress test traffic is in the range of (0, 2) and (2, 7), there will be a problem of large differences in the traffic achievement simulation evaluation values. Therefore, for the case where the ratio of the first interface predicted traffic to the first interface stress test traffic is (0, 2), let x = 1 / x, that is, let the ratio of the first interface predicted traffic to the first interface stress test traffic be equal to its reciprocal, and substitute it into the interface traffic achievement calculation formula to calculate the traffic achievement simulation evaluation value of the target interface, which can effectively avoid the above problems.

[0167] The following describes the beneficial effect of selecting the ratio of the first interface predicted flow rate to the first interface pressure measurement flow rate as an independent variable.

[0168] For the indicator of achieving simulation degree evaluation value of the traffic of the target interface, there are two ways to select independent variables: 1) using the absolute value of the difference between the predicted traffic of the first interface and the stress test traffic of the first interface as the independent variable, 2) using the ratio of the predicted traffic of the first interface to the stress test traffic of the first interface as the independent variable.

[0169] During the research and development process, the inventor of the present application found through a large number of experimental simulations that the above-mentioned method of using the absolute value of the difference between the predicted flow of the first interface and the pressure measurement flow of the first interface as the independent variable has too large a difference in sample points, resulting in the calculated flow simulation evaluation values ​​of each target interface being concentrated and scattered at one point, and most of them are full marks, such as Figure 5 As shown, Figure 5 The distribution of simulation evaluation values ​​of the traffic of each target interface is achieved by using the absolute value of the difference between the first interface predicted traffic of each target interface and the first interface stress test traffic as an independent variable.

[0170] When the ratio of the first interface predicted flow of the target interface to the first interface stress test flow is used as the independent variable, the problem of concentrated distribution of the flow achievement simulation evaluation values ​​of each target interface is greatly improved. Figure 6 As shown, Figure 6 The distribution of simulation evaluation values ​​of the traffic of each target interface is achieved by using the ratio of the first interface predicted traffic of each target interface to the first interface stress test traffic as an independent variable.

[0171] In addition, after selecting the ratio of the predicted traffic of the first interface to the stress test traffic of the first interface as the independent variable, it was found through experiments that the distribution of the traffic simulation evaluation values ​​of each target interface was still uneven, with a large number of scores ranging from 95 to 100 points. Analysis found that there were some outliers with too large ratios, which affected the overall distribution. Therefore, these outliers with too large ratios can be filtered out first, and then the traffic simulation evaluation value of the target interface can be calculated, which further improves the reliability of the simulation evaluation.

[0172] Step A2: the server performs statistical processing on the traffic simulation degree evaluation value of each target interface according to the interface weight of each target interface to obtain the interface traffic simulation degree evaluation value.

[0173] In the disclosed embodiment, each target interface has a corresponding interface weight according to its interface level. The server performs statistical processing on the traffic simulation degree evaluation value of each target interface according to the interface weight of each target interface. For example, it may be a weighted summation processing to obtain the interface traffic simulation degree evaluation value.

[0174] Exemplarily, referring to Formula 2, the server can use Formula 2 to calculate the interface traffic simulation degree evaluation value to implement the process of step A2:

[0175]

[0176] in, The i in represents the i-th target interface, ω i is the interface weight of target interface i.

[0177] In this way, using Formula 2, the server can perform weighted summation on the traffic simulation degree evaluation value of each target interface according to the interface weight of each target interface to obtain the interface traffic simulation degree evaluation value.

[0178] On the other hand, illustratively, the server may perform the following steps B1 and B2 to implement a process of obtaining an interface coverage evaluation value of a target business system according to stress testing traffic:

[0179] Step B1, for each target interface in the target business system, the server obtains the target interface coverage value of the target interface according to the stress testing traffic, wherein if the stress testing traffic indicates that the target interface is covered by the traffic during the full-link stress testing process, the target interface coverage value is a first value; if the stress testing traffic indicates that the target interface is not covered by the traffic during the full-link stress testing process, the target interface coverage value is a second value, and the first value and the second value are different.

[0180] The stress test traffic characterizes that the target interface is covered by the traffic during the full-link stress test process, that is, the stress test traffic includes the interface stress test traffic of the target interface. If the stress test traffic includes the interface stress test traffic of the target interface, the target interface coverage value is a first value, such as 1 or 100, and if the stress test traffic does not include the interface stress test traffic of the target interface, the target interface coverage value is a second value, such as 0.

[0181] Step B2: The server performs statistical processing on the target interface coverage value of each target interface according to the interface weight of each target interface to obtain an interface coverage evaluation value.

[0182] Exemplarily, referring to Formula 3, the server may calculate the interface coverage evaluation value using Formula 3 to implement the process of step B3:

[0183]

[0184] Among them, x ci That is, it is the target interface coverage value of the i-th target interface. The meanings of the remaining parameters in Formula 3 refer to the relevant description of the above embodiment and will not be repeated here.

[0185] In step 402, the server performs mathematical operations on the interface traffic simulation evaluation value and the interface coverage evaluation value to obtain a stress test simulation evaluation value corresponding to the interface coverage evaluation dimension.

[0186] After the server obtains the interface traffic simulation evaluation value and the interface coverage evaluation value according to the above implementation mode, the interface traffic simulation evaluation value and the interface coverage evaluation value are subjected to mathematical operation processing, which may be a dot product processing, to obtain the stress test simulation evaluation value corresponding to the interface coverage evaluation dimension.

[0187] Referring to Formula 4, the server can use Formula 4 to calculate the stress test simulation evaluation value E1 corresponding to the interface coverage evaluation dimension to implement the process of step 402:

[0188]

[0189] The meanings of the relevant parameters in Formula 4 refer to the relevant description of the above embodiment and will not be repeated here.

[0190] It should be noted that in the process of calculating the simulation evaluation value of the target interface's traffic, some interfaces have very high ratios under normal pressure because of too small traffic. The excessively high ratios affect the calculation results of Formula 1, resulting in a large final stress test simulation evaluation value even if other indicators are poorly achieved, which will affect the reliability of the stress test simulation evaluation.

[0191] In view of this, the embodiment of the present disclosure performs traffic filtering on the traffic data of the interface with too small traffic (the threshold of the traffic size used in filtering can be set by itself during implementation), thereby effectively avoiding the above problems.

[0192] For example, see Figure 7 and Figure 8 , Figure 7 This is a distribution diagram of the number of target interfaces corresponding to each stress test simulation evaluation value before traffic filtering. Figure 8 This is a schematic diagram of the score distribution of the stress test simulation evaluation value of each target interface before traffic filtering.

[0193] from Figure 7 and Figure 8 It can be seen that before traffic filtering, the scores of the stress test simulation evaluation values ​​of a large number of target interfaces are distributed around the full score of 100.

[0194] For example, see Fig. 9 and Fig.10 , Fig. 9 This is a distribution diagram of the number of target interfaces corresponding to each stress test simulation evaluation value after traffic filtering. Fig.10 The following is a schematic diagram showing the distribution of the stress test simulation evaluation values ​​of each target interface after traffic filtering. Fig. 9 and Fig.10 It can be seen that the score distribution of the stress testing simulation evaluation values ​​of each target interface has been significantly improved.

[0195] Based on the above embodiment, after the server obtains the stress testing simulation evaluation value corresponding to the interface coverage evaluation dimension in the above manner, it can also display the first display page; wherein the first display page includes the first evaluation entry of each target interface.

[0196] For example, see Fig.11 , Fig.11 FIG. 1 is a schematic diagram of an exemplary first display page. Fig.11 In the example, each row represents a first evaluation item, and the first evaluation item includes an interface identifier of the target interface (the interface identifier may be an interface name of the target interface, or may be represented by relevant parameters of the target interface, such as Fig.11 "Domain Name", "PATH" path and "Interface Level" shown in the figure), the first interface stress test flow of the target interface (such as Fig.11 The "stress test flow" shown in the figure), the first interface predicted flow of the target interface (such as Fig.11 The total traffic shown in the figure), the interface weight of the target interface (such as Fig.11 The "weight" shown in the figure), the traffic of the target interface reaches the simulation degree evaluation value (such as Fig.11 The "score" shown in the figure) and the target interface coverage value of the target interface (such as Fig.11 The “coverage situation” shown in the figure may also include the coverage situation of the target interface, etc. The specific display content in the first display page is not limited here.

[0197] Furthermore, the server can also display graphs of stress test traffic and predicted traffic simultaneously, see Fig.12 , Fig.12 It is a curve chart of stress test flow and predicted flow, which helps evaluators to intuitively understand the results of stress test simulation evaluation by combining the curve chart of stress test flow and predicted flow.

[0198] 1) The simulation evaluation dimension includes the module coverage evaluation dimension;

[0199] See also Fig.13 In the case where the simulation evaluation dimension includes the module coverage evaluation dimension, step 302 may include Fig.13 Step 1301 and step 1302 are shown:

[0200] Step 1301: For each business module in the target business system, the server obtains module fitness data corresponding to the business module according to the predicted traffic and the stress test traffic.

[0201] The module fit data represents the traffic fit of the business module and the resource consumption fit of the business module in the predicted traffic and stress test traffic.

[0202] For the module coverage evaluation dimension, the main focus is on analyzing whether the changes in module interface traffic in the predicted traffic and stress test traffic are consistent (i.e., traffic fit), and whether the changes in the total number of CPU cores consumed are consistent (i.e., resource consumption fit). Here, resource consumption mainly considers the number of CPUs used.

[0203] In a possible implementation of step 1301, the server may execute the following steps C1, C2, and C3 to implement step 1301:

[0204] In step C1, the server obtains the interface fitness value corresponding to each business interface in the business module and the resource consumption fitness value corresponding to the business module according to the predicted traffic and the stress test traffic.

[0205] Exemplarily, step C1 may include: the server determines the predicted resource consumption quantity of the business module from the predicted traffic, determines the resource consumption stress test quantity of the business module from the stress test traffic, and obtains the resource consumption fit value corresponding to the business module based on the predicted resource consumption quantity and the resource consumption stress test quantity; for each business interface in the business module, the server determines the second interface predicted traffic of the business interface from the predicted traffic, determines the second interface stress test traffic of the business interface from the stress test traffic, and obtains the interface fit value corresponding to the business interface based on the second interface predicted traffic and the second interface stress test traffic.

[0206] The following describes the process of the server obtaining the interface fitness value and resource consumption fitness value respectively:

[0207] 1) The server obtains the interface fitness value:

[0208] The process of the server obtaining the interface fitting value corresponding to the business interface based on the second interface predicted traffic and the second interface stress test traffic includes: the server calculates the first fitting coefficient of the business interface based on the second interface predicted traffic and the second interface stress test traffic, and compares the first fitting coefficient with a preset coefficient threshold; if the first fitting coefficient is greater than the coefficient threshold, the server calculates the interface fitting value corresponding to the business interface based on the second interface predicted traffic, the second interface stress test traffic and the first fitting formula; if the first fitting coefficient is less than or equal to the coefficient threshold, the server calculates the interface fitting value corresponding to the business interface based on the second interface predicted traffic, the second interface stress test traffic and the second fitting formula, and the first fitting formula is different from the second fitting formula.

[0209] Exemplarily, the server may calculate the first fitting coefficient of the service interface using Formula 5:

[0210]

[0211] Among them, R 2 is the first fitting coefficient, y i is the stress test flow of the second interface of the i-th business interface. is the predicted traffic of the second interface of the i-th service interface, and y is the average stress test traffic of the second interface of each service interface; where R 2 The result value is [-∞, 1], which is mapped to [0-100]. That is, if the first fitting coefficient is less than 0, it will be directly defaulted to 0.

[0212] The server compares the first fitting coefficient with a preset coefficient threshold, such as 0.7. If the first fitting coefficient is greater than the coefficient threshold, the server calculates the interface fitting value corresponding to the business interface based on the second interface predicted traffic, the second interface stress test traffic, and the first fitting formula. The first fitting formula may be, for example, an MSE variant calculation formula. See Formula 6 for the first fitting formula:

[0213] r = 1- 1 / n∑((Yy) / y)2 Formula 6

[0214] Among them, r is the interface fitting value, Y is the predicted flow of the second interface, and y is the pressure measurement flow of the second interface.

[0215] If the first fitting coefficient is less than or equal to the coefficient threshold, the server calculates the interface fitting value corresponding to the business interface based on the second interface predicted traffic, the second interface stress test traffic and the second fitting formula.

[0216] The second goodness of fit formula is shown in Formula 7:

[0217] r=0.5*abs(PreFlowAver-FlowAver) / FlowAver+0.5*abs(PreFlowMax-FlowMax) / FlowMax Formula 7

[0218] Among them, PreFlowAver is the average value of the second interface stress test flow of each business interface, FlowAver is the average value of the second interface predicted flow of each business interface, PreFlowMax is the second interface stress test flow, and FlowMax is the second interface predicted flow.

[0219] In this way, the server uses Formula 6 or Formula 7 to calculate the interface fit value r corresponding to each business interface.

[0220] 2) The server obtains the resource consumption fit value:

[0221] The server obtains the resource consumption fitting value corresponding to the business module according to the resource consumption prediction quantity and the resource consumption stress test quantity, including: the server calculates the second fitting coefficient of the business module according to the resource consumption prediction quantity and the resource consumption stress test quantity, and compares the second fitting coefficient with a preset coefficient threshold; if the second fitting coefficient is greater than the coefficient threshold, the server calculates the resource consumption fitting value corresponding to the business module based on the resource consumption prediction quantity, the resource consumption stress test quantity and the first fitting formula; if the second fitting coefficient is less than or equal to the coefficient threshold, the server calculates the resource consumption fitting value corresponding to the business module based on the resource consumption prediction quantity, the resource consumption stress test quantity and the second fitting formula, and the first fitting formula is different from the second fitting formula.

[0222] The process of the server obtaining the resource consumption fitness value is similar to the process of the server obtaining the interface fitness value, which will not be repeated here.

[0223] Step C2: the server performs statistical processing on the interface fitness values ​​corresponding to each service interface according to the interface weight of each service interface, and obtains the flow fitness values ​​corresponding to the service module.

[0224] It can be understood that in the above step C1, the resource consumption fit value obtained by the server is the resource consumption fit value based on the business module, and the interface fit value is the interface fit value of each business interface in the business module. The object of analysis under the module coverage evaluation dimension is the business module, so it is necessary to calculate the traffic fit value of each business module.

[0225] Exemplarily, referring to Formula 8, the server may use Formula 8 to implement the process of step C2 for each business module:

[0226]

[0227] Among them, x ci is the interface fitting value corresponding to each business interface in the business module, ω i The interface weight of each service interface.

[0228] In step C3, the server performs mathematical operations on the resource consumption fitness value and the traffic fitness value to obtain module fitness data corresponding to the business module.

[0229] The mathematical operation processing may be to add the resource consumption fitness value and the traffic fitness value and then divide by 2, see Formula 9. The server may calculate the module fitness data corresponding to the service module using Formula 9:

[0230]

[0231] in, is the flow fitting value corresponding to business module i, x cpui is the resource consumption fitness value corresponding to the business module i. The result calculated by formula 9 is the module fitness corresponding to the business module i.

[0232] In step 1302, the server performs statistical processing on the fitting data of each module according to the module weight of each business module to obtain a stress test simulation evaluation value corresponding to the module coverage evaluation dimension.

[0233]

[0234] See Formula 10, where ω nsi is the module weight of each business module. The server performs weighted summation on the fitting data of each module according to the module weight of each business module to obtain the stress test simulation evaluation value E2 corresponding to the module coverage evaluation dimension.

[0235] Based on the above embodiment, after the server obtains the stress testing simulation evaluation value corresponding to the module coverage evaluation dimension in the above manner, it can also display a second display page, which includes second evaluation items of each business module.

[0236] See also Fig.14 , Fig.14 FIG. 1 is a schematic diagram of an exemplary second display page. Fig.14 In , each row represents a second evaluation item, and the second evaluation item includes a module identifier of a business module (such as Fig.14 The "NS" shown in the figure), the traffic fit value of the business module (such as Fig.14 The "traffic fit" shown in the figure), the resource consumption fit value of the business module (such as Fig.14 The "CPU consumption fit" shown in the figure) and the stress test simulation evaluation results of the business module (such as Fig.14 The stress test simulation evaluation result of the business module is calculated based on the traffic fit value of the business module and the resource consumption fit value of the business module.

[0237] Furthermore, the server can also display the resource consumption forecast and resource consumption stress test graphs simultaneously, see Fig.15 , Fig.15 It is a curve chart of the predicted resource consumption quantity and the resource consumption stress test quantity, so that the evaluator can intuitively understand the results of the stress test simulation evaluation by combining the curve chart of the predicted resource consumption quantity and the resource consumption stress test quantity.

[0238] exist Figure 2On the basis of the illustrated embodiment, the following introduces the process of obtaining the stress test simulation evaluation value corresponding to the simulation evaluation dimension according to the stress test traffic by the server, based on the link coverage evaluation dimension, the scenario coverage evaluation dimension, and the category coverage evaluation dimension.

[0239] 1) The simulation evaluation dimension includes the link coverage evaluation dimension:

[0240] See also Fig.16 The process of the server obtaining the stress test simulation evaluation value corresponding to the simulation evaluation dimension according to the stress test traffic includes: Fig.16 Step 1601 and step 1602 are shown:

[0241] Step 1601: The server obtains the link coverage evaluation value of each service link in the target service system according to the stress test traffic.

[0242] In a possible implementation of step 1601, for each business link, the server can obtain the link interface coverage value of each link interface in the business link based on the stress test traffic. Then, the server performs statistical processing on the link interface coverage value of each link interface according to the interface weight of each link interface to obtain the link coverage evaluation value of the business link.

[0243] Referring to Formula 11, the server can perform weighted summation of the link interface coverage values ​​of each link interface through Formula 11 to obtain a link coverage evaluation value of a service link:

[0244]

[0245] Among them, x ci That is, the link interface coverage value of each link interface in a service link, ω i The interface weight of the link interface.

[0246] The method for obtaining the link interface coverage value of the link interface is similar to the method for obtaining the target interface coverage value of the target interface in the above embodiment, and will not be repeated here.

[0247] In step 1602, the server performs statistical processing on the link coverage evaluation value of each service link according to the link weight of each service link to obtain a stress test simulation evaluation value of the link coverage evaluation dimension.

[0248]

[0249] E3 is the stress test simulation evaluation value of the link coverage evaluation dimension, ω nsi is the link weight, and the other parameters in Formula 12 refer to the relevant description above and will not be repeated here.

[0250] In this way, through the above implementation method, the server calculates the stress test simulation evaluation value of the link coverage evaluation dimension.

[0251] It should be noted that in the process of link coverage evaluation dimension analysis, the business link includes the main link and sub-links. The main link may have multiple sub-links. Considering that there is no traffic or very low traffic on some link interface lines, filtering rules are designed. The size of the main link inlet traffic is used as a reference to set custom filtering rules. For example, less than 1% or 2% of the main link inlet traffic is used as a filtering condition. When the traffic of a link interface is less than the filtering condition, the traffic of the link interface is filtered out in the process of calculating the stress test simulation evaluation value of the link coverage evaluation dimension.

[0252] In addition, considering that the above filtering conditions will be raised when the main link inlet traffic is relatively large, the filtering conditions can also be set to a maximum of 100qps. When the traffic of a link interface exceeds 100, it will be uniformly processed as 100.

[0253] For example, see Fig.17 and Fig.18 , Fig.17 is the link coverage evaluation value of each service link when the above filtering rules are not adopted (such as Fig.17 The schematic diagram of the “score” score shown in Fig.18 After the traffic is filtered using the above filtering rules, the link coverage evaluation value of each service link (such as Fig.18 Schematic diagram of the “score” score shown).

[0254] Based on the above embodiment, after the server obtains the stress testing simulation evaluation value of the link coverage evaluation dimension in the above manner, it can also display a third display page, which includes third evaluation items of each link interface.

[0255] See also Fig.19 , Fig.19 is a schematic diagram of an exemplary third display page. Fig.19 In the example, each row represents a third evaluation entry, and the third evaluation entry at least includes the interface identifier of the link interface (such as Fig.19 Source USN) and the link interface override value of the link interface (such as Fig.19 "Score" shown).

[0256] 2) The simulation evaluation dimension includes the scene coverage evaluation dimension:

[0257] See also Fig. 20 The process of the server obtaining the stress test simulation evaluation value corresponding to the simulation evaluation dimension according to the stress test traffic includes: Fig. 20Step 2001 and step 2002 are shown:

[0258] Step 2001: The server determines the scenario stress testing traffic for each business scenario corresponding to the target business system based on the stress testing traffic.

[0259] In step 2002, the server performs statistical processing on the stress test traffic of each scenario according to the scenario weight of each business scenario to obtain a stress test simulation evaluation value of the scenario coverage evaluation dimension, wherein the scenario weight is the ratio of the scenario stress test traffic to the stress test traffic.

[0260] Referring to Formula 13, the server can use Formula 13 to calculate the stress test simulation evaluation value E4 of the scenario coverage evaluation dimension:

[0261]

[0262] Among them, f i is the stress test flow of business scenario i, ω i The larger the scenario stress test traffic, the larger the scenario weight.

[0263] 3) The simulation evaluation dimension includes the category coverage evaluation dimension:

[0264] See also Fig.21 The process of the server obtaining the stress test simulation evaluation value corresponding to the simulation evaluation dimension according to the stress test traffic includes: Fig.21 Step 2101 and step 2102 are shown:

[0265] In step 2101, the server determines the category stress testing flow of each business category corresponding to the target business system according to the stress testing flow.

[0266] In step 2102, the server performs statistical processing on the stress testing traffic of each category according to the category weight of each business category to obtain a stress testing simulation evaluation value of the category coverage evaluation dimension, wherein the category weight is the ratio of the category stress testing traffic to the stress testing traffic.

[0267] Among them, the implementation process of step 2101 and step 2102 can refer to the process of step 2001 and step 2002 above, and will not be repeated here.

[0268] In one embodiment, see Fig. 22 , Fig. 22 This is a schematic diagram of an exemplary fifth display interface of the fourth display page.

[0269] The server may also display a fourth display page, which includes fourth evaluation items for each business scenario, and the fourth evaluation items include at least a scenario identifier of the business scenario (such as Fig. 22The "scene" shown in the figure) and the stress test simulation evaluation results of the business scenario (such as Fig. 22 The "score" score in the fourth display page shown in the figure) The stress test simulation evaluation result of the business scenario is determined based on the scenario stress test traffic of the business scenario.

[0270] The server may also display a fifth display page, which includes fifth evaluation items of each business category, and the fifth evaluation items at least include a category identifier of the business category (such as Fig. 22 The "category" shown in the figure) and the stress test simulation evaluation results of the business category (such as Fig. 22 The "score" score in the fifth display page shown in the figure) The stress test simulation evaluation result of the business category is determined based on the category stress test traffic of the business category.

[0271] It should be understood that, although the various steps in the above flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above flowchart may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0272] In one embodiment, Fig.23 As shown, a stress test simulation evaluation device is provided, comprising:

[0273] An acquisition module 2301 is used to acquire the stress test traffic corresponding to the target business system, where the stress test traffic is obtained by performing a full-link stress test on the target business system;

[0274] The determination module 2302 is used to obtain each simulation evaluation dimension, and obtain the stress testing simulation evaluation value corresponding to the simulation evaluation dimension according to the stress testing flow;

[0275] The evaluation module 2303 is used to perform statistical processing on the stress testing simulation evaluation values ​​corresponding to each of the simulation evaluation dimensions to obtain a stress testing simulation evaluation result corresponding to the target business system.

[0276] In one embodiment, the simulation degree evaluation dimension includes at least two of an interface coverage evaluation dimension, a module coverage evaluation dimension, a link coverage evaluation dimension, a scenario coverage evaluation dimension, and a category coverage evaluation dimension.

[0277] In one of the embodiments, the simulation degree evaluation dimension includes the interface coverage evaluation dimension and the module coverage evaluation dimension, and the determination module 2302 is specifically used to obtain the predicted traffic corresponding to the target business system, and the predicted traffic is obtained by predicting the actual online traffic of the target business system within a preset time period; based on the predicted traffic and the stress test traffic, the stress test simulation degree evaluation value corresponding to the simulation degree evaluation dimension is obtained.

[0278] In one of the embodiments, the simulation degree evaluation dimension includes the interface coverage evaluation dimension, and the determination module 2302 is specifically used to obtain the interface traffic simulation degree evaluation value of the target business system based on the predicted traffic and the stress test traffic, and obtain the interface coverage evaluation value of the target business system based on the stress test traffic; perform mathematical operations on the interface traffic simulation degree evaluation value and the interface coverage evaluation value to obtain the stress test simulation degree evaluation value corresponding to the interface coverage evaluation dimension.

[0279] In one of the embodiments, the determination module 2302 is specifically used to, for each target interface in the target business system, substitute the ratio of the first interface predicted traffic to the first interface stress test traffic of the target interface as an independent variable into the interface traffic achievement calculation formula to obtain the traffic achievement simulation degree evaluation value of the target interface; and perform statistical processing on the traffic achievement simulation degree evaluation value of each target interface according to the interface weight of each target interface to obtain the interface traffic achievement simulation degree evaluation value.

[0280] In one of the embodiments, the determination module 2302 is specifically used to determine the ratio of the predicted traffic of the first interface to the stress test traffic of the first interface based on the predicted traffic and the stress test traffic; if the ratio is within a preset ratio interval, the inverse of the ratio is substituted as the independent variable into the interface traffic achievement calculation formula to obtain the traffic achievement simulation degree evaluation value of the target interface; if the ratio is not within the preset ratio interval, the ratio is substituted as the independent variable into the interface traffic achievement calculation formula to obtain the traffic achievement simulation degree evaluation value of the target interface.

[0281] In one of the embodiments, the determination module 2302 is specifically used to obtain, for each target interface in the target business system, a target interface coverage value of the target interface according to the stress testing traffic, wherein if the stress testing traffic indicates that the target interface is covered by traffic during the full-link stress testing process, the target interface coverage value is a first value; if the stress testing traffic indicates that the target interface is not covered by traffic during the full-link stress testing process, the target interface coverage value is a second value, and the first value and the second value are different; according to the interface weight of each target interface, the target interface coverage value of each target interface is statistically processed to obtain the interface coverage evaluation value.

[0282] In one embodiment, the device further comprises:

[0283] A display module, used to display the first display page;

[0284] Among them, the first display page includes the first evaluation entry of each target interface, and the first evaluation entry includes the interface identifier of the target interface, the first interface stress test flow of the target interface, the first interface predicted flow of the target interface, the interface weight of the target interface, the flow achievement simulation degree evaluation value of the target interface and the target interface coverage value of the target interface.

[0285] In one of the embodiments, the simulation degree evaluation dimension includes a module coverage evaluation dimension, and the determination module 2302 is specifically used to obtain the module fitting data corresponding to the business module for each business module in the target business system according to the predicted traffic and the stress test traffic; and perform statistical processing on the module fitting data of each business module according to the module weight of each business module to obtain the stress test simulation degree evaluation value corresponding to the module coverage evaluation dimension.

[0286] In one of the embodiments, the determination module 2302 is specifically used to obtain the interface fitness value corresponding to each business interface in the business module and the resource consumption fitness value corresponding to the business module based on the predicted traffic and the stress test traffic; perform statistical processing on the interface fitness value corresponding to each business interface according to the interface weight of each business interface to obtain the traffic fitness value corresponding to the business module; perform mathematical operations on the resource consumption fitness value and the traffic fitness value to obtain the module fitness data corresponding to the business module.

[0287] In one embodiment, the determination module 2302 is specifically used to determine the predicted resource consumption quantity of the business module from the predicted traffic, determine the resource consumption stress test quantity of the business module from the stress test traffic, and obtain the resource consumption fit value corresponding to the business module based on the resource consumption predicted quantity and the resource consumption stress test quantity; for each of the business interfaces in the business module, determine the second interface predicted traffic of the business interface from the predicted traffic, determine the second interface stress test traffic of the business interface from the stress test traffic, and obtain the interface fit value corresponding to the business interface based on the second interface predicted traffic and the second interface stress test traffic.

[0288] In one embodiment, the determination module 2302 is specifically used to calculate the first fitting coefficient of the business interface based on the predicted traffic of the second interface and the stress test traffic of the second interface, and compare the first fitting coefficient with a preset coefficient threshold; if the first fitting coefficient is greater than the coefficient threshold, the interface fitting value corresponding to the business interface is calculated based on the predicted traffic of the second interface, the stress test traffic of the second interface and the first fitting formula; if the first fitting coefficient is less than or equal to the coefficient threshold, the interface fitting value corresponding to the business interface is calculated based on the predicted traffic of the second interface, the stress test traffic of the second interface and the second fitting formula, and the first fitting formula is different from the second fitting formula.

[0289] In one embodiment, the determination module 2302 is specifically used to calculate the second fitting coefficient of the business module based on the predicted resource consumption quantity and the resource consumption stress test quantity, and compare the second fitting coefficient with a preset coefficient threshold; if the second fitting coefficient is greater than the coefficient threshold, then the resource consumption fitting value corresponding to the business module is calculated based on the predicted resource consumption quantity, the resource consumption stress test quantity and the first fitting formula; if the second fitting coefficient is less than or equal to the coefficient threshold, then the resource consumption fitting value corresponding to the business module is calculated based on the predicted resource consumption quantity, the resource consumption stress test quantity and the second fitting formula, and the first fitting formula is different from the second fitting formula.

[0290] In one embodiment, the display module is also used to display a second display page; wherein the second display page includes second evaluation items for each of the business modules, the second evaluation items include a module identifier of the business module, a traffic fit value of the business module, a resource consumption fit value of the business module, and a stress test simulation evaluation result of the business module, and the stress test simulation evaluation result of the business module is calculated based on the traffic fit value of the business module and the resource consumption fit value of the business module.

[0291] In one of the embodiments, the simulation degree evaluation dimension includes a link coverage evaluation dimension, and the determination module 2302 is specifically used to obtain the link coverage evaluation value of each business link in the target business system according to the stress test traffic; and perform statistical processing on the link coverage evaluation value of each business link according to the link weight of each business link to obtain the stress test simulation degree evaluation value of the link coverage evaluation dimension.

[0292] In one of the embodiments, the determination module 2302 is specifically used to obtain, for each of the business links, the link interface coverage value of each link interface in the business link according to the stress test traffic; and to perform statistical processing on the link interface coverage value of each link interface according to the interface weight of each link interface to obtain the link coverage evaluation value of the business link.

[0293] In one embodiment, the display module is also used to display a third display page; wherein the third display page includes a third evaluation entry for each of the link interfaces in each of the service links, and the third evaluation entry includes at least an interface identifier of the link interface and a link interface coverage value of the link interface.

[0294] In one of the embodiments, the simulation degree evaluation dimension includes a scenario coverage evaluation dimension, and the determination module 2302 is specifically used to determine the scenario stress testing flow of each business scenario corresponding to the target business system according to the stress testing flow; according to the scenario weight of each business scenario, the scenario stress testing flow is statistically processed to obtain the stress testing simulation degree evaluation value of the scenario coverage evaluation dimension, wherein the scenario weight is the ratio of the scenario stress testing flow to the stress testing flow.

[0295] In one of the embodiments, the display module is further used to display a fourth display page;

[0296] Among them, the fourth display page includes fourth evaluation items for each of the business scenarios, and the fourth evaluation items at least include the scenario identifier of the business scenario and the stress testing simulation evaluation result of the business scenario, and the stress testing simulation evaluation result of the business scenario is determined based on the scenario stress testing traffic of the business scenario.

[0297] In one of the embodiments, the simulation degree evaluation dimension includes a category coverage evaluation dimension, and the determination module 2302 is specifically used to determine the category stress testing flow of each business category corresponding to the target business system according to the stress testing flow; according to the category weight of each business category, statistical processing is performed on each category stress testing flow to obtain the stress testing simulation degree evaluation value of the category coverage evaluation dimension, wherein the category weight is the ratio of the category stress testing flow to the stress testing flow.

[0298] In one of the embodiments, the display module is further used to display a fifth display page;

[0299] Among them, the fifth display page includes the fifth evaluation item of each of the business categories, and the fifth evaluation item at least includes the category identification of the business category and the stress testing simulation evaluation result of the business category, and the stress testing simulation evaluation result of the business category is determined based on the category stress testing traffic of the business category.

[0300] For the specific limitations of the stress testing simulation evaluation device, please refer to the limitations of the stress testing simulation evaluation method above, which will not be repeated here. Each module in the above-mentioned stress testing simulation evaluation device can be implemented in whole or in part through software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the server in the form of hardware, or can be stored in the memory in the server in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0301] Fig.24 FIG. 1 is a block diagram of a server 1400 according to an exemplary embodiment. Fig.24 The server 1400 includes a processing component 1420, which further includes one or more processors, and a memory resource represented by a memory 1422 for storing instructions or computer programs, such as applications, that can be executed by the processing component 1420. The application stored in the memory 1422 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1420 is configured to execute instructions to perform the above-mentioned stress testing simulation evaluation method.

[0302] The server 1400 may also include a power supply component 1424 configured to perform power management of the device 1400, a wired or wireless network interface 1426 configured to connect the device 1400 to a network, and an input / output (I / O) interface 1428. The server 1400 may operate based on an operating system stored in the memory 1422, such as Windows 14 14erverTM, Mac O14 XTM, UnixTM, LinuxTM, FreeB14DTM or the like.

[0303] In an exemplary embodiment, a storage medium including instructions is also provided, such as a memory 1422 including instructions, which can be executed by a processor of the server 1400 to perform the above method. The storage medium can be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0304] In an exemplary embodiment, a computer program product is also provided, and when the computer program is executed by a processor, the above method can be implemented. The computer program product includes one or more computer instructions. When these computer instructions are loaded and executed on a computer, part or all of the above method can be implemented in whole or in part according to the process or function described in the embodiment of the present disclosure.

[0305] It should be noted that the solutions described in this specification and the embodiments, if they involve the processing of personal information, will be processed on the premise of having a legal basis (such as obtaining the consent of the subject of personal information, or being necessary for the performance of a contract, etc.), and will only be processed within the scope of regulations or agreements. If a user refuses to process personal information other than the necessary information for basic functions, it will not affect the user's use of basic functions.

[0306] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the embodiments of the present disclosure can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0307] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0308] The above-described embodiments only express several implementation methods of the embodiments of the present disclosure, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for those of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the embodiments of the present disclosure, and these all belong to the protection scope of the embodiments of the present disclosure. Therefore, the protection scope of the patent of the embodiments of the present disclosure shall be subject to the attached claims.

Claims

1. A stress test simulation evaluation method, characterized in that: The method comprises: Obtain the stress test traffic corresponding to the target business system, where the stress test traffic is obtained by performing full-link stress testing on the target business system; Obtain each simulation degree evaluation dimension, and obtain the stress testing simulation degree evaluation value corresponding to the simulation degree evaluation dimension according to the stress testing flow; The stress testing simulation degree evaluation values ​​corresponding to each of the simulation degree evaluation dimensions are statistically processed to obtain the stress testing simulation degree evaluation results corresponding to the target business system.

2. The method according to claim 1, characterized in that The simulation degree evaluation dimensions include at least two of the interface coverage evaluation dimensions, module coverage evaluation dimensions, link coverage evaluation dimensions, scenario coverage evaluation dimensions and category coverage evaluation dimensions.

3. The method according to claim 2, characterized in that The simulation degree evaluation dimension includes the interface coverage evaluation dimension and the module coverage evaluation dimension, and obtaining the stress testing simulation degree evaluation value corresponding to the simulation degree evaluation dimension according to the stress testing flow includes: Obtaining the predicted traffic corresponding to the target business system, where the predicted traffic is obtained by predicting the real online traffic of the target business system within a preset time period; According to the predicted flow and the stress test flow, a stress test simulation degree evaluation value corresponding to the simulation degree evaluation dimension is obtained.

4. The method according to claim 3, characterized in that The simulation evaluation dimension includes the interface coverage evaluation dimension, and obtaining the stress test simulation evaluation value corresponding to the simulation evaluation dimension according to the predicted traffic and the stress test traffic includes: According to the predicted traffic and the stress test traffic, obtaining a simulation degree evaluation value of the interface traffic of the target business system, and according to the stress test traffic, obtaining an interface coverage evaluation value of the target business system; The interface traffic achievement simulation degree evaluation value and the interface coverage evaluation value are subjected to mathematical operation to obtain a stress test simulation degree evaluation value corresponding to the interface coverage evaluation dimension.

5. The method according to claim 4, characterized in that The obtaining, according to the predicted traffic and the stress test traffic, a simulation degree evaluation value of the interface traffic of the target business system includes: For each target interface in the target business system, based on the predicted traffic and the stress test traffic, a ratio of the first interface predicted traffic of the target interface to the first interface stress test traffic is substituted as an independent variable into the interface traffic achievement calculation formula to obtain a traffic achievement simulation degree evaluation value of the target interface; According to the interface weight of each of the target interfaces, the traffic simulation degree evaluation value of each of the target interfaces is statistically processed to obtain the interface traffic simulation degree evaluation value.

6. The method according to claim 5, characterized in that The method of substituting the ratio of the first interface predicted flow rate of the target interface to the first interface stress measured flow rate as an independent variable into the interface flow achievement calculation formula based on the predicted flow rate and the stress measured flow rate to obtain the flow achievement simulation degree evaluation value of the target interface includes: Determine, according to the predicted flow and the stress test flow, a ratio of the predicted flow of the first interface to the stress test flow of the first interface; If the ratio is within the preset ratio interval, the reciprocal of the ratio is substituted as the independent variable into the interface flow achievement calculation formula to obtain the flow achievement simulation degree evaluation value of the target interface; If the ratio is not within the preset ratio interval, the ratio is substituted as the independent variable into the interface flow achievement calculation formula to obtain the flow achievement simulation degree evaluation value of the target interface.

7. The method according to claim 5, characterized in that The obtaining the interface coverage evaluation value of the target business system according to the stress testing flow includes: For each target interface in the target business system, a target interface coverage value of the target interface is obtained according to the stress testing traffic, wherein if the stress testing traffic indicates that the target interface is covered by traffic during the full-link stress testing process, the target interface coverage value is a first value; if the stress testing traffic indicates that the target interface is not covered by traffic during the full-link stress testing process, the target interface coverage value is a second value, and the first value and the second value are different; According to the interface weight of each of the target interfaces, the target interface coverage value of each of the target interfaces is statistically processed to obtain the interface coverage evaluation value.

8. The method according to claim 7, characterized in that The method further comprises: Display the first display page; Among them, the first display page includes the first evaluation entry of each target interface, and the first evaluation entry includes the interface identifier of the target interface, the first interface stress test flow of the target interface, the first interface predicted flow of the target interface, the interface weight of the target interface, the flow achievement simulation degree evaluation value of the target interface and the target interface coverage value of the target interface.

9. The method according to claim 3, characterized in that: The simulation evaluation dimension includes a module coverage evaluation dimension, and obtaining a stress test simulation evaluation value corresponding to the simulation evaluation dimension according to the predicted flow and the stress test flow includes: For each business module in the target business system, obtaining module fitness data corresponding to the business module according to the predicted traffic and the stress test traffic; According to the module weight of each of the business modules, statistical processing is performed on the fitting data of each of the modules to obtain a stress testing simulation evaluation value corresponding to the module coverage evaluation dimension.

10. The method according to claim 9, characterized in that The obtaining module fitness data corresponding to the business module according to the predicted traffic and the stress test traffic includes: According to the predicted flow and the stress test flow, obtaining an interface fitness value corresponding to each business interface in the business module and a resource consumption fitness value corresponding to the business module; According to the interface weight of each of the service interfaces, statistical processing is performed on the interface fitness values ​​corresponding to each of the service interfaces to obtain the flow fitness values ​​corresponding to the service modules; The resource consumption fitness value and the traffic fitness value are subjected to mathematical operation processing to obtain the module fitness data corresponding to the business module.

11. The method according to claim 10, characterized in that The obtaining, according to the predicted traffic and the stress test traffic, an interface fitness value corresponding to each business interface in the business module and a resource consumption fitness value corresponding to the business module, comprises: Determine the predicted resource consumption quantity of the business module from the predicted traffic, determine the resource consumption stress test quantity of the business module from the stress test traffic, and obtain the resource consumption fitting degree value corresponding to the business module according to the predicted resource consumption quantity and the resource consumption stress test quantity; For each of the business interfaces in the business module, the second interface predicted traffic of the business interface is determined from the predicted traffic, the second interface stress test traffic of the business interface is determined from the stress test traffic, and the interface fitness value corresponding to the business interface is obtained based on the second interface predicted traffic and the second interface stress test traffic.

12. The method according to claim 11, characterized in that The obtaining the interface fitness value corresponding to the business interface according to the second interface predicted traffic and the second interface stress test traffic includes: Calculate a first fitting coefficient of the service interface according to the predicted traffic of the second interface and the stress test traffic of the second interface, and compare the first fitting coefficient with a preset coefficient threshold; If the first fitting coefficient is greater than the coefficient threshold, the interface fitting value corresponding to the business interface is calculated based on the second interface predicted traffic, the second interface stress test traffic and the first fitting formula; If the first fitting coefficient is less than or equal to the coefficient threshold, the interface fitting value corresponding to the business interface is calculated based on the predicted traffic of the second interface, the stress test traffic of the second interface and the second fitting formula, and the first fitting formula is different from the second fitting formula.

13. The method according to claim 11, characterized in that The obtaining the resource consumption fitting value corresponding to the business module according to the resource consumption prediction quantity and the resource consumption stress test quantity includes: Calculate a second fitting coefficient of the business module according to the predicted resource consumption quantity and the resource consumption stress test quantity, and compare the second fitting coefficient with a preset coefficient threshold; If the second fitting coefficient is greater than the coefficient threshold, the resource consumption fitting value corresponding to the business module is calculated based on the predicted resource consumption quantity, the resource consumption stress test quantity and the first fitting formula; If the second fitting coefficient is less than or equal to the coefficient threshold, the resource consumption fitting value corresponding to the business module is calculated based on the predicted resource consumption quantity, the resource consumption stress test quantity and the second fitting formula, and the first fitting formula is different from the second fitting formula.

14. The method according to any one of claims 10 to 13, characterized in that: The method further comprises: Display the second display page; Among them, the second display page includes second evaluation items of each of the business modules, and the second evaluation items include the module identification of the business module, the traffic fit value of the business module, the resource consumption fit value of the business module, and the stress testing simulation evaluation result of the business module. The stress testing simulation evaluation result of the business module is calculated based on the traffic fit value of the business module and the resource consumption fit value of the business module.

15. The method according to claim 2, characterized in that The simulation degree evaluation dimension includes a link coverage evaluation dimension, and obtaining a stress test simulation degree evaluation value corresponding to the simulation degree evaluation dimension according to the stress test traffic includes: According to the stress test flow, obtaining a link coverage evaluation value of each service link in the target service system; According to the link weight of each of the service links, the link coverage evaluation value of each of the service links is statistically processed to obtain a stress test simulation evaluation value of the link coverage evaluation dimension.

16. The method according to claim 15, characterized in that The obtaining, according to the stress test flow, a link coverage evaluation value of each service link in the target service system includes: For each of the service links, obtaining a link interface coverage value of each link interface in the service link according to the stress test flow; According to the interface weight of each link interface, the link interface coverage value of each link interface is statistically processed to obtain the link coverage evaluation value of the service link.

17. The method according to claim 15 or 16, characterized in that The method further comprises: Display the third display page; The third display page includes a third evaluation entry of each link interface in each service link, and the third evaluation entry includes at least an interface identifier of the link interface and a link interface coverage value of the link interface.

18. The method according to claim 2, characterized in that The simulation evaluation dimension includes a scenario coverage evaluation dimension, and obtaining a stress testing simulation evaluation value corresponding to the simulation evaluation dimension according to the stress testing flow includes: Determine the scenario stress testing flow of each business scenario corresponding to the target business system according to the stress testing flow; According to the scenario weight of each of the business scenarios, the stress testing traffic of each scenario is statistically processed to obtain a stress testing simulation degree evaluation value of the scenario coverage evaluation dimension, wherein the scenario weight is the ratio of the scenario stress testing traffic to the stress testing traffic.

19. The method according to claim 18, characterized in that The method further comprises: Display the fourth display page; Among them, the fourth display page includes fourth evaluation items for each of the business scenarios, and the fourth evaluation items at least include the scenario identifier of the business scenario and the stress testing simulation evaluation result of the business scenario, and the stress testing simulation evaluation result of the business scenario is determined based on the scenario stress testing traffic of the business scenario.

20. The method according to claim 2, characterized in that The simulation evaluation dimension includes a category coverage evaluation dimension, and obtaining a stress testing simulation evaluation value corresponding to the simulation evaluation dimension according to the stress testing flow includes: Determine the category stress testing flow of each business category corresponding to the target business system according to the stress testing flow; According to the category weight of each of the business categories, statistical processing is performed on the stress testing traffic of each category to obtain a stress testing simulation evaluation value of the category coverage evaluation dimension, wherein the category weight is the ratio of the category stress testing traffic to the stress testing traffic.

21. The method according to claim 20, characterized in that The method further comprises: Display the fifth display page; Among them, the fifth display page includes the fifth evaluation item of each of the business categories, and the fifth evaluation item at least includes the category identification of the business category and the stress testing simulation evaluation result of the business category, and the stress testing simulation evaluation result of the business category is determined based on the category stress testing traffic of the business category.

22. A stress test evaluation device, characterized in that: The device comprises: An acquisition module is used to acquire the stress test traffic corresponding to the target business system, where the stress test traffic is obtained by performing a full-link stress test on the target business system; A determination module is used to obtain each simulation evaluation dimension, and obtain a stress testing simulation evaluation value corresponding to the simulation evaluation dimension according to the stress testing flow; The evaluation module is used to perform statistical processing on the stress testing simulation evaluation values ​​corresponding to each simulation evaluation dimension to obtain the stress testing simulation evaluation results corresponding to the target business system.

23. A server comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 21 are implemented.

24. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 21 are implemented.

25. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 21 are implemented.

Images