Automated test processing method and device

Through automated test processing methods and devices, manual operations are reduced, the efficiency and effectiveness of stress testing in the financial field are improved, and the problems of large and low efficiency of testing work in the prior art are solved.

CN113656313BActive Publication Date: 2024-12-03工银科技有限公司 +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110957275.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-08-19
Publication Date
2024-12-03
Estimated Expiration
2041-08-19

AI Technical Summary

Technical Problem

The lack of automatic scheduling strategies in the financial field of existing stress testing technologies has resulted in large testing workloads and low testing efficiency, and cannot meet the needs of many performance scenarios and high performance indicator accuracy requirements.

Method used

It provides an automated test processing method and device, by receiving automatic test instructions, obtaining test parameters, performing multiple stress tests, generating performance indicator data, and generating stress test reports based on these data, reducing manual operation dependence and improving testing efficiency.

Benefits of technology

It reduces the testing workload, improves testing efficiency and effectiveness, and is suitable for financial business areas with multi-performance scenarios and high-performance indicator requirements.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113656313B_ABST
    Figure CN113656313B_ABST
Patent Text Reader

Abstract

The present disclosure provides an automated test processing method, which is applied to the field of financial technology, and the method includes: in response to a received automatic test instruction, obtaining M groups of test parameters, wherein the test parameters include test scenario parameters and test flow parameters for stress testing a test object, and M is an integer greater than 1; performing N stress tests on the test object according to each test parameter in the M groups of test parameters, and obtaining N groups of performance indicator data, where N is an integer not less than M; and generating a stress test report for the test object based on the N groups of performance indicator data. The present disclosure also provides an automated test processing device, an electronic device, and a computer storage medium.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of financial technology, and in particular to an automated testing processing method, system and device. Background Art

[0002] With the rapid development of computer technology, more and more diverse Internet technologies are being applied to the financial field. The financial field has higher standards for the security and real-time performance of business operations. Stress testing is one of the important means to ensure the security of financial business operations and improve the real-time performance of financial business operations.

[0003] In the process of implementing the technical solution disclosed in the present invention, the inventor found that in the stress testing process in the related art, the stress testing operation lacks an automatic scheduling strategy and relies on manual configuration of stress testing scenarios and execution of stress testing sampling. With the increase in performance scenarios in the financial field and the improvement in the accuracy requirements of performance indicators, the current stress testing technology has shown the defects of large testing workload and low testing efficiency. Summary of the invention

[0004] One aspect of the present disclosure provides an automated test processing method, comprising: in response to a received automatic test instruction, obtaining M groups of test parameters, wherein the test parameters include test scenario parameters and test flow parameters for performing a stress test on a test object, and M is an integer greater than 1; performing N stress tests on the test object according to each test parameter in the M groups of test parameters to obtain N groups of performance indicator data, where N is an integer not less than M; and generating a stress test report for the test object based on the N groups of performance indicator data.

[0005] Optionally, according to each test parameter in the M groups of test parameters, N stress tests are performed on the test object to obtain N groups of performance indicator data, including: according to the test scenario parameters in each of the test parameters, M test scenarios for stress testing the test object are determined; according to the test flow parameters in each of the test parameters, the test flow value associated with each of the M test scenarios is determined; according to the test flow value associated with each of the test scenarios, N stress tests are performed on the test object to obtain N groups of performance indicator data.

[0006] Optionally, determining M test scenarios for stress testing the test object based on the test scenario parameters in each of the test parameters includes: determining at least one operation event for stress testing the test object in each of the M test scenarios based on the test scenario parameters in each of the test parameters; determining the test flow value associated with each of the M test scenarios based on the test flow parameters in each of the test parameters includes: determining the concurrency number associated with each of the operation events in each of the test scenarios based on the test flow parameters in each of the test parameters.

[0007] Optionally, the test scenario parameters include single-scenario test parameters and multi-scenario test parameters; and determining at least one operation event for stress testing the test object in each of the M test scenarios based on the test scenario parameters in each of the test parameters includes: determining a single operation event for stress testing the test object in the corresponding test scenario based on the single-scenario test parameters in the test scenario parameters; and / or determining multiple operation events for stress testing the test object in the corresponding test scenario based on the multi-scenario test parameters in the test scenario parameters, wherein a corresponding concurrency ratio is preset for each operation event in the multiple operation events, and the concurrency ratio is determined based on real user behavior data in the business analysis system.

[0008] Optionally, according to the test flow value associated with each of the test scenarios, N stress tests are performed on the test object to obtain N groups of performance indicator data, including: generating an automatic test script for each of the test scenarios according to at least one operation event associated with each of the test scenarios and according to the concurrency number associated with each of the operation events; looping the automatic test script and listening to the preset performance indicators associated with the test object during each execution to obtain the N groups of performance indicator data.

[0009] Optionally, the automatic test script is executed in a loop, and preset performance indicators associated with the test object are listened to during each execution process to obtain the N groups of performance indicator data, including: the automatic test script is executed in a loop to transmit each of the operation events of the corresponding concurrent number to the execution center of the test object through a preset event response link to implement N test execution processes for the test object; and the preset performance indicators associated with the test object are listened to during each test execution process to obtain the N groups of performance indicator data.

[0010] Optionally, when the multi-scenario test parameters indicate multiple operation events in the corresponding test scenario, the concurrency ratio associated with any of the operation events is positively correlated to the test priority for the operation event, and when the received automatic test instruction contains test priority information, the method further includes: determining at least one operation event for stress testing the test object based on the test priority information and based on the concurrency ratio associated with each of the multiple operation events.

[0011] Optionally, the method further includes: determining a performance limit state of the test object based on performance indicator data associated with each of the test parameters; and adjusting a test flow parameter in the test parameters based on the performance limit state.

[0012] Another aspect of the present disclosure provides an automated test processing device, including: an acquisition module, used to acquire M groups of test parameters in response to a received automatic test instruction, wherein the test parameters include test scenario parameters and test flow parameters for performing a stress test on a test object, and M is an integer greater than 1; a first processing module, used to perform N stress tests on the test object according to each test parameter in the M groups of test parameters, and obtain N groups of performance indicator data, where N is an integer not less than M; and a second processing module, used to generate a stress test report for the test object based on the N groups of performance indicator data.

[0013] Optionally, the first processing module includes: a first processing sub-module, used to determine M test scenarios for stress testing the test object according to the test scenario parameters in each of the test parameters; a second processing sub-module, used to determine the test flow value associated with each of the M test scenarios according to the test flow parameters in each of the test parameters; and a third processing sub-module, used to perform N stress tests on the test object according to the test flow value associated with each of the test scenarios, to obtain N sets of performance indicator data.

[0014] Optionally, the first processing submodule includes: a first processing unit, used to determine at least one operation event for stress testing the test object in each of the M test scenarios based on the test scenario parameters in each of the test parameters; the second processing submodule includes: a second processing unit, used to determine the concurrency number associated with each of the operation events in each of the test scenarios based on the test traffic parameters in each of the test parameters.

[0015] Optionally, the test scenario parameters include single-scenario test parameters and multi-scenario test parameters; the first processing unit includes: a first processing sub-unit, used to determine a single operation event in a corresponding test scenario for stress testing the test object according to the single-scenario test parameters in the test scenario parameters; a second processing sub-unit, used to determine multiple operation events in a corresponding test scenario for stress testing the test object according to the multi-scenario test parameters in the test scenario parameters, wherein a corresponding concurrency ratio is preset for each operation event in the multiple operation events, and the concurrency ratio is determined based on real user behavior data in the business analysis system.

[0016] Optionally, the third processing submodule includes: a third processing unit, used to generate an automatic test script for each test scenario based on at least one operation event associated with each test scenario and the number of concurrency associated with each operation event; a fourth processing unit, used to loop execute the automatic test script and listen to preset performance indicators associated with the test object during each execution to obtain the N groups of performance indicator data.

[0017] Optionally, the fourth processing unit includes: a third processing sub-unit, used to loop execute the automatic test script so as to transmit each of the operation events of the corresponding concurrent number to the execution center of the test object through a preset event response link, so as to implement N test execution processes for the test object; and a fourth processing sub-unit, used to monitor preset performance indicators associated with the test object during each of the test execution processes to obtain the N groups of performance indicator data.

[0018] Optionally, when the multi-scenario test parameters indicate multiple operation events in the corresponding test scenario, the concurrency ratio associated with any of the operation events is positively correlated to the test priority for the operation event, and when the received automatic test instruction contains test priority information, the first processing unit is further used to: determine at least one operation event for stress testing the test object based on the test priority information and based on the concurrency ratio associated with each of the multiple operation events.

[0019] Optionally, the device also includes: a third processing module, used to determine the performance limit state of the test object based on performance indicator data associated with each of the test parameters; and used to adjust the test flow parameter in the test parameters according to the performance limit state.

[0020] Another aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method of an embodiment of the present disclosure.

[0021] Another aspect of the present disclosure provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the method of an embodiment of the present disclosure.

[0022] Another aspect of the present disclosure provides a computer program product, including a computer program, wherein when the computer program is executed by a processor, the method of the embodiment of the present disclosure is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] For a more complete understanding of the present disclosure and its advantages, reference will now be made to the following description taken in conjunction with the accompanying drawings, in which:

[0024] Figure 1 The system architecture of the automated test processing method and device according to the embodiment of the present disclosure is schematically shown;

[0025] Figure 2 A flowchart of an automated test processing method according to an embodiment of the present disclosure is schematically shown;

[0026] Figure 3 A flowchart of another automated test processing method according to an embodiment of the present disclosure is schematically shown;

[0027] Figure 4 A flowchart of another automated test processing method according to an embodiment of the present disclosure is schematically shown;

[0028] Figure 5 A block diagram schematically shows an automated test processing device according to an embodiment of the present disclosure;

[0029] Figure 6 A block diagram of an electronic device according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0030] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0031] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "include", "comprising", etc. used herein indicate the existence of features, operations, operations and / or components, but do not exclude the existence or addition of one or more other features, operations, operations or components.

[0032] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0033] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0034] Some block diagrams and / or flow charts are shown in the accompanying drawings. It should be understood that some boxes or combinations thereof in the block diagrams and / or flow charts can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable automated test processing device, so that these instructions can create a device for implementing the functions / operations described in these block diagrams and / or flow charts when executed by the processor. The technology of the present disclosure can be implemented in the form of hardware and / or software (including firmware, microcode, etc.). In addition, the technology of the present disclosure can take the form of a computer program product on a computer-readable storage medium storing instructions, which can be used by an instruction execution system or used in conjunction with an instruction execution system.

[0035] With the rapid development of computer technology, more and more diverse Internet technologies (such as big data technology, distributed technology, blockchain technology, artificial intelligence technology, etc.) are being applied to the financial field. With the increasing popularity of personal terminals, more and more users are handling financial services through Internet terminals, and the number of online banking users has expanded dramatically. Since the financial field has strict requirements on the security and real-time operation of business, higher standards are required for the business environment performance of online banks. Simulating stress testing of the business environment of online banks is one of the important means to ensure the security of financial business operations and improve the real-time operation of financial business.

[0036] In the process of implementing the technical solution disclosed in the present invention, the inventor found that in the stress testing process in the related art, the stress testing operation lacks an automatic scheduling strategy and relies on manual configuration of stress testing scenarios and execution of stress testing sampling. With the increase in performance scenarios in the financial field and the improvement in the accuracy requirements of performance indicators, the current stress testing technology has shown the defects of large testing workload and low testing efficiency.

[0037] In order to overcome the defects of related stress testing technologies, such as strong dependence on manual operation, large testing workload, and low testing efficiency, the embodiments of the present disclosure provide an automated test processing method and a processing device capable of applying the method. In the method, first, in response to the received automatic test instructions, M groups of test parameters are obtained, wherein the test parameters include test scenario parameters and test flow parameters for stress testing the test object, and M is an integer greater than 1. Next, according to each test parameter in the M groups of test parameters, N stress tests are performed on the test object to obtain N groups of performance indicator data, where N is an integer not less than M. Finally, based on the N groups of performance indicator data, a stress test report for the test object is generated.

[0038] Figure 1 The system architecture of the automated test processing method and device according to the embodiment of the present disclosure is schematically shown. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0039] like Figure 1 As shown, the system architecture 100 may include a test request end 101, a network 102, a test processing platform 103, and a service platform (multiple of which are shown in the figure, such as service platforms 104, 105, and 106). The network 102 is a medium for providing a communication link between the test request end 101, the test processing platform 103, and the service platform (such as service platforms 104, 105, and 106). The network 102 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0040] The test processing platform 103 receives the automatic test instruction from the test request end 101, and the test request end 101 may include an application development platform 1011 and an operation and maintenance management platform 1012. Specifically, the automatic test instruction may be generated by the application developer through the user end in the application development platform 1011, or may be automatically triggered by the application development platform 1011 based on a preset time interval. In addition, the automatic test instruction may also be generated by the operation and maintenance management personnel through the user end in the operation and maintenance management platform 1012, or may be automatically triggered by the operation and maintenance management platform 1012 based on a preset time interval.

[0041] The test processing platform 103 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud computing, network services, and middleware services.

[0042] The business platforms (such as business platforms 104, 105, 106) constitute the test objects in the stress testing operation. By simulating real user operations in the business platforms (such as business platforms 104, 105, 106), it is determined whether the operating status of the business platforms meets the preset status requirements, and the stress testing results for the business platforms are obtained.

[0043] Specifically, the test processing platform 103 responds to the automatic test instructions received from the test request end 101, and obtains M groups of test parameters from the test database, where the test parameters include test scenario parameters and test traffic parameters for stress testing the test object, and M is an integer greater than 1. Next, the test processing platform 103 performs N stress tests on the test object (such as business platforms 104, 105, and 106) according to each test parameter in the M groups of test parameters, and obtains N groups of performance indicator data, where N is an integer not less than M. Finally, the test processing platform 103 generates a stress test report for the test object (such as business platforms 104, 105, and 106) based on the N groups of performance indicator data, and returns the stress test report to the test request end 101.

[0044] It should be noted that the automated test processing method and device of the embodiment of the present disclosure can be used in the financial field, and can also be used in any field other than the financial field. The present disclosure will be described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0045] Figure 2 The following schematically shows a flow chart of an automated test processing method according to an embodiment of the present disclosure. Figure 2 As shown, method 200 may include operations S210 to S230.

[0046] In operation S210, in response to the received automatic test instruction, M groups of test parameters are obtained, wherein the test parameters include test scenario parameters and test flow parameters for performing a stress test on a test object, and M is an integer greater than 1.

[0047] Next, in operation S220 , N stress tests are performed on the test object according to each test parameter in the M groups of test parameters to obtain N groups of performance indicator data, where N is an integer not less than M.

[0048] Next, in operation S230, a stress test report for the test object is generated based on the N groups of performance indicator data.

[0049] The specific process of each step of the automated test processing method of this embodiment is described in detail below.

[0050] In operation S210, in response to the received automatic test instruction, M groups of test parameters are obtained, wherein the test parameters include test scenario parameters and test flow parameters for performing a stress test on a test object, and M is an integer greater than 1.

[0051] In this embodiment, specifically, the test processing platform receives an automatic test instruction from a test request end, which may be an application development platform or an operation and maintenance management platform. The automatic test instruction may be a test instruction issued by a user port in the test request end, or may be a test instruction automatically triggered and generated by the test request end based on a preset time interval. The automatic test instruction indicates a test object to be stress tested, which may specifically be an application software or functional component in a financial service platform, such as a transaction page of an online banking application.

[0052] In response to the received automatic test instruction, the test processing platform obtains M groups of test parameters, each of which includes a test scenario parameter and a test flow parameter for performing a stress test on the test object, and M is an integer greater than 1. The test scenario parameter indicates a test scenario for performing a stress test on the test object, and the test flow parameter indicates a test flow value for performing a stress test on the test object in the corresponding test scenario. The test scenario may be a simulated business scenario for generating a response page according to an operation path, and the operation path may indicate a set of operations arranged in an execution order, and a set of network links for generating a response page.

[0053] Specifically, the test scenario parameters may indicate the operation event information used to perform stress testing on the test object in the corresponding test scenario, where the operation event information includes, for example, the operation event name, operation event code, operation event type, operation event execution order and other information, where the operation event type may include, for example, login operation, query operation, purchase operation, account opening operation, withdrawal operation, recharge operation, transfer operation, sale operation and other types.

[0054] In addition, the test scenario parameters may also indicate the data acquisition path for the stress test in the corresponding test scenario. The scenario test data required for the corresponding test scenario is obtained from the preset database according to the data acquisition path. The scenario test data may be obtained by recording the real data of the online interface. In addition, the test scenario parameters may also indicate information such as the test interface, test case, test script, and test result feedback path associated with the corresponding test scenario.

[0055] Correspondingly, the test traffic parameter may indicate the concurrent number of operation events used to perform stress testing on the test object in the corresponding test scenario, and the concurrent number may specifically be the number of operation event requests that the test object processes simultaneously within a preset time period. For example, 1,000 users are simulated to log in to the online banking transaction page at the same time, and the concurrent number associated with the login operation is 1,000.

[0056] In addition, the test traffic parameters can also indicate the traffic allocation ratio for stress testing the test object in the corresponding test scenario. According to the traffic allocation ratio indicated by the test traffic parameters, the user traffic within the preset time period is allocated to determine the user traffic used for stress testing in different test scenarios. Exemplarily, according to the traffic allocation ratio indicated by the test traffic parameters, 30%, 25%, 20%, 15%, and 10% of the user traffic are used in test scenarios associated with login operations, query operations, recharge operations, transfer operations, and purchase operations, respectively, to achieve stress testing for different test scenarios based on different user traffic. In addition, the test traffic parameters can also indicate information such as the type of test client used for stress testing in the corresponding test scenario, the maximum number of test clients, the number of test processes, and the running time of the test process.

[0057] The test scenario parameters can be obtained by analyzing the real user behavior data in the business analysis system. The test scenario parameters can restore the user behavior data in the real business scenario. By analyzing the real user behavior data in the business analysis system, the user behavior concurrency in the real business scenario is obtained, and the analysis results are used as the empirical value of the concurrent number of operation events for stress testing the test object. In addition, the test traffic parameters can also indicate the concurrency duration of the operation events used to stress test the test object and the concurrency per unit time.

[0058] Next, in operation S220 , N stress tests are performed on the test object according to each test parameter in the M groups of test parameters to obtain N groups of performance indicator data, where N is an integer not less than M.

[0059] In this embodiment, specifically, by executing the automatic test script in a loop, each operation event corresponding to the concurrent number can be transmitted to the execution center of the test object through a preset event response link, so as to realize N test execution processes for the test object, where N is an integer greater than or equal to M. There is a one-to-one correspondence between the test environment parameters and the test scenarios, and the same test environment parameters may correspond to multiple test traffic parameters. Therefore, more than one stress test process may be performed in the same test scenario, and the stress test conditions configured for different stress test processes may be different, that is, the test traffic parameters used in multiple stress test processes performed in the same test scenario may be different.

[0060] The automatic test script is a series of instructions for performing a specific test, which can be executed by an automated test tool. The automatic test script can be pre-generated by a test script generation engine of the test processing platform and stored in a script management center of the test processing platform.

[0061] Exemplarily, the obtained M groups of test parameters are sent to the process robot (i.e., RPA, Robotic Process Automation) program, so that the process robot program automatically logs into the transaction page of the online bank based on the operation events and concurrent numbers indicated by the test parameters, and simulates the user's real transaction operation by capturing the transaction elements in the transaction page, thereby realizing the automated stress test for the online bank transaction page. Among them, the process robot program is an application that realizes the automation of the user operation process by simulating the user's operation actions in the terminal.

[0062] Next, in operation S230, a stress test report for the test object is generated based on the N groups of performance indicator data.

[0063] In this embodiment, specifically, the business performance indicator data and / or hardware performance indicator data associated with the test object during N stress tests are monitored to obtain N groups of performance indicator data. Among them, the business performance indicator data include, for example, the number of operation event request responses, the number of concurrent responses, the response time, the response throughput, the network traffic, etc., and the hardware performance indicator data include, for example, the CPU occupancy rate, the memory occupancy rate, the CPU temperature, the battery temperature, etc.

[0064] Based on N groups of performance indicator data, a stress test report for the test object is generated. In addition, target indicator data associated with the test object can be obtained, and the target indicator data includes reference data corresponding to different stress test standards. The target indicator data corresponding to the same stress test standard is compared with the performance indicator data obtained from the actual test to obtain a comparison result, and the test result for the test object is output according to the comparison result.

[0065] Optionally, the performance limit state of the test object can be determined based on the performance indicator data associated with each test parameter, and then the test flow parameter in the test parameter can be adjusted based on the performance limit state of the test object. The test flow parameter indicates the test flow value used to perform a pressure test on the test object. In actual applications, multiple pressure tests on the test object can be performed based on successively increasing test flow values ​​in the same test scenario. When determining the performance limit state of the test object, after the operating state of the test object becomes abnormal for the first time, the corresponding test flow value can be reduced according to the preset decompression rule, and then it is determined whether the fluctuation amplitude of the performance indicator of the test object is greater than the preset threshold value. If so, it is determined that the test object has reached the performance limit state.

[0066] Through the disclosed embodiment, in response to the received automatic test instruction, M groups of test parameters are obtained, wherein the test parameters include test scenario parameters and test flow parameters for stress testing the test object, and M is an integer greater than 1; according to each test parameter in the M groups of test parameters, N stress tests are performed on the test object to obtain N groups of performance indicator data, and N is an integer not less than M; based on the N groups of performance indicator data, a stress test report for the test object is generated. According to the test scenario parameters and test flow parameters for stress testing the test object, N stress tests are performed on the test object. This design is conducive to reducing the dependence on manual operation in the process of test scenario construction and stress test sampling, which is conducive to reducing the workload of stress testing, improving the efficiency of stress testing, and improving the effect of stress testing. It can be well applied to the financial business field with many performance scenarios to be tested and high requirements for the accuracy of performance indicators.

[0067] Figure 3 The flowchart of another automated test processing method according to an embodiment of the present disclosure is schematically shown. Figure 3 As shown, operation S220 may include operations S310 to S330.

[0068] In operation S310, M test scenarios for performing a stress test on a test object are determined according to the test scenario parameters in each test parameter.

[0069] Next, in operation S320, a test flow value associated with each of the M test scenarios is determined according to the test flow parameter in each test parameter.

[0070] Next, in operation S330, N pressure tests are performed on the test object according to the test flow value associated with each test scenario to obtain N groups of performance indicator data.

[0071] The specific process of each step of the automated test processing method of this embodiment is described in detail below.

[0072] In operation S310, M test scenarios for performing a stress test on a test object are determined according to the test scenario parameters in each test parameter.

[0073] In this embodiment, specifically, the test scenario parameters in the M groups of test parameters indicate M test scenarios for performing stress testing on the test object. Figure 4 A flowchart of another automated test processing method according to an embodiment of the present disclosure is schematically shown. Figure 4 As shown, operation S220 may include operations S310', S320' and S330'. In operation S310', at least one operation event for performing stress testing on the test object in the corresponding test scenario is determined according to the test scenario parameters in each test parameter.

[0074] The test scenario parameters include single-scenario test parameters and multi-scenario test parameters. When determining at least one operation event for stress testing the test object according to the test scenario parameters, a single operation event for stress testing the test object in the corresponding test scenario can be determined according to the single-scenario test parameters in the test scenario parameters; and / or, multiple operation events for stress testing the test object in the corresponding test scenario can be determined according to the multi-scenario test parameters in the test scenario parameters. A corresponding concurrent number ratio is preset for each operation event in the aforementioned multiple operation events, and the concurrent number ratio is determined based on the real user behavior data in the business analysis system.

[0075] The test scenario parameters and test traffic parameters in the M group of test parameters are obtained by analyzing the real user behavior data in the business analysis system, which stores user behavior data in real business scenarios or real usage environments. By analyzing the user behavior data in the business analysis system, the empirical values ​​of the test scenario parameters and test traffic parameters used for stress testing the test object are obtained. This design is conducive to effectively improving the efficiency of stress testing and ensuring the effect of stress testing.

[0076] When the multi-scenario test parameters indicate multiple operation events for stress testing the test object, a corresponding concurrency ratio is preset for each operation event in the multiple operation events, and the concurrency ratio is determined based on the real user behavior data in the business analysis system. For example, when performing a stress test on an online banking transaction page, the multi-scenario test parameters indicate that the operation events used for stress testing include a login operation, an add-to-cart operation, an order operation, and a payment operation, a total of 4 operation events, and the concurrency ratios associated with each operation event are 60%, 25%, 10%, and 5%, respectively. When the number of concurrent operations per hour is 1,000, the number of login operation requests, the number of add-to-cart operation requests, the number of order operation requests, and the number of payment operation requests triggered on the online banking transaction page are 600, 250, 100, and 50, respectively.

[0077] The higher the corresponding concurrency ratio, the higher the importance or application frequency of the operation event. Considering that in actual applications, important functions or common components of the object to be tested are often stress tested, therefore, a higher test priority can be preset for operation events with a large concurrency ratio. That is, in the case where the multi-scenario test parameters indicate multiple operation events in the corresponding test scenario, the concurrency ratio associated with any operation event is positively correlated to the test priority for the operation event. In the case where the received automatic test instruction contains test priority information, at least one operation event for stress testing the test object can be determined based on the test priority information and the concurrency ratio associated with each operation event in the multiple operation events.

[0078] Next, in operation S320, a test flow value associated with each of the M test scenarios is determined according to the test flow parameter in each test parameter.

[0079] In this embodiment, specifically, in operation S320', the concurrency number associated with each operation event in each test scenario is determined based on the test traffic parameter in each test parameter. The concurrency number indicates the number of threads used to trigger the operation event, that is, the number of operation event requests to be triggered. In addition to being used to indicate the concurrency number of the operation event, the test traffic parameter can also be used to indicate the concurrency duration of the operation event. The test traffic parameter can be obtained based on real user behavior data in the business analysis system, or it can be determined based on the object type and scale characteristics of the object to be tested. In addition, it can also be obtained by querying a preset test model experience library, which is not limited in this embodiment.

[0080] Next, in operation S330, N pressure tests are performed on the test object according to the test flow value associated with each test scenario to obtain N groups of performance indicator data.

[0081] In this embodiment, specifically, in operation S330', the stress test implementation framework is called, and an automatic test script for each test scenario is generated based on at least one operation event associated with each test scenario and the number of concurrencies associated with each operation event. The automatic test script is executed in a loop, and during each execution, preset performance indicators associated with the test object are listened to to obtain N groups of performance indicator data.

[0082] Specifically, the automatic test script is executed cyclically to pass each operation event of the corresponding concurrent number to the execution center of the test object through the preset event response link. After capturing the operation event request, the execution center returns the event response result to the test processing platform through the event response link, thereby realizing N test execution processes for the test object. During each test execution process, the preset performance indicators associated with the test object are monitored to obtain N groups of performance indicator data, and the performance data analysis results are obtained by analyzing the performance indicator data, and a stress test report for the test is generated according to the performance data analysis results.

[0083] Figure 5 A block diagram of an automated test processing device according to an embodiment of the present disclosure is schematically shown.

[0084] like Figure 5 As shown, the apparatus 500 includes an acquisition module 501 , a first processing module 502 , and a second processing module 503 .

[0085] Among them, the acquisition module 501 is used to obtain M groups of test parameters in response to the received automatic test instructions, wherein the test parameters include test scenario parameters and test flow parameters for stress testing the test object, and M is an integer greater than 1; the first processing module 502 is used to perform N stress tests on the test object according to each test parameter in the M groups of test parameters, and obtain N groups of performance indicator data, where N is an integer not less than M; the second processing module 503 is used to generate a stress test report for the test object based on the N groups of performance indicator data.

[0086] Through the disclosed embodiment, in response to the received automatic test instruction, M groups of test parameters are obtained, wherein the test parameters include test scenario parameters and test flow parameters for stress testing the test object, and M is an integer greater than 1; according to each test parameter in the M groups of test parameters, N stress tests are performed on the test object to obtain N groups of performance indicator data, and N is an integer not less than M; based on the N groups of performance indicator data, a stress test report for the test object is generated. According to the test scenario parameters and test flow parameters for stress testing the test object, N stress tests are performed on the test object. This design is conducive to reducing the dependence on manual operation in the process of test scenario construction and stress test sampling, which is conducive to reducing the workload of stress testing, improving the efficiency of stress testing, and improving the effect of stress testing. It can be well applied to the financial business field with many performance scenarios to be tested and high requirements for the accuracy of performance indicators.

[0087] As a feasible approach, the first processing module includes: a first processing sub-module, used to determine M test scenarios for stress testing the test object based on the test scenario parameters in each test parameter; a second processing sub-module, used to determine the test flow value associated with each test scenario in the M test scenarios based on the test flow parameters in each test parameter; and a third processing sub-module, used to perform N stress tests on the test object based on the test flow value associated with each test scenario to obtain N sets of performance indicator data.

[0088] As a feasible approach, the first processing submodule includes: a first processing unit, used to determine at least one operation event for stress testing the test object in each of M test scenarios based on the test scenario parameters in each test parameter; the second processing submodule includes: a second processing unit, used to determine the concurrency number associated with each operation event in each test scenario based on the test traffic parameters in each test parameter.

[0089] As a feasible approach, the test scenario parameters include single-scenario test parameters and multi-scenario test parameters; the first processing unit includes: a first processing sub-unit, used to determine a single operation event for stress testing the test object in the corresponding test scenario according to the single-scenario test parameters in the test scenario parameters; a second processing sub-unit, used to determine multiple operation events for stress testing the test object in the corresponding test scenario according to the multi-scenario test parameters in the test scenario parameters, wherein a corresponding concurrency ratio is preset for each operation event in the multiple operation events, and the concurrency ratio is determined based on real user behavior data in the business analysis system.

[0090] As a feasible approach, the third processing submodule includes: a third processing unit, used to generate an automatic test script for each test scenario based on at least one operation event associated with each test scenario and the number of concurrencies associated with each operation event; a fourth processing unit, used to execute the automatic test script in a loop, and during each execution process, listen to preset performance indicators associated with the test object to obtain N groups of performance indicator data.

[0091] As a feasible approach, the fourth processing unit includes: a third processing sub-unit, used to execute the automatic test script in a loop, so as to transmit each operation event of the corresponding concurrent number to the execution center of the test object through a preset event response link, so as to realize N test execution processes for the test object; and a fourth processing sub-unit, used to monitor the preset performance indicators associated with the test object during each test execution process to obtain N groups of performance indicator data.

[0092] As a feasible approach, when the multi-scenario test parameters indicate multiple operation events in the corresponding test scenarios, the proportion of concurrency numbers associated with any operation event is positively correlated to the test priority for the operation event, and when the received automatic test instruction contains test priority information, the first processing unit is further used to: determine at least one operation event for stress testing the test object based on the test priority information and based on the proportion of concurrency numbers associated with each operation event in the multiple operation events.

[0093] As a feasible approach, the device further includes: a third processing module for determining the performance limit state of the test object based on performance indicator data associated with each test parameter; and for adjusting the test flow parameter in the test parameter based on the performance limit state.

[0094] It should be noted that in the embodiments of the present disclosure, the implementation of the device part is the same as or similar to the implementation of the method part, and will not be repeated here.

[0095] According to any one or more of the modules of the embodiments of the present disclosure, or at least part of the functions of any one of them, can be implemented in one module. According to any one or more of the modules of the embodiments of the present disclosure, it can be split into multiple modules for implementation. According to any one or more of the modules of the embodiments of the present disclosure, it can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware of any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Or according to one or more of the modules of the embodiments of the present disclosure, it can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be performed.

[0096] For example, any multiple of the acquisition module 501, the first processing module 502, and the second processing module 503 can be combined in one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the acquisition module 501, the first processing module 502, and the second processing module 503 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware or in any appropriate combination of any of them. At least one of the acquisition module 501, the first processing module 502, and the second processing module 503 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0097] Figure 6 A block diagram of an electronic device 600 suitable for implementing the processing method and processing device according to an embodiment of the present disclosure is schematically shown. Figure 6 The electronic device 600 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0098] like Figure 6As shown, the electronic device 600 according to an embodiment of the present disclosure includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 into a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include an onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0099] In RAM 603, various programs and data required for the operation of electronic device 600 are stored. Processor 601, ROM 602 and RAM 603 are connected to each other via bus 604. Processor 601 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 602 and / or RAM 603. It should be noted that the program can also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in one or more memories.

[0100] According to an embodiment of the present disclosure, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to the bus 604. The electronic device 600 may further include one or more of the following components connected to the I / O interface 605: an input portion 606 including a keyboard, a mouse, etc.; an output portion 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 608 including a hard disk, etc.; and a communication portion 609 including a network interface card such as a LAN card, a modem, etc. The communication portion 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage portion 608 as needed.

[0101] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0102] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0103] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 602 and / or RAM 603 described above and / or one or more memories other than ROM 602 and RAM 603.

[0104] An embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains a program code for executing the method provided by the embodiment of the present disclosure. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the method for detecting file upload vulnerabilities provided by the embodiment of the present disclosure.

[0105] When the computer program is executed by the processor 601, the above functions defined in the system / device of the embodiment of the present disclosure are executed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0106] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 609, and / or installed from a removable medium 611. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0107] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0108] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0109] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Without departing from the scope of the present disclosure, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. An automated test processing method, comprising: In response to the received automatic test instruction, M groups of test parameters are obtained, wherein the test parameters include test scenario parameters and test flow parameters for performing a stress test on the test object, and M is an integer greater than 1; According to each test parameter in the M groups of test parameters, perform N stress tests on the test object to obtain N groups of performance indicator data, where N is an integer not less than M; and generate a stress test report for the test object based on the N groups of performance indicator data; The method of performing N stress tests on the test object according to each test parameter in the M groups of test parameters to obtain N groups of performance indicator data includes: determining M test scenarios for performing stress tests on the test object according to the test scenario parameters in each of the test parameters; determining a test flow value associated with each of the M test scenarios according to the test flow parameters in each of the test parameters; performing N stress tests on the test object according to the test flow value associated with each of the test scenarios to obtain N groups of performance indicator data; The determining, according to the test scenario parameters in each of the test parameters, M test scenarios for performing a stress test on the test object comprises: determining, according to the test scenario parameters in each of the test parameters, at least one operation event for performing a stress test on the test object in each of the M test scenarios; The test scenario parameters include single scenario test parameters and multi-scenario test parameters; In the case where the multi-scenario test parameter indicates multiple operation events in the corresponding test scenario, the proportion of the number of concurrent operations associated with any of the operation events is positively correlated to the test priority for the operation event. In the case where the received automatic test instruction includes test priority information, the method further includes: determining at least one operation event for performing stress testing on the test object according to the test priority information and according to the concurrency ratio associated with each of the multiple operation events; The test scenario generates a simulated business scenario of a response page according to an operation path; the operation path indicates a set of operations arranged in an execution order, and indicates a set of network links used to generate a response page.

2. The method according to claim 1, wherein: The step of determining the test flow value associated with each of the M test scenarios according to the test flow parameter in each of the test parameters includes: According to the test traffic parameter in each of the test parameters, the concurrency number associated with each of the operation events in each of the test scenarios is determined.

3. The method according to claim 2, wherein: The determining, according to the test scenario parameters in each of the test parameters, at least one operation event for performing a stress test on the test object in each of the M test scenarios comprises: Determining, according to the single-scenario test parameters in the test scenario parameters, a single operation event for performing a stress test on the test object in the corresponding test scenario; and / or Determine, according to the multi-scenario test parameters in the test scenario parameters, a plurality of operation events for performing stress testing on the test object in the corresponding test scenario, Therein, a corresponding concurrent number ratio is preset for each operation event in the multiple operation events, and the concurrent number ratio is determined based on real user behavior data in the business analysis system.

4. The method according to claim 2, wherein: The step of performing N stress tests on the test object according to the test flow value associated with each of the test scenarios to obtain N groups of performance indicator data includes: Generate an automatic test script for each of the test scenarios according to at least one operation event associated with each of the test scenarios and according to the concurrency number associated with each of the operation events; The automatic test script is executed in a loop, and during each execution process, preset performance indicators associated with the test object are monitored to obtain the N groups of performance indicator data.

5. The method according to claim 4, wherein: The automatic test script is executed cyclically, and preset performance indicators associated with the test object are monitored during each execution process to obtain the N groups of performance indicator data, including: Executing the automatic test script in a loop to transmit the corresponding concurrent number of operation events to the execution center of the test object through a preset event response link, so as to implement N test execution processes for the test object; and During the execution of each of the tests, preset performance indicators associated with the test object are monitored to obtain the N groups of performance indicator data.

6. The method according to any one of claims 1 to 5, further comprising: Determining the performance limit state of the test object according to the performance indicator data associated with each of the test parameters; According to the performance limit state, the test flow parameters in the test parameters are adjusted.

7. An automated test processing device, comprising: an acquisition module, configured to acquire M groups of test parameters in response to a received automatic test instruction, wherein the test parameters include test scenario parameters and test flow parameters for performing a stress test on a test object, and M is an integer greater than 1; a first processing module, configured to perform N stress tests on the test object according to each test parameter in the M groups of test parameters, and obtain N groups of performance indicator data, where N is an integer not less than M; and A second processing module, configured to generate a stress test report for the test object based on the N groups of performance indicator data; The first processing module includes: a first processing submodule, used to determine M test scenarios for stress testing the test object according to the test scenario parameters in each of the test parameters; a second processing submodule, used to determine the test flow value associated with each of the M test scenarios according to the test flow parameters in each of the test parameters; a third processing submodule, used to perform N stress tests on the test object according to the test flow value associated with each of the test scenarios, to obtain N groups of performance indicator data; The first processing submodule includes: a first processing unit, configured to determine at least one operation event for performing a stress test on the test object in each of the M test scenarios according to a test scenario parameter in each of the test parameters; The test scenario parameters include single scenario test parameters and multi-scenario test parameters; In a case where the multi-scenario test parameter indicates a plurality of operation events in the corresponding test scenario, the proportion of the number of concurrent operations associated with any of the operation events is positively correlated with the test priority for the operation event, and in a case where the received automatic test instruction includes test priority information, the first processing unit is further used to: determine at least one operation event for performing stress testing on the test object according to the test priority information and according to the proportion of the number of concurrent operations associated with each of the plurality of operation events; The test scenario generates a simulated business scenario of a response page according to an operation path; the operation path indicates a set of operations arranged in an execution order, and indicates a set of network links used to generate a response page.

8. An electronic device comprising: one or more processors; as well as a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.