Software system testing method, electronic equipment and computer readable medium
By analyzing the operation log to generate test data sets and using the atomic use case library to create test plans and automated use cases, the problems of insufficient software testing efficiency and cost in the existing technology are solved, and more efficient test execution and data-driven testing are achieved, which improves testing Efficiency and data authenticity.
Patent Information
- Application Number
- CN202310751961.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-25
- Publication Date
- 2024-12-27
AI Technical Summary
Existing software testing solutions are insufficient in efficiency and cost optimization when applied to network management systems, lack of parsing product operation logs to build test cases and data, separation of test scripts and data leads to binding of parameter acquisition and execution time, and insufficient decomposition of back-end interface tests for automated use cases , and test cases and automation cases have poor reusability.
Generate test data sets by parsing the operation log of the software system, creating test plans and automated use cases using the atomic use case library, forming a test blueprint, and automatically execute them based on the test data set, output test reports, and implementing automation and data-driven testing environments.
Improves testing efficiency and data authenticity, saves maintenance costs of atomic and automated use cases, and can automatically create plans and execute tests based on the test environment, and output test reports that are closer to the actual environment.
Smart Images
Figure CN119201659A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical field of software testing, and in particular to a software system testing method, electronic equipment, and computer-readable medium. Background Art
[0002] Software testing is an important part of the software development process. It is done by analyzing and testing the software before it is put into operation to confirm whether the function or performance of a program meets the development requirements or application requirements.
[0003] However, conventional software testing solutions still have defects and limitations when applied to network management systems, and the efficiency and cost of software testing need to be further optimized. Summary of the invention
[0004] Embodiments of the present disclosure provide a software system testing method, an electronic device, and a computer-readable medium.
[0005] In a first aspect, an embodiment of the present disclosure provides a testing method for a software system, comprising: parsing an operation log of the software system to obtain a test data set including multiple test data; creating a test plan based on the scale and type of the test environment using atomic use cases in an atomic use case library, wherein the atomic use case library includes multiple atomic use cases, and one operation item in the operation log corresponds to one atomic use case; automating the test plan to obtain a test blueprint, wherein the test blueprint includes automated use cases corresponding to the atomic use cases in the test plan and scheduling the start time of the automated use cases; executing the test blueprint based on the test data set to obtain a test report.
[0006] In a second aspect, an embodiment of the present disclosure provides an electronic device, comprising: one or more processors; a memory on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the testing method described in the first aspect of the embodiment of the present disclosure.
[0007] In a third aspect, an embodiment of the present disclosure provides a computer-readable medium having a computer program stored thereon, and when the program is executed by a processor, the test method described in the first aspect of the embodiment of the present disclosure is implemented.
[0008] The software system testing method provided by the embodiment of the present disclosure generates a test data set by parsing the operation log of the software system. The test data in the test data set is more realistic, making the test scenario closer to the actual software operating environment. When creating a test plan, the atomic use cases in the atomic use case library can be reused, and when creating a test blueprint, the automated use cases can be reused, which can effectively save the maintenance cost of the atomic use cases and the automated use cases. The test plan and the test blueprint can be automatically created according to the test environment, and the test blueprint can be automatically executed according to the test data set to output a test report, which is conducive to improving the test efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] Figure 1 is a flow chart of a testing method in an embodiment of the present disclosure;
[0010] Figure 2 is a flow chart of another testing method in an embodiment of the present disclosure;
[0011] Figure 3 is a flowchart of some steps in another testing method in an embodiment of the present disclosure;
[0012] Figure 4 is a flowchart of some steps in another testing method in an embodiment of the present disclosure;
[0013] Figure 5 is a schematic diagram of the composition of an electronic device in an embodiment of the present disclosure;
[0014] Figure 6 is a schematic diagram of the composition of a computer-readable medium in an embodiment of the present disclosure;
[0015] Figure 7 is a schematic diagram of the architecture of a test system in an embodiment of the present disclosure;
[0016] Figure 8 is a schematic diagram of a testing process of a software system in an embodiment of the present disclosure;
[0017] Fig. 9 is a schematic diagram of a testing process of another software system in an embodiment of the present disclosure;
[0018] Fig.10 is a schematic diagram of a testing process of another software system in an embodiment of the present disclosure;
[0019] Fig.11 It is a schematic diagram of the architecture of another test system in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0020] In order to enable those skilled in the art to better understand the technical solution of the present disclosure, the software system testing method, electronic device, and computer-readable medium provided by the present disclosure are described in detail below with reference to the accompanying drawings.
[0021] Example embodiments will be described more fully below with reference to the accompanying drawings, but the example embodiments may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. On the contrary, the purpose of providing these embodiments is to make the present disclosure thorough and complete and to enable those skilled in the art to fully understand the scope of the present disclosure.
[0022] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.
[0023] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.
[0024] The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a", "an" and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "comprising" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof is not excluded.
[0025] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless explicitly defined as such herein.
[0026] The inventor of the present disclosure has found that in some related technologies, test cases, test scripts and test data are separated, and the test is performed by data-driven test script execution. Its defects are: first, it is less involved in parsing the running log of the product to build test cases and test data, and test data is generally generated based on pre-built test case data or existing test case data analysis; second, the separation of test scripts and test data, mostly through test data to control the execution of different test scripts, less is the extraction of parameters in the test script as the input parameter of the test, and the acquisition of parameters is bound to the time of test case execution; third, the automation case backend interface test is less decomposed into atomic operation granularity, and a complete task test is realized by sequentially combining atomic operations, and atomic operations are more common in front-end automation testing or keyword-driven testing; fourth, it is less involved in the situation where test cases and automation cases can be reused at the same time, and different scenario tests can be realized by only modifying the test data. Generally, different test scenarios will design different test cases, and there are many cases of reusing keywords when developing automation cases.
[0027] In view of this, firstly, refer to Figure 1 , the present disclosure provides a method for testing a software system, including:
[0028] S1. Analyze the operation log of the software system to obtain a test data set including multiple test data;
[0029] S2. Create a test plan using atomic use cases in an atomic use case library according to the scale and type of the test environment, wherein the atomic use case library includes a plurality of the atomic use cases, and one operation item in the operation log corresponds to one atomic use case;
[0030] S3, automatically processing the test plan to obtain a test blueprint, wherein the test blueprint includes the automated use cases corresponding to the atomic use cases in the test plan and the start time of scheduling the automated use cases;
[0031] S4. Execute the test blueprint according to the test data set to obtain a test report.
[0032] The embodiment of the present disclosure does not specifically limit the test data set. In some embodiments, the test data set is in the form of a test data driven file.
[0033] In the disclosed embodiment, an operation item in the operation log is an operation record. The disclosed embodiment defines an operation record in the operation log as an atomic operation, and an atomic operation corresponds to an atomic use case. In the disclosed embodiment, the operation task is decomposed into atomic operations, and an independent operation task can be completed by an atomic operation, and a non-independent operation task is completed in sequence by a number of interrelated atomic operations.
[0034] In the embodiment of the present disclosure, the automated use case is an automated script of the atomic use case, and one automated use case corresponds to one atomic use case.
[0035] In the embodiments of the present disclosure, test environments of different scales and types correspond to actual operating environments of different scales and types. In the embodiments of the present disclosure, creating a test plan refers to the process of formulating a plan for executing atomic use cases in test environments of different scales and types. In some embodiments, when creating a test plan, the corresponding atomic use case is selected from the atomic use case library based on the user operation behavior in the actual operating environment corresponding to the test environment. In the embodiments of the present disclosure, the user operation behavior characterizes different time periods, the user's operation of the software system's functions (i.e., operation entry) and the number of operations, etc., and the embodiments of the present disclosure do not make special limitations on this. In some embodiments, the test plan is created using the atomic use cases in the atomic use case library, that is, based on the correspondence between the atomic operations performed by the atomic use cases and the user's operation behavior, the atomic use case is selected and an execution plan is formulated.
[0036] In the disclosed embodiment, creating a test blueprint refers to the process of automating the test plan, by determining the automation use cases corresponding to the test plan and scheduling the start time of the automation use cases, thereby automatically executing the test plan.
[0037] In the embodiments of the present disclosure, when creating different test plans, atomic use cases in the atomic use case library can be reused; when creating test blueprints according to different test plans, automated use cases can be reused.
[0038] In the embodiment of the present disclosure, when executing the test blueprint according to the test data set, the test data corresponding to the test environment is obtained from the test data set according to the test environment, so as to implement the testing of different test scenarios.
[0039] The embodiments of the present disclosure do not specifically limit the software system. For example, the software system is a telecommunications network management product.
[0040] The software system testing method provided by the embodiment of the present disclosure generates a test data set by parsing the operation log of the software system. The test data in the test data set is more realistic, making the test scenario closer to the actual software operating environment. When creating a test plan, the atomic use cases in the atomic use case library can be reused, and when creating a test blueprint, the automated use cases can be reused, which can effectively save the maintenance cost of the atomic use cases and the automated use cases. The test plan and the test blueprint can be automatically created according to the test environment, and the test blueprint can be automatically executed according to the test data set to output a test report, which is conducive to improving the test efficiency.
[0041] In the disclosed embodiment, when testing the software system, the atomic use case library and automated use cases that have been created can be reused. When the atomic use case library and automated use cases do not exist, the atomic use case library and automated use cases can also be created according to the operation log.
[0042] Accordingly, in some embodiments, reference Figure 2 , the testing method further comprises:
[0043] S5. Create the atomic use case library according to the operation log, wherein the atomic use cases in the atomic use case library include the operation entry identifier and driving parameters of the operation entry corresponding to the atomic use case;
[0044] S6. Create an automation script for the atomic use case and obtain an automation use case corresponding to the atomic use case.
[0045] In the disclosed embodiment, an operation entry corresponds to a business logic of the software system; in the operation log, an operation entry may correspond to one or more operation records. In the disclosed embodiment, a business logic corresponds to an atomic use case, and the operation records corresponding to the same operation entry in the operation log correspond to the same atomic use case.
[0046] The embodiment of the present disclosure does not specifically limit the operation entry identifier. For example, the operation entry identifier is the name of the operation entry.
[0047] In the disclosed embodiment, the driving parameters of the atomic use case are variables. The disclosed embodiment does not specifically limit the driving parameters. For example, the driving parameters include parameters corresponding to the number of users, the number of operation objects, the operation frequency, etc. In some embodiments, the atomic use case also includes the operation time, the operation steps, etc.
[0048] In the embodiment of the present disclosure, the operation entry identifier of the automation use case is consistent with the operation entry identifier of the corresponding atomic use case, and the driving parameters of the automation use case are consistent with the driving parameters of the corresponding atomic use case.
[0049] In the disclosed embodiment, when creating an automated script for atomic use cases and obtaining corresponding automated use cases, the atomic use cases of independent operation tasks are placed in the same test suite; the atomic use cases of non-independent operation tasks that are interrelated are placed in a test suite; the atomic use cases in the same test suite are executed serially, and the atomic use cases in different test suites can be executed in parallel.
[0050] The disclosed embodiment does not specifically limit how to create an atomic use case library.
[0051] In some embodiments, creating the atomic use case library according to the operation log includes:
[0052] Extracting an operation entry identifier of an operation entry according to the operation log;
[0053] According to the operation steps corresponding to the operation entry, create an atomic use case corresponding to the operation entry identifier;
[0054] The atomic use cases corresponding to the different operation entry identifiers are combined into the atomic use case library.
[0055] The embodiments of the present disclosure do not specifically limit how to generate automated use cases corresponding to atomic use cases.
[0056] In the embodiment of the present disclosure, the operation entry identifier of the automation use case is consistent with the operation entry identifier of the corresponding atomic use case, and the driving parameters of the automation use case are consistent with the driving parameters of the corresponding atomic use case.
[0057] The embodiments of the present disclosure do not specifically limit how to establish the test data set.
[0058] In some embodiments, the operation log of the software system is parsed to obtain a test data set including multiple test data, including:
[0059] Collecting operation logs of the software system corresponding to operation environments of different scales and types;
[0060] Test data corresponding to operation environments of different scales and types are extracted from the operation log to obtain the test data set, wherein the test data includes an operation entry identifier and drive data of the operation entry.
[0061] The embodiment of the present disclosure does not impose any special limitation on the driving data. For example, the driving data includes data such as the number of users, the number of operation objects, and the frequency of operation.
[0062] In some embodiments, reference Figure 3 , extracting test data corresponding to operation environments of different scales and types from the operation log to obtain the test data set, including:
[0063] S11, cleaning the operation log according to the time granularity, and obtaining the drive data corresponding to each time granularity;
[0064] S12: Summarize the drive data according to the operation entry identifier and the corresponding time granularity to obtain the test data.
[0065] In some embodiments, by cleaning and summarizing the operation logs, an atomic operation analysis report including operation time, operation steps, operation entry, number of users, number of operation objects, operation frequency, original details, etc. is obtained, and a test data set consisting of multiple test data is further obtained.
[0066] In some embodiments, reference Figure 3 , extracting test data corresponding to operating environments of different scales and types from the operation log to obtain the test data set, further comprising:
[0067] S13, generating pre-operation data corresponding to the pre-operation in the operating environment;
[0068] S14, generating post-operation data corresponding to the post-operation in the operation environment;
[0069] S15: Add the pre-operation data and the post-operation data to the test data set.
[0070] In the disclosed embodiment, since the atomic use case for coherent operations cannot guarantee that the extracted operation data are all extracted from the beginning of the entire business logic, supplementing the preceding operation data can supplement the missing atomic operation data to ensure data completeness.
[0071] In the disclosed embodiment, supplementing the post-operation data refers to supplementing the operation of deleting the task class so that the entire environment is restored to its original state after the test is executed.
[0072] In some embodiments, the driving data includes at least one of the number of users, the number of operation objects, and the operation frequency.
[0073] The disclosed embodiment also supports manual construction and arrangement of data-driven test files based on operation entries.
[0074] Accordingly, in some embodiments, the operation log of the software system is parsed to obtain a test data set including multiple test data, including:
[0075] Extracting an operation entry identifier of an operation entry according to the operation log;
[0076] The test data is arranged according to the operation entry identifier to obtain the test data set.
[0077] In some embodiments, reference Figure 4 , based on the scale and type of the test environment, create a test plan using atomic use cases in the atomic use case library, including:
[0078] S21, determining a user operation behavior in an operation environment corresponding to the test environment according to the scale and type of the test environment;
[0079] S22. Create a test plan based on the user operation behavior using the atomic use cases in the atomic use case library.
[0080] For example, the operation logs of the software system corresponding to operating environments of different scales and types are collected. For example, the operating environments of different scales and types include three different types of operating environments, namely env1 (scale 1), env2 (scale 2), and env3 (scale 3). Each operating environment collects the operation logs of the first quarter and the second quarter; corresponding to the operating environments env1, env2, and env3, the test environments test_env1 (scale 1), test_env2 (scale 2), and test_env3 are set up; the test data obtained by parsing the operation log of env1 in the first quarter is selected to create test plan 1 in test_env1; the test data obtained by parsing the operation log of env2 in the first quarter is selected to create test plan 2 in test_env2; the test data obtained by parsing the operation log of env1 in the second quarter is selected to create test plan 3 in test_env1; the test data obtained by parsing the operation log of env2 in the second quarter is selected to create test plan 4 in test_env2; and so on.
[0081] The disclosed embodiment does not specifically limit how to create a test plan using atomic use cases in an atomic use case library based on user operation behaviors.
[0082] In some embodiments, creating the test plan using the atomic use cases in the atomic use case library according to the user operation behavior includes:
[0083] Creating test sub-plans corresponding to different test time periods according to the user operation behavior, the test sub-plans including an operation entry identifier of an operation entry corresponding to the user operation behavior;
[0084] Acquire a target atomic use case from the atomic use case library according to the operation entry identifier in the test sub-plan;
[0085] Add the target atomic use case to the corresponding test sub-plan, wherein the test plan includes the test sub-plans corresponding to different test time periods.
[0086] In some embodiments, reference Figure 4 , the test plan is automatically processed to obtain a test blueprint, including:
[0087] S31, acquiring a target automation use case corresponding to the target atomic use case according to the operation entry identifier of the target atomic use case in the test plan;
[0088] S32, determining the start time of scheduling the target automation use case according to the test plan;
[0089] S33. Arrange the target automation use case according to the startup time to obtain the test blueprint.
[0090] In some embodiments, reference Figure 4 , executing the test blueprint according to the test data set to obtain a test report, including:
[0091] S41, acquiring target test data from the test data set according to the test environment and the test time period corresponding to the target automation use case in the test blueprint;
[0092] S42: Schedule and execute the target automation case according to the target test data and the start time corresponding to the target automation case to obtain the test report.
[0093] In the disclosed embodiment, the target test data is passed into the automation use case as an input parameter of the target automation use case.
[0094] In the disclosed embodiments, by modifying or adjusting the target test data, different scenarios and different purpose tests can be implemented, that is, data-driven testing can be implemented. For example, setting the number of users, the number of operation objects, and the frequency to 1 can implement functional testing; the number of users, the number of operation objects, and the frequency are consistent with the external product system operation log to implement user behavior testing; the number of users, the number of operation objects, and the frequency are set to the maximum to implement system boundary testing, etc.
[0095] In an embodiment of the present disclosure, if there is test data corresponding to the test environment in the test data set, the test data corresponding to the test environment in the test data set is directly obtained as the target test data; if there is no test data corresponding to the test environment in the test data set, the target test data corresponding to the test environment is generated according to the benchmark test data.
[0096] Accordingly, in some embodiments, obtaining target test data from the test data set according to the test environment and the test time period corresponding to the target automation use case in the test blueprint includes:
[0097] Acquire benchmark test data from the test data set according to the test environment and the test time period corresponding to the target automation use case in the test blueprint;
[0098] The target test data is generated according to the benchmark test data.
[0099] Second, refer to Figure 5 , an embodiment of the present disclosure provides an electronic device, comprising:
[0100] One or more processors 101;
[0101] A memory 102 storing one or more programs, which, when executed by one or more processors, enables the one or more processors to implement the test method described in the first aspect of the embodiment of the present disclosure;
[0102] One or more I / O interfaces 103 are connected between the processor and the memory and are configured to implement information exchange between the processor and the memory.
[0103] Among them, the processor 101 is a device with data processing capabilities, including but not limited to a central processing unit (CPU) and the like; the memory 102 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH); the I / O interface (read-write interface) 103 is connected between the processor 101 and the memory 102, and can realize information interaction between the processor 101 and the memory 102, including but not limited to a data bus (Bus) and the like.
[0104] In some embodiments, the processor 101 , the memory 102 , and the I / O interface 103 are connected to each other via a bus 104 , and further connected to other components of the computing device.
[0105] Fourthly, refer to Figure 6 An embodiment of the present disclosure provides a computer-readable medium on which a computer program is stored. When the program is executed by a processor, the test method described in the first aspect of the embodiment of the present disclosure is implemented.
[0106] In order to enable those skilled in the art to more clearly understand the technical solutions provided by the embodiments of the present disclosure, the technical solutions provided by the embodiments of the present disclosure are described in detail below through specific embodiments:
[0107] Embodiment 1
[0108] In this embodiment, the overall architecture of the test system is as follows: Figure 7 As shown. In this embodiment, the collected operation log of the software system or the desensitized operation log meets the data protection requirements. The operation log is input into the test system, and the data-driven test file is stored in the database of the test system. The test system and the different test environments of the software system (test environment 1, test environment 2, ..., test environment N) can communicate with each other through the network, and can provide users with services such as creating test plans, viewing test progress and test reports through external web services (web browsers).
[0109] The software architecture of the test system in this embodiment mainly includes the following parts:
[0110] Generate a data-driven test file: the user obtains the operation logs of outsourced products of different scales into the test system of this embodiment, parses the operation logs of the software system, extracts and summarizes all different operation entry names in the product; the test system cleans the operation logs according to the time granularity (1h), and obtains the operation entry, user, number of operation objects, and frequency within each time granularity; the extracted data is extracted and summarized to obtain an atomic operation analysis report containing operation time, function / operation, product, APP, operation entry, number of users, number of operation objects, frequency, and original details; the atomic operation analysis report is supplemented with verification, and finally a complete executable data-driven test file is obtained.
[0111] Build an atomic use case library: The test system extracts and summarizes the names of all different operation entrances in the product; builds an atomic use case library based on the operation entrances and the corresponding product functions / operations; each atomic use case is named as "Atomic Use Case_Operation Entrance"; the operation steps describe M users, select N number of operation objects, specific operations to be performed, and expected results; the number of users M, the number of operation objects N, and the frequency of execution F are all open parameters, not fixed values.
[0112] Develop atomic automated use cases: When developing automated use cases, atomic use cases that operate independently are placed in the same test suite; atomic use cases that are interrelated are placed in a test suite and saved in the order of atomic operations; when the use cases are executed, the use cases in the same test suite can only be executed serially, while different test suites can be executed in parallel; the three parameters required for the execution of the use case: the number of users M, the number of operation objects N, and the frequency F are used as variables, and the incoming data needs to be dynamically loaded according to the execution time and execution environment.
[0113] Automatically create test plans and test blueprints: select the data-driven file corresponding to the environment according to the environment and scale (if there is no test file corresponding to the environment or scale, select the benchmark data test file and automatically generate the data-driven file of the corresponding scale); obtain the identification of the use case according to the operation entry and the use case title in the atomic use case library; create a test plan for the corresponding use case according to different operation times; create a test blueprint according to the specific operation time and test plan.
[0114] Blueprint scheduling use case execution: When the automated use case is executed, the data-driven file of the corresponding environment is obtained according to the environment in which the use case is executed; the number of users M, number of operation objects N, frequency F, and other parameters P required for the automated use case at the corresponding time point are obtained from the data-driven file according to the execution time and the operation entry in the use case operation name; when the atomic use case is executed, the number of users M, number of operation objects N, frequency F, and other parameters P are passed in as parameters of the automated use case to execute the use case.
[0115] Output test report: The test blueprint is executed as planned, the test results are summarized, and the test report is printed.
[0116] Embodiment 2
[0117] This embodiment is based on the test system of the first embodiment. The overall architecture of the test system is as follows: Figure 8 The process of testing the software system is as follows. Fig. 9 As shown, including:
[0118] Generate data-driven test files. Phase 1: Users collect system operation logs (field operation log 1, field operation log 2, field operation log 3) from various types of outbound environments (test scenario 1: busy hour playback; test scenario 2: daily playback; test scenario 3: weekly playback) of network management products at different locations, and upload them to this system; extract data from each operation log, separate the operation behavior (business logic) from the parameter data (drive data), and each operation log can be represented as an operation entry + drive data; the drive data includes the number of users, the number of operation objects, the frequency, and other parameters; define an operation item in the operation log record as an atomic operation, and an atomic operation corresponds to an atomic use case. A complete operation behavior of a user can be completed by one atomic operation or multiple atomic operations. Extract and summarize the different operation entry name entries in the product from all field log records to build an atomic use case library. Phase 2: Generate atomic operation analysis reports (atomic operation analysis report 1, atomic operation analysis report 2, ..., atomic operation analysis report N) based on the test purpose and test scenario. This example uses a test environment to simulate the maximum stress scenario of real field user behavior to illustrate the steps of generating an atomic operation analysis report: Combined with user commuting time and data analysis, the 7-hour busy time period (9:00-12:00; 14:00-18:00) is defined as busy time. Fig.10The specific steps are as follows: 1. Obtain operation logs of different bureaus; 2. Summarize the operation logs by granularity (e.g., hourly level), clean the original operation logs by busy hours, and obtain the operation entry, user, number of operation objects, frequency (frequency within 1H), and other parameters within each busy hour; 3. Extract the 7 busy hour time periods (9:00-12:00, 14:00-18:00) in step (2), and summarize the number of different operation entry items in each busy hour of each day; 4. Extract the list of busy hour operation entries with the most operation items in each busy hour in 3, and summarize the list of operation entries for the 7 busy hours; 5. Expand the busy hour data summarized in 4 according to the operation entry, and take the number of users of each operation item with the largest number of users (different user names) of the operation in the hourly playback as the number of concurrent user operations, the number of operation objects with the most single user operations in this hour, and the average value of this hour (the sum of frequency / number of users) for the frequency. Finally, an atomic operation analysis report including operation time, function / operation, product, APP, operation entry, number of users, number of operation objects, frequency, and original details is obtained; the test system supports cleaning the operation log according to different time granularities (1h, 15min, etc.). The third stage: supplement the pre- and post-atomic operation data. When extracting busy hour data in the second stage, the atomic use case for coherent operations cannot guarantee that all extracted operation data are extracted from the beginning of the entire business logic. Therefore, the missing atomic operation data is supplemented before the busy hour, and the data file is ensured to be complete and executable through the entire busy hour; after the busy hour, the operation of the deletion task class is supplemented, and the data-driven test is executed, and the entire environment is restored to its original state; in this stage, the atomic operations to be supplemented are first manually combed and verified, and the rules for supplementing the atomic operation data are summarized and organized (supplementing data rule set). The fourth stage: In the previous stage, the rules manually summarized and combed are automatically supplemented and verified through system coding, and the test system outputs the data-driven test files of the corresponding environment (test data file 1, test data file 2, ..., test data file N). Furthermore, the test system supports manual construction and arrangement of data-driven test files based on operation entry.
[0119] Build an atomic use case library. Phase 1: Define an operation item in the operation log record as an atomic operation. An atomic operation corresponds to an atomic use case. A complete user operation can be completed through one atomic operation or multiple atomic operations. For example: 1. If a user needs to add a new user to the system, he can do it through an atomic operation: product name-function name-add user, and enter different operation data to complete it; 2. If a user creates a task and exports data, he can do it through two atomic operations in succession: product name-function name-new task, product name-function name-export data, and enter different operation data to complete it. Phase 2: The test system extracts and summarizes all the different operation entry names in the product; builds an atomic use case library based on the operation entry combined with the corresponding product function / operation; each atomic use case name is named as: "Atomic Use Case_Operation Entry"; in the operation steps, describe the operation process in the form of M users, selecting N operation objects, and the number of specific function operations performed, as well as the expected effect to be achieved by the operation steps; the number of users M, the number of operation objects N, and the frequency of execution F are all open values, not fixed values.
[0120] Develop automated use cases. Atomic use cases are used when developing automated use cases. When executing use cases, use cases in the same test suite can only be executed serially, while use cases in different test suites can be executed in parallel. The three parameters required for use case execution: number of users M, number of operation objects N, and frequency F are used as variables, and the incoming data needs to be dynamically loaded according to the execution time and execution environment.
[0121] Create test plans and test blueprints. Select the data-driven files for the corresponding environment according to the environment and scale; obtain the identification of the use case according to the operation entry and the use case title in the atomic use case library; create test plans for the corresponding use cases according to different operation times; create test blueprints according to the specific operation time and test plan.
[0122] Use case scheduling. Combined Fig.11 When the automated use case is executed, the data-driven file of the corresponding environment is obtained according to the environment in which the use case is executed; the number of users M, the number of operation objects N, and the frequency F required for the automated use case at the corresponding time point are obtained from the data-driven file according to the execution time and the operation entry in the use case operation name; when the atomic use case is executed, the number of users M, the number of operation objects N, and the frequency F are used as the input parameters of the automated use case to execute the use case.
[0123] Output test report: The test blueprint is executed as planned, the test results are summarized, and the test report is printed.
[0124] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.
[0125] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for limiting purposes. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly noted, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, those skilled in the art will appreciate that various changes in form and detail may be made without departing from the scope of the present disclosure as set forth in the appended claims.
Claims
1. A software system testing method, comprising: Parse the operation log of the software system to obtain a test data set including multiple test data; Creating a test plan using atomic use cases in an atomic use case library according to the scale and type of the test environment, wherein the atomic use case library includes a plurality of the atomic use cases, and one operation item in the operation log corresponds to one atomic use case; Automating the test plan to obtain a test blueprint, wherein the test blueprint includes automated use cases corresponding to the atomic use cases in the test plan and a start time for scheduling the automated use cases; The test blueprint is executed according to the test data set to obtain a test report.
2. The testing method according to claim 1, characterized in that: Before creating a test plan using atomic use cases in an atomic use case library according to the scale and type of the test environment, the test method further includes: Creating the atomic use case library according to the operation log, wherein the atomic use cases in the atomic use case library include the operation entry identifier and driving parameters of the operation entry corresponding to the atomic use case; An automation script for the atomic use case is created to obtain an automation use case corresponding to the atomic use case.
3. The testing method according to claim 2, characterized in that: The step of creating the atomic use case library according to the operation log includes: Extracting an operation entry identifier of an operation entry according to the operation log; According to the operation steps corresponding to the operation entry, create an atomic use case corresponding to the operation entry identifier; The atomic use cases corresponding to the different operation entry identifiers are combined into the atomic use case library.
4. The testing method according to any one of claims 1 to 3, characterized in that: The operation log of the software system is parsed to obtain a test data set including multiple test data, including: Collecting operation logs of the software system corresponding to operating environments of different scales and types; Test data corresponding to operation environments of different scales and types are extracted from the operation log to obtain the test data set, wherein the test data includes an operation entry identifier and drive data of the operation entry.
5. The testing method according to claim 4, characterized in that: The extracting test data corresponding to operating environments of different scales and types from the operation log to obtain the test data set includes: Cleaning the operation log according to the time granularity, and obtaining the drive data corresponding to each time granularity; The drive data is aggregated according to the operation entry identifier and the corresponding time granularity to obtain the test data.
6. The testing method according to claim 5, characterized in that: After the drive data is aggregated according to the operation entry identifier and the corresponding time granularity to obtain the test data, the method further includes: Generating pre-operation data corresponding to the pre-operation in the operating environment; Generate post-operation data corresponding to the post-operation in the operation environment; The pre-operation data and the post-operation data are added to the test data set.
7. The testing method according to claim 4, characterized in that: The driving data includes at least one of the number of users, the number of operation objects, and the operation frequency.
8. The testing method according to any one of claims 1 to 3, characterized in that: The operation log of the software system is parsed to obtain a test data set including multiple test data, including: Extracting an operation entry identifier of an operation entry according to the operation log; The test data is arranged according to the operation entry identifier to obtain the test data set.
9. The testing method according to any one of claims 1 to 3, characterized in that: According to the scale and type of the test environment, the test plan is created using the atomic use cases in the atomic use case library, including: Determining user operation behavior in an operation environment corresponding to the test environment according to the scale and type of the test environment; According to the user operation behavior, the test plan is created using the atomic use cases in the atomic use case library.
10. The testing method according to claim 9, characterized in that: According to the user operation behavior, the test plan is created by using the atomic use case in the atomic use case library, including: Creating test sub-plans corresponding to different test time periods according to the user operation behavior, the test sub-plans including an operation entry identifier of an operation entry corresponding to the user operation behavior; Acquire a target atomic use case from the atomic use case library according to the operation entry identifier in the test sub-plan; Add the target atomic use case to the corresponding test sub-plan, wherein the test plan includes the test sub-plans corresponding to different test time periods.
11. The testing method according to claim 10, characterized in that: The step of automatically processing the test plan to obtain a test blueprint includes: According to the operation entry identifier of the target atomic use case in the test plan, obtaining the target automation use case corresponding to the target atomic use case; Determine the start time of scheduling the target automation use case according to the test plan; The target automation use case is arranged according to the start time to obtain the test blueprint.
12. The testing method according to claim 11, characterized in that: Executing the test blueprint according to the test data set to obtain a test report includes: Acquire target test data from the test data set according to the test environment and the test time period corresponding to the target automation use case in the test blueprint; According to the target test data and the start time corresponding to the target automation case, the target automation case is scheduled and executed to obtain the test report.
13. The testing method according to claim 12, characterized in that: Acquiring target test data from the test data set according to the test environment and the test time period corresponding to the target automation use case in the test blueprint, including: Acquire benchmark test data from the test data set according to the test environment and the test time period corresponding to the target automation use case in the test blueprint; The target test data is generated according to the benchmark test data.
14. An electronic device comprising: one or more processors; A memory having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the testing method according to any one of claims 1 to 13.
15. A computer readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the testing method according to any one of claims 1 to 13.