Software system test method, electronic device, and computer-readable medium

Abstract

Provided is a software system test method, comprising: analyzing an operation log of a software system to obtain a test data set comprising a plurality of pieces of test data; according to the scale and the type of a test environment, creating a test plan by means of using atomic cases in an atomic case library, the atomic case library comprising a plurality of atomic cases, and one operation item in the operation log corresponding to one atomic case; performing automatic processing on the test plan to obtain a test blueprint, the test blueprint comprising automation cases corresponding to the atomic cases in the test plan and a starting time for scheduling the automation cases; and executing the test blueprint according to the test data set to obtain a test report.

Description

Software system testing method, electronic device, and computer-readable medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to patent application No. 202310751961.5 filed with the China Patent Office on June 25, 2023, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to, but is not limited to, the field of software testing technology. Background Art

[0004] Software testing is an important part of the software development process. It is done by analyzing and testing the software before it is put into operation to confirm whether the function or performance of a program meets the development requirements or application requirements.

[0005] However, conventional software testing solutions still have defects and limitations when applied to network management systems, and the efficiency and cost of software testing need to be further optimized.

[0006] Summary of the Invention

[0007] Embodiments of the present disclosure provide a software system testing method, an electronic device, and a computer-readable medium.

[0008] In a first aspect, an embodiment of the present disclosure provides a testing method for a software system, comprising: parsing an operation log of the software system to obtain a test data set including multiple test data; creating a test plan using atomic use cases in an atomic use case library according to the scale and type of the test environment, wherein the atomic use case library includes multiple atomic use cases, and one operation item in the operation log corresponds to one atomic use case; automating the test plan to obtain a test blueprint, wherein the test blueprint includes automated use cases corresponding to the atomic use cases in the test plan and scheduling the start time of the automated use cases; executing the test blueprint according to the test data set to obtain a test report.

[0009] In a second aspect, an embodiment of the present disclosure provides an electronic device, comprising: one or more processors; a memory on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the testing method described in the first aspect of the embodiment of the present disclosure.

[0010] In a third aspect, an embodiment of the present disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the testing method described in the first aspect of the embodiment of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] FIG1 is a flow chart of a testing method according to an embodiment of the present disclosure;

[0012] FIG2 is a flow chart of another testing method according to an embodiment of the present disclosure;

[0013] FIG3 is a flow chart of some steps in another testing method according to an embodiment of the present disclosure;

[0014] FIG4 is a flow chart of some steps in another testing method according to an embodiment of the present disclosure;

[0015] FIG5 is a schematic diagram of the composition of an electronic device according to an embodiment of the present disclosure;

[0016] FIG6 is a schematic diagram of the composition of a computer-readable medium according to an embodiment of the present disclosure;

[0017] FIG7 is a schematic diagram of the architecture of a test system according to an embodiment of the present disclosure;

[0018] FIG8 is a schematic diagram of a testing process of a software system according to an embodiment of the present disclosure;

[0019] FIG9 is a schematic diagram of a testing process of another software system according to an embodiment of the present disclosure;

[0020] FIG10 is a schematic diagram of a testing process of another software system according to an embodiment of the present disclosure;

[0021] FIG11 is a schematic diagram of the architecture of another test system according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0022] To enable those skilled in the art to better understand the technical solutions of the present disclosure, the software system testing method, electronic device, and computer-readable medium provided by the present disclosure are described in detail below with reference to the accompanying drawings.

[0023] Example embodiments will be described more fully hereinafter with reference to the accompanying drawings, but the example embodiments may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the scope of this disclosure to those skilled in the art.

[0024] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.

[0025] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0026] The terms used herein are used only to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a," "an," and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "comprising" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements, and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or groups thereof is not excluded.

[0027] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless expressly defined as such herein.

[0028] Research has found that in some related technologies, test cases, test scripts, and test data are separated, and testing is performed through data-driven test script execution. The following are the drawbacks: First, there is little involvement in parsing product run logs to construct test cases and test data. Test data is generally generated based on pre-built test case data or existing test case data. Second, the separation of test scripts and test data is mostly controlled by test data, while parameters in the test script are rarely extracted as test input parameters, and the acquisition of parameters is tied to the time of test case execution. Third, automated test case backend interface testing is rarely decomposed to the granularity of atomic operations. A complete task test is achieved by sequentially combining atomic operations. Atomic operations are more common in front-end automated testing or keyword-driven testing. Fourth, there is little involvement in the simultaneous reuse of test cases and automated test cases. Different scenario tests can be achieved by simply modifying the test data. Generally, different test cases are designed for different test scenarios, and keywords are often reused when developing automated test cases.

[0029] In view of this, in a first aspect, referring to FIG1 , an embodiment of the present disclosure provides a method for testing a software system, including:

[0030] S1. Analyze the operation log of the software system to obtain a test data set including multiple test data;

[0031] S2. Creating a test plan using atomic use cases in an atomic use case library based on the scale and type of the test environment, wherein the atomic use case library includes a plurality of the atomic use cases, and each operation item in the operation log corresponds to one atomic use case;

[0032] S3. Automating the test plan to obtain a test blueprint, wherein the test blueprint includes automated use cases corresponding to the atomic use cases in the test plan and the start time of the automated use cases;

[0033] S4. Execute the test blueprint according to the test data set to obtain a test report.

[0034] The embodiment of the present disclosure does not impose any special limitation on the test data set. In some embodiments, the test data set is in the form of a test data driven file.

[0035] In the disclosed embodiments, an operation item in an operation log is an operation record. The disclosed embodiments define an operation record in an operation log as an atomic operation, and an atomic operation corresponds to an atomic use case. In the disclosed embodiments, operation tasks are broken down into atomic operations. Independent operation tasks can be completed through a single atomic operation, while dependent operation tasks are completed sequentially through multiple, interrelated atomic operations.

[0036] In the embodiment of the present disclosure, the automated use case is an automated script of the atomic use case, and one automated use case corresponds to one atomic use case.

[0037] In an embodiment of the present disclosure, test environments of different scales and types correspond to actual operating environments of different scales and types. In an embodiment of the present disclosure, creating a test plan refers to the process of formulating a plan for executing atomic use cases in test environments of different scales and types. In some embodiments, when creating a test plan, the corresponding atomic use case is selected from the atomic use case library based on the user operation behavior in the actual operating environment corresponding to the test environment. In an embodiment of the present disclosure, the user operation behavior represents different time periods, the user's operation of the software system's functions (i.e., operation entry) and the number of operations, etc., and the embodiment of the present disclosure does not make special restrictions on this. In some embodiments, a test plan is created using the atomic use cases in the atomic use case library, that is, based on the correspondence between the atomic operations performed by the atomic use case and the user's operation behavior, the atomic use case is selected and an execution plan is formulated.

[0038] In the embodiment of the present disclosure, creating a test blueprint refers to the process of automating the test plan, and automatically executing the test plan by determining the automation use cases corresponding to the test plan and scheduling the start time of the automation use cases.

[0039] In the embodiment of the present disclosure, when creating different test plans, the atomic use cases in the atomic use case library can be reused; when creating test blueprints according to different test plans, the automated use cases can be reused.

[0040] In the embodiment of the present disclosure, when executing a test blueprint according to a test data set, test data corresponding to the test environment is obtained from the test data set according to the test environment, thereby implementing testing of different test scenarios.

[0041] The embodiments of the present disclosure do not impose any special restrictions on the software system. For example, the software system is a telecommunications network management product.

[0042] The software system testing method provided by the embodiment of the present disclosure generates a test data set by parsing the operation log of the software system. The test data in the test data set is more realistic, making the test scenario closer to the actual software operating environment. When creating a test plan, the atomic use cases in the atomic use case library can be reused, and when creating a test blueprint, the automated use cases can be reused, which can effectively save the maintenance costs of atomic use cases and automated use cases. The test plan and test blueprint can be automatically created according to the test environment, and the test blueprint can be automatically executed according to the test data set to output a test report, which is conducive to improving testing efficiency.

[0043] In the disclosed embodiment, when testing a software system, the already created atomic use case library and automated use case can be reused. If the atomic use case library and automated use case do not exist, the atomic use case library and automated use case can also be created based on the operation log.

[0044] Accordingly, in some embodiments, referring to FIG. 2 , the testing method further includes:

[0045] S5. Create the atomic use case library according to the operation log, wherein the atomic use cases in the atomic use case library include the operation entry identifier and driving parameters of the operation entry corresponding to the atomic use case;

[0046] S6. Create an automation script for the atomic use case and obtain an automation use case corresponding to the atomic use case.

[0047] In the disclosed embodiments, an operation entry corresponds to a business logic of the software system; in the operation log, an operation entry may correspond to one or more operation records. In the disclosed embodiments, a business logic item corresponds to an atomic use case, and operation records corresponding to the same operation entry in the operation log correspond to the same atomic use case.

[0048] The embodiment of the present disclosure does not impose any special limitation on the operation entry identifier. For example, the operation entry identifier is the name of the operation entry.

[0049] In the disclosed embodiments, the driving parameters of an atomic use case are variables. The disclosed embodiments do not specifically limit the driving parameters. For example, the driving parameters include parameters corresponding to the number of users, the number of operation objects, the operation frequency, etc. In some embodiments, the atomic use case also includes the operation time, the operation steps, etc.

[0050] In the embodiment of the present disclosure, the operation entry identifier of the automation use case is consistent with the operation entry identifier of the corresponding atomic use case, and the driving parameters of the automation use case are consistent with the driving parameters of the corresponding atomic use case.

[0051] In the embodiment of the present disclosure, when creating an automation script for an atomic use case and obtaining the corresponding automation use case, the atomic use cases of independent operation tasks are placed in the same test suite; the atomic use cases of non-independent operation tasks that are mutually related are placed in a test suite; the atomic use cases in the same test suite are executed serially, and the atomic use cases in different test suites can be executed in parallel.

[0052] The embodiments of the present disclosure do not impose any special limitation on how to create an atomic use case library.

[0053] In some embodiments, creating the atomic use case library according to the operation log includes:

[0054] Extracting an operation entry identifier of an operation entry according to the operation log;

[0055] According to the operation steps corresponding to the operation entry, create an atomic use case corresponding to the operation entry identifier;

[0056] The atomic use cases corresponding to the different operation entry identifiers are combined into the atomic use case library.

[0057] The embodiments of the present disclosure do not impose any special limitation on how to generate automated use cases corresponding to atomic use cases.

[0058] In the embodiment of the present disclosure, the operation entry identifier of the automation use case is consistent with the operation entry identifier of the corresponding atomic use case, and the driving parameters of the automation use case are consistent with the driving parameters of the corresponding atomic use case.

[0059] The embodiments of the present disclosure do not impose any special limitation on how to establish the test data set.

[0060] In some embodiments, the operation log of the software system is parsed to obtain a test data set including multiple test data, including:

[0061] Collecting operation logs of the software system corresponding to operating environments of different scales and types;

[0062] Test data corresponding to operation environments of different scales and types are extracted from the operation log to obtain the test data set, wherein the test data includes an operation entry identifier and driver data of an operation entry.

[0063] The embodiment of the present disclosure does not impose any special restrictions on the driving data. For example, the driving data includes data such as the number of users, the number of operation objects, and the frequency of operations.

[0064] In some embodiments, referring to FIG. 3 , extracting test data corresponding to operating environments of different scales and types from the operation log to obtain the test data set includes:

[0065] S11, cleaning the operation log according to time granularity, and obtaining the drive data corresponding to each time granularity;

[0066] S12: Summarize the driving data according to the operation entry identifier and the corresponding time granularity to obtain the test data.

[0067] In some embodiments, by cleaning and summarizing the operation logs, an atomic operation analysis report including operation time, operation steps, operation entry, number of users, number of operation objects, operation frequency, original details, etc. is obtained, and a test data set consisting of multiple test data is further obtained.

[0068] In some embodiments, referring to FIG. 3 , extracting test data corresponding to operating environments of different scales and types from the operation log to obtain the test data set further includes:

[0069] S13, generating pre-operation data corresponding to the pre-operation in the operating environment;

[0070] S14, generating post-operation data corresponding to the post-operation in the operating environment;

[0071] S15: Add the pre-operation data and the post-operation data to the test data set.

[0072] In the disclosed embodiment, since the atomic use case for coherent operations cannot guarantee that the extracted operation data are all extracted from the beginning of the entire business logic, supplementing the preceding operation data can supplement the missing atomic operation data to ensure data completeness.

[0073] In the embodiment of the present disclosure, supplementing post-operation data refers to supplementing and deleting task-type operations so that the entire environment returns to its original state after the test is executed.

[0074] In some embodiments, the driving data includes at least one of the number of users, the number of operation objects, and the operation frequency.

[0075] The disclosed embodiment also supports manual construction and arrangement of data-driven test files based on operation entries.

[0076] Accordingly, in some embodiments, the operation log of the software system is parsed to obtain a test data set including multiple test data, including:

[0077] Extracting an operation entry identifier of an operation entry according to the operation log;

[0078] The test data is arranged according to the operation entry identifier to obtain the test data set.

[0079] In some embodiments, referring to FIG. 4 , a test plan is created using atomic use cases in an atomic use case library based on the scale and type of the test environment, including:

[0080] S21. Determine, based on the scale and type of the test environment, a user operation behavior in an operating environment corresponding to the test environment;

[0081] S22. Create a test plan based on the user operation behavior using the atomic use cases in the atomic use case library.

[0082] For example, the operation logs of the software system corresponding to operating environments of different scales and types are collected. For example, the operating environments of different scales and types include three different types of operating environments: env1 (scale 1), env2 (scale 2), and env3 (scale 3). Each operating environment collects the operation logs of the first quarter and the second quarter; corresponding to the operating environments env1, env2, and env3, test environments test_env1 (scale 1), test_env2 (scale 2), and test_env3 (scale 3) are set up; the test data obtained by parsing the operation log of env1 in the first quarter is selected to create test plan 1 in test_env1; the test data obtained by parsing the operation log of env2 in the first quarter is selected to create test plan 2 in test_env2; the test data obtained by parsing the operation log of env1 in the second quarter is selected to create test plan 3 in test_env1; the test data obtained by parsing the operation log of env2 in the second quarter is selected to create test plan 4 in test_env2; and so on.

[0083] The embodiment of the present disclosure does not specifically limit how to create a test plan using atomic use cases in an atomic use case library based on user operation behaviors.

[0084] In some embodiments, creating the test plan based on the user operation behavior using the atomic use case in the atomic use case library includes:

[0085] Creating test sub-plans corresponding to different test time periods according to the user operation behavior, the test sub-plans including an operation entry identifier of an operation entry corresponding to the user operation behavior;

[0086] Obtaining a target atomic use case from the atomic use case library according to the operation entry identifier in the test sub-plan;

[0087] The target atomic use case is added to the corresponding test sub-plan, wherein the test plan includes the test sub-plans corresponding to different test time periods.

[0088] In some embodiments, referring to FIG. 4 , the test plan is automatically processed to obtain a test blueprint, including:

[0089] S31. Acquire a target automation use case corresponding to the target atomic use case according to the operation entry identifier of the target atomic use case in the test plan;

[0090] S32. Determine the start time of scheduling the target automation use case according to the test plan;

[0091] S33. Arrange the target automation use case according to the startup time to obtain the test blueprint.

[0092] In some embodiments, referring to FIG. 4 , executing the test blueprint according to the test data set to obtain a test report includes:

[0093] S41. Obtain target test data from the test data set according to the test environment and the test time period corresponding to the target automation use case in the test blueprint;

[0094] S42: Schedule and execute the target automation use case according to the target test data and the start time corresponding to the target automation use case to obtain the test report.

[0095] In the embodiment of the present disclosure, the target test data is passed into the automation use case as an input parameter of the target automation use case.

[0096] In the disclosed embodiments, by modifying or adjusting the target test data, different scenarios and different purposes can be tested, i.e., data-driven testing can be achieved. For example, setting the number of users, number of operation objects, and frequency to 1 can achieve functional testing; ensuring that the number of users, number of operation objects, and frequency are consistent with the external product system operation log can achieve user behavior testing; setting the number of users, number of operation objects, and frequency to the maximum can achieve system boundary testing, etc.

[0097] In an embodiment of the present disclosure, if there is test data corresponding to the test environment in the test data set, the test data corresponding to the test environment in the test data set is directly obtained as the target test data; if there is no test data corresponding to the test environment in the test data set, the target test data corresponding to the test environment is generated based on the benchmark test data.

[0098] Accordingly, in some embodiments, obtaining target test data from the test data set according to the test environment and the test time period corresponding to the target automation use case in the test blueprint includes:

[0099] Obtaining benchmark test data from the test data set according to the test environment and the test time period corresponding to the target automation use case in the test blueprint;

[0100] The target test data is generated according to the benchmark test data.

[0101] In a second aspect, referring to FIG. 5 , an embodiment of the present disclosure provides an electronic device, comprising:

[0102] One or more processors 101;

[0103] A memory 102 storing one or more programs, which, when executed by one or more processors, enables the one or more processors to implement the testing method described in the first aspect of the embodiment of the present disclosure;

[0104] One or more I / O interfaces 103 are connected between the processor and the memory and are configured to implement information exchange between the processor and the memory.

[0105] Among them, the processor 101 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 102 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and flash memory (FLASH); the I / O interface (read-write interface) 103 is connected between the processor 101 and the memory 102, and can realize information interaction between the processor 101 and the memory 102, including but not limited to a data bus (Bus), etc.

[0106] In some embodiments, the processor 101 , the memory 102 , and the I / O interface 103 are connected to each other via a bus 104 , and further connected to other components of the computing device.

[0107] In a fourth aspect, referring to FIG. 6 , an embodiment of the present disclosure provides a computer-readable medium having a computer program stored thereon, which implements the testing method described in the first aspect of the embodiment of the present disclosure when the program is executed by a processor.

[0108] In order to enable those skilled in the art to more clearly understand the technical solutions provided by the embodiments of the present disclosure, the technical solutions provided by the embodiments of the present disclosure are described in detail below through exemplary embodiments:

[0109] First Exemplary Embodiment

[0110] In this embodiment, the overall architecture of the test system is shown in Figure 7. In this embodiment, the collected software system operation logs or desensitized operation logs meet data protection requirements. The operation logs are input into the test system, and the data-driven test files are stored in the test system's database. The test system and the different test environments of the software system (test environment 1, test environment 2, ..., test environment N) can communicate via the network. Through external web services (web browsers), users can create test plans, view test progress, and generate test reports.

[0111] The software architecture of the test system in this embodiment mainly includes the following parts:

[0112] Generate a data-driven test file: users obtain the operation logs of outsourced products of different scales into the test system of this embodiment, parse the operation logs of the software system, extract and summarize all different operation entry names in the product; the test system cleans the operation logs according to the time granularity (1h), and obtains the operation entry, user, number of operation objects, and frequency within each time granularity; the extracted data is extracted and summarized to obtain an atomic operation analysis report containing operation time, function / operation, product, APP, operation entry, number of users, number of operation objects, frequency, and original details; the atomic operation analysis report is then supplemented with verification to finally obtain a complete executable data-driven test file.

[0113] Build an atomic use case library: The test system extracts and summarizes the names of all different operation entry points in the product; build an atomic use case library based on the operation entry points and the corresponding product functions / operations; each atomic use case is named "atomic use case_operation entry"; the operation steps describe M users, select N number of operation objects, the specific operations performed, and the expected results; the number of users M, the number of operation objects N, and the frequency of execution F are all open parameters, not fixed values.

[0114] Develop atomic automated use cases: When developing automated use cases, atomic use cases that operate independently are placed separately in the same test suite; atomic use cases that are related to each other are placed in a test suite and saved in the order of atomic operations; when the use cases are executed, the use cases in the same test suite can only be executed serially, while different test suites can be executed in parallel; the three parameters required for use case execution: the number of users M, the number of operation objects N, and the frequency F are used as variables, and the incoming data needs to be dynamically loaded according to the execution time and execution environment.

[0115] Automatically create test plans and test blueprints: select the data-driven file for the corresponding environment based on the environment and scale (if there is no test file for the corresponding environment or scale, select the benchmark data test file and automatically generate the data-driven file for the corresponding scale); obtain the use case identifier based on the operation entry and the use case title in the atomic use case library; create test plans for corresponding use cases based on different operation times; create test blueprints based on specific operation times and test plans.

[0116] Blueprint scheduling use case execution: When the automated use case is executed, the data-driven file of the corresponding environment is obtained according to the environment in which the use case is executed; according to the execution time and the operation entry in the use case operation name, the number of users M, number of operation objects N, frequency F, and other parameters P required for the automated use case at the corresponding time point are obtained from the data-driven file; when the atomic use case is executed, the number of users M, number of operation objects N, frequency F, and other parameters P are passed in as parameters of the automated use case to execute the use case.

[0117] Output test report: The test blueprint is executed as planned, the test results are summarized, and the test report is printed.

[0118] Second Exemplary Embodiment

[0119] This embodiment is based on the test system of the first exemplary embodiment, and the overall architecture of the test system is shown in Figure 8. The process of testing the software system is shown in Figure 9, including:

[0120] Generate data-driven test files. Phase 1: Users collect system operation logs (Field Operation Log 1, Field Operation Log 2, Field Operation Log 3) from various types of outbound environments (Test Scenario 1: Busy Hour Replay; Test Scenario 2: Daily Replay; Test Scenario 3: Weekly Replay) from different locations of network management products and upload them to this system. Data is extracted from each operation log, separating the operation behavior (business logic) from the parameter data (driver data). Each operation log can be represented as an operation entry + driver data. Driver data includes the number of users, the number of operation objects, the frequency, and other parameters. An operation item in the operation log record is defined as an atomic operation. An atomic operation corresponds to an atomic use case. A complete user operation can be completed through one or more atomic operations. Different operation entry name entries in the product are extracted and summarized from all field log records to build an atomic use case library. Phase 2: Generate atomic operation analysis reports (Atomic Operation Analysis Report 1, Atomic Operation Analysis Report 2, ..., Atomic Operation Analysis Report N) based on the test objectives and test scenarios. This embodiment uses a test environment to simulate the maximum stress scenario of real field user behavior to illustrate the steps of generating an atomic operation analysis report: combining user commuting time and data analysis, the 7-hour busy time period each day (9:00-12:00; 14:00-18:00) is defined as busy time. Combined with Figure 10, the specific steps are as follows: 1. Obtain the operation logs of different sites; 2. Summarize the operation logs by granularity (such as hourly level), clean the original operation logs by busy hour, and obtain the operation entry, user, number of operation objects, frequency (frequency within 1 hour), and other parameters within each busy hour; 3. Extract the 7 busy hour time periods (9:00-12:00, 14:00-18:00) in step (2), and summarize the number of different operation entries in each busy hour of each day; 4. Extract the list of busy hour operation entries with the most operation items in each busy hour in step 3, and summarize the operation entry lists for the 7 busy hours; 5. Expand the busy hour data summarized in step 4 according to the operation entry, and take the number of users (different user names) of each operation item with the most users in the hourly playback as the number of concurrent user operations, the number of operation objects with the most operation objects operated by a single user in this hour as the operation object, and the average value of this hour (the sum of the frequency / the number of users) as the frequency. Finally, an atomic operation analysis report is obtained, which includes operation time, function / operation, product, APP, operation entry, number of users, number of operation objects, frequency, and original details; the test system supports cleaning operation logs according to different time granularities (1h, 15min, etc.).The third stage: Supplement the front and back atomic operation data. When extracting busy hour data in the second stage, the atomic use case for coherent operations cannot guarantee that all extracted operation data are extracted from the beginning of the entire business logic. Therefore, the missing atomic operation data is supplemented before the busy hour to ensure that the data file is complete and executable throughout the busy hour. After the busy hour, the operations of the deletion task class are supplemented. After the data-driven test is executed, the entire environment is restored to its original state. In this stage, the atomic operations to be supplemented are first manually sorted and verified, and the rules for supplementing the atomic operation data are summarized and organized (supplemented data rule set). The fourth stage: In the previous stage, the rules manually summarized and sorted are automatically supplemented and verified through system coding, and the test system outputs the data-driven test files for the corresponding environment (test data file 1, test data file 2, ..., test data file N). Furthermore, this test system supports manual construction and arrangement of data-driven test files based on operation entries.

[0121] Build an atomic use case library. Phase 1: Define an operation item in the operation log record as an atomic operation. One atomic operation corresponds to an atomic use case. A complete user operation can be completed through one atomic operation or multiple atomic operations. For example: 1. If the user needs to add a new user to the system, he can do so through an atomic operation: product name-function name-add user, and enter different operation data to complete it; 2. If the user creates a task and exports data, he can do so through two successive atomic operations: product name-function name-new task, product name-function name-export data, and enter different operation data to complete it. Phase 2: The test system extracts and summarizes all the different operation entry names in the product; based on the operation entry and the corresponding product function / operation, an atomic use case library is built; each atomic use case name is named: "atomic use case_operation entry"; the operation steps describe the operation process in the form of M users, N number of operation objects, and the number of times the specific function operation is performed, as well as the expected effect to be achieved by the operation steps; the number of users M, the number of operation objects N, and the frequency of execution F are all open values, not fixed values.

[0122] Develop automated use cases. Atomic use cases are used when developing automated use cases. Use cases within the same test suite can only be executed serially, while cases in different test suites can be executed in parallel. The three parameters required for use case execution—the number of users M, the number of operation objects N, and the frequency F—are variables, and data needs to be dynamically loaded based on the execution time and environment.

[0123] Create test plans and test blueprints. Select the data-driven files for the environment and scale. Obtain the use case identifier by mapping the operation entry with the use case title in the atomic use case library. Create test plans for the use cases based on different operation times. Create test blueprints based on the specific operation time and test plan.

[0124] Use case scheduling. Referring to Figure 11, when an automated use case is executed, the data-driven file for the corresponding environment is obtained based on the environment in which the use case is executed. Based on the execution time and the operation entry in the use case operation name, the number of users M, number of operation objects N, and frequency F required for the automated use case at the corresponding time point are obtained from the data-driven file. When an atomic use case is executed, the number of users M, number of operation objects N, and frequency F are used as input parameters for the automated use case, and the use case is executed.

[0125] Output test report: The test blueprint is executed as planned, the test results are summarized, and the test report is printed.

[0126] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As is well known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable, and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0127] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for purposes of limitation. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly indicated, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, it will be understood by those skilled in the art that various changes in form and detail may be made without departing from the scope of the present disclosure as set forth in the appended claims.

Claims

1. A software system testing method, comprising: Parse the operation log of the software system to obtain a test data set including multiple test data; Creating a test plan using atomic use cases in an atomic use case library according to the scale and type of the test environment, wherein the atomic use case library includes a plurality of the atomic use cases, and one operation item in the operation log corresponds to one atomic use case; Automating the test plan to obtain a test blueprint, wherein the test blueprint includes automated use cases corresponding to the atomic use cases in the test plan and a start time for scheduling the automated use cases; The test blueprint is executed according to the test data set to obtain a test report.

2. The testing method according to claim 1, wherein: Before creating a test plan using atomic use cases in an atomic use case library according to the scale and type of the test environment, the test method further includes: Creating the atomic use case library according to the operation log, wherein the atomic use cases in the atomic use case library include the operation entry identifier and driving parameters of the operation entry corresponding to the atomic use case; An automation script for the atomic use case is created to obtain an automation use case corresponding to the atomic use case.

3. The testing method according to claim 2, wherein: The step of creating the atomic use case library according to the operation log includes: Extracting an operation entry identifier of an operation entry according to the operation log; According to the operation steps corresponding to the operation entry, create an atomic use case corresponding to the operation entry identifier; The atomic use cases corresponding to the different operation entry identifiers are combined into the atomic use case library.

4. The test method according to any one of claims 1 to 3, wherein: The operation log of the software system is parsed to obtain a test data set including multiple test data, including: Collecting operation logs of the software system corresponding to operating environments of different scales and types; Test data corresponding to operation environments of different scales and types are extracted from the operation log to obtain the test data set, wherein the test data includes an operation entry identifier and drive data of the operation entry.

5. The testing method according to claim 4, wherein: The extracting test data corresponding to operating environments of different scales and types from the operation log to obtain the test data set includes: Cleaning the operation log according to the time granularity, and obtaining the drive data corresponding to each time granularity; The drive data is aggregated according to the operation entry identifier and the corresponding time granularity to obtain the test data.

6. The testing method according to claim 5, wherein: After the drive data is aggregated according to the operation entry identifier and the corresponding time granularity to obtain the test data, the method further includes: Generating pre-operation data corresponding to the pre-operation in the operating environment; Generate post-operation data corresponding to the post-operation in the operation environment; The pre-operation data and the post-operation data are added to the test data set.

7. The testing method according to claim 4, wherein: The driving data includes at least one of the number of users, the number of operation objects, and the operation frequency.

8. The test method according to any one of claims 1 to 3, wherein: The operation log of the software system is parsed to obtain a test data set including multiple test data, including: Extracting an operation entry identifier of an operation entry according to the operation log; The test data is arranged according to the operation entry identifier to obtain the test data set.

9. The testing method according to any one of claims 1 to 3, wherein: According to the scale and type of the test environment, the test plan is created using the atomic use cases in the atomic use case library, including: Determining user operation behavior in an operation environment corresponding to the test environment according to the scale and type of the test environment; According to the user operation behavior, the test plan is created using the atomic use cases in the atomic use case library.

10. The testing method according to claim 9, wherein: According to the user operation behavior, the test plan is created by using the atomic use case in the atomic use case library, including: Creating test sub-plans corresponding to different test time periods according to the user operation behavior, the test sub-plans including an operation entry identifier of an operation entry corresponding to the user operation behavior; Acquire a target atomic use case from the atomic use case library according to the operation entry identifier in the test sub-plan; Add the target atomic use case to the corresponding test sub-plan, wherein the test plan includes the test sub-plans corresponding to different test time periods.

11. The testing method according to claim 10, wherein: The step of automatically processing the test plan to obtain a test blueprint includes: According to the operation entry identifier of the target atomic use case in the test plan, obtaining the target automation use case corresponding to the target atomic use case; Determine the start time of scheduling the target automation use case according to the test plan; The target automation use case is arranged according to the start time to obtain the test blueprint.

12. The testing method according to claim 11, wherein: Executing the test blueprint according to the test data set to obtain a test report includes: Acquire target test data from the test data set according to the test environment and the test time period corresponding to the target automation use case in the test blueprint; According to the target test data and the start time corresponding to the target automation case, the target automation case is scheduled and executed to obtain the test report.

13. The testing method according to claim 12, wherein: Acquiring target test data from the test data set according to the test environment and the test time period corresponding to the target automation use case in the test blueprint, including: Acquire benchmark test data from the test data set according to the test environment and the test time period corresponding to the target automation use case in the test blueprint; The target test data is generated according to the benchmark test data.

14. An electronic device comprising: one or more processors; A memory having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the testing method according to any one of claims 1 to 13.

15. A computer readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the testing method according to any one of claims 1 to 13.

Images