Test case processing method and device, electronic equipment and storage medium

Abstract

This application discloses a test case processing method, apparatus, electronic device, and storage medium. The method includes: acquiring first business information; acquiring at least one test case matching the first business information from a test case library; performing tests using the at least one test case; when the first test case fails to execute, determining at least one feature from the log information generated by executing the first test case to obtain a first set; selecting at least one failure reason matching the feature in the first set from a second set; the second set contains at least one test case failure reason. The solution provided by this application can automatically select test cases matching the business information from the test case library based on the business information, and can automatically analyze the causes of failed test cases, reducing manual processing time and thus improving testing efficiency.

Description

Technical Field

[0001] This application relates to the field of automated testing technology, and in particular to a test case processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the development of artificial intelligence technology, intelligent voice interaction products are becoming increasingly common, such as smart speakers and robotic customer service. For these products, traditional testing methods require building a large set of test cases to verify various functions and assess the product's level of intelligence in order to ensure the reliability of test results. However, when code changes necessitate regression testing, the large number of test cases in the set leads to low testing efficiency.

[0003] In related technologies, predictive models for test cases are built to select test cases for regression verification, achieving accurate testing of product functions with a smaller number of test cases and improving test case selection efficiency. However, testers still need to spend a significant amount of time analyzing different test results before they can improve and optimize product functions based on the analysis. This results in low analysis efficiency during testing, leading to low overall testing efficiency.

[0004] Therefore, there is currently no effective solution for improving the overall efficiency of product testing. Summary of the Invention

[0005] To address the related technical issues, embodiments of this application provide a test case processing method, apparatus, electronic device, and storage medium.

[0006] The technical solution of this application embodiment is implemented as follows:

[0007] This application provides a test case processing method, including:

[0008] Obtain the first business information;

[0009] Obtain at least one test case from the test case library that matches the first business information;

[0010] Use at least one of the test cases to perform the test;

[0011] When the execution of the first test case in the at least one test case fails, at least one feature is determined from the log information generated by the execution of the first test case to obtain a first set;

[0012] Select at least one failure reason from the second set that matches a feature in the first set; the second set contains at least one test case failure reason; wherein...

[0013] Based on the first business information and the test, a profile is created for at least one of the matched test cases.

[0014] In the above scheme, determining at least one feature from the log information generated during the execution of the first test case includes:

[0015] Re-test using the first test case;

[0016] Identify at least one feature from the log information generated during the first execution and re-execution of the first test case.

[0017] In the above scheme, selecting at least one failure reason from the second set that matches the features in the first set includes:

[0018] Based on a similarity algorithm, at least one failure reason that matches the features in the first set is selected from the second set.

[0019] The method in the above scheme further includes:

[0020] Select at least one profile of a failed use case from the use case profile library;

[0021] The second set is determined based on the profile of at least one selected failed use case.

[0022] The method in the above scheme further includes:

[0023] Based on the similarity algorithm, at least one test case that failed to execute and matches the profile of the first test case is selected from the test case library to obtain a third set; and based on the similarity algorithm, at least one test case that successfully executed and matches the profile of the first test case is selected from the test case library to obtain a fourth set.

[0024] Based on the third and fourth sets, it is determined whether the first test case has a code defect, and the determination result is obtained;

[0025] Based on the judgment result, update the profile of the first test case.

[0026] In the above scheme,

[0027] The step of determining whether the first test case has a code defect based on the third and fourth sets includes:

[0028] The code path coverage blocks of the code in the third and fourth sets are determined;

[0029] Based on the code path coverage block of the code in the third set, the code difference between the first test case and the code in the third set is determined to obtain the fifth set. Based on the code path coverage block of the code in the fourth set, the code intersection between the first test case and the code in the fourth set is determined to obtain the sixth set.

[0030] Determine whether the fifth and sixth sets involve code changes;

[0031] When it is determined that the fifth and sixth sets involve code changes, it is determined that the first test case has a code defect.

[0032] The method in the above scheme further includes:

[0033] Once the profile is complete, update the test case profile to the test case profile library.

[0034] This application also provides a test case processing device, including:

[0035] The first acquisition unit is used to acquire the first business information;

[0036] The second acquisition unit is used to acquire at least one test case that matches the first business information from the test case library;

[0037] A test unit, used to perform testing using the at least one test case;

[0038] A determining unit is configured to determine at least one feature from the log information generated during the execution of the first test case when the first test case fails to execute in the at least one test case, thereby obtaining a first set.

[0039] A selection unit is configured to select at least one failure reason from a second set that matches a feature in the first set; the second set contains at least one test case failure reason; wherein...

[0040] A profiling unit is used to create a profile of at least one matched test case based on the first business information and the test.

[0041] This application also provides an electronic device, including: a processor and a memory for storing a computer program capable of running on the processor.

[0042] Wherein, the processor is used to execute the steps of any of the test case processing methods when running the computer program.

[0043] This application also provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of any of the test case processing methods.

[0044] The test case processing method, apparatus, electronic device, and storage medium provided in this application embodiment obtain first business information, then retrieve at least one test case matching the first business information from a test case library; perform testing using the at least one test case; when the first test case in the at least one test case fails to execute, determine at least one feature from the log information generated by executing the first test case to obtain a first set; select at least one failure reason matching the feature in the first set from a second set; the second set contains at least one test case failure reason; wherein, a profile is created for the at least one matching test case based on the first business information and the test. The technical solution provided in this application embodiment, based on the obtained business information, can accurately filter out test cases to be executed that match the business information from the test case library, improving the efficiency of test case filtering. Simultaneously, it can automatically analyze the failure reasons when test case execution fails, reducing the time for manual analysis and troubleshooting, thus improving the overall efficiency of testing and also enhancing the level of testing automation. Attached Figure Description

[0045] Figure 1 This is a schematic flowchart illustrating the method for processing test cases in an embodiment of this application.

[0046] Figure 2 This is a structural diagram illustrating the test cases in the embodiments of this application;

[0047] Figure 3 A schematic diagram of the system structure for illustrating test cases in the application embodiments of this application;

[0048] Figure 4 This is a schematic diagram of the method for intelligent recommendation of test cases in an application embodiment of this application;

[0049] Figure 5 This is a schematic diagram of the method for attributing test case failures in an application embodiment of this application;

[0050] Figure 6 This is a flowchart illustrating the method for locating defects in test case code in an application embodiment of this application.

[0051] Figure 7 This is a schematic diagram of the test case processing device structure in an embodiment of this application;

[0052] Figure 8 This is a schematic diagram of the electronic device structure according to an embodiment of this application. Detailed Implementation

[0053] The present application will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0054] Before providing a further detailed description of this application, the nouns and terms used in the embodiments of this application will be explained, and the nouns and terms used in the embodiments of this application shall be interpreted as follows.

[0055] (1) Regression testing: For software products, when the code corresponding to a certain function of the product is modified, test cases need to be used to repeatedly test the function of the product to determine whether the modified content introduces new defects or causes errors in the code corresponding to other functions, affecting the original function of the product.

[0056] (2) Edit distance: This refers to the number of editing operations required to transform one object into another, such as insertion, replacement, and deletion. Generally speaking, the smaller the edit distance, the higher the similarity between the two objects; the larger the edit distance, the lower the similarity between the two objects.

[0057] (3) Term Frequency-Inverse Document Frequency (TF-IDF) technology: a weighted technique used for information retrieval and data mining that can determine the importance of words in a text relative to the text itself.

[0058] When using test cases to perform functional testing on a software product, bug fixes or version updates may be necessary based on product defects or changes in requirements. This process involves modifications to the software code. In such cases, it is necessary to perform regression testing on the software product's functionality using the already executed test cases to ensure that the modifications have not introduced new errors and have not affected other functions of the software product.

[0059] In this process, traditional testing methods build large-scale test datasets. When performing regression testing, a large number of test cases are used to test the product's functionality. However, these test cases may contain inaccurate regression test results, leading to low testing efficiency. Therefore, improving testing efficiency when regression testing is required has become an urgent problem to be solved.

[0060] To address the aforementioned issues, a training sample set can be constructed using historical test cases, entity elements within the software product, and defects, thereby establishing a predictive model for test cases and defects. Then, by inputting the entity element to be tested, the predicted test cases are obtained, and the test cases to be executed are determined based on these predictions, thus enabling accurate testing of the software product. In this approach, the test cases to be executed are selected based on historical test cases and defect models. Compared to traditional testing methods, this method uses a smaller number of test cases to achieve accurate testing of the software product, improving the efficiency of test case selection.

[0061] However, when using test cases to test software products, testers also need to manually troubleshoot and analyze the reasons for failures and locate defective code when test cases fail. This consumes a lot of time, resulting in low test analysis efficiency and consequently low overall test efficiency.

[0062] Overall, current testing methods still suffer from low overall efficiency and insufficient automation.

[0063] Based on this, in various embodiments of this application, test cases matching the business information are retrieved from the test case library and tested according to the business information. Features of failed test cases are extracted to analyze the reasons for their failure. This not only automatically filters out test cases to be executed, improving the efficiency of test screening, but also automatically attributes the causes of failed test cases, reducing manual processing time and improving the overall efficiency and automation level of testing.

[0064] This application provides a test case processing method, such as... Figure 1 As shown, applied to an electronic device, the method includes:

[0065] Step 101: Obtain the first business information;

[0066] Step 102: Obtain at least one test case that matches the first business information from the test case library;

[0067] Step 103: Perform testing using at least one of the test cases;

[0068] Step 104: When the execution of the first test case in the at least one test case fails, determine at least one feature from the log information generated by executing the first test case to obtain a first set;

[0069] Step 105: Select at least one failure reason from the second set that matches a feature in the first set; the second set contains at least one test case failure reason;

[0070] Specifically, a profile is created for at least one matched test case based on the first business information and the test.

[0071] In practical applications, before step 101, N test cases can be pre-constructed and stored in the test case library; where N is an integer greater than 1; the number of N can be set as needed, such as 500.

[0072] In practical applications, test cases can be automatically constructed using regular expressions (REs) or by manual input. This application does not limit the specific methods used.

[0073] In practical applications, when constructing N test cases, corresponding business requirement description information and tags can be set for each test case; the business requirement description information may include: title, test case steps, business function description, test case verification points, and expected results; the tags are used to indicate the category of the test case.

[0074] Then, for the test case library, a word segmentation algorithm can be used to analyze and extract features from the business requirement description information of each test case in the test case library to obtain a feature pool corresponding to the test case library. For example, the FMM algorithm is used to perform corpus analysis on the business requirement description information of each test case, extract features corresponding to the test case, and store them in the feature pool; wherein, the features in the feature pool have a mapping relationship with the test cases.

[0075] When regression testing is required, the first business information can be obtained by receiving manually input business information from testers; the first business information may include information such as business function name and program interface name.

[0076] In practical applications, in step 102, after obtaining the first business information, target features corresponding to the first business information can be extracted first. Then, using a similarity algorithm, the extracted target features are compared with each feature in the feature pool to obtain the similarity between the target features and each feature. Based on the obtained similarity, at least one feature is determined from the feature pool. Then, based on the mapping relationship between features and test cases, at least one test case matching the first business information is determined, and at least one test case matching the first business information is obtained from the test case library.

[0077] The similarity algorithm may include the Naive Bayes algorithm.

[0078] Specifically, after obtaining the similarity between the target feature and each feature in the feature pool using a similarity algorithm, the features can be arranged in descending order based on the size of the similarity values. This makes it easier to prioritize features with high similarity values ​​to the target feature when determining at least one feature based on similarity.

[0079] Here, after obtaining at least one test case that matches the first business information from the test case library, the tester can also review the obtained test cases to determine at least one test case to be executed in step 103.

[0080] In practical applications, after retrieving at least one test case from the test case library, a profile of the corresponding test case can be created based on the initial business information. Furthermore, when using at least one test case for testing, a profile of the corresponding test case can also be created based on the testing process.

[0081] Among them, such as Figure 2 As shown, profiling each test case can be understood as follows:

[0082] On the one hand, based on the acquired first business information, static features corresponding to the test cases can be extracted, which can specifically include basic test case information, business function description, test environment information, data configuration information, etc. Specifically, the basic test case information represents the test case name, execution steps, etc.; the business function description represents the business features associated with the test case; the test environment information represents the environment information corresponding to the test case, such as network conditions and third-party resources; and the data configuration information represents the configuration information corresponding to the test case.

[0083] On the other hand, based on the testing process of the corresponding test cases, the dynamic characteristics of the corresponding test cases can be extracted (which can be understood as the characteristics corresponding to the execution process of the test cases). Specifically, these characteristics can include execution results, end-to-end logs, execution code coverage blocks, daily execution information, and reasons for execution failure. Among them, the execution result status represents the final execution result of the test case; the end-to-end log represents the relevant information recorded during the execution process of the test case; the execution code coverage blocks represent the code involved in the execution process of the test case; the daily execution information includes the time information and system status information corresponding to the execution of the test case; and the execution failure attribution represents the reason for the failure of the test case execution.

[0084] For example, during the testing process using test cases, the test cases can be processed to generate corresponding functional scripts. Then, the generated functional scripts are used to execute the corresponding software testing process. Specifically, by setting instrumentation information (which can be understood as probes) in the functional scripts, information can be collected during the execution of the test cases, thereby enabling the extraction of the dynamic characteristics of the test cases.

[0085] In summary, by extracting the feature information of the entire lifecycle of test cases, a profile of the test cases was achieved.

[0086] In practical applications, once the test case profile is completed, it can be updated to the test case profile library in a timely manner. This will enable the analysis of subsequent failure reasons to accurately determine the cause of test case execution failure based on the updated test case profile library.

[0087] Based on this, in one embodiment, the method may further include:

[0088] Once the profile is complete, update the test case profile to the test case profile library.

[0089] In practical applications, if the execution result of each test case in at least one test case determines that the first test case in at least one test case has failed, then the first test case can be used to retest in order to determine whether the failure is related to environmental factors.

[0090] If the first test case executes successfully upon retesting, it indicates that the initial test failure may be due to environmental factors. These environmental factors can include environment configuration, third-party resources, network issues, etc. In this case, testers can analyze the environmental factors and troubleshoot the root cause based on the profile of the first test case.

[0091] If the first test case fails to execute when retested, key characteristics can be identified from the log information generated during the execution of the first test case to facilitate the analysis of the cause of the failure.

[0092] In one embodiment, determining at least one feature from the log information generated during the execution of the first test case includes:

[0093] Re-test using the first test case;

[0094] Identify at least one feature from the log information generated during the first execution and re-execution of the first test case.

[0095] Here, when determining key features, at least one feature can be determined from the log information of the first execution and re-execution of the first test case to obtain the first set.

[0096] For example, TF-IDF technology can be used to extract at least one word that appears a preset number of times from the log information of the first execution and re-execution of the first test case, as at least one definite feature, thereby obtaining a first set; wherein, the preset number of times can be set as needed, such as 5 times.

[0097] In practical applications, in order to determine the reason for the failure of the first test case, it is also necessary to determine the second set.

[0098] Based on this, in one embodiment, the method may further include:

[0099] Select at least one profile of a failed use case from the use case profile library;

[0100] The second set is determined based on the profile of at least one selected failed use case.

[0101] Here, since the test case profile library stores profiles of at least one test case, and each test case profile contains the execution result of the test case (including success or failure), the profile of at least one failed test case can be selected from the test case profile library based on the execution result in the test case profile, thus determining the second set.

[0102] The profile of each test case in the second set includes at least the execution failure attribution (i.e. the reason for the test case failure) and the full-link log.

[0103] In practical applications, in order to determine the failure reason of at least one test case that matches the first test case, when profiling the test cases, a similar test case set and a derived test case set can be generated based on the business information of the test cases; the similar test case set represents a set of test cases used to test normal conditions; the derived test case set represents a set of test cases used to test abnormal conditions.

[0104] In this way, when determining the second set, the profile of at least one failed use case can be selected based on the similar use case set and the derived use case set.

[0105] In practical applications, once the first set and the second set are determined, the failure reasons of the first test case can be automatically analyzed, thereby reducing the time spent on manual analysis and troubleshooting and improving the efficiency of test analysis.

[0106] In one embodiment, selecting at least one failure reason from the second set that matches a feature in the first set may include:

[0107] Based on a similarity algorithm, at least one failure reason that matches the features in the first set is selected from the second set.

[0108] The similarity algorithm can be set to the minimum edit distance algorithm or other similarity algorithms, and this application embodiment does not limit it.

[0109] For example, by using the minimum edit distance algorithm, the features in the first set are compared with the end-to-end logs of each profile in the second set to obtain the similarity (i.e., edit distance) between the features in the first set and each profile in the second set. Then, based on the obtained similarity, the profile of at least one test case matching the features in the first set can be determined. Finally, based on the execution failure attribution in the determined at least one test case profile, at least one failure reason corresponding to the features in the first set can be determined.

[0110] Specifically, after obtaining the similarity between the features in the first set and each portrait in the second set using the minimum edit distance algorithm, they can be arranged in descending order based on the size of the similarity value. This is so that when determining the portrait of at least one test case based on similarity, the portrait of the test case with a high similarity value (i.e., a small edit distance) to the features in the first set can be selected first.

[0111] Here, after identifying at least one failure reason that matches the features in the first set, manual feedback and evaluation can be performed on the identified at least one failure reason; that is, testers can verify the identified at least one failure reason to ensure the accuracy of the failure reasons of the first test case.

[0112] In practical applications, after determining at least one reason for the failure of the first test case, further automatic location of code defects can be achieved to reduce the time spent manually checking the code.

[0113] Based on this, in one embodiment, the method may further include:

[0114] Based on the similarity algorithm, at least one test case that failed to execute and matches the profile of the first test case is selected from the test case library to obtain a third set; and based on the similarity algorithm, at least one test case that successfully executed and matches the profile of the first test case is selected from the test case library to obtain a fourth set.

[0115] Based on the third and fourth sets, it is determined whether the first test case has a code defect, and the determination result is obtained;

[0116] Based on the judgment result, update the profile of the first test case.

[0117] The similarity algorithm can be set to the minimum edit distance algorithm or other types of similarity algorithms, and this application embodiment does not limit it.

[0118] In practical applications, when it is determined that the first test case fails to execute, the profiles of all failed test cases and all successful test cases can be selected based on the execution results of each test case in the test case profile library.

[0119] For example, using the minimum edit distance algorithm, the similarity between the profile of the first test case and each failed test case profile in the failed test case profile can be obtained. Then, based on the obtained similarity, the profile of at least one failed test case that matches the profile of the first test case can be determined. Then, using the profile of the determined at least one failed test case, the corresponding at least one failed test case can be determined from the test case library, thereby obtaining the third set.

[0120] In practical applications, after obtaining the similarity between the profile of the first test case and each failed test case profile, they can be arranged in descending order based on the size of the similarity value. This is to prioritize the profile of the failed test case with a high similarity value to the profile of the first test case when determining the profile of at least one failed test case.

[0121] For example, using the minimum edit distance algorithm, the similarity between the profile of the first test case and each successful test case profile can be obtained. Then, based on the obtained similarity, the profile of at least one successfully executed test case that matches the profile of the first test case can be determined. Then, using the profile of the determined at least one successfully executed test case, the corresponding at least one successfully executed test case can be determined from the test case library, resulting in a fourth set.

[0122] In practical applications, after obtaining the similarity between the profile of the first test case and each successful test case profile, they can be arranged in descending order based on the size of the similarity value. This is to prioritize the profile of the successful test case with the highest similarity value to the profile of the first test case when determining the profile of at least one successfully executed test case.

[0123] In practical applications, after obtaining the third and fourth sets, the code of the test cases in the third and fourth sets can be used to determine whether the first test case has any code defects.

[0124] Based on this, in one embodiment, determining whether the first test case has a code defect based on the third set and the fourth set includes:

[0125] The code path coverage blocks of the code in the third and fourth sets are determined;

[0126] Based on the code path coverage block of the code in the third set, the code difference between the first test case and the code in the third set is determined to obtain the fifth set. Based on the code path coverage block of the code in the fourth set, the code intersection between the first test case and the code in the fourth set is determined to obtain the sixth set.

[0127] Determine whether the fifth and sixth sets involve code changes;

[0128] When it is determined that the fifth and sixth sets involve code changes, it is determined that the first test case has a code defect.

[0129] In practical applications, the code path coverage blocks of the code in the third and fourth sets of the test case profile library can be determined by combining the profiles of each test case in the third and fourth sets. Specifically, the code path coverage blocks of the code in the third set are determined by using the execution code coverage blocks of the test case profiles in the third set; and the code path coverage blocks of the code in the fourth set are determined by using the execution code coverage blocks of the test case profiles in the fourth set.

[0130] In practical applications, by combining the profile of the first test case in the test case profile library, it is also possible to determine the code path coverage block of the first test case.

[0131] By comparing the code path coverage block of the first test case with the code path coverage block of each test case in the third set, the parts of the code of the first test case that are different from the code of each test case in the third set can be obtained as the code difference set, and then the fifth set can be obtained.

[0132] Accordingly, by comparing the code path coverage block of the first test case with the code path coverage block of each test case in the fourth set, the code that is the same as the code of the first test case and the code of each test case in the fourth set is obtained as the code intersection, and then the sixth set is obtained.

[0133] Then, based on the fifth and sixth sets, it is determined whether the code involves code changes (which can be understood as whether the code has been modified). Specifically, for modified code, comments can be added to the code during the modification process to indicate the area corresponding to the modified code. Thus, when it is necessary to determine whether the code involves code changes, it can be determined by checking whether corresponding comments exist in the code.

[0134] Here, when it is determined that sets five and six involve code changes, it can be concluded that the first test case has a code defect. In this case, by locating the areas involving code changes to indicate the potentially defective code to testers, the time spent by testers in troubleshooting code issues can be reduced, thus improving testing efficiency.

[0135] The test case processing method provided in this application embodiment involves obtaining first business information, acquiring at least one test case matching the first business information from a test case library, performing tests using the at least one test case, determining at least one feature from the log information generated by executing the first test case to obtain a first set, selecting at least one failure reason matching the feature in the first set from a second set, and the second set containing at least one test case failure reason; wherein, a profile is created for the at least one matching test case based on the first business information and the test. The technical solution provided in this application embodiment, based on the acquired business information, can automatically filter out test cases to be executed that match the business information from the test case library, improving the efficiency and accuracy of test case filtering. Simultaneously, it can automatically analyze the failure reasons when test case execution fails, reducing the time cost of manual analysis and troubleshooting, thereby improving the overall efficiency of testing and enhancing the level of automation.

[0136] The present application will be further described in detail below with reference to application examples.

[0137] This application embodiment proposes an intelligent testing system based on test case profiling, which enables intelligent testing of software products. When regression testing is required, the system can not only automatically filter test cases that meet business requirements from the test case set, but also automatically attribute test cases to causes and locate code defects when test case execution fails, reducing the cost of manual filtering and location, and improving the efficiency of software product testing.

[0138] Specifically, such as Figure 3 As shown, an intelligent testing system may include: a test case intelligent recommendation module, an execution module, a test case profiling module, and a result analysis module; among which,

[0139] The test case intelligent recommendation module is used to construct a test case library and recommend corresponding test cases based on business requirements.

[0140] The execution module is used to execute test cases and collect information during the execution of test cases;

[0141] The test case profiling module is used to profile test cases and manage these profiles.

[0142] The results analysis module is used to automatically attribute the causes of failed test cases and locate defective code when the failed test cases involve modified code.

[0143] The process of processing test cases in this application example includes the following steps:

[0144] Step 1: The test case intelligent recommendation module constructs multiple test cases and sets corresponding business requirement description information and tags (which can be understood as categories) for each constructed test case to form a test case library;

[0145] The test cases can be constructed manually or automatically, such as using test cases (REs).

[0146] The business requirement description information may include information such as title, use case steps, business function description, use case verification points or expected results.

[0147] In practical applications, after the test case intelligent recommendation module forms a test case library, it uses jieba analysis and FMM algorithm to perform corpus analysis, feature extraction and vectorization of the business requirement description information corresponding to each test case in the test case library, and generates a feature pool corresponding to the test case library.

[0148] Step 2: After receiving the business information (i.e., the first business information), the test case intelligent recommendation module selects M test cases that match the business information from the test case library.

[0149] Where M is an integer greater than 1, and the value of M can be set as needed.

[0150] Specifically, such as Figure 4 As shown, the process by which the test case construction module selects M test cases that match the business information can include the following steps:

[0151] Step 401: The test case construction module obtains business information;

[0152] Here, when regression verification is required, the test case intelligent recommendation module obtains business information by receiving business information manually input by testers, such as title, business description, test case verification points, business function name, program interface name, etc.

[0153] Step 402: The use case construction module extracts the characteristics of the target use case;

[0154] In practical applications, the use case construction module extracts the corresponding target feature information based on the acquired business information.

[0155] Step 403: The test case construction module matches the target test case features with the feature pool;

[0156] Specifically, the Naive Bayes algorithm is used to compare the features of the target test case with the features of each test case in the feature pool to obtain the corresponding similarity results.

[0157] The formula for the Naive Bayes algorithm can be expressed as:

[0158] Here, A represents the category feature of the test cases in the feature pool; B represents the target test case feature; P(A|B) represents the probability that B belongs to A; P(B|A) represents the probability that A contains B; P(A) represents the probability that A appears in all categories; P(B) represents the probability that B appears in all features.

[0159] Step 404: The test case construction module selects M test cases from the test case library;

[0160] In practical applications, the test case construction module can arrange the similarity results in descending order of value, and select the M test cases with the largest similarity values ​​according to the testers' needs, such as 3 cases.

[0161] Step 405: Determine the final test cases.

[0162] Here, after the test case intelligent recommendation module selects M matching test cases from the test case library, testers can manually review them to determine the final test cases for executing step 3.

[0163] Meanwhile, the intelligent test case recommendation module can send the received business information and the determined M test cases to the test case profiling module, so that the test case profiling module can create a profile for the M test cases.

[0164] Step 3: The execution module executes the M test cases determined by the test case intelligent recommendation module and collects information during the execution of each test case.

[0165] In practical applications, after the execution module receives the M test cases determined by the intelligent test case recommendation module, it processes each test case in the test case library to obtain the corresponding functional script. Then, it calls the interface testing engine of the execution module to execute the corresponding functional script.

[0166] The execution module collects dynamic information during the execution of each test case by setting instrumentation information in the functional scripts; the dynamic information includes at least full-link log information, code coverage information, assertion result information, and test case result status.

[0167] Step 4: The test case profiling module creates a profile for each test case based on business information and information from the test case execution process;

[0168] Specifically, the test case profiling module can extract static features corresponding to test cases based on business information, such as basic test case information and business characteristics. Simultaneously, the module can also extract dynamic features corresponding to test cases based on information during test case execution, such as test case code coverage blocks, test case result status, and failure attribution. Based on the static and dynamic features of test cases—that is, the features throughout the entire lifecycle of a test case—the test case profiling module can create a profile of the test cases and store the profile in a test case profiling library after completion.

[0169] Step 5: When test case A (i.e., the first test case) fails to execute among the M test cases, the result analysis module automatically analyzes the reason for the failure of test case A;

[0170] Specifically, such as Figure 5 As shown, the process by which the results analysis module analyzes the reasons for the failure of test case A may include the following steps:

[0171] Step 501: The result analysis module sends execution information to the execution module so that the execution module can execute test case A again.

[0172] Here, during the re-execution of test case A by the execution module, at least the full-link log information, execution results, and code coverage block information are collected during the re-execution process.

[0173] The end-to-end log information may include assertion results, exception information, test environment detection results, and baseline configuration detection results.

[0174] Step 502: Based on the execution result of test case A being executed again, the result analysis module determines whether test case A was executed successfully;

[0175] If test case A executes successfully, then step 503 is executed; if test case A fails to execute, then step 504 is executed.

[0176] Step 503: The results analysis module performs environmental attribution investigation;

[0177] Specifically, environmental causes can be analyzed based on the end-to-end log information collected by the execution module, such as network problems, environment configuration information problems, or third-party resource problems.

[0178] Step 504: The result analysis module obtains the full-link log information for the first and subsequent executions of test case A;

[0179] Step 505: The result analysis module extracts log features from the end-to-end log information;

[0180] For example, the results analysis module can use the TF-IDF algorithm to extract log features and form a failure attribution set F.

[0181] Step 506: The results analysis module uses a similarity algorithm to find 5 failed test cases that match the log characteristics.

[0182] In practical applications, the results analysis module can select the profiles of all failed test cases from the test case profile library to form a historical failure test case attribution set L.

[0183] Then, the minimum edit distance algorithm is used to calculate the edit distance (i.e., similarity) between the attribution feature F of the failed test cases and the profile of each test case in the historical failure test case attribution set L. By arranging the calculated edit distances in ascending order, the profiles of the 5 test cases with small edit distances (i.e., large similarity) are selected first, thereby determining the failure reason of each test case and forming the corresponding failure attribution list.

[0184] Step 507: The results analysis module will provide feedback on the failure attribution list;

[0185] Specifically, the results analysis module feeds back the failure attribution list to the testers to assess the accuracy of the automatic attribution.

[0186] Step 508: The results analysis module updates the profile of test case A based on the results of manual evaluation.

[0187] Here, after the manual evaluation is completed, the results analysis module sends the failure attribution list of test case A to the test case profiling module to update the profile of test case A.

[0188] Step 6: The results analysis module automatically locates the defective code in test case A;

[0189] Specifically, such as Figure 6 As shown, the process by which the results analysis module automatically locates defective code in test case A can include the following steps:

[0190] Step 601: The result analysis module determines a set of similar test cases that match test case A from the test case profile library based on the profile of test case A;

[0191] In practical applications, the results analysis module first determines the profiles of all successfully executed test cases and all failed test cases from the test case profile library. Then, using the minimum edit distance algorithm, it calculates the similarity between the profile of test case A and the profile of each successfully executed test case, obtaining the corresponding similarity results. Simultaneously, using the minimum edit distance algorithm, it calculates the similarity between the profile of test case A and the profile of each failed test case, obtaining the corresponding similarity results.

[0192] Here, the similarity results between the profile of test case A and the profiles of successfully executed test cases can be sorted from largest to smallest. The five profiles of successfully executed test cases with the highest similarity scores are selected to form the set S of successful test cases. Simultaneously, the similarity results between the profile of test case A and the profiles of failed test cases can be sorted from largest to smallest. The five profiles of failed test cases with the highest similarity scores are selected to form the set F of failed test cases.

[0193] Step 602: The result analysis module obtains the code path overlay blocks of the successful execution test case set S and the failed execution test case set F.

[0194] Here, the results analysis module uses the test case profile library to obtain the code path coverage block of each test case in the successfully executed test case set S, and can also obtain the code path coverage block of each test case in the failed test case set F.

[0195] Then, by comparing the code path coverage block of test case A with the code path coverage block of each test case in the set of successfully executed test cases S, the parts of the code of test case A that are different from the code of each test case in the set of successfully executed test cases S are obtained, and thus the code difference set Ts is obtained.

[0196] Accordingly, by comparing the code path coverage block of test case A with the code path coverage block of each test case in the set of failed test cases F, the parts of the code of test case A that are the same as the code of each test case in the set of failed test cases F are obtained, and thus the code intersection Tf is obtained.

[0197] Step 603: The result analysis module determines whether the code difference Ts and the code intersection Tf involve modified code.

[0198] If it is determined that the code involved is modified, then step 604 is executed; if it is determined that the code is not modified, then step 606 is executed.

[0199] In practical applications, the results analysis module can determine whether the code difference Ts and code intersection Tf involve modified code based on the comments in the test case code. Specifically, for modified code, comments can be added to indicate the area of ​​the modified code. Thus, when it's necessary to determine whether code changes have occurred, the presence of corresponding comments in the code can be used to ascertain whether changes have been made.

[0200] Step 604: When the result analysis module determines that the code difference Ts and the code intersection Tf involve code changes, it determines that there is a suspicious defective code in the code of test case A, and then automatically locates the code involving code changes.

[0201] Step 605: The results analysis module sends the identified suspicious defect code information to the test case profiling module to update the profile of test case A.

[0202] Step 606: End the current process.

[0203] This application proposes an intelligent testing method based on test case profiling. A test case profiling model is established based on the characteristic information throughout the entire lifecycle of a test case. When regression verification is required after code changes, the test case profiling model can automatically filter out test cases that match the business description information, allowing testers to use these selected test cases for testing. This achieves the goal of accurate testing using a smaller number of test cases, avoids the increased time costs caused by manual screening, and improves the efficiency of test screening.

[0204] Furthermore, when test cases fail to execute, the system can automatically attribute the failures to their causes based on the test case profiling model, accelerating the efficiency of root cause analysis and reducing the time spent on manual analysis. Additionally, the solution described above can use similarity algorithms to identify code defects when test cases fail, automatically locating defective code and reducing the time spent manually reviewing program code, thereby improving the efficiency of test analysis.

[0205] To implement the solutions in the embodiments of this application, the embodiments of this application also provide a test case processing device, such as... Figure 7 As shown, the device includes:

[0206] The first acquisition unit 701 is used to acquire first business information;

[0207] The second acquisition unit 702 is used to acquire at least one test case that matches the first business information from the test case library;

[0208] Test unit 703 is used to perform tests using the at least one test case;

[0209] The determining unit 704 is configured to determine at least one feature from the log information generated by executing the first test case when the first test case fails to execute in the at least one test case, and obtain a first set.

[0210] Selection unit 705 is configured to select at least one failure reason from a second set that matches a feature in the first set; the second set contains at least one test case failure reason; wherein...

[0211] The profiling unit 706 is used to create a profile of at least one matched test case based on the first business information and the test.

[0212] It should be noted that the functions of the first acquisition unit 701 and the second acquisition unit 702 are equivalent to the functions of the intelligent use case recommendation module in the application embodiment; the function of the testing unit 703 is equivalent to the function of the execution module in the application embodiment; the function of the profiling unit 706 is equivalent to the function of the use case profiling module in the application embodiment; and the functions of the determining unit 704 and the selecting unit 705 are equivalent to the functions of the result analysis module in the application embodiment.

[0213] In one embodiment, the determining unit 704 is configured to:

[0214] Re-test using the first test case;

[0215] Identify at least one feature from the log information generated during the initial execution and re-execution of the first test case.

[0216] In one embodiment, the selection unit 705 is used for:

[0217] Based on a similarity algorithm, at least one failure reason that matches the features in the first set is selected from the second set.

[0218] In one embodiment, the determining unit 704 is further configured to:

[0219] Select at least one profile of a failed use case from the use case profile library;

[0220] The second set is determined based on the profile of at least one selected failed use case.

[0221] In one embodiment, the selection unit 705 is further configured to:

[0222] Based on the similarity algorithm, at least one test case that failed to execute and matches the profile of the first test case is selected from the test case library to obtain a third set; and based on the similarity algorithm, at least one test case that successfully executed and matches the profile of the first test case is selected from the test case library to obtain a fourth set.

[0223] Based on the third and fourth sets, it is determined whether the first test case has a code defect, and the determination result is obtained;

[0224] Based on the judgment result, update the profile of the first test case.

[0225] In one embodiment, the selection unit 705 is used for:

[0226] The code path coverage blocks of the code in the third and fourth sets are determined;

[0227] Based on the code path coverage block of the code in the third set, the code difference between the first test case and the code in the third set is determined to obtain the fifth set. Based on the code path coverage block of the code in the fourth set, the code intersection between the first test case and the code in the fourth set is determined to obtain the sixth set.

[0228] Determine whether the fifth and sixth sets involve code changes;

[0229] When it is determined that the fifth and sixth sets involve code changes, it is determined that the first test case has a code defect.

[0230] In one embodiment, the image-collecting unit 706 is further configured to:

[0231] Once the profile is complete, the profile of the test case is stored in the test case profile library.

[0232] In practical applications, the first acquisition unit 701 can be implemented by a processor in the test case processing device combined with a communication interface; the second acquisition unit 702, the test unit 703, the determination unit 704, and the selection unit 705 can be implemented by a processor in the test case processing device.

[0233] It should be noted that the test case processing device provided in the above embodiments is only illustrated by the division of the above-described program units when processing test cases. In actual applications, the above processing can be assigned to different program units as needed, that is, the internal structure of the device can be divided into different program units to complete all or part of the processing described above. In addition, the test case processing device and the test case processing method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0234] Based on the hardware implementation of the above program modules, and in order to implement the test case processing method of the embodiments of this application, the embodiments of this application also provide an electronic device, such as... Figure 8 As shown, the electronic device 800 includes:

[0235] The communication interface 801 enables interaction with other devices;

[0236] The processor 802 is connected to the communication interface 801 to enable interaction with other devices and to execute the methods provided by one or more of the above-mentioned technical solutions when running a computer program;

[0237] The computer program is stored in memory 803.

[0238] Specifically, the processor 802 is used for:

[0239] First service information is obtained through the communication interface 801;

[0240] Obtain at least one test case from the test case library that matches the first business information;

[0241] Use at least one of the test cases to perform the test;

[0242] When the execution of the first test case in the at least one test case fails, at least one feature is determined from the log information generated by the execution of the first test case to obtain a first set;

[0243] Select at least one failure reason from the second set that matches a feature in the first set; the second set contains at least one test case failure reason; wherein...

[0244] Based on the first business information and the test, a profile is created for at least one of the matched test cases.

[0245] In one embodiment, the processor 802 is configured to:

[0246] Re-test using the first test case;

[0247] Identify at least one feature from the log information generated during the first execution and re-execution of the first test case.

[0248] In one embodiment, the processor 802 is configured to select at least one failure reason from a second set that matches a feature in the first set, based on a similarity algorithm.

[0249] In one embodiment, the processor 802 is further configured to:

[0250] Select at least one profile of a failed use case from the use case profile library;

[0251] The second set is determined based on the profile of at least one selected failed use case.

[0252] In one embodiment, the processor 802 is further configured to:

[0253] Based on the similarity algorithm, at least one test case that failed to execute and matches the profile of the first test case is selected from the test case library to obtain a third set; and based on the similarity algorithm, at least one test case that successfully executed and matches the profile of the first test case is selected from the test case library to obtain a fourth set.

[0254] Based on the third and fourth sets, it is determined whether the first test case has a code defect, and the determination result is obtained;

[0255] Based on the judgment result, update the profile of the first test case.

[0256] In one embodiment, the processor 802 is configured to:

[0257] The code path coverage blocks of the code in the third and fourth sets are determined;

[0258] Based on the code path coverage block of the code in the third set, the code difference between the first test case and the code in the third set is determined to obtain the fifth set. Based on the code path coverage block of the code in the fourth set, the code intersection between the first test case and the code in the fourth set is determined to obtain the sixth set.

[0259] Determine whether the fifth and sixth sets involve code changes;

[0260] When it is determined that the fifth and sixth sets involve code changes, it is determined that the first test case has a code defect.

[0261] In one embodiment, the processor 802 is further configured to:

[0262] Once the profile is complete, the profile of the test case is stored in the test case profile library.

[0263] It should be noted that the specific processing procedure of processor 802 can be understood by referring to the above method.

[0264] Of course, in practical applications, the various components in electronic device 800 are coupled together through bus system 804. It can be understood that bus system 804 is used to realize the connection and communication between these components. In addition to a data bus, bus system 804 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, in... Figure 8 The general labeled all buses as Bus System 804.

[0265] The memory 803 in this embodiment is used to store various types of data to support the operation of the electronic device 800. Examples of such data include any computer program used to operate on the electronic device 800.

[0266] The methods disclosed in the embodiments of this application can be applied to the processor 802, or implemented by the processor 802. The processor 802 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in the processor 802 or by instructions in the form of software. The processor 802 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 802 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in the memory 803. The processor 802 reads the information in the memory 803 and completes the steps of the aforementioned method in combination with its hardware.

[0267] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned method.

[0268] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a memory 803 storing a computer program, which can be executed by the processor 802 of the electronic device 800 to complete the steps described in the aforementioned test case processing. The computer-readable storage medium can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk storage device or a magnetic tape storage device.

[0269] It should be noted that terms such as "first" and "second" are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0270] Furthermore, the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0271] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A test case processing method, characterized in that, include: Obtain the first business information; Obtain at least one test case from the test case library that matches the first business information; Use at least one of the test cases to perform the test; When the execution of the first test case in the at least one test case fails, at least one feature is determined from the log information generated by the execution of the first test case to obtain a first set; Select at least one failure reason from the second set that matches a feature in the first set; the second set contains at least one test case failure reason; wherein... Based on the first business information and the test, a profile is created for at least one matched test case; the method further includes: Based on the similarity algorithm, at least one test case that failed to execute and matches the profile of the first test case is selected from the test case library to obtain a third set; and based on the similarity algorithm, at least one test case that successfully executed and matches the profile of the first test case is selected from the test case library to obtain a fourth set. Based on the third and fourth sets, it is determined whether the first test case has a code defect, and the determination result is obtained; Based on the judgment result, the profile of the first test case is updated; wherein, judging whether the first test case has code defects based on the third set and the fourth set includes: The code path coverage blocks of the code in the third and fourth sets are determined; Based on the code path coverage block of the code in the third set, the code difference between the first test case and the code in the third set is determined to obtain the fifth set. Based on the code path coverage block of the code in the fourth set, the code intersection between the first test case and the code in the fourth set is determined to obtain the sixth set. Determine whether the fifth and sixth sets involve code changes; When it is determined that the fifth and sixth sets involve code changes, it is determined that the first test case has a code defect.

2. The method according to claim 1, characterized in that, Determining at least one feature from the log information generated during the execution of the first test case includes: Re-test using the first test case; Identify at least one feature from the log information generated during the first execution and re-execution of the first test case.

3. The method according to claim 1, characterized in that, The step of selecting at least one failure reason from the second set that matches a feature in the first set includes: Based on a similarity algorithm, at least one failure reason that matches the features in the first set is selected from the second set.

4. The method according to claim 1, characterized in that, The method further includes: Select at least one profile of a failed use case from the use case profile library; The second set is determined based on the profile of at least one selected failed use case.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: Once the profile is complete, update the test case profile to the test case profile library.

6. A test case processing device, characterized in that, include: The first acquisition unit is used to acquire the first business information; The second acquisition unit is used to acquire at least one test case that matches the first business information from the test case library; A test unit, used to perform testing using the at least one test case; A determining unit is configured to determine at least one feature from the log information generated during the execution of the first test case when the first test case fails to execute in the at least one test case, thereby obtaining a first set. A selection unit is configured to select at least one failure reason from a second set that matches a feature in the first set; the second set contains at least one test case failure reason; wherein... A profiling unit is used to create a profile of at least one matched test case based on the first business information and the test. The selection unit is further configured to: select at least one failed test case from the test case library that matches the profile of the first test case based on a similarity algorithm, to obtain a third set; and select at least one successfully executed test case from the test case library that matches the profile of the first test case based on a similarity algorithm, to obtain a fourth set; determine whether the first test case has a code defect based on the third and fourth sets, and obtain a determination result; and update the profile of the first test case based on the determination result; wherein... The selection unit is used to determine the code path coverage blocks of the third set and the fourth set of code; based on the code path coverage blocks of the third set of code, determine the code difference between the first test case and the code of the third set of code to obtain a fifth set; and based on the code path coverage blocks of the fourth set of code, determine the code intersection between the first test case and the code of the fourth set of code to obtain a sixth set; determine whether the fifth set and the sixth set involve code changes; when it is determined that the fifth set and the sixth set involve code changes, determine that the first test case has a code defect.

7. An electronic device, characterized in that, include: The processor and the memory used to store computer programs that can run on the processor. When the processor runs the computer program, it executes the steps of the test case processing method according to any one of claims 1 to 5.

8. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the test case processing method according to any one of claims 1 to 5.

Images