Test case analysis method and device, processor and electronic device

Abstract

The application discloses a test case analysis method and device, a processor and an electronic device. It relates to the field of artificial intelligence, and the method comprises the following steps: obtaining time sequence data, text data and attribute data of a failed test case, wherein the time sequence data is data representing the stability of the execution process of the test case, the text data is data representing the particularity of logs in the execution process of the test case, and the attribute data is data representing the uniqueness of the test case; inputting the time sequence data, the text data and the attribute data into a target model to obtain a failure cause, wherein the target model is trained by a plurality of sets of training samples, and each set of training samples comprises time sequence data, text data, attribute data and a failure cause of a historical failed test case; and adjusting the execution method of the test case based on the failure cause. Through the application, the problem of low efficiency in analyzing the failure cause of a failed test case in the related art is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, in particular, to a test case analysis method and device, a processor and an electronic device. BACKGROUND

[0002] There are a large number of failed test cases in the automated testing process of software. When analyzing the failure causes of these failed test cases, there are problems such as complex failure cause investigation links, multiple failure causes, and a large number of repeated failure causes. In related technologies, the failure cause analysis methods for failed test cases mainly include the following methods: the first method is a failure case attribution based on keyword / word rule matching. Test personnel filter log message information by configuring keywords / words. If there is a keyword / word configured by the test personnel, the corresponding reason tag is given to this failed test case. The second method is a failure case attribution based on error stack classification. Console logs and error stacks involved in the failed test case are analyzed to achieve the purpose of obtaining the failure cause of the test case. The third method is a failure case attribution based on Bug reports. In the software maintenance process of large software programs, Bug reports become an important medium for helping developers to solve Bugs. Users can report software errors through a fixed format (i.e., Bug reports) and upload them to a Bug tracking system. Then, a senior developer is assigned to understand and fix the reported errors according to the information displayed in the submitted report.

[0003] However, during the execution of the automated test script, it is often affected by the low environmental stability, invalid test data state, and the like, thereby causing problems such as complex failure cause investigation links, repeated analysis of similar failure causes, and multiple failure causes. The above-mentioned several failure case attribution methods all have their own defects. For example, the first method based on manual rule matching has high cost and no generalization ability. The second method based on error stack classification cannot consider problems that occur in the entire automated testing process, especially some stability problems that occur in the entire automated execution process. The third method based on Bug report analysis needs to consume a certain amount of human cost to generate Bug reports and perform analysis, and when a large number of failed test cases occur, the analysis efficiency of this method is low.

[0004] At present, there is no effective solution to the problem of low efficiency in analyzing the failure causes of failed test cases in related technologies. SUMMARY

[0005] The main purpose of the present application is to provide a test case analysis method, device, processor and electronic device to solve the problem of low efficiency in analyzing the failure causes of failed test cases in related technologies.

[0006] In order to achieve the above object, according to one aspect of the present application, a test case analysis method is provided. The method comprises: obtaining timing data, text data and attribute data of a failed test case, wherein the timing data is data representing stability in the execution process of the test case, the text data is data representing the particularity of logs in the execution process of the test case, and the attribute data is data representing the uniqueness of the test case; inputting the timing data, the text data and the attribute data into a target model to obtain a failure cause, wherein the target model is trained by a plurality of training samples, and each training sample comprises timing data, text data, attribute data and a failure cause of a historical failed test case; and adjusting the execution method of the test case based on the failure cause.

[0007] Optionally, obtaining the timing data of the failed test case comprises: obtaining execution time and execution state of each execution process of the failed test case in a preset period, wherein the execution state comprises execution success and execution failure; determining a first execution time when the failed test case is found, determining a group of second execution times with execution failure, and determining a third execution time with execution success adjacent to each second execution time to obtain a plurality of third execution times; calculating a difference between each third execution time and the first execution time to obtain a group of difference values; and determining a target value of the group of difference values as the timing data, wherein the target value comprises at least one of the following: maximum value, minimum value, median and variance.

[0008] Optionally, obtaining the text data of the failed test case comprises: obtaining a target log associated with the failed test case and response body information returned by a console running the failed test case, wherein the target log comprises at least one of the following: integration tool scheduling log, console log and error stack log; extracting at least one keyword from the target log and inputting the keyword into a preset semantic extraction model to obtain semantic features; and determining the keyword, the semantic features and the response body information as the text data.

[0009] Optionally, obtaining the attribute data of the failed test case comprises: determining a test framework type, an application type, a test progress stage, a plate mounting period state, a communication problem state, a response body return code, a uniform resource locator feature and an interface type of the failed test case; and determining at least one of the test framework type, the application type, the test progress stage, the plate mounting period state, the communication problem state, the response body return code, the uniform resource locator feature and the interface type as the attribute data.

[0010] Optionally, before inputting the timing data, the text data and the attribute data into the target model, the method further comprises: obtaining each historical failed test case and a corresponding historical failure reason, and extracting the timing data, the text data and the attribute data in the historical failed test case; taking the timing data, the text data, the attribute data and the historical failure reason in the historical failed test case as a group of training samples, to obtain a plurality of groups of training samples; inputting the plurality of groups of training samples into the preset model for training, to obtain the target model.

[0011] Optionally, before adjusting the execution method of the test case based on the failure reason, the method further comprises: determining an actual failure reason of the failed test case through a preset rule; judging whether the failure reason and the actual failure reason are the same; in a case where the failure reason and the actual failure reason are not the same, adjusting the execution method of the test case based on the actual failure reason.

[0012] Optionally, in a case where the failure reason and the actual failure reason are not the same, the method further comprises: obtaining a plurality of groups of correction samples in a preset period, wherein each group of correction samples comprises one failed test case and an actual failure reason; correcting the target model based on the plurality of groups of correction samples, to obtain a corrected target model; and replacing the target model with the corrected target model.

[0013] In order to achieve the above-mentioned purpose, according to another aspect of the present application, a test case analysis device is provided. The device comprises: an acquisition unit configured to acquire timing data, text data and attribute data of a failed test case, wherein the timing data is data representing stability in a test case execution process, the text data is data representing particularity of logs in the test case execution process, and the attribute data is data representing uniqueness of the test case; an input unit configured to input the timing data, the text data and the attribute data into a target model to obtain a failure reason, wherein the target model is trained by a plurality of groups of training samples, and each group of training samples comprises timing data, text data, attribute data and a failure reason of a historical failed test case; and an adjustment unit configured to adjust an execution method of the test case based on the failure reason.

[0014] By the present application, the following steps are adopted: obtaining timing data, text data and attribute data of a failed test case, wherein the timing data is data representing stability in the execution process of the test case, the text data is data representing the particularity of logs in the execution process of the test case, and the attribute data is data representing the uniqueness of the test case; inputting the timing data, the text data and the attribute data into a target model to obtain a failure cause, wherein the target model is trained by a plurality of sets of training samples, and each set of training samples includes timing data, text data, attribute data and a failure cause of a historical failed test case; and adjusting the execution method of the test case based on the failure cause, thereby solving the problem of low efficiency in analyzing the failure cause of a failed test case in the related art. By inputting the timing data, the text data and the attribute data of the failed test case into the target model to comprehensively analyze the failure cause, the effect of more comprehensive and efficient analysis of the failure cause of the test case is achieved. BRIEF DESCRIPTION OF DRAWINGS

[0015] The accompanying drawings, which form a part of the present application, are intended to provide further understanding of the present application, and are used to interpret the illustrative embodiments of the present application and their descriptions, and do not constitute improper limitations to the present application. In the drawings:

[0016] Figure 1 is a flowchart of a test case analysis method provided according to an embodiment of the present application;

[0017] Figure 2 is a schematic diagram of feature engineering provided according to an embodiment of the present application;

[0018] Figure 3 is a flowchart of a failure cause prediction method of an automated test case provided according to an embodiment of the present application;

[0019] Figure 4 is a schematic diagram of a test case analysis device provided according to an embodiment of the present application;

[0020] Figure 5 is a schematic diagram of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0022] In the following, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application, so that those skilled in the art can better understand the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work should fall within the scope of protection of the present application.

[0023] It should be noted that the terms "first", "second" and the like in the description and claims of the present application and the above drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0024] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for display, analyzed data, etc.) involved in the present disclosure are all information and data authorized by the user or authorized by all parties.

[0025] The present application will be described below in conjunction with preferred implementation steps, Figure 1 The flowchart of the analysis method of the test case provided according to the embodiments of the present application is shown in Figure 1 The method comprises the following steps:

[0026] In step S101, the timing data, text data and attribute data of the failed test case are obtained, wherein the timing data is data representing the stability of the test case execution process, the text data is data representing the particularity of the log involved in the test case execution process, and the attribute data is data representing the uniqueness of the test case.

[0027] Specifically, since only the keywords / words and error stack information are considered, the failure reason of the failed test case in the entire test case running process cannot be determined. In order to further capture the multi-source reasons of the failed test case, time sequence analysis can be performed through the state and time of the test case running, that is, the stability of the test case in a test monthly version is measured through the time sequence data of the failed test case, so as to capture the environmental stability problem. The keywords, semantic features of the console log, error stack and response body return information, that is, the particularity of different logs in the test case execution process is measured through the text data of the failed test case; other possible reasons of the failed test case are analyzed by combining the inherent attributes of a single automated scheduling process, that is, attribute data.

[0028] In step S102, the time sequence data, the text data and the attribute data are input into a target model to obtain the failure reason, wherein the target model is trained by a plurality of training samples, and each training sample includes time sequence data, text data, attribute data and a failure reason of a historical failed test case.

[0029] Specifically, the target model can be a neural network model. The time sequence data, the text data, the attribute data and the failure reason of the historical failed test case are trained to obtain the target model which can directly output the failure reason of the failed test case according to the input time sequence data, text data and attribute data.

[0030] It should be noted that in order to overcome the problems of the prior art in terms of human cost, failure reason blind area and the like, and to improve the troubleshooting efficiency of the test personnel in the entire automated process, the present application utilizes the key data of the automated test full process for feature engineering construction, comprehensively considers the stability of the test case running, the particularity of the log and the inherent characteristics of the test case to perform all-round mining on the failed test case, and adopts natural language processing, time sequence analysis and the like to perform modeling analysis to obtain feature data. According to the feature data, the target model is trained, so that it can improve the environmental problems which are difficult to capture in the automated test full process without human intervention in reason analysis, and help the test personnel to troubleshoot the failure reason from the aspects of environment, data, functional change and script.

[0031] In step S103, the execution method of the test case is adjusted based on the failure reason.

[0032] Specifically, after the failure reason of the failed test case is determined, the failure reason is recorded, and the execution method of the subsequent test case is adjusted correspondingly to avoid the same failure reason from occurring again.

[0033] The test case analysis method provided by the embodiments of the present application obtains the time sequence data, text data and attribute data of the failed test case, wherein the time sequence data is data representing the stability in the execution process of the test case, the text data is data representing the particularity of the log in the execution process of the test case, and the attribute data is data representing the uniqueness of the test case; the time sequence data, the text data and the attribute data are input into a target model to obtain the failure reason, wherein the target model is trained by a plurality of sets of training samples, and each set of training samples includes the time sequence data, the text data, the attribute data and the failure reason of a historical failed test case; and the execution method of the test case is adjusted based on the failure reason, thereby solving the problem of low efficiency in analyzing the failure reason of the failed test case in the related art. The time sequence data, the text data and the attribute data of the failed test case are input into the target model to comprehensively analyze the failure reason, thereby achieving the effect of more comprehensive and efficient analysis of the failure reason of the test case.

[0034] In view of the need to obtain the time sequence data of the failed test case considering the stability of the test case, optionally, in the test case analysis method provided by the embodiments of the present application, obtaining the time sequence data of the failed test case includes: obtaining the execution time and the execution state of each execution process of the failed test case in a preset period, wherein the execution state includes execution success and execution failure; determining a first execution time when the failed test case is found, determining a set of second execution times of which the execution state is execution failure, and determining a third execution time of which the execution state adjacent to each second execution time is execution success, to obtain a plurality of third execution times; calculating the difference between each third execution time and the first execution time to obtain a set of difference values; and determining a target value of the set of difference values, wherein the target value is determined as the time sequence data, and the target value at least includes one of the following: maximum value, minimum value, median and variance.

[0035] Specifically, the preset period can be one month, by collecting the timing data in the automated test process, the execution time and execution state of the failed test case in a test monthly version in multiple different execution processes are obtained, and the collected timing data is processed and analyzed. In a test monthly version, a plurality of execution processes are included, each execution process records a corresponding execution state and execution time, the different execution times of the failed test case in the monthly version include the execution time at time t-n, the time at the current time t, the execution time at time t-n corresponds to the historical running state, the time at the current time t corresponds to the execution state at time t when the current failed test case appears, and each historical running state includes the states of execution success and execution failure. A group of difference values between the current time t, i.e., the first execution time and the third execution time, is calculated, a target value of the group of difference values is determined as timing data, the target value can be the maximum value, the minimum value, the median, the variance, etc. in the group of difference values, the environment stability change of the current test case in the entire monthly version execution process is measured by the timing data, and the failure reason caused by the environmental factors is analyzed.

[0036] In order to analyze the particularity of the log when the failed test case, the text data of the failed test case needs to be obtained. Optionally, in the test case analysis method provided in the embodiments of the present application, obtaining the text data of the failed test case comprises: obtaining a target log associated with the failed test case, and response body information returned by a console running the failed test case, wherein the target log at least includes one of the following: integrated tool scheduling log, console log and error stack log; at least one keyword is extracted from the target log, and the keyword is input into a preset semantic extraction model to obtain semantic features; the keyword, the semantic features and the response body information are determined as the text data.

[0037] Specifically, the integrated tool scheduling log can be a Jenkins scheduling log, the preset semantic extraction model can be a BERT (Bidrectional Encoder Representation from Transformers, a pre-trained natural language processing model) model, some text information involved in the execution process of a failed test case in a test monthly version of an automated test process is obtained, and the text information comes from different target logs. Key information is extracted and analyzed and mined for different log records. In the entire automated test process, Jenkins scheduling logs, console logs, and error stack logs are involved. Among them, the Jenkins scheduling log is usually large, so a keyword extraction method is used for information mining, that is, whether the keyword appears in the integrated tool scheduling log is analyzed, for example, the final keyword list is obtained by taking the union of the keyword weight and the word frequency. Console logs generally have fixed formats, such as msg and code fields. The msg is very important key information, and the text information in the console log is extracted through the msg keyword. Error stack logs also have information specificity, such as important information related to the root cause of failure in sentences involving the word except. After important text is extracted from the target log by using a regular matching method, semantic features are extracted by using a BERT model in the field of natural language processing, and the keywords, semantic features, and response body information are determined as text data. The text data is obtained to provide input parameters considering the specificity of the log for analyzing the failure cause.

[0038] Considering the factors of the unique nature of the test case, the attribute data of the failed test case needs to be obtained. Optionally, in the test case analysis method provided in the embodiments of the present application, obtaining the attribute data of the failed test case comprises: determining the test framework type, application type, test progress stage, plate mounting period state, communication problem state, response body return code, uniform resource locator feature, and interface type of the failed test case; and determining at least one of the test framework type, application type, test progress stage, plate mounting period state, communication problem state, response body return code, uniform resource locator feature, and interface type as the attribute data.

[0039] Specifically, the test framework type is, for example, a modular test framework, a data-driven framework, a keyword-driven framework, a hybrid model, etc. Different framework types, different application types, different test progress stages, different plate mounting period states, different communication problem states, different response body return codes, different uniform resource locator features, and different interface types can all cause different test case failure causes, so at least one of the above data can be selected as the attribute data. The attribute data is obtained to provide input parameters considering the unique nature of the test case for analyzing the failure cause.

[0040] Before the target model outputs the failure cause, the target model needs to be trained. Optionally, in the test case analysis method provided in the embodiments of the present application, before the time sequence data, the text data and the attribute data are input into the target model, the method further includes: obtaining each historical failed test case and the corresponding historical failure cause, and extracting the time sequence data, the text data and the attribute data in the historical failed test case; taking the time sequence data, the text data, the attribute data and the historical failure cause in the historical failed test case as a group of training samples, and obtaining a plurality of groups of training samples; inputting the plurality of groups of training samples into a preset model for training, and obtaining the target model.

[0041] Specifically, when obtaining the training samples, the stability of the case running, the particularity of the log and the inherent characteristics of the case are comprehensively considered through feature engineering, and the data of the historical failed test case is subjected to feature derivation and integration, including case time sequence data, text data and attribute data. The case time sequence data comprehensively considers the historical running state, the current running state and the derived features of the failed test case, the text data comprehensively considers the keywords, semantic features and response body information returned by the console of the log, and the attribute data comprehensively considers the test progress stage, the communication problem state, the response body return code, the uniform resource locator feature and the interface type. Figure 2 is a schematic diagram of the feature engineering according to the embodiments of the present application, as shown in Figure 2 The word frequency feature, the word vector feature, the time difference feature, the keyword feature and the statistical feature are extracted from the data of the historical failed test case. The time sequence data, the text data and the attribute data are obtained based on the feature engineering, and the time sequence data, the text data and the attribute data and the corresponding failure cause are taken as a group of training samples. A plurality of groups of training samples corresponding to a plurality of historical failed test cases are input into a preset neural network model for training, and a target model is obtained. The failure cause of the current failed test case is analyzed by training the target model.

[0042] In order to guarantee the accuracy of the failure cause, the actual failure cause needs to be determined according to a preset rule. Optionally, in the test case analysis method provided in the embodiments of the present application, before the execution method of the test case is adjusted based on the failure cause, the method further includes: determining the actual failure cause of the failed test case through a preset rule; judging whether the failure cause is the same as the actual failure cause; and in the case where the failure cause is not the same as the actual failure cause, adjusting the execution method of the test case based on the actual failure cause.

[0043] Specifically, the preset rule can be a rule summarized by a test personnel according to expert experience. The actual failure cause is obtained by analyzing the failed test case through the preset rule, and the execution method of the test case is adjusted according to the actual failure cause. The analysis accuracy of the target model is guaranteed by determining the actual failure cause.

[0044] Optionally, in the test case analysis method provided by the embodiments of the present application, in the case where the failure cause is not the same as the actual failure cause, the method further comprises: obtaining a plurality of groups of correction samples in a preset period, wherein each group of correction samples comprises a failure test case and an actual failure cause; correcting the target model based on the plurality of groups of correction samples to obtain a corrected target model; and replacing the target model with the corrected target model.

[0045] Specifically, in the case where the failure cause is not the same as the actual failure cause, it indicates that the target model has an error in analyzing the failure cause of the current failure test case, and at this time, the target model needs to be corrected in time. The failure test case with different failure cause and actual failure cause is used as a correction sample, and a plurality of groups of correction samples in a day are collected to train the target model to obtain a corrected target model. The analysis accuracy of the target model is ensured by correcting the target model.

[0046] According to another embodiment of the present application, a test case failure cause prediction method is provided. Figure 3 is a flowchart of the test case failure cause prediction method provided by the embodiments of the present application. As shown in Figure 3 , the method comprises:

[0047] After the failure case matching the failure cause and the model predicted failure cause on the first day, the test personnel verify the failure case and input the alpha version model for training to obtain a beta version model, and the beta version model is used as a prediction model of the second day. The test case not matching the failure cause obtained in real time on the second day and all test cases not matching the failure cause on the first day are input into the beta version model to predict the result, and the failure cause is updated according to the predicted result, and the test personnel verify the failure cause, and the failure case for verification is used as a training sample of the next day.

[0048] The test case failure cause prediction method provided by the embodiments of the present application comprehensively considers the whole life cycle of the current failure case. Not only the bug content is analyzed, but also the stability of the case running, the particularity of the log and the inherent characteristics of the case are considered, so that the failure cause of the test case is analyzed more comprehensively and efficiently.

[0049] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a group of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0050] The embodiment of the present application further provides a test case analysis device. It should be noted that the test case analysis device of the embodiment of the present application can be used to execute the test case analysis method provided by the embodiment of the present application. The test case analysis device provided by the embodiment of the present application is introduced as follows.

[0051] Figure 4 FIG. 1 is a schematic diagram of a test case analysis device provided by the embodiment of the present application. As shown in the figure, the device comprises: Figure 4

[0052] The acquisition unit 10 is configured to acquire time sequence data, text data and attribute data of a failed test case, wherein the time sequence data is data representing stability in a test case execution process, the text data is data representing particularity of logs in the test case execution process, and the attribute data is data representing uniqueness of the test case.

[0053] The input unit 20 is configured to input the time sequence data, the text data and the attribute data into a target model to obtain a failure reason, wherein the target model is trained by a plurality of training samples, and each training sample comprises time sequence data, text data, attribute data and a failure reason of a historical failed test case.

[0054] The adjustment unit 30 is configured to adjust an execution method of the test case based on the failure reason.

[0055] The test case analysis device provided by the embodiment of the present application acquires, by the acquisition unit 10, time sequence data, text data and attribute data of a failed test case, wherein the time sequence data is data representing stability in a test case execution process, the text data is data representing particularity of logs in the test case execution process, and the attribute data is data representing uniqueness of the test case. The input unit 20 inputs the time sequence data, the text data and the attribute data into a target model to obtain a failure reason, wherein the target model is trained by a plurality of training samples, and each training sample comprises time sequence data, text data, attribute data and a failure reason of a historical failed test case. The adjustment unit 30 adjusts an execution method of the test case based on the failure reason, thereby solving the problem of low efficiency in analyzing a failure reason of a failed test case in the related art, comprehensively analyzing the failure reason by inputting the time sequence data, the text data and the attribute data of the failed test case into the target model, and achieving the effect of more comprehensive and efficient analysis of the failure reason of the test case.

[0056] ​Optionally, in the test case analysis apparatus provided by the embodiment of the present application, the obtaining unit 10 comprises: a first obtaining module, configured to obtain the execution time and the execution state of the failed test case in the preset period, wherein the execution state comprises execution success and execution failure; a first determining module, configured to determine a group of first execution times of each execution process of the execution failure and determine a plurality of second execution times of the execution success; a calculating module, configured to calculate the difference between each third execution time and the first execution time to obtain a group of difference values; and a second determining module, configured to determine a target value of the group of difference values, and determine the target value as the timing data, wherein the target value comprises at least one of the following: the maximum value, the minimum value, the median and the variance.

[0057] Optionally, in the test case analysis apparatus provided by the embodiment of the present application, the obtaining unit 10 comprises: a second obtaining module, configured to obtain the target log associated with the failed test case and the response body information returned by the console running the failed test case, wherein the target log comprises at least one of the following: the integrated tool scheduling log, the console log and the error stack log; an extracting module, configured to extract at least one keyword from the target log and input the keyword into a preset semantic extraction model to obtain semantic features; and a third determining module, configured to determine the keyword, the semantic features and the response body information as the text data.

[0058] Optionally, in the test case analysis apparatus provided by the embodiment of the present application, the obtaining unit 10 comprises: a fourth determining module, configured to determine the test framework type, the application type, the test progress stage, the plate mounting period state, the communication problem state, the response body return code, the uniform resource locator feature and the interface type of the failed test case; and a fifth determining module, configured to determine at least one of the test framework type, the application type, the test progress stage, the plate mounting period state, the communication problem state, the response body return code, the uniform resource locator feature and the interface type as the attribute data.

[0059] Optionally, in the test case analysis apparatus provided by the embodiment of the present application, the apparatus further comprises: a historical failure reason obtaining unit, configured to obtain each historical failed test case and the corresponding historical failure reason, and extract the timing data, the text data and the attribute data in the historical failed test case; a first determining unit, configured to determine the timing data, the text data, the attribute data and the historical failure reason in the historical failed test case as a group of training samples to obtain a plurality of groups of training samples; and a training unit, configured to input the plurality of groups of training samples into a preset model for training to obtain a target model.

[0060] Optionally, in the test case analysis apparatus provided by the embodiment of the present application, the apparatus further comprises: a second determination unit configured to determine an actual failure cause of the failed test case according to a preset rule; a judgment unit configured to judge whether the failure cause and the actual failure cause are the same; and an actual failure cause adjustment unit configured to adjust the execution method of the test case based on the actual failure cause when the failure cause and the actual failure cause are not the same.

[0061] Optionally, in the test case analysis apparatus provided by the embodiment of the present application, the apparatus further comprises: a correction sample acquisition unit configured to acquire a plurality of groups of correction samples in a preset period, wherein each group of correction samples comprises a failed test case and an actual failure cause; a correction unit configured to correct the target model based on the plurality of groups of correction samples to obtain a corrected target model; and a replacement unit configured to replace the target model with the corrected target model.

[0062] The test case analysis apparatus comprises a processor and a memory, and the above-mentioned acquisition unit 10, input unit 20 and adjustment unit 30 are all stored in the memory as program units, and the corresponding functions are realized by the processor executing the program units stored in the memory.

[0063] The processor comprises a core, and the core retrieves the corresponding program units from the memory. One or more than one core can be set, and the failure cause of the test case can be analyzed more comprehensively and efficiently by adjusting the core parameters.

[0064] The memory can comprise a non-permanent memory in a computer readable medium, random access memory (RAM) and / or non-volatile memory such as read-only memory (ROM) or flash memory (flash RAM), and the memory comprises at least one memory chip.

[0065] The embodiment of the present application provides a computer readable storage medium, which stores a program, and the program is executed by a processor to realize the test case analysis method.

[0066] The embodiment of the present application provides a processor, which is used for running a program, and the program is executed to realize the test case analysis method.

[0067] The embodiment of the present application provides an electronic device, Figure 5 is a schematic diagram of the electronic device provided by the embodiment of the present application. As shown in the figure, Figure 5 The device 501 comprises a processor, a memory and a program stored in the memory and executable on the processor, and the processor realizes the following steps when executing the program: the test case analysis method. The device in the present application can be a server, a PC, a PAD, a mobile phone and the like.

[0068] The present application also provides a computer program product adapted to perform a program that initializes the method steps of testing a case analysis method when executed on a data processing device.

[0069] Those skilled in the art will appreciate that embodiments of the present application can be readily used as a method, apparatus (system) or computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0070] The present application is described in reference to the flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing system or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks.

[0071] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks.

[0072] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks.

[0073] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0074] Memory can include non-persistent memory, Random Access Memory (RAM), and / or non-volatile memory, such as read only memory (ROM) or flash memory, etc., in a computer readable medium. Memory is an example of computer readable media.

[0075] Computer readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile discs (DVD) or other optical storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer readable media does not include transitory media, such as modulated data signals and carrier waves.

[0076] It should also be noted that the terms "comprising", "containing", or any other similar term are intended to encompass non-exclusive inclusions, such that a process, method, article, or apparatus that comprises a list of elements does not include those elements solely, but can also include other elements not expressly listed, or also include inherent elements of such process, method, article, or apparatus. Without more limitations, an element defined by the phrase "comprising a" does not exclude the existence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0077] Those skilled in the art will understand that embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product implemented on one or more computer available storage media (including, but not limited to, disk memory, CD-ROMs, optical storage media, etc.) containing computer usable program code.

[0078] The above merely provides an example of the present application and is not intended to limit the present application. Various modifications and changes can be made by those skilled in the art to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of claims of the present application.

Claims

1. A method of analyzing test cases, characterized by, The method comprises: obtaining time sequence data, text data and attribute data of a failed test case, wherein the time sequence data is data representing stability in the execution process of the test case, the text data is data representing the particularity of logs in the execution process of the test case, and the attribute data is data representing the uniqueness of the test case; inputting the time sequence data, the text data and the attribute data into a target model to obtain a failure cause, wherein the target model is trained by a plurality of training samples, and each training sample comprises time sequence data, text data, attribute data and a failure cause of a historical failed test case; adjusting the execution method of the test case based on the failure cause; wherein the time sequence data of the failed test case is obtained by obtaining the execution time and execution state of each execution process of the failed test case within a preset period, wherein the execution state comprises execution success and execution failure; determining a first execution time when the failed test case is found, determining a group of second execution times with execution failure, and determining a third execution time adjacent to each second execution time with execution success, to obtain a plurality of third execution times; calculating the difference between each third execution time and the first execution time to obtain a group of difference values; determining a target value of the group of difference values, and determining the target value as the time sequence data, wherein the target value comprises at least one of the following: maximum value, minimum value, median value and variance; obtaining the text data of the failed test case comprises: obtaining a target log associated with the failed test case, and response body information returned by a console running the failed test case, wherein the target log comprises at least one of the following: integrated tool scheduling log, console log and error stack log; extracting at least one keyword from the target log, and inputting the keyword into a preset semantic extraction model to obtain semantic features; and determining the keyword, the semantic features and the response body information as the text data; the attribute data comprises at least one of the following: test framework type, application type, test progress stage, plate mounting period state, communication problem state, response body return code, uniform resource locator feature and interface type of the failed test case.

2. The method of claim 1, wherein, Obtaining attribute data of a failed test case comprises: determining the test framework type, the application type, the test progress stage, the plate mounting period state, the communication problem state, the response body return code, the uniform resource locator feature and the interface type of the failed test case; determining at least one of the test framework type, the application type, the test progress stage, the plate mounting period state, the communication problem state, the response body return code, the uniform resource locator feature and the interface type as the attribute data.

3. The method of claim 1, wherein, Before inputting the time sequence data, the text data and the attribute data into the target model, the method further comprises: obtaining each historical failed test case and a corresponding historical failure cause, and extracting time sequence data, text data and attribute data from the historical failed test case; The time sequence data, the text data, the attribute data and the historical failure reason in the historical failure test case are taken as a set of training samples, and a plurality of sets of training samples are obtained; The plurality of sets of training samples are input into a preset model for training, and the target model is obtained.

4. The method of claim 1, wherein, Before the method of adjusting the execution method of the test case based on the failure reason, the method further comprises: determining an actual failure reason of the failure test case through a preset rule; determining whether the failure reason is the same as the actual failure reason; in the case that the failure reason is not the same as the actual failure reason, adjusting the execution method of the test case based on the actual failure reason.

5. The method of claim 4, wherein, In the case that the failure reason is not the same as the actual failure reason, the method further comprises: obtaining a plurality of sets of correction samples in a preset period, wherein each set of correction samples comprises a failure test case and an actual failure reason; correcting the target model based on the plurality of sets of correction samples to obtain a corrected target model; replacing the target model with the corrected target model.

6. An analysis apparatus of a test case, characterized by, comprises: an acquisition unit, configured to acquire time sequence data, text data and attribute data of a failure test case, wherein the time sequence data is data representing stability in a test case execution process, the text data is data representing particularity of logs in the test case execution process, and the attribute data is data representing uniqueness of the test case; an input unit, configured to input the time sequence data, the text data and the attribute data into a target model to obtain a failure reason, wherein the target model is trained by a plurality of sets of training samples, and each set of training samples comprises time sequence data, text data, attribute data and a failure reason of a historical failure test case; an adjustment unit, configured to adjust an execution method of a test case based on the failure reason; wherein the acquisition unit comprises: a first acquisition module, configured to acquire execution time and execution state of each execution process of the failure test case in a preset period, wherein the execution state comprises execution success and execution failure; a first determination module, configured to determine a first execution time when the failure test case is found, determine a plurality of second execution times with execution failure, and determine a plurality of third execution times with execution success adjacent to each second execution time; a calculation module, configured to calculate a difference between each third execution time and the first execution time to obtain a set of differences; and a second determination module, configured to determine a target value of the set of differences, and determine the target value as the time sequence data, wherein the target value at least comprises one of the following: maximum value, minimum value, median and variance. The acquisition unit further comprises: a second acquisition module, configured to acquire a target log associated with the failed test case and response body information returned by a console running the failed test case, wherein the target log comprises at least one of the following: an integrated tool scheduling log, a console log and an error stack log; an extraction module, configured to extract at least one keyword from the target log and input the keyword into a preset semantic extraction model to obtain semantic features; and a third determination module, configured to determine the keyword, the semantic features and the response body information as the text data. The attribute data comprises at least one of the following: a test framework type, an application type, a test progress stage, a plate mounting period state, a communication problem state, a response body return code, a uniform resource locator feature and an interface type of the failed test case.

7. A processor, comprising: The processor is configured to run a program, wherein the program, when running, performs the test case analysis method of any one of claims 1 to 5.

8. An electronic device, comprising: The processor is configured to run a program, wherein the program, when running, performs the test case analysis method of any one of claims 1 to 5. The processor is configured to run a program, wherein the program, when running, performs the test case analysis method of any one of claims 1 to 5.

Images