Test case priority decision-making method and device, electronic equipment and storage medium

Abstract

The invention discloses a test case priority decision-making method, device and system and electronic equipment, and belongs to the technical field of computers. The method comprises the following steps: selecting at least one sorted test case set as a source test case set according to the similarity between sorted test case sets and a target test case set; obtaining a first sorting index value of the test cases in the target test case set according to the similarity between the test cases in the source test case set and the test cases in the target test case set and the known sorting of the single test cases in the source test case set; according to historical performance data of the test cases in the sorted test case set, obtaining at least one second sorting index value of the test cases in the target test case set; and performing priority decision on the test cases in the target test case set in combination with the first sorting index value and the second sorting index value. According to the method, the accuracy of the priority decision of the test case is effectively improved, so that the regression test effect can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a test case priority decision method, a test case priority decision device, an electronic device, and a computer-readable storage medium. Background Art

[0002] The decision of test case priority is to determine the order in which test cases are executed according to certain rules, which is an important test management task. Test case sorting can significantly improve test efficiency, so that test cases with high priority can be executed as early as possible, thereby discovering defects faster. Regression testing plays an extremely important role in the life cycle of software development. It can ensure software quality and stability, support continuous integration and continuous delivery, and effectively reduce the risk of introducing new code. However, when the test cases involved in regression testing are too complicated, and the time or manpower currently used for regression testing is seriously insufficient, all test cases in the test case set cannot be fully executed. In the prior art, when all test cases in the test case set cannot be fully executed due to factors such as time or manpower shortage, and only some test cases can be executed, there is a solution to solve the above problem by selecting test cases with high priority for priority execution. The decision of test case priority directly affects the selection of test cases, thereby affecting the test effect.

[0003] It can be seen that the decision-making method for test case priority needs to be continuously optimized. Summary of the invention

[0004] The embodiments of the present application provide a method and device for deciding test case priority, which helps to improve the accuracy of test case priority setting, thereby improving the test effect.

[0005] In a first aspect, an embodiment of the present application provides a method for determining a priority of a test case, including:

[0006] According to the similarity between the sorted test case set and the target test case set, selecting at least one of the sorted test case sets as a source test case set;

[0007] Obtaining a first ranking index value of the test case in the target test case set according to the similarity between the test cases in the source test case set and the test cases in the target test case set, and the known ranking of a single test case in the source test case set;

[0008] Acquire at least one second ranking index value of the test cases in the target test case set according to the historical performance data of the test cases in the sorted test case set;

[0009] According to the first sorting index value and the second sorting index value, priority decisions are made on the test cases in the target test case set.

[0010] In a second aspect, an embodiment of the present application provides a test case priority decision device, including:

[0011] A source test case set selection module, configured to select at least one of the sorted test case sets as a source test case set according to the similarity between the sorted test case sets and the target test case set;

[0012] A first ranking index value acquisition module is used to acquire a first ranking index value of a test case in the target test case set according to the similarity between the test cases in the source test case set and the test cases in the target test case set, and a known ranking of a single test case in the source test case set;

[0013] A second ranking index value acquisition module, configured to acquire at least one second ranking index value of a test case in the target test case set according to historical performance data of the test case in the sorted test case set;

[0014] A priority decision module is used to make a priority decision on the test cases in the target test case set according to the first sorting index value and the second sorting index value.

[0015] In a third aspect, an embodiment of the present application further discloses an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the test case priority decision method described in the embodiment of the present application when executing the computer program.

[0016] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the test case priority decision method disclosed in the embodiment of the present application.

[0017] The test case priority decision method disclosed in the embodiment of the present application selects at least one sorted test case set as a source test case set based on the similarity between the sorted test case set and the target test case set; obtains a first sorting index value of the test case in the target test case set based on the similarity between the test cases in the source test case set and the target test case set, and the known sorting of a single test case in the source test case set; on the other hand, obtains at least one second sorting index value of the test case in the target test case set based on the historical performance data of the test cases in the sorted test case set; finally, priority decisions are made on the test cases in the target test case set based on priority factors of multiple dimensions such as the first sorting index value and the second sorting index value, thereby effectively improving the accuracy of test case priority decisions, thereby helping to improve regression testing results.

[0018] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0020] Figure 1 It is a flow chart of the decision method for test case priority disclosed in the embodiment of the present application;

[0021] Figure 2 It is a schematic diagram of the principle of the test case priority decision method disclosed in the embodiment of the present application;

[0022] Figure 3 is a topological diagram of the LDA model disclosed in the embodiment of the present application;

[0023] Figure 4 It is a structural diagram of the LDA model disclosed in the embodiment of the present application;

[0024] Figure 5 It is a schematic diagram of the structure of the decision-making device for the test case priority disclosed in the embodiment of the present application;

[0025] Figure 6A block diagram schematically shows an electronic device for executing the method according to the present application; and

[0026] Figure 7 A storage unit for holding or carrying a program code for implementing the method according to the present application is schematically shown. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0028] Reference Figure 1 , a test case priority decision method disclosed in an embodiment of the present application includes: steps 110 to 140.

[0029] Figure 4 Schematic diagram of the principle of the test case priority decision method described in the embodiment of this application. Figure 1 and Figure 4 The specific implementation steps of the test case priority decision method described in the embodiments of the present application are explained.

[0030] Step 110: Select at least one of the sorted test case sets as a source test case set according to the similarity between the sorted test case sets and the target test case set.

[0031] Wherein, the target test case set is a set of test cases used for regression testing. The sorted test case set refers to a set of test cases that have executed the test task. In an embodiment of the present application, the sorted test case set includes one or more test cases, and the test cases included in the sorted test case set are known sorting results, wherein the sorting results include but are not limited to: priority values, sorting positions, etc. The sorting results are used to indicate the priority at which the corresponding test case is used for testing. In addition, the test cases included in the sorted test case set are associated with historical test data, wherein each of the historical test data associated with the test case includes but is not limited to the test code coverage and defect detection capability index value of the test case. Wherein, the test code coverage is positively correlated with the number of times the test case is used in the test task, and the defect detection capability index value is positively correlated with the number of times the test case is used to detect program defects.

[0032] The historical test data associated with the test case is obtained by analyzing the test results obtained by executing the test task using the test case. The method for obtaining the historical test data associated with the test case refers to the prior art, and the method for obtaining the historical test data associated with the test case is not limited in the embodiments of the present application.

[0033] In some optional embodiments, selecting at least one of the sorted test case sets as a source test case set based on the similarity between the sorted test case set and the target test case set includes: sub-step 1101, sub-step 1102, sub-step 1103 and sub-step 1104.

[0034] The specific implementation of each sub-step is described below.

[0035] Sub-step 1101, taking a single test case set as a document, and obtaining the text content vectors of each test case set respectively through a word frequency-inverse document frequency model.

[0036] Wherein, the test case set is selected from the sorted test case set and the target test case set.

[0037] The core of TF-IDF (term frequency - inverse document frequency) is to use the term frequency (TF) indicator to measure the importance of keywords in the test case set, and to use the inverse document frequency (IDF) indicator to evaluate the importance of keywords in the entire corpus. TF and IDF are combined to calculate the TF-IDF value. The similarity between test case sets can be measured by comparing the TF-IDF values ​​of different test case sets.

[0038] In some optional embodiments, taking a single test case set as a document, obtaining the text content vectors of each test case set respectively through a word frequency-reverse file frequency model, including: obtaining a first corpus consisting of the sorted test case set and the target test case set; taking a single test case set as a document, obtaining the keywords included in the first corpus, the first number of occurrences of each keyword in a specified test case set, the second number of occurrences of each keyword in the first corpus, the total number of test case sets, and the number of test case sets in which each keyword appears; for each keyword, obtaining the word frequency-reverse file frequency of the keyword according to the first number of occurrences, the second number of occurrences, the total number of test case sets, and the number of test case sets in which the keyword appears; and obtaining the text content vectors of the specified test case set according to the word frequency and the reverse file frequency of each keyword in the specified test case set.

[0039] First, obtain a sorted test case set and a target test case set (i.e., a test case set for regression testing), and form a corpus with the sorted test case set and the target test case set, which is referred to as the "first corpus" in the embodiment of the present application. Afterwards, preprocess each test case in the first corpus to obtain a list of keywords corresponding to the first corpus. For example, each test case in the first corpus can be segmented, and then punctuation marks, special conjunctions, and other meaningless words are deleted, and the remaining segmented words are used as keywords corresponding to the first corpus. On the other hand, the keywords included in each test case set, the number of occurrences of each keyword in each test case set, and the number of occurrences of each keyword in the first corpus can be obtained according to the method.

[0040] Next, the TF-IDF (Term Frequency–Inverse Document Frequency) word vectorization model is used to obtain the text content vectors of each of the test case sets. The TF-IDF word vectorization model consists of two parts: word frequency and inverse document frequency. The TF-IDF word vectorization model is used to measure the importance of keywords in the corpus. In an embodiment of the present application, the word frequency-inverse document frequency of each keyword in the test case set is used to characterize the text content vector of the test case set.

[0041] The text content vector of the j-th test case set can be represented by a sequence of the word frequency-inverse document frequency of the keywords included in the j-th test case set. For example, the word frequency-inverse document frequency TF-IDF of the i-th keyword in the j-th test case set is ij It can be expressed as: TF-IDF ij =TF ij ×IDF i , where TF ij It represents the frequency of the ith keyword in the jth test case set, which can be expressed by the formula Calculate, IDF i The inverse document frequency of the ith keyword can be expressed by the formula calculate.

[0042] In the above formula, n ij represents the number of times the i-th keyword appears in the j-th test case set, that is, the first occurrence number; K represents the total number of keywords corresponding to the first corpus, represents the total number of times the keyword appears in the jth test case set, that is, the second number of occurrences, N1 represents the total number of sorted test case sets and target test case sets in the first corpus, and a represents the number of test case sets in which the i-th keyword appears in the first corpus.

[0043] In some optional embodiments, in order to further simplify subsequent processing, the TF-IDF of the keyword can be normalized so that the value range of TF-IDF is unified to [0, 1]. The normalization formula for the TF-IDF of the keyword is as follows: in, represents the normalized TF-IDF value of the i-th keyword in the j-th test case set, and MAX(TF-IDF) represents the maximum TF-IDF value of all keywords.

[0044] In some optional embodiments, for the jth test case set, the normalized TF-IDF values ​​corresponding to each keyword in the test case set can be arranged in the order in which the keywords appear in the jth test case set to generate a one-dimensional vector as the text content vector of the jth test case set.

[0045] Sub-step 1102, using text topic mining technology to obtain the text topic vectors of each of the test case sets.

[0046] Latent Dirichlet Allocation (LDA) is a probabilistic topic generation model. The LDA model can effectively mine the potential information in the text or find the relationship between two texts without the same words. In the LDA model, all the texts in a test case set can be regarded as the topics in the latent topic set combined according to a certain probability. The LDA model can be divided into three layers: text collection layer, text layer and feature word layer. The topological structure of the LDA model is as follows: Figure 2 As shown in Figure 2. Each document is represented as a set of words, and each topic is composed of a set of specific words. The goal of the LDA model is to discover some hidden topics from the document set, which can explain the distribution of words in the document set.

[0047] The LDA model is based on the following assumptions: there are k independent topics in the entire document collection; each topic is a multinomial distribution over words; each document is composed of a random mixture of k topics; each document is a multinomial distribution over k topics; the prior distribution of the topic probability distribution of each document is the Dirichlet distribution (i.e., Dirichlet distribution); the prior distribution of the probability distribution of words in each topic is the Dirichlet distribution.

[0048] In the LDA model, the document is the basic unit of LDA model processing, the topic is the abstract concept hidden in the document, each topic is associated with a specific set of words; the word is the specific vocabulary in the document and is the basic element that constitutes the topic.

[0049] like Figure 4 Taking the structural diagram of the LDA model shown as an example, in the prior art, the training steps of the LDA model are as follows: given a document set, the model parameters α, β and θ are learned through an optimization algorithm (such as Gibbs sampling or variational inference); then, the learned model parameters are used to infer the topic distribution of new documents. For example, for the N1th word in the test case M, an implicit topic Z_m of the document M is first sampled according to the θ distribution.

[0050] From the above training steps of the LDA model, it can be concluded that the LDA model can be used to extract text topics from test cases as feature vectors of the test cases to calculate the similarity between test cases.

[0051] In some optional embodiments, the text topic mining technology is used to obtain the text topic vectors of each of the test case sets, including: obtaining a second corpus consisting of the sorted test case set and the target test case set; taking a single test case as a document, training a topic extraction model based on the second corpus to obtain the text topic vectors of each of the test case sets. Among them, the topic extraction model is an LDA model. In specific implementation, a test case can be regarded as a document, and the topic is the abstract concept hidden in the test case set. Each topic is associated with a specific set of words; words are specific vocabulary in the test case and are the basic elements that constitute the topic.

[0052] First, a second corpus is formed based on the sorted test case set and the target test case set. Then, each test case in the second corpus is used as a document to preprocess the test case set to obtain a test case expressed as a keyword sequence, for example, as: pre ={dp1,dp2,…,dp n}, where dp i represents the keyword sequence corresponding to the i-th test case set, and n represents the total number of test case sets. Finally, the keyword sequence dp obtained after preprocessing a single test case is i Considering it as a text, the LDA model is trained on the second corpus for topic modeling, and the topic distribution p(z|d) corresponding to each test case set is calculated, where z represents the topic and d represents the test case set. Furthermore, for each test case set, according to the topic distribution of all hidden topics on the test case set, the text topic vector representation E of the test case set can be obtained.

[0053] Taking a single test case as a document, training a topic extraction model based on the second corpus to obtain the text topic vector of each test case set can be referred to the implementation method of extracting topics based on several documents in the prior art, which will not be repeated in the embodiments of this application.

[0054] Sub-step 1103, for each of the test case sets, combining the corresponding text content vector and the text topic vector, respectively calculate the similarity score between each of the sorted test case sets and the target test case set.

[0055] Next, the correlation between the test case sets is calculated, and the most appropriate sorted test case set is selected as the source test case set.

[0056] In order to further explore the correlation between the sorted test case set and the target test case set, the embodiment of the present application combines the traditional TF-IDF with the LDA model-based method to mine text information from the two dimensions of feature words and text topics, thus making up for the shortcomings of the two methods.

[0057] For any test case set d i According to the above steps, the text content vector obtained by using the TF-IDF algorithm can be expressed as: i(TF-IDF) ={w1, W2, ..., w n}, where n represents the test case set d i The number of feature words in w j Represents the test case set d i The TF-IDF value of the jth keyword in .

[0058] On the other hand, for any test case set d i , the text topic vector obtained based on the LDA topic model in the previous step can be expressed as: i(LDA) ={e1, e2, ..., e T}, where T represents the number of implicit topics corresponding to the second corpus, e1, e2, …, e T Represents the test case set d i The distribution of each hidden topic. The hidden topics corresponding to the second corpus and the number of hidden topics are obtained by training the LDA model based on the second corpus.

[0059] After that, the sorted test case set d can be calculated using the following formulas: i and the target test case set d j The similarity based on the text content vector and the similarity based on the text topic vector are used to determine the test case set d based on the two similarities. iand d j The similarity score L(f i ,f j ):

[0060]

[0061] Among them, f i(IF-IDF) Represents the test case set d i The text content vector, f i(LDA) Represents the test case set d i The text topic vector, f j(TF-IDF) Represents the test case set d j The text content vector, f j(LDA) Represents the test case set d j The text topic vector, L TF-IDF () represents the text content similarity calculation function, L LDA () represents the text topic similarity calculation function, represents the linear correlation coefficient, The value is set according to application requirements.

[0062] According to the above method, the similarity scores between the target test case set and each sorted test case set can be calculated.

[0063] Sub-step 1104: selecting at least one of the sorted test case sets as a source test case set according to the similarity score.

[0064] In some optional embodiments, the sorted test case set with the largest pre-target test case set similarity score may be used as the source test case set.

[0065] Step 120: Obtain a first ranking index value of the test case in the target test case set based on the similarity between the test cases in the source test case set and the test cases in the target test case set, and the known ranking of a single test case in the source test case set.

[0066] In some optional embodiments, obtaining a first ranking index value of the test cases in the target test case set based on the similarity between the test cases in the source test case set and the target test case set, and the known ranking of a single test case in the source test case set, includes: sub-step 1201, sub-step 1202, sub-step 1203 and sub-step 1204.

[0067] Sub-step 1201, taking a single test case as a document, and using a word frequency-inverse document frequency model, respectively obtain the text content vector of each test case in the source test case set and the target test case set.

[0068] Wherein, the single test case is selected from the source test case set and the target test case set.

[0069] In specific implementation, the source test case set and the target test case set can be used to form a new corpus, for example, recorded as the "third corpus". Afterwards, each test case in the third corpus is preprocessed to obtain a list of keywords corresponding to the third corpus. For example, each test case in the third corpus can be segmented separately, and then punctuation marks, special conjunctions and other meaningless words are deleted, and the remaining segmented words are used as keywords corresponding to the third corpus. On the other hand, the keywords included in each test case, the number of occurrences of each keyword in each test case, and the number of occurrences of each keyword in the third corpus can be obtained according to the above method.

[0070] Next, the TF-IDF (Term Frequency–Inverse Document Frequency) word vectorization model is used to obtain the text content vectors of each of the test cases in the third corpus. In an embodiment of the present application, the term frequency-inverse document frequency of each keyword in the test case set is used to characterize the text content vector of the test case set.

[0071] The text content vector of the qth test case may be represented by a sequence of word frequency in the qth test case and inverse document frequency of the keywords included in the qth test case.

[0072] For example, the term frequency of the p-th keyword in the q-th test case is the inverse document frequency TF-IDF pq It can be expressed as: TF-IDF pq =TF pq ×IDF pq , where TF pq It represents the frequency of the p-th keyword in the q-th test case, which can be expressed by the formula Calculate, IDF p The inverse document frequency of the pth keyword can be expressed by the formula Calculate. In the above formula, n pq represents the number of times the p-th keyword appears in the q-th test case; Y represents the total number of keywords corresponding to the third corpus, represents the total number of times the keyword appears in the qth test case, M1 represents the total number of test cases used and to-be-used in the third corpus, and b represents the number of test cases in which the pth keyword appears.

[0073] In some optional embodiments, in order to further simplify subsequent processing, the TF-IDF of the keyword can be normalized so that the value range of TF-IDF is unified to [0, 1]. The specific implementation method of the TF-IDF normalization of the keyword is described above and will not be repeated here.

[0074] In some optional embodiments, for the qth test case, the normalized TF-IDF values ​​corresponding to each keyword in the test case can be arranged in the order in which the keywords appear in the qth test case to generate a one-dimensional vector as the text content vector of the qth test case.

[0075] Sub-step 1202, using text topic mining technology to obtain text topic vectors of each test case in the source test case set and the target test case set respectively.

[0076] In this step, a single test case in the source test case set and the target test case set is taken as a document, and an LDA model is trained on the third corpus to perform topic modeling, and a text topic vector corresponding to each test case is calculated.

[0077] The specific implementation method of using text topic mining technology to obtain the text topic vector of each test case in the source test case set and the target test case set can be found in the prior art and will not be repeated in the embodiments of this application.

[0078] Sub-step 1203, for each of the test cases in the source test case set, combining the text content vector and the text topic vector, respectively calculates the similarity score between each of the test cases in the target test case set and each of the test cases in the source test case set.

[0079] In some optional embodiments, for each test case in the target test case set, the L TF-IDF () function calculates the text content similarity between the test case in the target test case set and each test case in the source test case set, using the L LDA () function calculates the text topic similarity between the test case in the target test case set and each test case in the source test case set.

[0080] Next, for two test cases, the weighted sum of the text content similarity and the text topic similarity between the two test cases can be used as the similarity score of the two test cases.

[0081] Sub-step 1204: acquiring a first ranking index value of the test cases in the target test case set according to the similarity score and the known ranking of each test case in the source test case set.

[0082] In some optional embodiments, the ranking score or ranking position of the test case in the source test case set can be used as the ranking score or ranking position of the test case with the highest similarity score to the test case in the target test case set, as the first ranking index value of the test case in the target test case set. Taking the test case with the highest priority in the source test case set as TermS1 as an example, assuming that the priority score of the test case TermS1 is Score1, the priority score of the test case with the highest similarity score to the test case TermS1 in the test case to be used can be set to Score1.

[0083] After this step is processed, the first ranking index value of each test case in the target test case set can be obtained.

[0084] Step 130: Obtain at least one second ranking index value of the test cases in the target test case set according to the historical performance data of the test cases in the sorted test case set.

[0085] In some optional embodiments, the historical performance data includes, but is not limited to: test code coverage, and defect detection capability index value.

[0086] In an embodiment of the present application, the ranking index value of the test case in the target test case set, that is, the second ranking index value, can also be predicted based on the historical performance data of the used test cases similar to the test case to be used in the historical testing process.

[0087] In some optional embodiments, obtaining at least one second sorting index value of the test cases in the target test case set based on the historical performance data of the test cases in the sorted test case set includes one or more of the following: obtaining the second sorting index value of the corresponding test cases in the target test case set based on the test code coverage of the test cases in the sorted test case set; obtaining the second sorting index value of the corresponding test cases in the target test case set based on the defect detection capability index value of the test cases in the sorted test case set.

[0088] In an embodiment of the present application, after executing the test case, the test code coverage and defect detection capability index value of each test case can be obtained by analyzing the execution effect of the test case. When determining the priority of unused test cases, for any candidate test case in the target test case set, the test code coverage and defect detection capability index value of the test case with the highest similarity to the candidate test case in the used test case set can be used as the test code coverage and defect detection capability index value of the candidate test case, respectively, to facilitate the subsequent calculation of the priority of the candidate test case in combination with the test code coverage and defect detection capability index value.

[0089] Step 140: Make a priority decision on the test cases in the target test case set according to the first sorting index value and the second sorting index value.

[0090] In some optional embodiments, priority decisions are made for the test cases in the target test case set based on the first sorting index value and the second sorting index value, including: obtaining the average value of the first sorting index value and the at least one second sorting index value, and using the average value as the comprehensive sorting index value of the corresponding test cases in the test case set; and determining the priority of the corresponding test cases in the test case set based on the comprehensive sorting index value.

[0091] In some optional embodiments, priority decisions are made for the test cases in the target test case set based on the first sorting index value and the second sorting index value, including: obtaining a weighted average of the first sorting index value and the at least one second sorting index value, and using the weighted average as a comprehensive sorting index value for the corresponding test cases in the test case set; and determining the priority of the corresponding test cases in the test case set based on the comprehensive sorting index value.

[0092] As mentioned above, the second sorting index value includes: test code coverage and defect detection capability index value.

[0093] In software testing, if a test case detects a program defect, it is generally believed that the test case has a high probability of detecting the program defect again. Therefore, in order to avoid the test case from reappearing, the test case is set to a higher priority in regression testing. In the embodiment of the present application, in order to quantify the impact of defect detection capability on the test case priority ranking, the defect detection capability of the test case is represented by a defect detection capability index value. Optionally, the defect detection capability index value of the test case can be calculated by the following formula: Among them, (Sort3) i represents the defect detection capability index value of the i-th test case, P 1irepresents the total number of times the i-th test case appears in the sorted test case set, p 2i It represents the total number of program defects detected when the i-th test case history is used for testing, and N represents the total number of test cases in the sorted test case set.

[0094] Test code coverage is a metric in software testing that describes the proportion and degree of source code in a program being tested, and the resulting proportion is called test code coverage. In the embodiments of the present application, the calculation method of the test code coverage of the test case is referred to in the prior art and will not be repeated here.

[0095] Optionally, the first sorting index value of the i-th test case is expressed as (Sort1) i , test code coverage is expressed as (Sort2) i , the defect detection capability index value is expressed as (Sort3) i For example, we can use the formula (Sort) i =((Sort1) i +(Sort2) i +(Sort3) i ) / 3 Calculate the priority of the i-th test case (Sort) i , or, by formula (Sort) i =(α(Sort1) i +β(Sort2) i +γ(Sort3) i ) / 3 Calculate the priority of the i-th test case (Sort) i , where α, β and γ represent the weights of the priority factors at different latitudes, and the values ​​of α, β and γ are determined according to application requirements. i It can be used as the comprehensive ranking index value of the i-th test case.

[0096] After calculating the priorities of the test cases according to the above method, test cases with high statement coverage and strong defect detection capabilities can be preferentially selected for regression testing, thereby improving the performance of regression testing.

[0097] At this point, the priority setting for each test case in the target test case set has been completed.

[0098] In summary, the test case priority decision method disclosed in the embodiment of the present application selects at least one sorted test case set as a source test case set based on the similarity between the sorted test case set and the target test case set; obtains a first sorting index value of the test case in the target test case set based on the similarity between the test cases in the source test case set and the target test case set, and the known sorting of a single test case in the source test case set; on the other hand, obtains at least one second sorting index value of the test case in the target test case set based on the historical performance data of the test case in the sorted test case set; finally, priority factors of multiple dimensions such as the first sorting index value and the second sorting index value are combined to make priority decisions for the test cases in the target test case set, thereby effectively improving the accuracy of test case priority decisions, thereby helping to improve the regression testing effect.

[0099] Furthermore, the present application builds a bridge between source test cases and target test cases by combining the text content similarity and text topic similarity of test cases, and migrates the implicit information in historical test data to the priority sorting of target test cases, providing a new solution for test case priority sorting and realizing resource reuse.

[0100] Based on the above embodiments, the present application also discloses a test case priority decision device, such as Figure 5 As shown, the device comprises:

[0101] A source test case set selection module 510, configured to select at least one of the sorted test case sets as a source test case set according to the similarity between the sorted test case sets and the target test case set;

[0102] A first ranking index value acquisition module 520, configured to acquire a first ranking index value of a test case in the target test case set according to the similarity between the test cases in the source test case set and the test cases in the target test case set, and a known ranking of a single test case in the source test case set;

[0103] A second ranking index value acquisition module 530 is used to acquire at least one second ranking index value of the test cases in the target test case set according to the historical performance data of the test cases in the sorted test case set;

[0104] The priority decision module 540 is used to make a priority decision on the test cases in the target test case set according to the first sorting index value and the second sorting index value.

[0105] In some optional embodiments, the source test case set selection module 510 is further used to:

[0106] Taking a single test case set as a document, respectively obtaining text content vectors of each test case set through a word frequency-inverse document frequency model, wherein the test case set is selected from the sorted test case set and the target test case set;

[0107] Using text topic mining technology to obtain text topic vectors of each test case set;

[0108] For each of the test case sets, combining the corresponding text content vector and the text topic vector, respectively calculating the similarity score between each of the sorted test case sets and the target test case set;

[0109] At least one of the sorted test case sets is selected as a source test case set according to the similarity score.

[0110] In some optional embodiments, taking a single test case set as a document and obtaining text content vectors of each test case set respectively through a word frequency-inverse document frequency model includes:

[0111] Acquire a first corpus consisting of the sorted test case set and the target test case set;

[0112] Taking a single test case set as a document, obtaining keywords included in the first corpus, the first occurrence number of each keyword in each test case set, the second occurrence number of each keyword in the first corpus, the total number of test case sets, and the number of test case sets in which each keyword appears;

[0113] For each of the keywords, according to the first number of occurrences, the second number of occurrences, the total number of the test case sets, and the number of the test case sets in which the keyword appears, obtain the word frequency-inverse file frequency of the keyword;

[0114] According to the word frequency and the reverse file frequency of each of the keywords in the specified test case set, the text content vectors of the specified test case set are respectively obtained.

[0115] In some optional embodiments, the adopting of text topic mining technology to obtain the text topic vectors of each of the test case sets includes:

[0116] Acquire a second corpus consisting of the sorted test case set and the target test case set;

[0117] Taking a single test case as a document, a topic extraction model is trained on the basis of the second corpus to obtain a text topic vector for each of the test case sets.

[0118] In some optional embodiments, the first ranking index value obtaining module 520 is further used to:

[0119] Taking a single test case as a document, obtaining text content vectors of each test case in the source test case set and the target test case set respectively through a word frequency-inverse document frequency model;

[0120] Using text topic mining technology, respectively obtain the text topic vector of each test case in the source test case set and the target test case set;

[0121] For each of the test cases in the source test case set, combining the text content vector and the text topic vector, respectively calculate a similarity score between each of the test cases in the target test case set and each of the test cases in the source test case set;

[0122] A first ranking index value of the test cases in the target test case set is obtained according to the similarity score and the known ranking of each test case in the source test case set.

[0123] In some optional embodiments, the second sorting index value obtaining module 530 is further used to:

[0124] According to the test code coverage of the test cases in the sorted test case set, obtaining a second sorting index value of the corresponding test case in the target test case set; and / or,

[0125] According to the defect detection capability index value of the test case in the sorted test case set, a second sorting index value of the corresponding test case in the target test case set is obtained.

[0126] The test case priority decision-making device disclosed in the embodiment of the present application is used to implement the test case priority decision-making method described in the embodiment of the present application. The specific implementation methods of each module of the device will not be repeated here, and reference can be made to the specific implementation methods of the corresponding steps in the method embodiment.

[0127] The test case priority decision-making device disclosed in the embodiment of the present application selects at least one sorted test case set as a source test case set based on the similarity between the sorted test case set and the target test case set; obtains a first sorting index value of the test case in the target test case set based on the similarity between the test cases in the source test case set and the target test case set, and the known sorting of a single test case in the source test case set; on the other hand, obtains at least one second sorting index value of the test case in the target test case set based on the historical performance data of the test case in the sorted test case set; finally, priority factors of multiple dimensions such as the first sorting index value and the second sorting index value are combined to make priority decisions on the test cases in the target test case set, thereby effectively improving the accuracy of test case priority decisions, thereby helping to improve regression testing results.

[0128] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0129] The above is a detailed introduction to a test case priority decision-making method and device provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

[0130] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0131] The various component embodiments of the present application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all functions of some or all components in the electronic device according to the embodiment of the present application. The present application can also be implemented as a device or apparatus program (e.g., computer program and computer program product) for executing part or all of the methods described herein. Such a program implementing the present application can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0132] For example, Figure 6 An electronic device that can implement the method according to the present application is shown. The electronic device can be a PC, a mobile terminal, a personal digital assistant, a tablet computer, etc. The electronic device traditionally includes a processor 610 and a memory 620 and a program code 630 stored on the memory 620 and can be run on the processor 610, and the processor 610 implements the method described in the above embodiment when executing the program code 630. The memory 620 can be a computer program product or a computer-readable medium. The memory 620 can be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk or a ROM. The memory 620 has a storage space 6201 for the program code 630 of a computer program for executing any method step in the above method. For example, the storage space 6201 for the program code 630 can include various computer programs for implementing the various steps in the above method respectively. The program code 630 is a computer-readable code. These computer programs can be read from one or more computer program products or written into the one or more computer program products. These computer program products include program code carriers such as hard disks, compact disks (CDs), memory cards or floppy disks. The computer program includes a computer readable code, and when the computer readable code is executed on an electronic device, the electronic device is caused to execute the method according to the above embodiment.

[0133] An embodiment of the present application also discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the test case priority decision method as described in the embodiment of the present application.

[0134] Such a computer program product may be a computer-readable storage medium, which may have a computer-readable storage medium having a Figure 6The memory 620 in the electronic device shown in the figure is similar to the storage segment, storage space, etc. The program code can be compressed and stored in the computer-readable storage medium in an appropriate form. The computer-readable storage medium is generally as shown in the reference Figure 7 The portable or fixed storage unit. Generally, the storage unit includes computer readable code 630', which is a code read by a processor. When the code is executed by the processor, each step in the method described above is implemented.

[0135] The term "one embodiment", "embodiment" or "one or more embodiments" herein means that a particular feature, structure or characteristic described in conjunction with the embodiment is included in at least one embodiment of the present application. In addition, please note that the examples of the term "in one embodiment" here do not necessarily all refer to the same embodiment.

[0136] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0137] In the claims, any reference signs placed between brackets shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present application may be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

[0138] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A test case priority decision method, characterized in that: The method comprises: According to the similarity between the sorted test case set and the target test case set, selecting at least one of the sorted test case sets as a source test case set; Obtaining a first ranking index value of the test case in the target test case set according to the similarity between the test cases in the source test case set and the test cases in the target test case set, and the known ranking of a single test case in the source test case set; Acquire at least one second ranking index value of the test cases in the target test case set according to the historical performance data of the test cases in the sorted test case set; According to the first sorting index value and the second sorting index value, priority decisions are made on the test cases in the target test case set.

2. The method according to claim 1, characterized in that The selecting at least one of the sorted test case sets as a source test case set according to the similarity between the sorted test case set and the target test case set comprises: Taking a single test case set as a document, respectively obtaining text content vectors of each test case set through a word frequency-inverse document frequency model, wherein the test case set is selected from the sorted test case set and the target test case set; Using text topic mining technology to obtain text topic vectors of each test case set; For each of the test case sets, combining the corresponding text content vector and the text topic vector, respectively calculating the similarity score between each of the sorted test case sets and the target test case set; At least one of the sorted test case sets is selected as a source test case set according to the similarity score.

3. The method according to claim 2, characterized in that The method of taking a single test case set as a document and obtaining the text content vectors of each test case set respectively through the word frequency-inverse document frequency model includes: Acquire a first corpus consisting of the sorted test case set and the target test case set; Taking a single test case set as a document, obtaining keywords included in the first corpus, the first occurrence number of each keyword in each test case set, the second occurrence number of each keyword in the first corpus, the total number of test case sets, and the number of test case sets in which each keyword appears; For each of the keywords, according to the first number of occurrences, the second number of occurrences, the total number of the test case sets, and the number of the test case sets in which the keyword appears, obtain the word frequency-inverse file frequency of the keyword; According to the word frequency and the reverse file frequency of each of the keywords in the specified test case set, the text content vectors of the specified test case set are respectively obtained.

4. The method according to claim 2, characterized in that: The text topic mining technology is used to obtain the text topic vectors of each test case set, including: Acquire a second corpus consisting of the sorted test case set and the target test case set; Taking a single test case as a document, a topic extraction model is trained on the basis of the second corpus to obtain a text topic vector for each of the test case sets.

5. The method according to claim 1, characterized in that The obtaining, according to the similarity between the test cases in the source test case set and the test cases in the target test case set, a first ranking index value of the test cases in the target test case set, includes: Taking a single test case as a document, obtaining text content vectors of each test case in the source test case set and the target test case set respectively through a word frequency-inverse document frequency model; Using text topic mining technology, respectively obtain the text topic vector of each test case in the source test case set and the target test case set; For each of the test cases in the source test case set, combining the text content vector and the text topic vector, respectively calculate a similarity score between each of the test cases in the target test case set and each of the test cases in the source test case set; A first ranking index value of the test cases in the target test case set is obtained according to the similarity score and the known ranking of each test case in the source test case set.

6. The method according to claim 1, characterized in that The acquiring, according to the historical performance data of the test cases in the sorted test case set, at least one second sorting index value of the test cases in the target test case set includes one or more of the following: According to the test code coverage of the test cases in the sorted test case set, obtaining a second sorting index value of the corresponding test case in the target test case set; According to the defect detection capability index value of the test case in the sorted test case set, a second sorting index value of the corresponding test case in the target test case set is obtained.

7. A test case priority decision device, characterized in that: The device comprises: A source test case set selection module, configured to select at least one of the sorted test case sets as a source test case set according to the similarity between the sorted test case sets and the target test case set; A first ranking index value acquisition module is used to acquire a first ranking index value of a test case in the target test case set according to the similarity between the test cases in the source test case set and the test cases in the target test case set, and a known ranking of a single test case in the source test case set; A second ranking index value acquisition module, configured to acquire at least one second ranking index value of a test case in the target test case set according to historical performance data of the test case in the sorted test case set; A priority decision module is used to make a priority decision on the test cases in the target test case set according to the first sorting index value and the second sorting index value.

8. The device according to claim 7, characterized in that The second sorting index value acquisition module is further used to: According to the test code coverage of the test cases in the sorted test case set, obtaining a second sorting index value of the corresponding test case in the target test case set; and / or, According to the defect detection capability index value of the test case in the sorted test case set, a second sorting index value of the corresponding test case in the target test case set is obtained.

9. An electronic device comprising a memory, a processor, and a program code stored in the memory and executable on the processor, wherein: When the processor executes the program code, the method according to any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium having program code stored thereon, characterized in that: When the program code is executed by a processor, the steps of the method described in any one of claims 1 to 6 are implemented.

Images