A Sorting Similarity Software Abnormal Behavior Modeling Method and a Computer Readable Medium

Abstract

The present invention provides a method for modeling abnormal behaviors of sorting similarity software and a computer-readable medium. The method of the present invention enables the effectiveness of applying a multi-fault program dataset and multiple external index evaluation fault characterization functions to the fault separation process, calculates fitness scores for fault sorting characterization based on the evaluation results, and based on the fitness scores, evolves the population of fault characterization functions through crossover, replication, and mutation operations of the genetic programming algorithm until a termination condition is reached. The evolved individual of the fault sorting characterization with the highest fitness is used as a fault classification model for fault separation of real multi-fault programs. The present invention solves the problems that there is currently no available automated generation method for fault sorting characterization for fault separation and it is impossible to establish a mapping relationship from fault sorting characterization to the ability of failure test case characterization. Moreover, using the method of the present invention, efficient automated generation of fault sorting characterization can be achieved and used for fault separation work in actual production.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular, to a method for modeling abnormal software behavior of sorting similarity and a computer-readable medium. Background Art

[0002] Software defect localization technology aims to find out the root cause of software abnormal behavior, which is of great significance for ensuring and improving software quality and enhancing the effectiveness of software repair technology. During the software testing process, the actual output of test cases being different from the expected output indicates that there is at least one defect hidden in the software. Developers must quickly repair the defect to restore the normal function of the software. However, the prerequisite for repairing the defect is to locate the defect. Only by accurately finding the internal cause that triggers the abnormal behavior of software execution can an effective basis be provided for the subsequent repair process. For this reason, many software defect localization methods have been proposed successively, such as the localization method based on program mutation technology, the localization method based on program spectrum technology, etc. Although these methods have been proven to have good defect localization effects, they almost all are based on the single-fault assumption, that is, it is assumed that there is only one defect in a faulty program. With the continuous growth of software scale and the increasing complexity of software systems, it is almost impossible for a faulty program to contain only one defect. In other words, the multi-fault environment is the normal state faced by software debugging tasks in the real environment.

[0003] The simultaneous existence of multiple faults will seriously reduce the effectiveness of traditional single-fault localization methods. Therefore, multi-fault software fault localization technology has become one of the hotspots in the current research field of fault localization. This type of technology first uses a risk assessment function that estimates the suspiciousness for program elements in single-fault localization to generate a sorted list representation for each failed test case, establishing the connection between the failed test cases and software faults, so as to divide the failed test cases according to the fault source to isolate the failed test cases caused by different faults and weaken the mutual influence between software faults. Then, the single-fault localization method is applied in parallel on each class of failed test cases, decomposing the multi-fault localization problem into multiple single-fault localization problems. Among them, the effectiveness of the fault division process determines the accuracy of parallel single-fault localization, and the risk assessment function used has a significant impact on the effectiveness of fault division.

[0004] However, the risk assessment functions used in existing multi-fault location techniques for fault partitioning are all designed for single-fault location application scenarios. The factors to be considered in multi-fault isolation and single-fault location are not exactly the same. Therefore, the optimal risk assessment function in a single-fault scenario does not necessarily have good multi-fault isolation performance. Intuitively, if the risk assessment function has a stronger ability to extract fault features, the sorted list it generates will be able to more correctly simulate the failed test cases, thus obtaining better parallel debugging effects. Therefore, it is very important to use a risk assessment function that can correctly represent faults in a multi-fault scenario.

[0005] According to the purpose of use, a fault characterization function (risk assessment function) can be designed manually or using an automated method with the goal of generating a failed test case characterization with strong characterization ability and suitable for the partitioning algorithm, so as to improve the effectiveness of the fault partitioning process and further improve the accuracy of multi-fault location.

[0006] However, there is still a lack of methods to measure the characterization ability of failed test cases, and the characterization of failed test cases can only be indirectly evaluated through the effectiveness of the fault partitioning process. On the other hand, after determining the risk assessment function to be used, it is also necessary to combine the execution information of each failed test case with all successful test cases, calculate the suspiciousness for all program elements and sort them according to the calculated suspiciousness. Only the finally obtained sorted list can be used as the characterization of the failed test case. This complex characterization generation method and indirect evaluation method make it difficult to determine the mapping relationship between the fault characterization function and the characterization ability of the failed test case, so methods such as artificial neural networks cannot be used to train the fault characterization function. At the same time, if the method of manually designing the fault characterization function is adopted, due to the design relying only on human intuition, the characterization ability of the obtained fault characterization function is far from ideal, and the design efficiency is much lower than that of automated generation. How to design a reliable method for generating a fault characterization function for fault partitioning is an important issue. Summary of the Invention

[0007] To solve the above technical problems, the present invention provides a sorting similarity software abnormal behavior modeling method and a computer-readable medium.

[0008] The present invention provides a software abnormal behavior modeling method based on sorting similarity, and the specific steps are as follows:

[0009] Step 1: Introduce multiple multi-fault programs, randomly generate multiple fault characterization functions using a genetic programming algorithm, take each fault characterization function as each individual in the population, and construct an initial population through multiple fault characterization functions;

[0010] Step 2: Combine each fault characterization function in the initial population to generate a sorted list representation of each failed test case generated by applying each fault characterization function to each multi-fault program. Cluster and partition the sorted list representations of all failed test cases generated by applying each fault characterization function to each multi-fault program to obtain the classification results of the sorted list representations of multiple failed test cases generated by applying each fault characterization function to each multi-fault program. Evaluate the effectiveness scores of each external metric corresponding to the classification results generated by applying each fault characterization function to each multi-fault program according to multiple external metrics. Combine the effectiveness scores of multiple external metrics corresponding to the classification results generated by applying each fault characterization function to each multi-fault program to obtain the fitness scores at the time of fault separation for each fault characterization function applied to each multi-fault program. Obtain the fitness of each individual in the initial population according to the fitness scores at the time of fault separation of each fault characterization function applied to multiple multi-fault programs;

[0011] Step 3: Select the parental population from the population according to the fitness of each individual in the initial population. Create each offspring individual using the parental population according to the crossover rate, replication rate, and mutation rate. Take all the created offspring individuals as the offspring population, and calculate the fitness of each individual in the offspring population through Step 2;

[0012] Step 4: Repeat Step 2 and Step 3 until the user-specified termination condition is reached. Take the fault characterization function with the highest individual fitness in the last population that reaches the termination condition as the fault classification model;

[0013] Step 5: Use the fault classification model in the multi-fault program that needs to be fault-partitioned to generate a sorted list representation for each failed test case, calculate the distance between each pair of sorted list representations, and cluster the failed test cases in combination with the distance between each pair of sorted list representations to realize the modeling of software abnormal behaviors and the partitioning of multiple faults.

[0014] Preferably, each fault characterization function described in Step 1 is specifically defined as follows:

[0015] The fault characterization function REF is:

[0016] Suspiciousness k，f，i = REF k (Spectrum f，i )

[0017] Spectrum f，i ={NCF f，i , NCS i , NUF f，i , NUS i}

[0018] Where NCFf，i The number of times the i-th program statement of each multi-fault program is covered by the f-th failing test case, NCS i The number of times the i-th program statement of each multi-fault program is covered by successful test cases, NUF f，i The number of times the i-th program statement of each multi-fault program is not covered by the f-th failing test case, NUS i The number of times the i-th program statement of each multi-fault program is not covered by successful test cases, where the value of i ranges from 1 to K, and K is the number of program statements in each multi-fault program, NCF f，i , NCS i , NUF f，i , NUS i Together as the spectrum information Spectrum of the f-th failing test case of each multi-fault program at the i-th program statement f，i , Sspiciousness k，f，i For using the fault characterization function REF k The suspiciousness of the existence of a fault source at the i-th program statement of each multi-fault program calculated by combining the spectrum information of the f-th failing test case at the i-th program statement;

[0019] The initial population constructed in Step 1 is as follows:

[0020] Use the genetic programming algorithm to randomly generate a number of fault characterization functions and use them together as the initial population:

[0021]

[0022] Among them, Population p Represents the p-th generation population, and the constructed initial population is Population1, pop is the number of population individuals or the population size, Represents the k-th fault characterization function in the p-th generation population;

[0023] Preferably, the sorted list representation of each failing test case generated by applying each fault characterization function to each multi-fault program by combining each fault characterization function in the initial population in Step 2 is as follows:

[0024] Each multi-fault program contains multiple failing test cases and multiple successful test cases;

[0025] The multiple failing test cases are specifically defined as follows:

[0026] {Ftc1, Ftc2, Ftc3,..., Ftc fail}

[0027] Among them, fail is the number of failed test cases, and Ftc f represents the f-th failed test case in each multi-fault program, where f ranges from 1 to fail;

[0028] The multiple successful test cases are specifically defined as follows:

[0029] {Stc1, Stc2, Stc3,..., Stc success};

[0030] Among them, success is the number of successful test cases, and Stc s represents the s-th successful test case in each multi-fault program, where s ranges from 1 to success;

[0031] Collect the coverage information of each failed test case and each successful test case at each program statement in each multi-fault program;

[0032] Use the coverage information of each failed test case and each successful test case at each program statement in each multi-fault program to calculate the spectrum value of each failed test case at each corresponding program statement in each multi-fault program;

[0033] The spectrum value Spectrum collected by the f-th failed test case of each multi-fault program at the i-th program statement f，i is:

[0034] Spectrum f，i ={NCF f，i , NCS i , NUF f，i , NUS i}

[0035] where i ranges from 1 to K;

[0036] Use the k-th fault characterization function in the p-th generation of the population and the spectrum value collected by the f-th failed test case of each multi-fault program at the i-th program statement to calculate the suspiciousness of the i-th program statement of each multi-fault program calculated using the f-th failed test case as:

[0037]

[0038] where p ranges from 1 to end, end is the generation number of the last generation of the population, k ranges from 1 to pop, f ranges from 1 to fail, and i ranges from 1 to K;

[0039] According to the above suspiciousness calculation method, the suspiciousness characterized by using the k-th fault characterization function in the p-th generation population and the f-th failed test case for each multi-fault program is as follows:

[0040] Rep_Suspiciousness p，k，f

[0041] ={Suspiciousness p，k，f，1 ,Suspiciousness p，k，f，2 ,...,Suspiciousness p，k，f，is}

[0042] where is is the number of program statements of each multi-fault program;

[0043] Sort the suspiciousness of each program statement in the suspiciousness characterization calculated by using the k-th fault characterization function in the p-th generation population and the f-th failed test case for each multi-fault program. The sorted list characterization generated by using the k-th fault characterization function in the p-th generation population for the f-th failed test case of each multi-fault program is as follows:

[0044] Rep_Ranking p，k，f ={Index p，k，f，1 ,Index p，k，f，2 ,...,Index p，k，f，is}

[0045] where Index p，k，f，rank represents the actual index position of the program statement at the rank-th position in each multi-fault program when sorted in descending order of the suspiciousness calculated by using the k-th fault characterization function in the p-th generation population and the f-th failed test case corresponding to the program statement;

[0046] The classification result of the sorted list characterizations of all failed test cases generated by applying each fault characterization function to each multi-fault program described in step 2 is as follows;

[0047] Result p，k ={numOfClusters p，k ,Clusters p，k}

[0048] where Result p，k is the classification result of the sorted list characterizations of multiple failed test cases generated by applying the k-th fault characterization function in the p-th generation population to each multi-fault program, and numOfClusters p，kis the number of clusters predicted by using the k-th fault characterization function in the p-th generation population. When the number of clusters is not equal to the actual number of faults in each multi-fault program, skip the subsequent evaluation steps of the current fault characterization function on the current multi-fault program and set its fitness score on the corresponding multi-fault version to 0;

[0049] Clusters p，k is the cluster number assigned to each failed test case of each multi-fault program after clustering using the k-th fault characterization function in the p-th generation population. There is

[0050] Clusters p，k ={Group p，k，1 ,Group p，k，2 ,...,Group p，k，fail}

[0051] Among them, Group p，k，f is the sorted list representation corresponding class number generated by using the k-th fault characterization function in the p-th generation population for the f-th failed test case of each multi-fault program;

[0052] The effectiveness scores of each external metric corresponding to the classification results generated by applying each fault characterization function to each multi-fault program as described in step 2 are:

[0053] Score_PR p，k =PR(Clusters p，k , Oracle)

[0054] Score_RR p，k =RR(Clusters p，k , Oracle)

[0055] Score_FMI p，k =FMI(Clusters p，k , Oracle)

[0056] Score_JC p，k =JC(Clusters p，k , Oracle)

[0057] Among them, Oracle is the true correspondence between each failed test case and the fault source in each multi-fault program, and Score_PR p，k is the effectiveness score of the PR metric corresponding to the classification result generated by applying the k-th fault characterization function in the p-th generation population to each multi-fault program, and Score_RR p，kThe effectiveness score of the RR metric corresponding to the classification result generated by applying the k-th fault characterization function in the p-th generation population to each multi-fault program, Score_FMI p，k The effectiveness score of the FMI metric corresponding to the classification result generated by applying the k-th fault characterization function in the p-th generation population to each multi-fault program, Score_JC p，k The effectiveness score of the JC metric corresponding to the classification result generated by applying the k-th fault characterization function in the p-th generation population to each multi-fault program;

[0058] PR is the precision rate, RR is the recall rate, FMI is the Fowlkes-Mallows index, and JC is the Jaccard coefficient. The above four external metrics take the cluster numbers assigned to each failed test case of each multi-fault program after clustering using the k-th fault characterization function in the p-th generation population and the true correspondence between each failed test case and the fault source in each multi-fault program as inputs, and output the effectiveness score of this clustering. The score ranges from [0, 1], and the larger the value, the higher the effectiveness of the clustering process;

[0059] To calculate the four external metrics, it is necessary to establish the correspondence between each cluster in the cluster numbers assigned to each failed test case of each multi-fault program after clustering using the k-th fault characterization function in the p-th generation population and the true fault source. For each external metric, calculate the effectiveness scores under all possible correspondences, and take the highest score as the calculation result of this metric;

[0060] The fitness score when separating faults for each fault characterization function applied to each multi-fault program by combining the effectiveness scores of multiple external metrics corresponding to the classification results generated by each fault characterization function applied to each multi-fault program in step 2 is:

[0061] When calculating each external metric for the classification result generated by each fault characterization function applied to each multi-fault program, it is necessary to take the effectiveness score under the correspondence with the highest effectiveness score as the calculation result. At this time, the current metric is said to vote for this correspondence, and the number of votes for each correspondence is:

[0062]

[0063] where, represents whether to vote for the d-th correspondence for Score_PR p，k represents whether to vote for the d-th correspondence for Score_RR p，k represents whether to vote for the d-th correspondence for Score_FMI p，k ​​​Representing Score_JC p，k Whether to vote for the d-th correspondence, Vote p，k，d Represents the total number of votes obtained by the k-th fault characterization function in the p-th generation of the population at the d-th correspondence. The value of d ranges from 1 to V, where V is the number of possible correspondences for the multi-fault program;

[0064] The total number of votes for the correspondence with the most votes is:

[0065] Vote p，k，most = Max(Vote p，k，1 , Vote p，k，2 ,..., Vote p，k，V )

[0066] Among them, Vote p，k，most Is the total number of votes for the correspondence with the most votes in the evaluation of the effectiveness index of the classification results generated by applying the k-th fault characterization function in the p-th generation of the population to each multi-fault program, which is determined by the number of fault sources included in the multi-fault program;

[0067] The fitness score calculation method for the k-th fault characterization function in the p-th generation of the population when applied to the fault isolation of each multi-fault program is:

[0068] FitnessScore p，k = TotalMetrics p，k * PenaltyFactor p，k

[0069] Among them, FitnessScore p，k Is the fitness score when the k-th fault characterization function in the p-th generation of the population is applied to the fault isolation of each multi-fault program;

[0070] TotalMetrics p，k = Score_PR p，k + Score_RR p，k + Score_FMI p，k + Score_JC p，k

[0071] Among them, TotalMetrics p，k Is the sum of the effectiveness scores of the four external metrics of the classification results generated by applying the k-th fault characterization function in the p-th generation of the population to each multi-fault program;

[0072] PenaltyFactor p，k = 1 - 0.05 * (4 - Vote p，k，most )

[0073] Among them, PenaltyFactor p，k is the fitness penalty factor when the k-th fault characterization function in the p-th generation population is applied to the fault isolation of each multi-fault program, which is determined by Vote p，k，most ;

[0074] The fitness of each individual in the initial population obtained according to the fitness scores when each fault characterization function is applied to the fault isolation of multiple multi-fault programs in step 2 is as follows:

[0075] The individual fitness of each fault characterization function in the initial population is:

[0076]

[0077] Among them is the fitness of the k-th fault characterization function in the p-th generation population, Version ver represents the ver-th multi-fault program, nv is the number of multi-fault programs included in the multiple multi-fault programs introduced in step 1, is the fitness score when the k-th fault characterization function in the p-th generation population is applied to the fault isolation of the ver-th multi-fault program;

[0078] Preferably, the selection of the parent population from the population according to the fitness of each individual in step 3 is as follows:

[0079] Using the roulette wheel algorithm to select pop*proportion distinct individuals from the population according to the fitness of each individual to form the parent population:

[0080]

[0081] Among them, pop is the population size, proportion is the ratio of the next parent population to the parental population, Population p represents the p-th generation population, represents the fitness score of the k-th fault characterization function in the p-th generation population, Roulette is the roulette wheel algorithm, Father_Population p+1 represents the selection from Population p for generating Population p+1 of the parent population, the value of p ranges from 1 to end, and the value of k ranges from 1 to pop;

[0082] The creation of offspring individuals using the parent population according to the crossover rate, replication rate, and mutation rate in step 3 is as follows:

[0083] Use the parental population Father_Population according to the crossover rate, replication rate, and mutation rate p+1 Create each offspring individual

[0084] Determine how to create each offspring individual according to the crossover rate, replication rate, and mutation rate The creation method is as follows:

[0085]

[0086] Among them, c, o, and m are the crossover rate, replication rate, and mutation rate, and there is

[0087] c + o + m = 1

[0088] 0 ≤ c, o, m ≤ 1

[0089] random is a random number in the interval [0, 1], Method is the generation method of the current offspring individual, Crossover represents generation by crossover, Copy represents generation by replication, Mutate represents generation by mutation, and determine the individual creation method according to the value of random and the magnitudes of the crossover rate, replication rate, and mutation rate;

[0090] The creation of the offspring individual is as follows:

[0091]

[0092] Among them, is the k-th fault characterization function of the offspring population of the p-th generation, Create_Crossover is the crossover algorithm, REF r1 , REF r2 is two different individuals randomly selected from the parental population Father_Population p+1 , Create_Copy and Create_Mutate are the replication and mutation algorithms, REF r is an individual randomly selected from the parental population Father_Population p+1 ;

[0093] Each individual in the population, in addition to having the attribute of its own as the fault characterization function, also has an age attribute:

[0094]

[0095]

[0096] Among them, is the age of the k-th individual in the p-th generation population, is all individuals in the initial population, For the newly created individuals in the offspring population, Method is the corresponding creation method, REF r created by the replication algorithm the individuals randomly selected from the parent population for replication when creating, Age(*) represents the age of *;

[0097] When the age of an individual reaches 3, it cannot be selected as a parent population, define Survive_Population p as the surviving population composed of all individuals with an age less than 3 in the p-th generation population;

[0098] That is, the selection method of the father population Father_Population of the p-th generation population p+1 should be modified to:

[0099]

[0100] Among them, the k-th fault characterization function of the p-th generation population is an individual in Survive_Population p if and only if

[0101] When the number of generated offspring individuals reaches the population size pop, stop generating, and use these pop individuals as the next generation population Population p+1 ;

[0102] The method described in step 3 for evaluating the fitness of the offspring population using the method in step 2 and continuing to evolve the next generation population until the termination condition is reached is:

[0103] Before meeting the user-defined termination condition SC (such as having iterated a fixed number of times or obtained a fault characterization function that meets the requirements), continuously use the method in step 2 to evaluate the fitness scores of each fault characterization function in the current p-th generation population Population p and use the method in step 3 to generate the offspring population Population p+1 ;

[0104] Preferably, the method of using the fault characterization function with the highest fitness among the individuals in the last generation population that reaches the termination condition as the fault classification model described in step 4 is:

[0105] Population end is the population of the last generation that has completed fitness evaluation and reached the termination condition SC, is for Population end with the highest fitness score in The k-th fault characterization function, which can be used for fault characterization generation and fault isolation of real multi-fault programs;

[0106] Preferably, in the multi-fault program that needs to be fault-partitioned using the fault classification model, generating a sorted list representation for each failed test case in step 5 is as follows:

[0107] Using the population Population that completed fitness evaluation and reached the termination condition SC in the last generation in step 4 end The k-th fault characterization function with the highest fitness score selected from it, that is, the fault classification model Generating a sorted list representation Rep_Ranking for each failed test case of the target multi-fault program according to the method in step 2 end，k，f , Rep_Ranking end，k，f Is the sorted list representation generated for the f-th failed test case of the target multi-fault program using the fault classification model ;

[0108] The distance calculation between each pair of sorted list representations in step 5 is as follows:

[0109] The pre-selected distance metric function Distance_Metric is:

[0110] Distance fo，ft = Distance_Metric(Rep_Ranking end，k，fo , Rep_Ranking end，k，ft )

[0111] Where, Rep_Ranking end，k，fo , Rep_Ranking end，k，fo Are the sorted list representations generated for two failed test cases fo and ft of the target multi-fault program according to the fault isolation model , and Distance fo，ft Is the distance between the failed test cases fo and ft calculated using the distance metric function Distance_Metric;

[0112] Using the calculated distance to cluster the failed test cases in step 5 is as follows;

[0113] The pre-selected clustering strategy includes:

[0114] numOfClusters end，k

[0115] = Predict(Distance f1，f1, Distance f1，f2 , ..., Distance fo，ft , ..., Distance fail，fail )

[0116] And:

[0117] Clusters end，k = Cluster(Distance f1，f1 , Distance f1，f2 , ..., Distance fo，ft , ..., Distance fail，fail )

[0118] Wherein, Distance fo，ft is the distance between the failed test cases fo and ft calculated using the distance metric function Distance_Metric, Predict is a function for predicting the number of clusters, numOfClusters end，k is the number of clusters predicted using the fault classification model, Cluster is a clustering algorithm, and Clusters end，k is the cluster number assigned to each failed test case of the target multi-fault program after clustering using the fault classification model. There is

[0119] Clusters end，k = {Group end，k，1 , Group end，k，2 , ..., Group end，k，fail}

[0120] Wherein, Group end，k，f is the sorted list representation corresponding to the class number generated for the f-th failed test case of the target multi-fault program using the fault classification model;

[0121] Both Predict and Cluster take the distance between every two failed test cases fo and ft in the target multi-fault program as input.

[0122] The present invention also provides a computer-readable medium storing a computer program executed by an electronic device. When the computer program runs on the electronic device, the electronic device is caused to execute the steps of the sorting similarity software abnormal behavior modeling method.

[0123] The objective of the present invention is to provide a method for automatically generating a fault sorting representation for a fault isolation scenario. First, a genetic programming algorithm is used to generate a population of random initial fault sorting representations. Then, the individuals in the population are applied to a multi-fault program dataset specified by the user for fault isolation. Each fault sorting representation is used to generate a sorted list representation for the failing test cases in the multi-fault program. The distance between the representations is calculated, and a clustering algorithm is applied to divide the failing test case representations according to the fault source. Multiple external metrics are used to evaluate the effectiveness of the division process. Based on the evaluation results, the fitness of the fault sorting representation for fault isolation in the multi-fault program is calculated. The overall performance of the fault sorting representation on the dataset is statistically analyzed to obtain the fitness score of the fault sorting representation. Finally, the parent population is selected according to the fitness scores of each fault sorting representation individual in the population, and a child population is generated through crossover, replication, and mutation operations according to the preset crossover rate, replication rate, and mutation rate. The child population is continuously generated in the same way until the method execution reaches the termination condition. The fault sorting representation individuals with high fitness in the last generation of the population that has completed fitness evaluation are output as the result and used for fault isolation in real multi-fault programs. The present invention finally forms a complete software abnormal behavior modeling method based on sorting similarity.

[0124] The software abnormal behavior modeling method based on sorting similarity provided by the present invention provides a method for automatically generating a fault representation function through a genetic programming algorithm and including multiple interfaces that can be intervened manually to improve flexibility. This solves the problem that there is currently no available method for automatically generating a fault sorting representation for fault isolation. Based on the above method for automatically generating a fault representation function, the present invention uses a genetic programming algorithm to solve the problem of being unable to establish a mapping relationship from a fault sorting representation to the ability of a failing test case representation, realizes the automation of generating a fault representation function, and improves the adaptability of the generated fault representation function to fault isolation applications of different types of multi-fault programs. Finally, the present invention also adds multiple manually adjustable interfaces to the method process to enable the method to meet various types of requirements. The present invention finally forms a complete software abnormal behavior modeling method based on sorting similarity. Users can use this method to achieve efficient automatic generation of fault sorting representations and use them for fault isolation work in actual production. Description of the Drawings

[0125] Figure 1 : Flowchart of the method of the embodiment of the present invention;

[0126] Figure 2 : Tree structure diagram of the fault representation function generated by the genetic programming algorithm of the embodiment of the present invention;

[0127] Figure 3 : Algorithm schematic diagram of the overall process of the method of the embodiment of the present invention;

[0128] Figure 4 : Schematic algorithm diagram of the process for evaluating the fitness of individuals in the population of fault characterization functions in the embodiments of the present invention. Detailed implementation manners

[0129] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0130] Specifically in implementation, the method proposed by the technical solution of the present invention can be automatically run by those skilled in the art using computer software technology. The system device for implementing the method, such as a computer-readable storage medium storing the corresponding computer program of the technical solution of the present invention and a computer device including running the corresponding computer program, should also be within the protection scope of the present invention.

[0131] Figure 1 It is the flowchart of the method of the present invention. Next, in conjunction with Figures 1 to 4 The technical solution of the method in the embodiments of the present invention is introduced as a method for modeling software abnormal behavior of sorting similarity, and the specific steps are as follows:

[0132] Step 1: Introduce multiple multi-fault programs, randomly generate multiple fault characterization functions using the genetic programming algorithm, take each fault characterization function as each individual in the population, and construct an initial population through multiple fault characterization functions;

[0133] Each fault characterization function described in Step 1 is specifically defined as follows:

[0134] The fault characterization function REF is:

[0135] Suspiciousness k，f，i = REF k (Spectrum f，i )

[0136] Spectrum f，i = {NCF f，i , NCS i , NUF f，i , NUS i}

[0137] Wherein, NCF f，i is the number of times that the i-th program statement of each multi-fault program is covered by the f-th failed test case, and NCS iThe number of times the i-th program statement of each multi-fault program is covered by successful test cases, NUF f，i The number of times the i-th program statement of each multi-fault program is not covered by the f-th failed test case, NUS i The number of times the i-th program statement of each multi-fault program is not covered by successful test cases, where the value of i ranges from 1 to K, and K is the number of program statements in each multi-fault program, NCF f，i , NCS i , NUF f，i , NUS i Together serve as the spectrum information Spectrum of the f-th failed test case of each multi-fault program at the i-th program statement f，i , Suspiciousness k，f，i Is for using the fault characterization function REF k The suspiciousness of the existence of a fault source at the i-th program statement of each multi-fault program calculated by combining the spectrum information of the f-th failed test case at the i-th program statement;

[0138] In one embodiment, the fault characterization function can be generated in a tree form through a genetic programming algorithm, and one of the generated fault characterization functions REF is as shown in the appendix Figure 2 As shown, in the figure, rectangular nodes represent operators that combine their sub-nodes below, and circular nodes represent program spectrum values and constants participating in the operation. The appendix Figure 2 The shown fault characterization function is:

[0139]

[0140] The initial population constructed in step 1 is:

[0141] Use the genetic programming algorithm to randomly generate several fault characterization functions and use them together as the initial population:

[0142]

[0143] Among them, Population p Represents the p-th generation population, and the constructed initial population is Population1, pop is the number of population individuals or the population size, Represents the k-th fault characterization function in the p-th generation population;

[0144] In one embodiment, since a population composed of 160 fault characterization function individuals can generate individuals with high effectiveness for fault isolation under the premise of controllable population evolution cost, the population size can be specified as 160, that is, set pop = 160.

[0145] Step 2: Combine each fault characterization function in the initial population to generate a sorted list representation of each failed test case generated by applying each fault characterization function to each multi-fault program. Cluster and partition the sorted list representations of all failed test cases generated by applying each fault characterization function to each multi-fault program to obtain the classification results of the sorted list representations of multiple failed test cases generated by applying each fault characterization function to each multi-fault program. Evaluate the effectiveness scores of each external metric corresponding to the classification results generated by applying each fault characterization function to each multi-fault program according to multiple external metrics. Merge the effectiveness scores of multiple external metrics corresponding to the classification results generated by applying each fault characterization function to each multi-fault program to obtain the fitness scores at the time of fault separation for each fault characterization function applied to each multi-fault program. Obtain the fitness of each individual in the initial population according to the fitness scores at the time of fault separation for each fault characterization function applied to multiple multi-fault programs;

[0146] The combination in Step 2 of generating a sorted list representation of each failed test case generated by applying each fault characterization function in the initial population to each multi-fault program is specifically as follows:

[0147] Each multi-fault program contains multiple failed test cases and multiple successful test cases;

[0148] The multiple failed test cases are specifically defined as follows:

[0149] {Ftc1, Ftc2, Ftc3,..., Ftc fail}

[0150] where fail is the number of failed test cases, and Ftc f represents the f-th failed test case in each multi-fault program, and the value of f ranges from 1 to fail;

[0151] The multiple successful test cases are specifically defined as follows:

[0152] {Stc1, Stc2, Stc3,..., Stc success};

[0153] where success is the number of successful test cases, and Stc s represents the s-th successful test case in each multi-fault program, and the value of s ranges from 1 to success;

[0154] In one embodiment, if there are a large number of successful test cases in the test suite that are irrelevant to the faulty module, the successful test cases can be screened to reduce their number and improve the efficiency of fitness evaluation during the population evolution process.

[0155] Collect the coverage information of each failed test case and each successful test case at each program statement in each multi-fault program;

[0156] Use the coverage information of each failed test case and each successful test case at each program statement in each multi-fault program to calculate the spectrum value of each failed test case at each corresponding program statement in each multi-fault program;

[0157] The spectrum value Spectrum collected by the f-th failed test case of each multi-fault program at the i-th program statement f，i is:

[0158] Spectrum f，i ={NCF f，i , NCS i , NUF f，i , NUS i}

[0159] The value range of i is between 1 and K;

[0160] Use the k-th fault characterization function in the p-th generation population and the spectrum value collected by the f-th failed test case of each multi-fault program at the i-th program statement to calculate the suspiciousness of the i-th program statement of each multi-fault program calculated using the f-th failed test case as:

[0161]

[0162] The value range of p is between 1 and end, where end is the generation number of the last generation population, the value range of k is between 1 and pop, the value range of f is between 1 and fail, and the value range of i is between 1 and K;

[0163] According to the above suspiciousness calculation method, the suspiciousness characterization calculated by using the k-th fault characterization function in the p-th generation population and the f-th failed test case of each multi-fault program is:

[0164] Rep_Suspiciousness p，k，f

[0165] ={Suspiciousness p，k，f，1 , Suspiciousness p，k，f，2 ,..., Suspiciousness p，k，f，is}

[0166] where is is the number of program statements of each multi-fault program;

[0167] The suspiciousness of each program statement in the suspiciousness characterization calculated according to the k-th failure characterization function and the f-th failed test case in the p-th generation population for each multi-fault program is used to sort the program statements, and the sorted list characterization generated for the f-th failed test case of each multi-fault program using the k-th failure characterization function in the p-th generation population is:

[0168] Rep_Ranking p，k，f ={Index p，k，f，1 , Index p，k，f，2 ,..., Index p，k，f，is}

[0169] Among them, Index p，k，f，rank represents the actual index position of the program statement at the rank-th position in each multi-fault program when sorted in descending order of the suspiciousness calculated according to the k-th failure characterization function and the f-th failed test case in the p-th generation population corresponding to the program statement;

[0170] The classification result of the sorted list characterizations of all failed test cases generated by applying each failure characterization function to each multi-fault program described in step 2 is;

[0171] Result p，k ={numOfClusters p，k , Clusters p，k}

[0172] Among them, Result p，k is the classification result of the sorted list characterizations of multiple failed test cases generated by applying the k-th failure characterization function in the p-th generation population to each multi-fault program, numOfClusters p，k is the number of clusters predicted using the k-th failure characterization function in the p-th generation population. When the number of clusters is not equal to the actual number of faults in each multi-fault program, skip the subsequent evaluation steps of the current failure characterization function on the current multi-fault program and set its fitness score on the corresponding multi-fault version to 0;

[0173] Clusters p，k is the cluster number assigned to each failed test case of each multi-fault program after clustering using the k-th failure characterization function in the p-th generation population. There is

[0174] Clusters p，k ={Group p，k，1 , Group p，k，2 ,..., Group p，k，fail}

[0175] where Group p，k，f is the sorted list generated by using the k-th fault characterization function in the p-th generation of the population for the f-th failed test case of each multi-fault program, representing the number of the corresponding class;

[0176] The effectiveness score of each external metric corresponding to the classification result generated by applying each fault characterization function to each multi-fault program as described in step 2 is:

[0177] Score_PR p，k = PR(Clusters p，k , Oracle)

[0178] Score_RR p，k = RR(Clusters p，k , Oracle)

[0179] Score_FMI p，k = FMI(Clusters p，k , Oracle)

[0180] Score_JC p，k = JC(Clusters p，k , Oracle)

[0181] where Oracle is the true correspondence between each failed test case and the fault source in each multi-fault program, and Score_PR p，k is the effectiveness score of the PR metric corresponding to the classification result generated by applying the k-th fault characterization function in the p-th generation of the population to each multi-fault program, Score_RR p，k is the effectiveness score of the RR metric corresponding to the classification result generated by applying the k-th fault characterization function in the p-th generation of the population to each multi-fault program, Score_FMI p，k is the effectiveness score of the FMI metric corresponding to the classification result generated by applying the k-th fault characterization function in the p-th generation of the population to each multi-fault program, and Score_JC p，k is the effectiveness score of the JC metric corresponding to the classification result generated by applying the k-th fault characterization function in the p-th generation of the population to each multi-fault program;

[0182] PR is the precision rate, RR is the recall rate, FMI is the Fowlkes-Mallows index, and JC is the Jaccard coefficient. The above four external metrics take the cluster numbers assigned to each failed test case of each multi-fault program after clustering using the k-th fault characterization function in the p-th generation population and the true correspondence between each failed test case and the fault source in each multi-fault program as inputs, and output the effectiveness score of this clustering. The score ranges from [0, 1], and the larger the value, the higher the effectiveness of the clustering process.

[0183] PR and RR are single-case metrics, which aim to compare whether the classification results of each failed test case are consistent with the true classification. For each fault class, the classification of a failed test case can be divided into four situations, including true positive (the failed test case is classified into this class by the fault separation framework and actually belongs to this class), false positive (the failed test case is classified into this class by the fault separation framework but actually does not belong to this class), true negative (the failed test case is not classified into this class by the fault separation framework and actually does not belong to this class), and false negative (the failed test case is not classified into this class by the fault separation framework but actually belongs to this class). Let TP, FP, TN, and FN be the sums of the occurrence times of the four situations of true positive, false positive, true negative, and false negative respectively. The calculation formulas for PR and RR are:

[0184]

[0185]

[0186] FMI and JC are pair metrics, which aim to compare whether the classification situations of each pair of failed test cases are consistent with the true classification. For each fault class, the classification consistency of a pair of failed test cases can be divided into four situations, including true same (both / neither of the two failed test cases are classified into this class by the fault separation framework and actually both belong to / do not belong to this class), false same (both / neither of the two failed test cases are classified into this class by the fault separation framework, but actually one failed test case belongs to this class and the other does not), false different (one failed test case is classified into this class by the fault separation framework and the other is not, but actually both / neither of the two failed test cases belong to this class), and true different (one failed test case is classified into this class by the fault separation framework and the other is not, and actually one failed test case belongs to this class and the other does not). Let SS, SD, DS, and DD be the sums of the occurrence times of the four situations of true same, false same, false different, and true different respectively. The calculation formulas for FMI and JC are:

[0187]

[0188]

[0189] The calculation of the four external indicators requires establishing the correspondence between each cluster number assigned to each failed test case of each multi-fault program after clustering using the k-th fault characterization function in the p-th generation population, and the true fault source. For each external indicator, calculate the effectiveness scores under all possible correspondences, and take the highest score as the calculation result of this indicator;

[0190] The fitness score when separating faults for each fault characterization function applied to each multi-fault program, obtained by combining the effectiveness scores of multiple external indicators corresponding to the classification results generated by applying each fault characterization function to each multi-fault program as described in step 2, is:

[0191] When calculating each external indicator for the classification results generated by applying each fault characterization function to each multi-fault program, it is necessary to take the effectiveness score under the correspondence with the highest effectiveness score as the calculation result. At this time, the current indicator is said to vote for this correspondence, and the number of votes for each correspondence is:

[0192]

[0193] Among them, represents Score_PR p，k Whether to vote for the d-th correspondence, represents Score_RR p，k Whether to vote for the d-th correspondence, represents Score_FMI p，k Whether to vote for the d-th correspondence, represents Score_JC p，k Whether to vote for the d-th correspondence, Vote p，k，d represents the total number of votes of the k-th fault characterization function in the p-th generation population at the d-th correspondence. The value of d ranges from 1 to V, where V is the number of possible correspondences of the multi-fault program;

[0194] In one embodiment, the training data set contains programs with 2, 3, 4, and 5 faults. For these faulty programs, there may be 2, 6, 24, and 120 correspondences respectively between the classification of failed test cases and the true fault classes, that is, the values of V are 2, 6, 24, and 120 respectively.

[0195] The total number of votes of the correspondence with the most votes is:

[0196] Vote p，k，most = Max(Vote p，k，1 , Vote p，k，2,..., Vote p，k，V )

[0197] where Vote p，k，most is the total number of votes for the corresponding relationship with the most votes in the evaluation of the effectiveness index for the classification results generated by applying the k-th fault characterization function in the p-th generation population to each multi-fault program, which is determined by the number of fault sources included in the multi-fault program;

[0198] The calculation method of the fitness score when applying the k-th fault characterization function in the p-th generation population to the fault isolation of each multi-fault program is as follows:

[0199] FitnessScore p，k = TotalMetrics p，k * PenaltyFactor p，k

[0200] where FitnessScore p，k is the fitness score when applying the k-th fault characterization function in the p-th generation population to the fault isolation of each multi-fault program;

[0201] TotalMetrics p，k = Score_PR p，k + Score_RR p，k + Score_FMI p，k + Score_JC p，k

[0202] where TotalMetrics p，k is the sum of the effectiveness scores of four external metrics for the classification results generated by applying the k-th fault characterization function in the p-th generation population to each multi-fault program;

[0203] PenaltyFactor p，k = 1 - 0.05 * (4 - Vote p，k，most )

[0204] where PenaltyFactor p，k is the fitness penalty factor when applying the k-th fault characterization function in the p-th generation population to the fault isolation of each multi-fault program, which is determined by Vote p，k，most ;

[0205] The fitness score calculation method can be adjusted according to the need for fault ranking characterization. If more attention is paid to the ideal value of the external index of clustering rather than whether the four indexes can be consistent in the corresponding relationship between the clustered clusters and the true fault classes, the influence of PenaktyFactor can be weakened by taking a lower value for 0.05 therein; if it is desired to ensure the authenticity of the clustering effectiveness evaluation, the fitness score calculation method described above may not be adopted, but after determining the corresponding relationship between the clustered clusters and the true fault classes, the sum of the four external index values on this corresponding relationship can be used as the fitness score.

[0206] The fitness of each individual in the initial population obtained according to the fitness score when each fault characterization function is applied to the fault separation of multiple multi-fault programs described in step 2 is as follows:

[0207] The individual fitness of each fault characterization function in the initial population is:

[0208]

[0209] where is the fitness of the k-th fault characterization function in the p-th generation population, Version ver represents the ver-th multi-fault program, nv is the number of multi-fault programs included in the multiple multi-fault programs introduced in step 1, is the fitness score when the k-th fault characterization function in the p-th generation population is applied to the fault separation of the ver-th multi-fault program;

[0210] In one embodiment, since 600 multi-fault programs can generally simulate multi-fault programs with different fault types, different program types and different numbers of faults and are suitable as the training data set for generating fault ranking characterization, the number of versions can be set to 600, that is, nv = 600.

[0211] In one embodiment, the fitness score evaluation process of the fault characterization function individuals is as shown in Figure 4 the example.

[0212] Step 3: Select the parental population from the population according to the fitness of each individual in the initial population, create each offspring individual using the parental population according to the crossover rate, replication rate and mutation rate, take all the created offspring individuals as the offspring population, and calculate the fitness of each individual in the offspring population through step 2;

[0213] The selection of the parental population from the population according to the fitness of each individual in the initial population described in step 3 is:

[0214] Use the roulette wheel algorithm to select pop*proportion distinct individuals from the population according to the fitness of each individual to form the parental population:

[0215]

[0216] Among them, pop is the population size, proportion is the ratio of the next-generation parent population to the parental population, and Population p represents the p-th generation population, represents the fitness score of the k-th fault characterization function in the p-th generation population, Roulette is the roulette wheel algorithm, and Father_Population p+1 represents the selection from Population p to generate the parental population for Population p+1 . The value range of p is from 1 to end, and the value range of k is from 1 to pop;

[0217] In one embodiment, since selecting half of the individuals from the population as the parental population can ensure the efficiency of the evolutionary process and prevent the premature generation of local optima, the ratio of the parental population to the population can be set to 0.5, that is, proportion = 0.5.

[0218] The individual selection algorithm for the parental population is not unique and can be flexibly adjusted to an algorithm other than Roulette according to needs.

[0219] The creation of offspring individuals using the parental population according to the crossover rate, replication rate, and mutation rate in step 3 is as follows:

[0220] Create each offspring individual using the parental population Father_Population p+1 according to the crossover rate, replication rate, and mutation rate

[0221] Determine the creation method of each offspring individual according to the crossover rate, replication rate, and mutation rate :

[0222]

[0223] Among them, c, o, and m are the crossover rate, replication rate, and mutation rate, and there is

[0224] c + o + m = 1

[0225] 0 ≤ c, o, m ≤ 1

[0226] random is a random number within the range of [0, 1], Method is the generation method of the current offspring individual, Crossover represents generation through crossover, Copy represents generation through copying, Mutate represents generation through mutation, and the individual creation method is determined according to the value of random and the magnitudes of the crossover rate, copy rate, and mutation rate;

[0227] In one embodiment, setting the crossover rate, copy rate, and mutation rate to 0.7, 0.2, and 0.1 respectively can ensure that the population evolves stably towards the direction of having a stronger ability to generate failure test case representations and has the ability to jump out of local optima. Therefore, the crossover rate, copy rate, and mutation rate can be set to 0.7, 0.2, and 0.1, that is, c = 0.7, o = 0.2, m = 0.1.

[0228] Based on satisfying the above - mentioned restrictive conditions, the settings of the crossover rate, copy rate, and mutation rate can be adjusted according to the actual situation. If the population cannot continuously improve the overall representation ability during the evolution process, the copy rate can be appropriately increased to retain the excellent individuals of the previous - generation population; if the population evolution falls into a local optimum and the optimal individuals of the population have not changed for multiple generations, the copy rate can be appropriately decreased and the mutation rate can be increased to break through the local optimum through mutating excellent individuals.

[0229] In one embodiment, since the replication of the optimal individual in the population easily leads to the population falling into a local optimum, a mutation rate of 0.5 can be additionally added in the case where the offspring population individual is confirmed to be generated by replicating the optimal individual of the parent population, in order to apply a mutation operation to it before replicating the optimal individual and improve the ability of the population evolution to break through the local optimum.

[0230] The creation of the offspring individual is as follows:

[0231]

[0232] Among them, is the k - th failure representation function of the offspring population of the p - th generation population, Create_Crossover is the crossover algorithm, REF r1 , REF r2 are two different individuals randomly selected from the parent population Father_Population p+1 , Create_Copy and Create_Mutate are the copy and mutation algorithms, REF r is an individual randomly selected from the parent population Father_Population p+1 ;

[0233] Each individual in the population, in addition to having the attribute of being itself as a failure representation function, also has an age attribute:

[0234]

[0235]

[0236] Among them, is the age of the k-th individual in the p-th generation population, is all individuals in the initial population, is the newly created individual in the offspring population, and Method is the corresponding creation method, REF r is created by the copy algorithm the individuals randomly selected from the parent population for copying when creating, and Age(*) represents the age of *;

[0237] When the age of an individual reaches 3, it cannot be selected as the parent population. Define Survive_Population p as the surviving population composed of all individuals with an age less than 3 in the p-th generation population;

[0238] That is, the selection method of the father population Father_Population of the p-th generation population p+1 should be modified to:

[0239]

[0240] Among them, the k-th fault characterization function of the p-th generation population is an individual in Survive_Population p if and only if

[0241] When the number of generated offspring individuals reaches the population size pop, stop generating, and use these pop individuals as the next generation population Population p+1 ;

[0242] The method described in step 3 for evaluating the fitness of the offspring population using the method in step 2 and continuing to evolve the next generation population until the termination condition is reached is:

[0243] Before meeting the user-defined termination condition SC (such as having iterated a fixed number of times or obtained a fault characterization function that meets the requirements), continuously use the method in step 2 to evaluate the fitness scores of each fault characterization function in the current p-th generation population Population p and use the method in step 3 to generate the offspring population Population p+1 ;

[0244] Regarding the selection of the termination condition, if the time cost of population evolution is low, it can be set to a fixed number of evolution generations; if each generation of population evolution consumes a lot of time, the user can flexibly decide whether to stop the evolution process according to the fitness evaluation of each generation.

[0245] Step 4: Repeat Step 2 and Step 3 until the user-specified termination condition is reached, and use the fault characterization function with the highest individual fitness in the last generation of the population that reaches the termination condition as the fault classification model.

[0246] What is described in Step 4 as using the fault characterization function with the highest individual fitness in the last generation of the population that reaches the termination condition as the fault classification model is as follows:

[0247] Population end is the population that has completed fitness evaluation for the last generation and reaches the termination condition SC. is Population end with the highest fitness score for the k-th fault characterization function, and this characterization function can be used for fault characterization generation and fault isolation of real multi-fault programs.

[0248] Step 5: Use the fault classification model in the multi-fault program that needs to be fault-partitioned to generate a sorted list representation for each failed test case, calculate the distance between each pair of sorted list representations, and cluster the failed test cases based on the distance between each pair of sorted list representations to achieve modeling of software abnormal behaviors and partitioning of multiple faults.

[0249] What is described in Step 5 as using the fault classification model in the multi-fault program that needs to be fault-partitioned to generate a sorted list representation for each failed test case is as follows:

[0250] Use the k-th fault characterization function with the highest fitness score selected from the population Population end that has completed fitness evaluation for the last generation and reaches the termination condition SC in Step 4, that is, the fault classification model to generate a sorted list representation Rep_Ranking end，k，f for each failed test case of the target multi-fault program according to the method in Step 2. end，k，f Rep_Ranking is the sorted list representation generated for the f-th failed test case of the target multi-fault program using the fault classification model.

[0251] What is described in Step 5 as calculating the distance between each pair of sorted list representations is as follows:

[0252] The pre-selected distance metric function Distance_Metric is as follows:

[0253] Distance fo，ft = Distance_Metric(Rep_Ranking end，k，fo , Rep_Ranking end，k，ft )

[0254] where Rep_Ranking end，k，fo , Rep_Ranking end，k，fo are the sorted list representations generated from two failed test cases fo and ft of the target multi-fault program according to the fault isolation model , and Distance fo，ft is the distance between the failed test cases fo and ft calculated using the distance metric function Distance_Metric;

[0255] In one embodiment, to more accurately measure the similarity between sorted list representations, a distance metric function proposed for Ranking, such as Jaccard distance, can be used.

[0256] Clustering the failed test cases using the calculated distance as described in step 5 is as follows;

[0257] The pre-selected clustering strategies include:

[0258] numOfClusters end，k

[0259] = Predict(Distance f1，f1 , Distance f1，f2 ,..., Distance fo，ft ,..., Distance fail，fail )

[0260] And:

[0261] Clusters end，k = Cluster(Distance f1，f1 , Distance f1，f2 ,..., Distance fo，ft ,..., Distance fail，fail )

[0262] where Distance fo，ft is the distance between the failed test cases fo and ft calculated using the distance metric function Distance_Metric, Predict is a function for predicting the number of clusters, and numOfClusters end，kis the number of clusters predicted using the fault classification model, Cluster is the clustering algorithm, Clusters end，k is the cluster number assigned to each failed test case of the target multi-fault program after clustering using the fault classification model, and there are

[0263] Clusters end，k ={Group end，k，1 ,Group end，k，2 ,...,Group end，k，fail}

[0264] Among them, Group end，k，f is the serial number of the class corresponding to the sorted list generated for the f-th failed test case of the target multi-fault program using the fault classification model;

[0265] Both Predict and Cluster take the distance between every two failed test cases fo and ft in the target multi-fault program as input.

[0266] A specific embodiment of the present invention also provides a computer-readable medium.

[0267] The computer-readable medium is a server workstation;

[0268] The server workstation stores a computer program executed by an electronic device. When the computer program runs on the electronic device, it causes the electronic device to execute the steps of the sorting similarity software abnormal behavior modeling method of the embodiment of the present invention.

[0269] It should be understood that the parts not elaborated in detail in this specification all belong to the prior art.

[0270] It should be understood that the above description of the preferred embodiment is relatively detailed, and it should not be considered as a limitation to the protection scope of the present invention patent. Under the inspiration of the present invention, those of ordinary skill in the art can also make substitutions or deformations without departing from the protection scope defined by the claims of the present invention, and all fall within the protection scope of the present invention. The scope of the present invention claimed should be subject to the appended claims.

Claims

1. A software abnormal behavior modeling method based on sorting similarity, characterized in that Including the following steps: Step 1: Introduce multiple multi-fault programs, randomly generate multiple fault characterization functions using a genetic programming algorithm, take each fault characterization function as each individual in the population, and construct an initial population through multiple fault characterization functions; Step 2: Combine each fault characterization function in the initial population to generate a sorted list representation of each failed test case generated by each fault characterization function applied to each multi-fault program, perform clustering division to obtain the classification result of the sorted list representation of multiple failed test cases generated by each fault characterization function applied to each multi-fault program, evaluate the effectiveness score of each external metric corresponding to the classification result generated by each fault characterization function applied to each multi-fault program, and obtain the fitness score at the time of fault separation of each fault characterization function applied to each multi-fault program and the fitness of each individual in the initial population; Step 3: Select a parental population from the population according to the fitness of each individual in the initial population, create each offspring individual using the parental population according to the crossover rate, replication rate, and mutation rate, take all the created offspring individuals as the offspring population, and calculate the fitness of each individual in the offspring population through Step 2; Step 4: Repeat Step 2 and Step 3 until the user-specified termination condition is reached, and take the fault characterization function with the highest individual fitness in the last population that reaches the termination condition as the fault classification model; Step 5: Use the fault classification model in the multi-fault program that needs to be fault-divided to generate a sorted list representation for each failed test case, calculate the distance between each pair of sorted list representations, and cluster the failed test cases in combination with the distance between each pair of sorted list representations to realize the modeling of software abnormal behavior and the division of multiple faults.

2. The software abnormal behavior modeling method based on sorting similarity according to claim 1, wherein Each fault characterization function described in Step 1 is specifically defined as follows: The fault characterization function REF is: Suspiciousness k，f，i = REF k (Spectrum f，i ) Spectrum f，i = {NCF f，i , NCS i , NUF f，i , NUS i} Among them, NCF f，i is the number of times the i-th program statement of each multi-fault program is covered by the f-th failing test case, and NCS i is the number of times the i-th program statement of each multi-fault program is covered by passing test cases, NUF f，i is the number of times the i-th program statement of each multi-fault program is not covered by the f-th failing test case, and NUS i is the number of times the i-th program statement of each multi-fault program is not covered by passing test cases. The value of i ranges from 1 to K, where K is the number of program statements in each multi-fault program. NCF f，i , NCS i , NUF f，i , and NUS i together serve as the spectral information Spectrum f，i at the i-th program statement of the f-th failing test case for each multi-fault program. Suspiciousness k，f，i is the suspicion degree of the existence of a fault source at the i-th program statement of each multi-fault program calculated by using the fault characterization function REF k in combination with the spectral information at the i-th program statement of the f-th failing test case; The construction of the initial population described in Step 1 is: Randomly generate several fault characterization functions using a genetic programming algorithm and take them together as the initial population: Among them, Population p represents the p-th generation population. The constructed initial population is Population1, and pop represents the number of individuals or the scale of the population. represents the k-th fault characterization function in the p-th generation population.

3. The software abnormal behavior modeling method based on sorting similarity according to claim 1, characterized in that The combination of each fault characterization function in the initial population to generate a sorted list representation of each failed test case generated by each fault characterization function applied to each multi-fault program described in Step 2 is specifically as follows: Each multi-fault program contains multiple failed test cases and multiple successful test cases; The multiple failed test cases are specifically defined as follows: {Ftc1, Ftc2, Ftc3, ..., Ftc fail} where fail is the number of failed test cases, and Ftc f represents the f-th failed test case in each multi-fault program, where the value of f ranges from 1 to fail; The multiple successful test cases are specifically defined as follows: {Stc1, Stc2, Stc3, ..., Stc success}; where success is the number of successful test cases, and Stc s represents the s-th successful test case in each multi-fault program, where s ranges from 1 to success; Collect the coverage information of each failed test case and each successful test case at each program statement in each multi-fault program; Calculate the spectrum value of each failed test case at each corresponding program statement in each multi-fault program using the coverage information of each failed test case and each successful test case at each program statement in each multi-fault program; The spectrum value Spectrum collected at the i-th program statement for the f-th failing test case of each multi-fault program f，i is as follows: Spectrum f，i = {NCF f，i , NCS i , NUF f，i , NUS i} The value of i ranges from 1 to K; Calculate the suspiciousness of the i-th program statement calculated by the f-th failed test case of each multi-fault program using the spectrum value collected by the k-th fault characterization function in the p-th population combined with the f-th failed test case of each multi-fault program as: The value of p ranges from 1 to end, where end is the generation number of the last generation of the population. The value of k ranges from 1 to pop, the value of f ranges from 1 to fail, and the value of i ranges from 1 to K; According to the above suspiciousness calculation method, the suspiciousness characterization obtained by using the k-th failure characterization function and the f-th failed test case in the p-th generation of the population for each multi-fault program is: Rep_Suspiciousness p，k，f ={Suspiciousness p，k，f，1 , Suspiciousness p，k，f，2 ,..., Suspiciousness p，k，f，is} where is is the number of program statements of each multi-fault program; Sort the suspiciousness of each program statement in the suspiciousness characterization obtained by using the k-th failure characterization function in the p-th generation of the population and the f-th failed test case for each multi-fault program. The sorted list characterization generated for the f-th failed test case of each multi-fault program by using the k-th failure characterization function in the p-th generation of the population is: Rep_Ranking p，k，f = {Index p，k，f，1 , Index p，k，f，2 ,..., Index p，k，f，is} Among them, Index p，k，f，rank represents the actual index position of the program statement ranked at the rank-th position in the descending order of suspiciousness calculated according to the k-th fault characterization function and the f-th failed test case in the p-th generation population for each multi-fault program.

4. The method for modeling software abnormal behavior based on sorting similarity according to claim 1, wherein The classification result of the sorted list characterization of multiple failed test cases generated by applying each failure characterization function to each multi-fault program obtained by clustering and partitioning in step 2 is as follows: Result p，k = {numOfClusters p，k , Clusters p，k} Among them, Result p，k is a sorted list representation classification result of multiple failure test cases generated by applying the k-th failure characterization function in the p-th generation population to each multi-failure program, and numOfClusters p，k is the number of clusters predicted using the k-th failure characterization function in the p-th generation population. When the number of clusters is not equal to the actual number of failures of each multi-failure program, skip the subsequent evaluation steps of the current failure characterization function on the current multi-failure program and set its fitness score on the corresponding multi-failure version to 0; Clusters p，k is the cluster number assigned to each failed test case of each multi-fault program after clustering using the k-th fault characterization function in the p-th generation population, and there is Clusters p，k = {Group p，k，1 , Group p，k，2 ,..., Group p，k，fail} where Group p，k，f is the serial number of the class corresponding to the sorted list generated by using the k-th fault characterization function in the p-th generation population for the f-th failed test case of each multi-fault program.

5. The software abnormal behavior modeling method based on sorting similarity according to claim 1, characterized in that The effectiveness score of each external index corresponding to the classification result generated by applying each failure characterization function to each multi-fault program evaluated according to multiple external indexes in step 2 is: Score_PR p，k = PR(Clusters p，k , Oracle) Score_RR p，k = RR(Clusters p，k , Oracle) Score_FMI p，k = FMI(Clusters p，k , Oracle) Score_JC p，k = JC(Clusters p，k , Oracle) Among them, Oracle is the true correspondence between each failed test case and the fault source in each multi-fault program, Score_PR p，k is the effectiveness score of the PR metric corresponding to the classification result generated by applying the k-th fault characterization function in the p-th generation population to each multi-fault program, Score_RR p，k is the effectiveness score of the RR metric corresponding to the classification result generated by applying the k-th fault characterization function in the p-th generation population to each multi-fault program, Score_FMI p，k is the effectiveness score of the FMI metric corresponding to the classification result generated by applying the k-th fault characterization function in the p-th generation population to each multi-fault program, Score_JC p，k is the effectiveness score of the JC metric corresponding to the classification result generated by applying the k-th fault characterization function in the p-th generation population to each multi-fault program; PR is the precision rate, RR is the recall rate, FMI is the Fowlkes-Mallows index, and JC is the Jaccard coefficient. The above four external indexes take the cluster number assigned to each failed test case of each multi-fault program after clustering using the k-th failure characterization function in the p-th generation of the population and the true correspondence between each failed test case and the fault source in each multi-fault program as inputs, and output the effectiveness score of this clustering. The score value is in the range of [0, 1]. The larger the value, the higher the effectiveness of the clustering process; The calculation of the four external indexes requires establishing the correspondence between each cluster in the cluster number assigned to each failed test case of each multi-fault program after clustering using the k-th failure characterization function in the p-th generation of the population and the true fault source. For each external index, calculate the effectiveness scores under all possible correspondence relationships, and take the highest score as the calculation result of this index.

6. The method for modeling software abnormal behavior based on sorting similarity according to claim 1, characterized in that, The fitness score when obtaining the fault separation of each failure characterization function applied to each multi-fault program in step 2 is as follows: Merge the effectiveness scores of multiple external indexes corresponding to the classification results generated by applying each failure characterization function to each multi-fault program to obtain the fitness score when the fault separation of each failure characterization function applied to each multi-fault program; The fitness score of each individual in the initial population obtained in step 2 is as follows: Obtain the fitness of each individual in the initial population according to the fitness scores when the fault separation of each failure characterization function applied to multiple multi-fault programs; 7. The software abnormal behavior modeling method based on sorting similarity according to claim 6, characterized in that The fitness score when merging the effectiveness scores of multiple external indexes corresponding to the classification results generated by applying each failure characterization function to each multi-fault program to obtain the fitness score when the fault separation of each failure characterization function applied to each multi-fault program is: When calculating each external metric for the classification results generated by each fault characterization function applied to each multi-fault program, it is necessary to take the effectiveness score under the corresponding relationship with the highest effectiveness score as the calculation result. At this time, the current metric is said to vote for this corresponding relationship. The votes for each corresponding relationship are as follows: Among them, represents Score_PR p，k whether to vote on the d-th corresponding relationship represents Score_RR p，k whether to vote on the d-th corresponding relationship represents Score-FMI p，k whether to vote on the d-th corresponding relationship represents Score_JC p，k whether to vote on the d-th corresponding relationship, Vote p，k，d represents the total number of votes obtained by the k-th fault characterization function in the p-th generation population at the d-th corresponding relationship, where the value of d ranges from 1 to V, and V is the number of possible corresponding relationships for the multi-fault program; The total number of votes for the corresponding relationship with the most votes is: Vote p，k，nost = Max(Vote p，k，1 , Vote p，k，2 ,..., Vote p，k，V ) Among them, Vote p，k，most is the total number of votes for the corresponding relationship with the most votes in the evaluation process of the effectiveness index of the classification results generated by applying the k-th fault characterization function in the p-th generation population to each multi-fault program, which is determined by the number of fault sources included in the multi-fault program; The fitness score calculation method when the k-th fault characterization function in the p-th generation population is applied to the fault isolation of each multi-fault program is: FitnessScore p，k = TotalMetrics p，k * PenaltyFactor p，k Among them, FitnessScore p，k is the fitness score when the k-th fault characterization function in the p-th generation population is applied to the fault isolation of each multi-fault program; TotalMetrics p，k = Score_PR p，k + Score_RR p，k + Score_FMI p，k + Score_JC p，k Among them, TotalMetrics p，k is the sum of the effectiveness scores of four external metrics of the classification results generated by applying the k-th fault characterization function in the p-th generation population to each multi-fault program; PenaltyFactor p，k = 1 - 0.05 * (4 - Vote p，k，most ) Among them, PenaltyFactor p，k is the fitness penalty factor when the k-th fault characterization function in the p-th generation population is applied to the fault isolation of each multi-fault program, and is determined by Vote p，k，most decide; The fitness of each individual in the initial population obtained according to the fitness scores when each fault characterization function is applied to the fault isolation of multiple multi-fault programs is: The individual fitness of each fault characterization function in the initial population is: where is the fitness of the k-th fault characterization function in the p-th generation population, Version ver represents the ver-th multi-fault program, and nv is the number of multi-fault programs included in the multiple multi-fault programs introduced in step 1 is the fitness score when the k-th fault characterization function in the p-th generation population is applied to fault isolation of the ver-th multi-fault program 8. The software abnormal behavior modeling method based on sorting similarity according to claim 1, characterized in that The method for selecting the parental population from the population according to the fitness of each individual in the initial population described in step 3 is: Use the roulette wheel algorithm to select pop*proportion distinct individuals from the population according to the fitness of each individual to form the parental population: Among them, pop is the population size, proportion is the ratio of the next parental population to the parental population, Population p represents the p-th generation population, represents the fitness score of the k-th fault characterization function in the p-th generation population, Roulette is the roulette wheel algorithm, Father_Population p+1 represents the selection from Population p to generate the parental population for Population p+1 The value of p ranges from 1 to end, and the value of k ranges from 1 to pop; The method for creating offspring individuals using the parental population according to the crossover rate, replication rate, and mutation rate described in step 3 is: Use the parent population Father - Population according to the crossover rate, replication rate, and mutation rate p+1 Create each offspring individual Determine how each offspring individual is created based on the crossover rate, replication rate, and mutation rate as follows: Among them, c, o, and m are the crossover rate, replication rate, and mutation rate, and there is c + o + m = 1 0 ≤ c, o, m ≤ 1 random is a random number in the interval [0, 1], Method is the generation method of the current offspring individual, Crossover represents generation by crossover, Copy represents generation by replication, Mutate represents generation by mutation, and the individual creation method is determined according to the value of random and the magnitudes of the crossover rate, replication rate, and mutation rate; The creation of offspring individuals is: Among them, is the k-th fault characterization function of the offspring population of the p-th generation population, Create_Crossover is the crossover algorithm, REF r1 , REF r2 is two different individuals randomly selected from the parent population Father_Population p+1 , Create_Copy and Create_Mutate are the copy and mutation algorithms, REF r is an individual randomly selected from the parent population Father_Population p+1 ; Each individual in the population, in addition to having the attribute of itself as a fault characterization function, also has an age attribute: Among them, is the age of the k-th individual in the p-th generation population, is all individuals in the initial population, is the newly created individual in the offspring population, Method is the corresponding creation method, REF r is created by the replication algorithm when randomly selecting individuals for replication from the parent population during creation; Age(*) represents the age of *. When the age of an individual reaches 3, it will not be selectable as the parental population, and Survive_Population is defined p as the surviving population composed of all individuals with an age less than 3 in the p-th generation population; That is, the selection method of the parental population Father_Population of the p-th generation population p+1 should be modified to: Among them, the k-th fault characterization function of the p-th generation population is an individual in Survive_Population p if and only if When the number of generated offspring individuals reaches the population size pop, stop generating, and use these pop individuals as the next-generation population Population p+1 ; The method described in step 3 for evaluating the fitness of the offspring population using the method in step 2 and continuing to evolve the next generation of the population until the termination condition is reached is: Before the user-defined termination condition SC is met, that is, before a fixed number of iterations have been performed or a fault characterization function that meets the requirements has been obtained, continuously use the method in step 2 to evaluate the fitness scores of each fault characterization function in the current p-th generation population Population, and use the method in step 3 to generate the offspring population Population p and use the method in step 3 to generate the offspring population Population p+1 .

9. The software abnormal behavior modeling method based on sorting similarity according to claim 1, characterized in that The method for using the fault characterization function with the highest individual fitness in the last generation population that reaches the termination condition as the fault classification model described in step 4 is: Population end is the population for which the fitness evaluation is completed for the last generation and the termination condition SC is reached, is Population end the k-th fault characterization function with the highest fitness score in, which can be used for fault characterization generation and fault isolation of real multi-fault programs; The method for generating a sorted list for each failed test case in the multi-fault program that needs to be fault partitioned using the fault classification model described in step 5 is: Use the population Population that completed fitness evaluation in the last generation in step 4 and reached the termination condition SC end The k-th fault characterization function with the highest fitness score selected from it, that is, the fault classification model Generate a sorted list representation Rep_Ranking for each failed test case of the target multi-fault program according to the method in step 2 end，k，f , Rep_Ranking end，k，f Is the sorted list representation generated for the f-th failed test case of the target multi-fault program using the fault classification model ; The method for calculating the distance between each pair of sorted list representations described in step 5 is: The pre-selected distance metric function Distance_Metric is: Distance fo，ft = Distance_Metric(Rep_Ranking end，k，fo , Rep_Ranking end，k，ft ) Among them, Rep_Ranking end，k，fo , Rep_Ranking end，k，fo is the sorted list representation generated by the two failed test cases fo and ft of the target multi-fault program according to the fault isolation model , Distance fo，ft is the distance between the failed test cases fo and ft calculated using the distance metric function Distance_Metric; The method for clustering the failed test cases using the calculated distance described in step 5 is; The pre-selected clustering strategies include: numOfClusters end，k = Predict(Distance f1，f1 , Distance f1，f2 , ..., Distance fo，ft , ..., Distance fail，fail ) And: Clusters end，k = Cluster(Distance f1，f1 , Distance f1，f2 ,..., Distance fo，ft ,..., Distance fail，fail ) Among them, Distance fo，ft is the distance between the failed test cases fo and ft calculated using the distance metric function Distance_Metric, Predict is the function for predicting the number of clusters, and numOfClusters end，k is the number of clusters predicted using the fault classification model, Cluster is the clustering algorithm, and Clusters end，k is the cluster number assigned to each failed test case of the target multi-fault program after clustering using the fault classification model, and there is Clusters end，k = {Group end，k，1 , Group end，k，2 ,..., Group end，k，fail} Among them, Group end，k，f is the serial number of the class corresponding to the sorted list generated by using the fault classification model for the f-th failed test case of the target multi-fault program; Both Predict and Cluster take the distance between every two failed test cases fo and ft in the target multi-fault program as input.

10. A computer-readable medium, characterized in that, The computer program executed by the storage electronic device, when the computer program runs on the electronic device, enables the electronic device to execute the steps of the method according to any one of claims 1-9.

Images