Test case filtering method based on reachability prediction model and related equipment

Abstract

The invention relates to the technical field of system testing, and provides a test case filtering method and related equipment based on an accessibility prediction model, and the method comprises the steps: generating a function call graph and a plurality of control flow graphs of a target system, and carrying out the integration to obtain an interprocess control flow graph; calculating the reachability representation of the test case based on the interprocess control flow diagram; performing feature extraction on the test cases to obtain case features, and constructing an association diagram of the test cases; carrying out reachability prediction by utilizing a reachability prediction model based on the use case characteristics and the association graph to obtain a prediction reachability representation; optimizing the reachability prediction model by using all the prediction reachability representations and all the reachability representations, and performing reachability prediction on the target test case by using the optimized reachability prediction model to obtain a final reachability representation; and filtering the target test case according to the final accessibility representation to obtain a filtering result. According to the method, the universality of test case filtering can be improved.

Description

Technical Field

[0001] The present application relates to the field of system testing technology, and in particular to a test case filtering method based on a reachability prediction model and related equipment. Background Art

[0002] In the context of the rapid development of cloud computing, the quality of the operating system kernel, as the core technology foundation, is directly related to the stability and security of cloud services. Cloud service providers such as Alibaba Cloud, Huawei Cloud, and Tencent Cloud all build their server operating systems based on the Linux kernel, serving multiple fields such as public cloud and hybrid cloud. However, with the increasing application of cloud computing, the security issues brought about by kernel defects are becoming increasingly prominent, which not only threatens business continuity and data security, but may also have a serious impact on the national economy.

[0003] According to statistics, the Linux kernel is one of the most vulnerable software, and as new features are constantly added, new defects are also increasing. Current open source operating system projects still face many challenges in kernel quality assurance, especially the lack of technical capabilities for kernel defect detection.

[0004] Test case execution is one of the steps with the largest test resource overhead in the fuzz testing process and is a key process in kernel fuzz testing. During the test case mutation execution process, kernel fuzz testing generates a large number of test cases for the operating system to execute through seed mutation operations. However, due to the lack of effective evaluation of test case quality, a large number of test cases cannot cover the target location during the test case execution process, resulting in a large amount of test resources being wasted on the execution of target-irrelevant test cases, reducing the execution probability of test cases covering the target location and restricting the kernel fuzz testing defect detection efficiency.

[0005] Current use case filtering technologies mainly include methods based on code coverage, mutation testing, symbolic execution, and other methods based on statistics or machine learning. Although these methods have improved the effectiveness and pertinence of test cases to a certain extent, they each have certain limitations. For example, although the code coverage-based method can help testers understand which parts of the code have been fully tested, this method often cannot distinguish which coverage is meaningful. Especially when facing complex systems, high coverage does not necessarily equate to high test quality. Mutation testing methods evaluate the effectiveness of test case sets by introducing "faults". Although they can provide deeper testing, they have high computational costs and difficult variant selection. Symbolic execution technology attempts to execute program paths through symbolic variables to automatically discover potential defects, but when dealing with non-deterministic inputs and complex path constraints, it may lead to an explosive growth of a large number of state spaces, thus affecting efficiency. Although methods based on statistics or machine learning can improve the efficiency of test case generation to a certain extent, they are still limited by data dependence and model generalization capabilities in practical applications. It can be seen that there is currently a problem of poor versatility in test case filtering methods. Summary of the invention

[0006] The present application provides a test case filtering method and related equipment based on a reachability prediction model, which can solve the problem of poor versatility of the test filtering method.

[0007] In a first aspect, an embodiment of the present application provides a test case filtering method based on a reachability prediction model, the test case filtering method comprising:

[0008] Generate a function call graph and multiple control flow graphs of the target system, and integrate the function call graph and all control flow graphs to obtain an inter-process control flow graph; multiple nodes in the function call graph correspond one-to-one to multiple kernel functions of the target system, and the edges between the nodes are the call relationships between the corresponding two kernel functions; multiple control flow graphs correspond one-to-one to multiple kernel functions; multiple nodes in the control flow graph correspond one-to-one to multiple basic blocks in the corresponding kernel function, and the edges between the nodes are the control relationships between the corresponding two basic blocks;

[0009] Obtain multiple test cases for testing the target system, and calculate the reachability representation of each test case based on the inter-procedural control flow graph; the reachability representation is used to describe the coverage of the test case for the target function, where the target function is the kernel function that needs to be tested;

[0010] For each test case, feature extraction is performed on the test case to obtain the test case feature, and a correlation graph is constructed based on the test case; multiple nodes in the correlation graph correspond to multiple call resources of the test case one by one, and the edges between the nodes are the correlation relationships between the corresponding two call resources;

[0011] Based on all use case features and all association graphs, reachability prediction is performed using the reachability prediction model to obtain the predicted reachability representation of each test case.

[0012] The reachability prediction model is optimized using all predicted reachability representations and all reachability representations to obtain an optimized reachability prediction model, and the reachability prediction of the target test case is performed using the optimized reachability prediction model to obtain a final reachability representation of the target test case;

[0013] The target test cases are filtered according to the final reachability representation to obtain filtering results; the filtering results are used to describe whether the target test cases can be used as test cases for the target system.

[0014] Optionally, compute a reachability representation for each test case based on the interprocedural control flow graph, including:

[0015] For each test case, perform the following steps:

[0016] Determine the use case node corresponding to each use case coverage position of the test case from all nodes of the inter-procedural control flow graph;

[0017] For each use case node, obtain the distance between the use case node and the node corresponding to the target function, and use the distance as the initial reachability representation of the use case node;

[0018] All initial reachability representations are integrated to obtain the reachability representation of the test case.

[0019] Optionally, feature extraction is performed on the test case to obtain the test case features, including:

[0020] The test cases include file description sequence, system call description sequence, and function description sequence;

[0021] Perform feature extraction on the file description sequence to obtain file semantic features, perform feature extraction on the system call description sequence to obtain call semantic features, perform feature extraction on the function description sequence to obtain function semantic features;

[0022] The file semantic features, call semantic features and function semantic features are concatenated to obtain the use case features of the test case.

[0023] Optionally, based on all use case features and all association graphs, a reachability prediction model is used to perform reachability prediction to obtain a predicted reachability representation of each test case, including:

[0024] For each test case, perform the following steps in sequence:

[0025] The input layer of the reachability prediction model is used to process the association graph of the test cases to obtain the case association matrix;

[0026] Based on the use case association matrix and the use case features of the test cases, the graph convolution layer of the reachability prediction model is used to obtain the use case high-order association features of the test cases;

[0027] The output layer of the reachability prediction model is used to process the high-order correlation features of the use case to obtain the predicted reachability representation.

[0028] Optionally, the association graph of the test case is processed using the input layer of the reachability prediction model to obtain a case association matrix, including:

[0029] By formula:

[0030]

[0031] A'=A+I

[0032] Calculating the Use Case Correlation Matrix

[0033] in, It represents the matrix used to perform symmetric normalization on the adjacency matrix, which is obtained from the degree matrix. A' represents the self-connection association matrix. D'(i,i) represents the degree of the i-th node in the association graph. A'(i,j) represents the adjacency relationship between the i-th node and the j-th node in the association graph. A represents the adjacency matrix of the association graph. i,j∈N, where N represents the total number of nodes in the association graph.

[0034] Optionally, the accessibility prediction model is optimized using all predicted accessibility representations and all accessibility representations to obtain an optimized accessibility prediction model, including:

[0035] Construct a loss function using all predicted reachability representations and all reachability representations;

[0036] The reachability prediction model is optimized using the loss function to obtain the optimized reachability prediction model.

[0037] Optionally, the loss function is:

[0038]

[0039] Among them, Err represents the value of the loss function, V represents the set of all test cases, M represents the number of categories of reachability labels, and Y p,q represents the probability that the accessibility of the p-th test case is represented by the q-th accessibility label, Z p,q represents the probability that the predicted reachability of the p-th test case is characterized by the q-th reachability label.

[0040] Optionally, the reachability prediction model is optimized using a loss function to obtain an optimized reachability prediction model, including:

[0041] Determine whether the value of the loss function is less than the preset value of the loss function;

[0042] If yes, the accessibility prediction model is used as the optimized accessibility prediction model;

[0043] Otherwise, adjust the parameters of the reachability prediction model, and return to the step of performing reachability prediction based on all use case features and all association graphs using the reachability prediction model to obtain the predicted reachability representation of each test case.

[0044] Optionally, the target test cases are filtered according to the final reachability representation to obtain filtering results, including:

[0045] If the final reachability representation indicates that the target test case covers the target function, then the filtering result of applying the target test case to the target system is obtained.

[0046] In a second aspect, an embodiment of the present application provides a test case filtering device based on a reachability prediction model, comprising:

[0047] A generation module generates a function call graph and multiple control flow graphs of the target system, and integrates the function call graph and all control flow graphs to obtain an inter-process control flow graph; multiple nodes in the function call graph correspond one-to-one to multiple kernel functions of the target system, and the edges between the nodes are the call relationships between the corresponding two kernel functions; multiple control flow graphs correspond one-to-one to multiple kernel functions; multiple nodes in the control flow graph correspond one-to-one to multiple basic blocks in the corresponding kernel functions, and the edges between the nodes are the control relationships between the corresponding two basic blocks;

[0048] A calculation module obtains multiple test cases for testing the target system, and calculates the reachability representation of each test case based on the inter-procedural control flow graph; the reachability representation is used to describe the coverage of the test case on the target function, and the target function is the kernel function that needs to be tested;

[0049] The construction module extracts features of each test case, obtains the test case features, and constructs an association graph based on the test case; multiple nodes in the association graph correspond to multiple call resources of the test case one by one, and the edges between the nodes are the association relationships between the corresponding two call resources;

[0050] The reachability prediction module uses the reachability prediction model to perform reachability prediction based on all use case features and all association graphs, and obtains the predicted reachability representation of each test case.

[0051] An optimization module optimizes the reachability prediction model using all predicted reachability representations and all reachability representations to obtain an optimized reachability prediction model, and uses the optimized reachability prediction model to perform reachability prediction on a target test case to obtain a final reachability representation of the target test case;

[0052] The filtering module filters the target test cases according to the final reachability representation to obtain filtering results; the filtering results are used to describe whether the target test cases can be used as test cases for the target system.

[0053] In a third aspect, an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned test case filtering method based on the reachability prediction model when executing the above-mentioned computer program.

[0054] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, it implements the above-mentioned test case filtering method based on the reachability prediction model.

[0055] The above solution of the present application has the following beneficial effects:

[0056] In an embodiment of the present application, a function call graph and multiple control flow graphs of a target system are generated, and the function call graph and all control flow graphs are integrated to obtain an inter-procedural control flow graph, and then multiple test cases for testing the target system are obtained, and the reachability representation of each test case is calculated based on the inter-procedural control flow graph, and then for each test case, feature extraction is performed on the test case to obtain the test case feature of the test case, and an association graph is constructed based on the test case, and then a reachability prediction is performed based on all the use case features and all the association graphs to obtain a predicted reachability representation of each test case, and then all the predicted reachability representations and all the reachability representations are used to optimize the reachability prediction model to obtain an optimized reachability prediction model, and the optimized reachability prediction model is used to perform reachability prediction on the target test case to obtain a final reachability representation of the target test case, and finally the target test case is filtered according to the final reachability representation to obtain a filtering result. Among them, the reachability representation is obtained according to the inter-procedural control flow graph, and the reachability of the system at two granularities, function and basic block, is focused on to improve the accuracy of the reachability representation. The reachability prediction model is optimized to improve the performance of the reachability prediction model and effectively improve the accuracy of the predicted reachability representation. Predictions are made based on use case features and association graphs and test cases are filtered without generating variants, which effectively improves the efficiency of test case filtering. At the same time, there is no need for specific processing of test cases, which reduces the data dependence on test cases and improves the versatility of test case filtering.

[0057] Other beneficial effects of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0059] Figure 1 A flowchart of a test case filtering method based on a reachability prediction model provided in one embodiment of the present application;

[0060] Figure 2 A schematic diagram of a function call graph provided in an embodiment of the present application;

[0061] Figure 3 A schematic diagram of a control flow graph provided in an embodiment of the present application;

[0062] Figure 4 A schematic diagram of an inter-process control flow graph provided by an embodiment of the present application;

[0063] Figure 5 A schematic diagram of an association diagram provided in an embodiment of the present application;

[0064] Figure 6 A specific flow chart of a test case filtering method based on a reachability prediction model provided in an embodiment of the present application;

[0065] Figure 7 A schematic diagram of the structure of a test case filtering device based on a reachability prediction model provided in one embodiment of the present application;

[0066] Figure 8 A schematic diagram of the structure of a terminal device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0067] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0068] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0069] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0070] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0071] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0072] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0073] In view of the problem of poor versatility of existing test case filtering methods, an embodiment of the present application provides a test case filtering method based on a reachability prediction model. The test case filtering method generates a function call graph and multiple control flow graphs of a target system, and integrates the function call graph and all control flow graphs to obtain an inter-process control flow graph, then obtains multiple test cases for testing the target system, calculates the reachability representation of each test case based on the inter-process control flow graph, and then extracts features of each test case to obtain the test case features of the test case, and constructs an association graph based on the test case, and then performs reachability prediction based on all the use case features and all the association graphs to obtain a predicted reachability representation of each test case, and then optimizes the reachability prediction model using all the predicted reachability representations and all the reachability representations to obtain an optimized reachability prediction model, and uses the optimized reachability prediction model to perform reachability prediction on the target test case to obtain a final reachability representation of the target test case, and finally filters the target test case according to the final reachability representation to obtain a filtering result. Among them, the reachability representation is obtained according to the inter-procedural control flow graph, and the reachability of the system at two granularities, function and basic block, is focused on to improve the accuracy of the reachability representation. The reachability prediction model is optimized to improve the performance of the reachability prediction model and effectively improve the accuracy of the predicted reachability representation. Predictions are made based on use case features and association graphs and test cases are filtered without generating variants, which effectively improves the efficiency of test case filtering. At the same time, there is no need for specific processing of test cases, which reduces the data dependence on test cases and improves the versatility of test case filtering.

[0074] The following is an explanation of the relevant professional terms of this application.

[0075] A target function refers to a function that implements a specific function and requires special attention and verification during software or system testing. It is usually a critical part or a place where defects are prone to occur.

[0076] A basic block is a maximally sequential sequence of statements in a function's program, with only one entry and exit, the entry being its first statement and the exit being its last statement.

[0077] A test case is a description of a test task for a software or system, reflecting the test plan, method, technology and strategy. Its contents include test objectives, test environment, input data, test steps, expected results, test scripts, etc., and finally form a document.

[0078] Kernel functions are functions that belong to the kernel of the system.

[0079] Next, an exemplary description is given of the test case filtering method based on the reachability prediction model provided in this application.

[0080] like Figure 1 As shown, the test case filtering method based on the reachability prediction model provided by the present application includes the following steps:

[0081] Step 11, generate a function call graph and multiple control flow graphs of the target system, and integrate the function call graph and all control flow graphs to obtain an inter-procedural control flow graph.

[0082] The multiple nodes in the above function call graph correspond one-to-one to the multiple kernel functions of the target system, the edges between the nodes are the call relationships between the corresponding two kernel functions, the multiple control flow graphs correspond one-to-one to the multiple kernel functions, a control flow graph is constructed for each kernel function with multiple basic blocks, the multiple nodes in the control flow graph correspond one-to-one to the multiple basic blocks in the kernel function, and the edges between the nodes are the control relationships between the corresponding two basic blocks. The above target system is the system to be tested, such as the server operating system of Alibaba Cloud, etc. The kernel function can be a list management function, a kernel library function, etc.

[0083] In some embodiments of the present application, static code analysis can be performed on the kernel code of the target system to obtain a function call graph and multiple control flow graphs, where the number of control flow graphs is the same as the number of kernel functions with basic blocks. When the function call graph and all control flow graphs are integrated, edges are generated between the control flow graph and the nodes of the kernel function corresponding to the control flow graph.

[0084] This step is described below with reference to a specific example.

[0085] Function call diagram Figure 2 As shown in the figure, the circle represents the node corresponding to the kernel function, the arrow represents the calling relationship between the nodes, and main, a, b, c, d, and e all represent kernel functions.

[0086] Taking kernel function b as an example, the control flow graph corresponding to the kernel function is as follows Figure 3 As shown, Figure 3 The squares in the figure represent the statements in the basic blocks of the kernel function, the arrows represent the control relationship between the statements, and the numbers are the numbers of the basic blocks.

[0087] Step 12, obtaining multiple test cases for testing the target system, and calculating the reachability representation of each test case based on the inter-procedural control flow graph.

[0088] The above reachability representation is used to describe the coverage of the target function by the test case, and the target function is the kernel function that needs to be tested.

[0089] Specifically, for each test case, perform the following steps:

[0090] In the first step, the use case node corresponding to each use case coverage position of the test case is determined from all the nodes of the inter-procedural control flow graph.

[0091] It should be noted that the above-mentioned use case coverage positions are kernel functions, basic blocks, etc. run by the target system when the target system is tested using the test case. The nodes corresponding to the use case coverage positions in the inter-procedural control flow graph are used as use case nodes.

[0092] In the second step, for each use case node, the distance between the use case node and the node corresponding to the target function is obtained, and the distance is used as the initial reachability representation of the use case node.

[0093] It should be noted that for the distance between the node corresponding to the kernel function and the node corresponding to the target function, the shortest path between the nodes is obtained from the process control flow graph, and the number of edges of the shortest path is used as the distance between the nodes. For the distance between the node corresponding to the basic block and the node corresponding to the target function, first calculate the distance between the kernel function corresponding to the basic block and the target function, the exit distance of the basic block from the corresponding kernel function, and the entrance distance of the basic block from the corresponding kernel function. Then, the weighted sum of the three distances is taken to obtain the distance between the node corresponding to the basic block and the node corresponding to the target function. Figure 3 Taking the control flow graph shown as an example, basic block 1 is the entry and basic block 7 is the exit. For basic block 2, the distance between basic block 2 and basic block 1 is 1, that is, the entry distance is 1, and the distance between basic block 2 and basic block 7 is 1, that is, the exit distance is 1.

[0094] In the third step, all initial reachability representations are integrated to obtain the reachability representation of the test case.

[0095] Exemplarily, all initial reachability representations are integrated into a set to obtain a reachability representation. The reachability representation is expressed as the distance between the test case coverage position and the target function of the test case.

[0096] It is worth mentioning that the reachability representation is obtained based on the inter-procedural control flow graph, focusing on the reachability of the system at two granularities: function and basic block, improving the accuracy of the reachability representation, accurately evaluating the effectiveness of the test cases, and avoiding the problem that high coverage does not necessarily equate to high test quality. Not only does it focus on the coverage of kernel function code lines, but also on the actual reachability of the test cases to specific code paths of kernel functions, thereby improving the effectiveness and pertinence of the test.

[0097] The above distance is exemplarily described below with reference to a specific example.

[0098] Will Figure 2 The function call graph shown is similar to Figure 3 The inter-procedural control flow graph obtained by integrating the control flow graph shown in Figure 4 As shown, Figure 4 The numbers of the nodes in the control flow graph represent the distance between the node and the target function. The distances between all nodes and the target function in the inter-procedural control flow graph are integrated to obtain Figure 4 As shown in the table, ICFG nodes (i.e. nodes in the inter-procedural control flow graph) include n1, n2, etc. The type of n1 is function, and the corresponding distance is 8.7. The type of n2 is basic block, and the corresponding distance is 30. Figure 4 N / A means the distance was not calculated.

[0099] Step 13, for each test case, extract the test case features to obtain the test case features, and build a correlation graph based on the test case.

[0100] The multiple nodes in the above association graph correspond to the multiple call resources of the test case one by one, and the edge between the nodes is the association relationship between the corresponding two call resources. The call resource is the function or basic block that the target system needs to call when executing the test case. The test case includes a file description sequence, a system call description sequence, and a function description sequence. The file description sequence is used to describe the file name of the file where the test case is located, the system call description sequence is used to describe the system call name of the test case, and the function description sequence is used to describe the function of the test case.

[0101] In some embodiments of the present application, the steps of extracting features from the test cases, obtaining the test case features, and constructing the association graph of the test cases are specifically as follows:

[0102] In the first step, feature extraction is performed on the file description sequence to obtain file semantic features, feature extraction is performed on the system call description sequence to obtain call semantic features, and feature extraction is performed on the function description sequence to obtain function semantic features.

[0103] Exemplarily, a deep language model (such as a large language model, a deep bag-of-words model, etc.) may be used to perform feature extraction to obtain file semantic features, call semantic features, and function semantic features.

[0104] For example, take the fuzzy test template file dev_snd_midi.txt as an example, which stores the system call templates related to the Linux MIDI interface (i.e., test cases, such as ioctl$SNDRV_RAWMIDI_IOCTL_INFO). For the system call template ioctl$SNDRV_RAWMIDI_IOCTL_INFO, ioctl is the system call name, and SNDRV_RAWMIDI_IOCTL_INFON is the function description sequence. Taking the system call template ioctl$SNDRV_RAWMIDI_IOCTL_INFO as an example, the file name (file description sequence) is dev_snd_midi, the system call name (system call description sequence) is ioctl, and the function description (function description sequence) is SNDRV_RAWMIDI_IOCTL_INFO.

[0105] In the second step, the file semantic features, call semantic features and function semantic features are concatenated to obtain the use case features of the test case.

[0106] The third step is to build an association graph based on the test cases.

[0107] The association diagram describes the association relationship between all call resources corresponding to the test case.

[0108] For example, the association diagram of the test case is as follows Figure 5 As shown in the figure, the test case consists of system calls open(), read(), and close(). The file opening, reading, and closing operations are completed through system calls in sequence. Among them, r0 is the file pointer, which is generated by open and used by read and close. In order to mine the association relationship of the use cases, the sequence relationship is first described by connecting the unidirectional edges in the graph structure (open points to read, and read points to close). Then, the resource dependency relationship between system calls is described according to the resource generation and use process between system calls (open points to read and close respectively). The code of the test case in this example is:

[0109] r0=open(&(0x7f0000000000)=”. / file0”,0x3,0x9);

[0110] read(r0,&(0x7f0000000000),42);

[0111] close(r0).

[0112] It is worth mentioning that by constructing the association graph of the test case, the resource calling sequence and dependency relationship of the test case can be described, and the key information of the test case can be obtained without generating a large number of variants, thereby reducing the computational cost and complexity and improving efficiency.

[0113] Step 14: Based on all use case features and all association graphs, reachability prediction is performed using a reachability prediction model to obtain a predicted reachability representation of each test case.

[0114] The above-mentioned reachability prediction model includes an input layer, a graph convolution layer, and an output layer connected in sequence. The input ends of the input layer and the graph convolution layer are both input ends of the reachability prediction model, and the output end of the output layer is the output end of the reachability prediction model. The input end of the input layer receives the association graph, and the input end of the graph convolution layer receives the use case features and the output data of the input layer.

[0115] In some embodiments of the present application, the above-mentioned steps of performing reachability prediction based on all use case features and all association graphs using the reachability prediction model to obtain the predicted reachability representation of each test case are specifically as follows:

[0116] For each test case, perform the following steps in sequence:

[0117] In the first step, the input layer of the reachability prediction model is used to process the association graph of the test case to obtain the case association matrix.

[0118] Specifically, the input layer is expressed as:

[0119]

[0120] A'=A+I

[0121] Calculating the Use Case Correlation Matrix

[0122] in, It represents the matrix used to perform symmetric normalization on the adjacency matrix, which is obtained from the degree matrix. A' represents the self-connection association matrix. D'(i,i) represents the degree of the i-th node in the association graph (the number of edges connected to the node). A'(i,j) represents the adjacency relationship between the i-th node and the j-th node in the association graph. A represents the adjacency matrix of the association graph, i,j∈N, and N represents the total number of nodes in the association graph.

[0123] It can be understood that the above formula for calculating the use case association matrix is ​​an expression of the input layer.

[0124] In the second step, based on the use case association matrix and the use case features of the test cases, the graph convolutional layer of the reachability prediction model is used to obtain the use case high-order association features of the test cases.

[0125] Specifically, the use case association matrix and use case features are input into the graph convolution layer for processing to obtain the high-order association features of the use case.

[0126] It should be noted that the graph convolution layer includes multiple layers of graph convolutional networks, and the propagation rules of two adjacent graph convolutional layer networks are expressed as:

[0127]

[0128] Among them, L (i+1) represents the input feature matrix of the i-th layer graph convolutional network. When i=0, L (0) represents the use case characteristics, W (i) Represents the weight matrix, i=0,1,2,...,I, and I represents the last layer of graph convolutional network.

[0129] In the third step, the output layer of the reachability prediction model is used to process the high-order correlation features of the use case to obtain the predicted reachability representation.

[0130] It should be noted that the output layer is an activation function, and the calculation expression of the graph convolution layer and the output layer is:

[0131]

[0132] Among them, F(X,A) represents the predicted reachability representation, represents the use case association matrix, X represents the use case characteristics, and W (1) and W (2) Both represent weight matrices.

[0133] It is worth mentioning that the reachability prediction model built based on graph convolutional networks can handle complex paths and non-deterministic input problems, avoid the problem of state space explosion, and capture the complex relationships of test cases.

[0134] Step 15, optimizing the reachability prediction model using all predicted reachability representations and all reachability representations to obtain an optimized reachability prediction model, and using the optimized reachability prediction model to perform reachability prediction on the target test case to obtain the final reachability representation of the target test case.

[0135] The above target test cases are test cases that need to be filtered.

[0136] In some embodiments of the present application, the above steps of optimizing the reachability prediction model using all predicted reachability representations and all reachability representations to obtain an optimized reachability prediction model, and performing reachability prediction on a target test case using the optimized reachability prediction model to obtain a final reachability representation of the target test case are specifically as follows:

[0137] In the first step, a loss function is constructed using all predicted reachability representations and all reachability representations.

[0138] Specifically, the loss function is:

[0139]

[0140] Among them, Err represents the value of the loss function, V represents the set of all test cases, M represents the number of categories of reachability labels, and Y p,q represents the probability that the accessibility of the p-th test case is represented by the q-th accessibility label, Z p,q represents the probability that the predicted reachability of the p-th test case is characterized by the q-th reachability label.

[0141] It should be noted that the reachability representation and predicted reachability representation reflect the distance between the test case and the target function. When there are multiple target functions, the target function that is closest to the test case can be described. In this step, in order to construct the loss function, through the process from step 12 to step 13, and by changing the target function, the reachability representation and predicted reachability representation of different target functions and each test case are obtained. The above reachability label describes the closest distance between the test case and the target function. For example, the qth reachability label describes the information that "the test case is closest to the qth target function".

[0142] In the second step, the reachability prediction model is optimized using the loss function to obtain the optimized reachability prediction model.

[0143] Specifically, it is determined whether the value of the loss function is less than a preset value of the loss function.

[0144] If yes, the accessibility prediction model is used as the optimized accessibility prediction model.

[0145] Otherwise, adjust the parameters of the reachability prediction model, and return to the step of performing reachability prediction based on all use case features and all association graphs using the reachability prediction model to obtain the predicted reachability representation of each test case.

[0146] For example, in order to prevent overfitting of model training, a regularization (Dropout) network layer can be added between graph convolutional layers.

[0147] In the third step, the optimized reachability prediction model is used to perform reachability prediction on the target test case to obtain the final reachability representation of the target test case.

[0148] Specifically, firstly, feature extraction is performed on the target test case to obtain the use case features of the target test case, and an association graph of the target test case is constructed. Based on the use case features and the association graph, the reachability prediction of the target test case is performed using the optimized reachability prediction model to obtain the final reachability representation.

[0149] It is worth mentioning that by training the reachability prediction model, the performance of the reachability prediction model can be improved, thereby improving the accuracy of the final reachability representation.

[0150] Step 16: Filter the target test cases according to the final reachability representation to obtain filtering results.

[0151] The above filtering results are used to describe whether the target test case can be used as a test case for the target system.

[0152] Specifically, if the final reachability representation indicates that the target test case covers the target function, then the target test case can be used as a test case for the target system. If the final reachability representation indicates that the target test case does not cover the target function, then the target test case cannot be used as a test case for the target system. When testing the target function of the target system, the target test case is removed.

[0153] It is worth mentioning that by obtaining the reachability representation based on the inter-procedural control flow graph, the reachability of the system at two granularities, function and basic block, is focused on, the accuracy of the reachability representation is improved, and the reachability prediction model is optimized, which can improve the performance of the reachability prediction model and effectively improve the accuracy of the predicted reachability representation. Predictions are made based on use case features and association graphs, and test cases are filtered without generating variants, which effectively improves the efficiency of test case filtering. At the same time, there is no need for specific processing of test cases, which reduces the data dependence on test cases and improves the versatility of test case filtering.

[0154] The test case filtering method based on the reachability prediction model provided in this application is illustrated below with reference to a specific example.

[0155] like Figure 6As shown, the input data includes an inter-procedural control flow graph (ICFG), a target function, and a test case. The ICFG and the target function are subjected to a target reachability relationship characterization analysis to obtain a reachability characterization. The test case is subjected to a use case feature extraction to obtain a use case feature. The test case is subjected to association mining to obtain an association description (i.e., an association graph). The use case features and the association graph are then input into a reachability prediction model for processing. The model is trained using the reachability characterization. The reachability prediction model outputs a targeted reachability prediction result of the test case (i.e., a predicted reachability characterization). A seed filtering decision is made based on the targeted reachability prediction result (i.e., the step of obtaining the filtering result in step 16 above). When the method of the present application is implemented as follows Figure 6 As shown in the targeted fuzz testing sub-process - case evaluation filtering, the test cases undergo seed filtering (i.e., reachability prediction model and seed filtering decision), followed by seed execution (i.e., executing the filtered test cases) and coverage information collection.

[0156] It should be noted that the method provided in this application aims to overcome the limitations of the existing technology, especially the problems that the current use case filtering method has difficulty in distinguishing meaningful coverage in complex systems, high computational cost, difficulty in selecting variants, and state space explosion. The method provided in this application effectively solves these problems by introducing a reachability prediction model, combining system call type feature extraction and graph convolutional network technology.

[0157] First, the method provided by this application analyzes the reachability relationship between the test case and the target through reachability characterization, and provides a compact feature representation for each test case. Compared with the traditional code coverage-based method, the method of this application not only focuses on the coverage of code lines, but also focuses on the actual reachability of test cases to specific code paths. It can more accurately evaluate the effectiveness of the use case and avoid the problem that high coverage does not necessarily equate to high test quality.

[0158] Secondly, the method of the present application extracts use case features based on system call types, and mines use case associations based on system call dependencies. This method extracts key features of use cases by analyzing system call types, and further mines associations between use cases through system call dependencies. Compared with mutation testing methods, this method does not require the generation of a large number of variants, thereby reducing computational costs and complexity. At the same time, this method can more accurately identify which use cases can cover specific program paths, improving the effectiveness and pertinence of test cases.

[0159] Finally, the method of the present application constructs a use case reachability prediction model based on a graph convolutional network. By training the model with use cases in the corpus, the trained model can predict the reachability representation of the use case to be executed. This method utilizes the powerful capabilities of graph convolutional networks, can handle complex paths and non-deterministic input problems, and avoids the problem of state space explosion in symbolic execution methods. Graph convolutional networks can capture the complex relationships between use cases, and predict the reachability of use cases through trained models, thereby filtering out use cases that are not related to the target function before the test case is executed, improving test efficiency and quality.

[0160] Compared with the existing methods, the method of the present application overcomes the shortcomings of the prior art in the following aspects:

[0161] (1) Accuracy:

[0162] Through reachability characterization, this method can more accurately evaluate the effectiveness of the use case and avoid the problem that high coverage does not necessarily equate to high test quality. This method not only focuses on the coverage of code lines, but also focuses on the actual reachability of the use case to a specific code path, thereby improving the effectiveness and pertinence of the test.

[0163] (2) Computational efficiency:

[0164] By extracting use case features through system call types and mining use case associations based on system call dependencies, the method of the present application can extract key features of use cases without generating a large number of variants. This method reduces computational cost and complexity and improves test efficiency.

[0165] (3) Handling complex paths:

[0166] Through the reachability prediction model of the graph convolutional network, the method of the present application can handle complex paths and non-deterministic input problems, avoiding the problem of state space explosion. The graph convolutional network can capture the complex relationship between use cases and predict the target reachability of the use case through the trained model, thereby filtering out use cases that are not related to the target before the use case is executed, improving the test efficiency and quality.

[0167] (4) Comprehensive advantages:

[0168] The method of this application combines the advantages of multiple technologies, and provides a more comprehensive and efficient use case filtering method through reachability characterization, system call type feature extraction and graph convolutional network technology. It not only improves the effectiveness and pertinence of test cases, but also reduces the computational cost and complexity, thus having higher practical value and promotion potential in actual applications.

[0169] The following is an exemplary description of the test case filtering device based on the reachability prediction model provided in the present application.

[0170] like Figure 7 As shown, the embodiment of the present application provides a test case filtering device based on a reachability prediction model, and the test case filtering device 700 based on a reachability prediction model includes:

[0171] A generation module 701 generates a function call graph and multiple control flow graphs of the target system, and integrates the function call graph and all control flow graphs to obtain an inter-process control flow graph; multiple nodes in the function call graph correspond one-to-one to multiple kernel functions of the target system, and the edges between the nodes are the call relationships between the corresponding two kernel functions; multiple control flow graphs correspond one-to-one to multiple kernel functions; multiple nodes in the control flow graph correspond one-to-one to multiple basic blocks in the corresponding kernel functions, and the edges between the nodes are the control relationships between the corresponding two basic blocks;

[0172] The calculation module 702 obtains multiple test cases for testing the target system, and calculates the reachability representation of each test case based on the inter-procedural control flow graph; the reachability representation is used to describe the coverage of the test case to the target function, and the target function is the kernel function that needs to be tested;

[0173] The construction module 703 extracts features of each test case to obtain the test case features, and constructs an association graph based on the test case; multiple nodes in the association graph correspond to multiple call resources of the test case one by one, and the edges between the nodes are the association relationships between the corresponding two call resources;

[0174] The reachability prediction module 704 performs reachability prediction based on all use case features and all association graphs using a reachability prediction model to obtain a predicted reachability representation of each test case;

[0175] The optimization module 705 optimizes the reachability prediction model using all predicted reachability representations and all reachability representations to obtain an optimized reachability prediction model, and uses the optimized reachability prediction model to perform reachability prediction on the target test case to obtain a final reachability representation of the target test case;

[0176] The filtering module 706 filters the target test case according to the final reachability representation to obtain a filtering result; the filtering result is used to describe whether the target test case can be used as a test case for the target system.

[0177] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0178] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0179] like Figure 8 As shown, an embodiment of the present application provides a terminal device. The terminal device D10 of this embodiment includes: at least one processor D100 ( Figure 8 Only one processor is shown in the figure), a memory D101, and a computer program D102 stored in the memory D101 and executable on the at least one processor D100, wherein the processor D100 implements the steps of any of the above-mentioned method embodiments when executing the computer program D102.

[0180] Specifically, when the processor D100 executes the computer program D102, it generates a function call graph and multiple control flow graphs of the target system, and integrates the function call graph and all control flow graphs to obtain an inter-procedural control flow graph, then obtains multiple test cases for testing the target system, calculates the reachability representation of each test case based on the inter-procedural control flow graph, and then performs feature extraction on each test case to obtain the test case feature of the test case, and builds an association graph based on the test case, and then performs reachability prediction based on all the use case features and all the association graphs to obtain a predicted reachability representation of each test case, and then optimizes the reachability prediction model using all the predicted reachability representations and all the reachability representations to obtain an optimized reachability prediction model, and uses the optimized reachability prediction model to perform reachability prediction on the target test case to obtain the final reachability representation of the target test case, and finally filters the target test case according to the final reachability representation to obtain a filtering result. Among them, the reachability representation is obtained according to the inter-procedural control flow graph, and the reachability of the system at two granularities, function and basic block, is focused on to improve the accuracy of the reachability representation. The reachability prediction model is optimized to improve the performance of the reachability prediction model and effectively improve the accuracy of the predicted reachability representation. Predictions are made based on use case features and association graphs and test cases are filtered without generating variants, which effectively improves the efficiency of test case filtering. At the same time, there is no need for specific processing of test cases, which reduces the data dependence on test cases and improves the versatility of test case filtering.

[0181] The processor D100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0182] In some embodiments, the memory D101 may be an internal storage unit of the terminal device D10, such as a hard disk or memory of the terminal device D10. In other embodiments, the memory D101 may also be an external storage device of the terminal device D10, such as a plug-in hard disk, a smart memory card (SMC, SmartMedia Card), a secure digital (SD, Secure Digital) card, a flash card (Flash Card), etc. equipped on the terminal device D10. Further, the memory D101 may also include both an internal storage unit of the terminal device D10 and an external storage device. The memory D101 is used to store an operating system, an application program, a boot loader (BootLoader), data and other programs, such as the program code of the computer program, etc. The memory D101 may also be used to temporarily store data that has been output or is to be output.

[0183] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0184] An embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0185] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium that can carry the computer program code to the test case filtering method device / terminal device based on the reachability prediction model. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk.

[0186] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0187] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0188] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles described in the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A test case filtering method based on a reachability prediction model, characterized in that: include: Generate a function call graph and multiple control flow graphs of the target system, and integrate the function call graph and all control flow graphs to obtain an inter-procedural control flow graph; The multiple nodes in the function call graph correspond one-to-one to the multiple kernel functions of the target system, the edges between the nodes are the call relationships between the corresponding two kernel functions, the multiple control flow graphs correspond one-to-one to the multiple kernel functions, the multiple nodes in the control flow graph correspond one-to-one to the multiple basic blocks in the corresponding kernel functions, and the edges between the nodes are the control relationships between the corresponding two basic blocks; Acquire multiple test cases for testing the target system, and calculate the reachability representation of each of the test cases based on the inter-procedural control flow graph; The reachability representation is used to describe the coverage of the target function by the test case, and the target function is the kernel function that needs to be tested; For each of the test cases, feature extraction is performed on the test case to obtain the test case feature of the test case, and an association graph is constructed based on the test case; the multiple nodes in the association graph correspond one-to-one to the multiple call resources of the test case, and the edges between the nodes are the association relationships between the corresponding two call resources; Based on all use case features and all association graphs, reachability prediction is performed using a reachability prediction model to obtain a predicted reachability representation of each of the test cases; Optimizing the reachability prediction model using all predicted reachability representations and all reachability representations to obtain an optimized reachability prediction model, and performing reachability prediction on a target test case using the optimized reachability prediction model to obtain a final reachability representation of the target test case; Filter the target test cases according to the final reachability representation to obtain a filtering result; The filtering result is used to describe whether the target test case can be used as a test case for the target system.

2. The test case filtering method according to claim 1, characterized in that: The calculating the reachability representation of each of the test cases based on the inter-procedural control flow graph comprises: For each of the test cases, perform the following steps: Determine, from all nodes of the inter-procedural control flow graph, a use case node corresponding to each use case coverage position of the test case; For each of the use case nodes, respectively, obtain the distance between the use case node and the node corresponding to the target function, and use the distance as the initial reachability representation of the use case node; All initial reachability representations are integrated to obtain the reachability representation of the test case.

3. The test case filtering method according to claim 1, characterized in that: The extracting features of the test case to obtain the test case features includes: The test cases include a file description sequence, a system call description sequence, and a function description sequence; Performing feature extraction on the file description sequence to obtain file semantic features, performing feature extraction on the system call description sequence to obtain call semantic features, and performing feature extraction on the function description sequence to obtain function semantic features; The file semantic feature, the call semantic feature and the function semantic feature are concatenated to obtain the use case feature of the test case.

4. The test case filtering method according to claim 1, characterized in that: The reachability prediction is performed based on all use case features and all association graphs using a reachability prediction model to obtain a predicted reachability representation of each test case, including: For each test case, perform the following steps in sequence: Processing the association graph of the test case using the input layer of the reachability prediction model to obtain a case association matrix; Based on the use case association matrix and the use case features of the test case, using the graph convolution layer of the reachability prediction model to obtain the use case high-order association features of the test case; The output layer of the reachability prediction model is used to process the high-order correlation features of the use case to obtain a predicted reachability representation.

5. The test case filtering method according to claim 4, characterized in that: The step of processing the association graph of the test case using the input layer of the reachability prediction model to obtain a case association matrix includes: By formula: A'=A+I Calculating the Use Case Correlation Matrix in, represents the matrix used to perform symmetric normalization on the adjacency matrix, which is obtained from the degree matrix, A' represents the self-connection association matrix, D'(i,i) represents the degree of the i-th node in the association graph, A'(i,j) represents the adjacency relationship between the i-th node and the j-th node in the association graph, A represents the adjacency matrix of the association graph, i,j∈N, and N represents the total number of nodes in the association graph.

6. The test case filtering method according to claim 1, characterized in that: The optimizing the accessibility prediction model by using all predicted accessibility representations and all accessibility representations to obtain an optimized accessibility prediction model includes: Construct a loss function using all predicted reachability representations and all reachability representations; The reachability prediction model is optimized using the loss function to obtain an optimized reachability prediction model.

7. The test case filtering method according to claim 6, characterized in that: The loss function is: Among them, Err represents the value of the loss function, V represents the set of all test cases, M represents the number of categories of reachability labels, and Y p,q represents the probability that the accessibility of the p-th test case is represented by the q-th accessibility label, Z p,q represents the probability that the predicted reachability of the p-th test case is characterized by the q-th reachability label.

8. The test case filtering method according to claim 7, characterized in that: The step of optimizing the reachability prediction model by using the loss function to obtain an optimized reachability prediction model includes: Determine whether the value of the loss function is less than a preset value of the loss function; If yes, the accessibility prediction model is used as the optimized accessibility prediction model; Otherwise, adjust the parameters of the reachability prediction model, and return to the step of performing reachability prediction based on all use case features and all association graphs using the reachability prediction model to obtain a predicted reachability representation of each test case.

9. The test case filtering method according to claim 1, characterized in that: The filtering of the target test case according to the final reachability representation to obtain a filtering result includes: If the final reachability representation indicates that the target test case covers the target function, a filtering result is obtained that the target test case can be used as a test case for the target system.

10. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the test case filtering method based on the reachability prediction model as described in any one of claims 1 to 9 is implemented.

Images