Differential Testing Method and System for X.509 Certificate Verification Based on Differential Information Transfer Graph

Abstract

The present invention discloses an X.509 certificate verification differential testing method and system based on a differential information transfer graph. Construction stage of the differential information transfer graph: Collect real DER format certificate chains captured from the network as initial seeds and store them in a corpus, and construct a differential information transfer graph based on this. Certificate generation stage: Traverse the differential information transfer graph, select differential information vertices and seed certificate chains according to the potential value, and generate test cases through fuzzification. Certificate verification stage: Send the test cases to all SSL / TLS protocol implementation programs to be tested, record the discovered verification differences, and update the corpus and the differential information transfer graph. The present invention makes full use of the differential information generated when different implementations verify the same certificate, and effectively solves the problems of insufficient diversity of test cases and insufficient effectiveness of certificate generation strategies existing in the existing work.

Description

Technical Field

[0001] The present invention relates to the technical field of network communication protocol testing. Specifically, it relates to a differential testing method and system for X.509 certificate verification based on a differential information transfer graph. Background Art

[0002] Fuzz testing is a method for vulnerability discovery by sending a large number of test cases with a certain degree of randomness but conforming to the basic input format to the target program to be tested and observing whether abnormal results occur. With the increasing importance of network protocol security, network protocol fuzz testing has also become a research hotspot.

[0003] The SSL / TLS protocol is a protocol for secure communication between a Web server and a browser. It determines whether the communication endpoints are correct by verifying whether the X.509 certificate is issued by a trusted certificate authority (CA) during the handshake phase to ensure the secure establishment of the connection. Existing vulnerability detection work for X.509 certificate verification in the SSL / TLS protocol implementation mainly achieves it through differential fuzz testing. Its core idea is to input the same mutated certificate to multiple protocol implementations to be tested simultaneously. If different verification results occur after they verify the mutated certificate, it indicates that there may be potential errors in one of the programs.

[0004] Existing vulnerability detection for X.509 certificate verification in the SSL / TLS protocol implementation mainly includes the following work: Frankencert is a black-box unguided work that generates test certificates through random combination mutations. Mucert improves Frankencert's random mutation strategy, optimizes the certificate generation process according to the code coverage executed by each input in a single application, and selects the seed with the highest coverage for testing through Markov chain Monte Carlo sampling. NEZHA is a domain-independent general differential testing tool that guides the testing using the behavioral asymmetry information between multiple programs. Transcert is a coverage-oriented differential testing method that introduces a concept called a coverage transfer graph to abstract the coverage transfer from seeds to mutated certificates. However, the above existing work still has the following two problems:

[0005] (1) Insufficient test case diversity: Existing work does not fully utilize the information generated when different protocol implementations verify the same certificate, resulting in insufficient diversity in the generated test cases to cover the behaviors in all target protocol implementations. Specifically, Frankencert is a black box without guidance, and it does not make any use of the information generated by the protocol implementation during certificate verification. Mucert and transcert only guide certificate generation through code coverage on a single reference implementation, without noticing that in differential testing, there is still a lot of unused coverage information on other programs, which may help trigger differences. NEZHA is the only one of these tools that considers measuring the behavioral differences in different protocol implementations through multiple indicators. However, in order to ensure domain independence, it uses an uncontrollable random mutation strategy based on mutation to generate certificates. This method is too blind and does not make good use of the collected behavioral difference information. Therefore, it cannot guarantee that the random mutation test cases can cover the behaviors in all target protocol implementations.

[0006] (2) Insufficient validity of certificate generation strategy: X.509 certificates use the DER (Distinguished Encoding Rules) structure, and its syntax always follows<Tag,Length,Value> , where Length refers to the length of the Value field. However, the existing certificate generation strategy mainly mutates the value of the Value field of the certificate, which may change the length of the field, but ignores the synchronous update of the Length field, so it cannot ensure that the mutated certificate complies with the DER syntax. This will reduce the efficiency of the generated mutated certificate, which may be rejected in the early stage of verification and cannot be tested in depth. Summary of the invention

[0007] In view of the above problems, the present invention proposes a differential fuzzy testing method for X.509 certificate verification based on a difference information transfer graph. The method is divided into a difference information transfer graph construction phase, a certificate generation phase and a certificate verification phase. In the difference information transfer graph construction phase, the real DER format certificate chains captured from the network are collected as initial seeds and stored in the corpus, and the difference information transfer graph is constructed based on this. In the certificate generation phase, the difference information transfer graph is traversed, the difference information vertices and seed certificate chains are selected according to the potential values, and test cases are generated through fuzzification. In the certificate verification phase, the test cases are sent to all SSL / TLS protocol implementation programs to be tested, the verification differences found are recorded, and the corpus and the difference information transfer graph are updated. The present invention makes full use of the difference information generated by different implementations when verifying the same certificate, and effectively solves the problems of insufficient test case diversity and insufficient effectiveness of certificate generation strategies in existing work.

[0008] The X.509 certificate verification differential fuzzy testing method based on the difference information transfer graph proposed in the present invention mainly includes three stages: a difference information transfer graph construction stage, a certificate generation stage and a certificate verification stage.

[0009] In the differential information transfer graph construction phase, the real DER format certificate chains captured from the Internet are collected as initial seeds and stored in the corpus, and the differential information transfer graph is constructed based on them;

[0010] In the certificate generation phase, the difference information transfer graph is traversed, the difference information vertices and seed certificate chains are selected according to the potential value, and test cases are generated through fuzzification;

[0011] During the certificate verification phase, the test cases are sent to all SSL / TLS protocol implementations to be tested, the verification differences found are recorded, and the corpus and difference information transfer graph are updated.

[0012] Furthermore, the difference information transfer graph in the difference information transfer graph construction phase is defined as follows;

[0013] 1-1. The difference information transfer graph is defined as a weighted directed graph G = (V, E, W), where:

[0014] (1) V represents a set of difference information vertices; each difference information vertex represents a unique difference information; in order to facilitate acquisition and comparison, each difference information vertex is identified by a unique hash value; each difference information vertex is associated with a certificate chain pool, which stores one or more certificate chains; the difference information that can be generated by each certificate chain in the certificate chain pool is the same as the difference information represented by the vertex;

[0015] (2) E represents the set of difference information transfer edges. If the certificate chain corresponding to a difference information vertex u mutates into a test case of a difference information vertex v, a difference information transfer edge u→v is added, indicating the transfer from difference information vertex u to difference information vertex v.

[0016] (3)W represents the set of edge weights; the weight of the difference information transfer edge u→v is defined by the following formula:

[0017]

[0018] Among them, Certs(u→v) represents the number of mutated certificate chains of difference information vertex v obtained by mutating the certificate chain in difference information vertex u, and Certs(v) represents the number of all certificate chains in the certificate chain pool corresponding to difference information vertex v.

[0019] Furthermore, the difference information transfer graph construction phase specifically includes the following steps:

[0020] 1-2-1. Seed collection;

[0021] Collect DER real certificate chains from the Internet and store them in the seed certificate chain corpus as initial seeds;

[0022] 1-2-2. Seed verification and difference information collection

[0023] Send each unmutated seed certificate chain in the certificate corpus to all SSL / TLS protocol implementation programs to be tested for certificate verification, and collect all difference information generated after all SSL / TLS protocol implementation programs to be tested verify the certificate chain;

[0024] 1-2-3. Construction of difference information transfer graph

[0025] Determine whether the vertex corresponding to the difference information exists in the difference information transfer graph. If so, add the current seed certificate chain to the certificate chain pool corresponding to the vertex. Otherwise, create a new difference information vertex and add the certificate chain to its certificate chain pool.

[0026] Furthermore, the difference information described in step 1-2-2 refers to the difference between the response, coverage, and path information generated by all protocol implementation programs when performing certificate verification on the same test case, which is quantified by the following three indicators:

[0027] (1) Response indicator: Given a set of target protocol implementation programs SUT = [sut_1, sut_2, ..., sut_n] and a test case Certs, each target protocol implementation program sut_i in the SUT set will generate a verification result r after verifying the certificate of the test case Certs. sut_i ; The tuple composed of the above verification results <r sut_1 , r sut_2 , ..., r sut_n >Defined as a response indicator of the test case Certs;

[0028] (2) Coverage index: Given a set of target protocol implementation programs SUT = [sut 1 , sut_2, ..., sut_n] and a test case Certs, collect the coverage information cov when each target protocol implementation program sut_i in SUT performs certificate verification on the test case Certs sut_i ; The tuple composed of the above coverage <cov sut_1 , cov sut_2 , ..., cov sut_n >Defined as a coverage metric for test case Certs;

[0029] (3) Path Index: Given a set of target protocol implementation programs SUT = [sut_1, sut_2,..., sut_n] and a test case Certs, collect the path information path of the control flow graph accessed when each target protocol implementation program sut_i in SUT performs certificate verification on the test case Certs. sut_i ; The tuple <|path sut_1 |, |path sut_2 |,..., |path sut_n |> composed of the number of edges of the above paths is defined as a path index of the test case Certs;

[0030] The difference information generated by testing a set of target protocol implementation programs SUT with a test case is represented by response metrics, coverage metric rates, and path metrics as: {<rs ut_1 , r sut_2 ,..., r sut_n >, <cov sut_1 , cov sut_2 ,..., cov sut_n >, <|path sut_1 |, |path sut_2 |,..., |path sut_n |>}, and the response metrics, coverage metric rates, and path metric contents are converted into a hash value through a hash operation to uniquely represent a difference information vertex.

[0031] Furthermore, in the certificate generation stage, traverse the difference information transfer graph, select difference information vertices and seed certificate chains according to the potential values, and generate test cases through fuzzification. The specific implementation is as follows:

[0032] 2-1. Selection of Difference Information Vertices

[0033] Vertex selection is performed by assigning a potential value to each vertex in the difference information transfer graph; the higher the potential value of a vertex, the more likely it is to be selected for testing; specifically, the calculation formula for the potential value is as follows:

[0034]

[0035] Among them, trans_num(v) represents the number of transfer paths starting from the difference information vertex v, p(trans(v)) represents the weight of each transfer path starting from the difference information vertex v, and choose(v) represents the number of times the difference information vertex v is selected for testing;

[0036] From the formula, the potential value of each differential information vertex depends on the number of transfer paths starting from this differential information vertex, the sum of the weights of each transfer path starting from this differential information vertex, and the proportion of this differential information vertex being selected; therefore, it is found that if a differential information vertex has more differential information transfer paths and fewer times of being selected for testing, then the potential value of this differential information vertex is higher and it is more likely to be selected for testing;

[0037] Normalize the potential values of all differential information vertices and convert them into a probability distribution in [0, 1], and select differential information vertices by randomly selecting numbers in [0, 1];

[0038] 2-2. Certificate chain selection

[0039] Since the differential information generated by all certificate chains in the certificate chain pool corresponding to each differential information vertex is the same, after selecting the differential information vertex, randomly select a certificate chain in its corresponding certificate chain pool as the seed for testing;

[0040] 2-3. Test case generation

[0041] After selecting the certificate chain as the seed, it is necessary to mutate it to generate test cases; first mutate the certificate chain, then randomly select multiple certificates on the certificate chain, convert each selected certificate into a tree structure, then mutate the leaf nodes of the tree structure, and then perform a bottom-up repair on the mutated certificate through backtracking repair to ensure that the mutated certificate still conforms to the DER structure.

[0042] Furthermore, the specific operations for generating test cases in step 2-3 are as follows:

[0043] 2-3-1 Certificate chain mutation

[0044] In the certificate chain mutation stage, only perform mutations at the chain level on the certificate chain and do not mutate the certificate fields; specific certificate chain mutation operations: perform mutation operations a random number of times within the range of [1, Q] on the certificate chain, including operations of inserting a certificate, deleting a certificate, and replacing a certain certificate in the certificate chain with another certificate;

[0045] When performing chain mutation on the certificate chain, also update the issuer and subject fields of the certificates on the certificate chain to ensure that each certificate is issued by the subsequent certificate;

[0046] 2-3-2 Certificate field mutation

[0047] The certificate field mutation stage mainly mutates individual certificates on the certificate chain;

[0048] First, randomly select one or more certificates in the certificate chain and add them to the queue of certificates to be mutated. Then, parse each certificate in the queue into a tree structure. After that, perform a depth-first traversal of the tree structure and randomly select leaf nodes for mutation. The mutation strategy includes modifying the signature algorithm or field of the certificate, deleting the certificate field, and adding the certificate field.

[0049] 2-3-3 Backtracking Repair

[0050] In the backtracking repair phase, the DER structure damaged by the mutation is repaired recursively from the mutated leaf nodes from the bottom up. The specific repair steps are as follows:

[0051] (1) Determine whether the current mutation operation changes the length of the Value field. If not, it means that the mutation will not damage the DER structure and the repair is complete. Otherwise, proceed to the next step.

[0052] (2) Correct the Length field of the current leaf node so that the Length value is consistent with the length of the Value field of the current node;

[0053] (3) Recursively correct the Length field of the parent node of the current node to make the Length value consistent with the length of the Value field of the current node, until the current node is the root node, and the repair is complete;

[0054] (4) Restore the repaired certificate from the tree structure to the mutated certificate in DER format.

[0055] Furthermore, in the certificate verification phase, the mutated certificate chain is sent as a test case to all SSL / TLS protocol implementations to be tested, the difference information generated after they execute the test case is collected, and the difference information transfer graph is updated; the specific implementation is as follows:

[0056] 4.1 Test case verification and difference information collection

[0057] Send the mutated certificate chain as a test case to all SSL / TLS protocol implementations to be tested, and collect the difference information I generated after they execute the test case;

[0058] Then add the current test case to the certificate corpus for saving;

[0059] 4.2 Update of difference information transfer diagram

[0060] Breadth-first traverse the entire difference information transfer graph to determine whether there is a difference information vertex that is the same as the difference information I generated by this test;

[0061] If the same difference information vertex is found, synchronize the current test case to the certificate chain pool corresponding to the difference information vertex;

[0062] If not, update the difference information generated in this round of testing as a new difference information vertex to the difference information transfer graph, synchronize the current test case to the certificate chain pool corresponding to the new difference information vertex, and add a transfer path from the difference information vertex of the original certificate to the new difference information vertex;

[0063] Finally, update the edge weights of the difference information transfer graph.

[0064] The X.509 certificate verification differential testing system based on the difference information transfer graph specifically includes:

[0065] Certificate collection module: The fuzz testing system uses the Zmap tool to perform network scanning on port 443 and establish an SSL connection, and obtains a corpus containing 1005 DER certificates provided by Frankencert developers as the initial certificate seeds;

[0066] Difference information transfer graph construction module: The fuzz testing system uses the script invent.sh provided by transcert to sequentially send the seed certificate chains in the certificate corpus to all SSL / TLS protocol implementation programs to be tested, and collect all the difference information generated after verification; use the improved seed_init method to construct the initial difference information transfer graph, which will judge whether each difference information exists in the difference information transfer graph. If it exists, add the current seed certificate chain to the certificate chain pool corresponding to the vertex, otherwise create a new difference information vertex and add the certificate chain to its certificate chain pool;

[0067] Test case generation module: The fuzz testing system traverses the difference information transfer graph, selects difference information vertices and seed certificate chains according to the potential value, generates test cases through fuzzification, and sends them to all SSL / TLS protocol implementations to be tested for certificate verification;

[0068] Difference information recording module: The fuzz testing system collects the difference information generated after verifying the test cases by all SSL / TLS protocol implementations to be tested, and adds the current test case to the certificate corpus for storage; then, traverse the entire difference information transfer graph, add the current test case to the certificate chain pool corresponding to the vertex in the graph, and update the weights of the edges in the graph.

[0069] The beneficial effects of the present invention are:

[0070] 1. The present invention proposes an X.509 certificate verification differential fuzz testing method based on a differential information transfer graph. By using the differential information transfer graph for test guidance, it makes more full use of the differential information generated by different implementations when verifying the same certificate than existing work. At the same time, through backtracking repair, it performs a bottom-up repair on the mutated certificate, so that the generated test cases can ensure the syntactic correctness of the X.509 certificate.

[0071] 2. The test guidance strategy based on the differential information transfer graph proposed by the present invention makes full use of the differential information generated by different implementations when verifying the same certificate. The diversity of the generated test cases can cover more behaviors in the target protocol implementation, effectively solving the problem of insufficient diversity of test cases existing in existing work.

[0072] 3. The tree-based test case generation strategy proposed by the present invention converts the seed certificate into a tree structure for mutation, and performs a bottom-up repair on the mutated certificate through backtracking repair, making the mutated certificate conform to the syntactic correctness of X.509, effectively solving the problem of insufficient effectiveness of the certificate generation strategy existing in existing work.

[0073] In summary, in view of the two deficiencies described in the background art, for the first deficiency, the present invention uses the differential information transfer graph for test guidance, making more full use of the differential information generated by different implementations when verifying the same certificate than existing work. For the second deficiency, through backtracking repair, it performs a bottom-up repair on the mutated certificate, so that the generated test cases can ensure the syntactic correctness of the X.509 certificate. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 It is the overall test flow chart of the present invention.

[0075] Figure 2 It is an example diagram of differential information in an X.509 certificate verification differential fuzz testing method based on a differential information transfer graph of the present invention.

[0076] Figure 3 It is an explanatory schematic diagram of the differential information transfer graph of the present invention.

[0077] Figure 4 It is an explanatory schematic diagram of test case generation of the present invention.

[0078] Figure 5 It is an experimental comparison of the number of verification differences triggered by four test tools after the same number of iterations.

[0079] Figure 6 The variation of the test case diversity of four test tools with the number of test cases.

[0080] Figure 7 The line coverage generated by four testing tools in OpenSSL varies with the number of iterations. Specific implementation manners

[0081] The technical solutions of the embodiments of the present invention will be described in detail below with reference to the accompanying drawings of the present invention.

[0082] As Figure 1 shown, the stateful protocol differential fuzz testing method based on behavior difference guidance proposed by the present invention mainly includes three stages: a difference information transfer graph construction stage, a certificate generation stage, and a certificate verification stage.

[0083] 1 Difference information transfer graph construction stage

[0084] Collect real DER format certificate chains captured from the network as initial seeds and store them in the corpus, and construct a difference information transfer graph based on this. The specific content includes the following:

[0085] 1.1 Seed collection

[0086] Collect real DER certificate chains from the network and store them in the seed certificate chain corpus as initial seeds;

[0087] 1.2 Seed verification and difference information collection

[0088] Send each seed certificate chain (without mutation) in the certificate corpus to all SSL / TLS protocol implementation programs to be tested for certificate verification, and collect all difference information generated after the certificate chain is verified by all SSL / TLS protocol implementation programs to be tested;

[0089] 1.3 Difference information transfer graph construction

[0090] Determine whether the vertex corresponding to the difference information exists in the difference information transfer graph. If it exists, add the current seed certificate chain to the certificate chain pool corresponding to the vertex; otherwise, create a new difference information vertex and add the certificate chain to its certificate chain pool.

[0091] The pseudocode of the above process is shown as follows:

[0092]

[0093] The difference information in the difference information transfer graph refers to the differences in responses, coverage rates, paths, etc. generated by all protocol implementation programs when verifying certificates for the same test case, as Figure 2 shown.

[0094] More formally, the above difference information is quantified by the following three metrics:

[0095] (1) Response metric: Given a set of target protocol implementers SUT = [sut_1, sut_2,..., sut_n] and a test case Certs, each target protocol implementer sut_i in SUT will generate a verification result r after performing certificate verification on the test case Certs. sut_i . The tuple <r sut_1 , r sut_2 ,..., r sut_n > composed of the above verification results is defined as a response metric for the test case Certs.

[0096] (2) Coverage metric: Given a set of target protocol implementers SUT = [sut 1 , sut_2,..., sut_n] and a test case Certs, collect the coverage information cov sut_i when each target protocol implementer sut_i in SUT performs certificate verification on the test case Certs. sut_1 . The tuple <cov sut_2 , cov sut_n > composed of the above coverages is defined as a coverage metric for the test case Certs.

[0097] (3) Path metric: Given a set of target protocol implementers SUT = [sut_1, sut_2,..., sut_n] and a test case Certs, collect the path information path sut_i of the control flow graph accessed when each target protocol implementer sut_i in SUT performs certificate verification on the test case Certs. sut_1 . The tuple <|path sut_2 |, |path sut_n |,... |path sut_1 |> composed of the number of edges of the above paths is defined as a path metric for the test case Certs.

[0098] The difference information generated by testing a set of target protocol implementers SUT with a test case is represented by metrics such as response, coverage, and path as: {<r sut_2 , r sut_n ,..., r sut_1 >, <cov sut_2 , cov sut_n >, <|path sut_1 |, |path sut_2 |,... |path sur_n |>}.

[0099] For example Figure 3As shown in the figure, the differential information transfer graph is defined as a weighted directed graph G = (V, E, W), where:

[0100] (1) V represents the set of differential information vertices. Each differential information vertex represents a unique piece of differential information. For the convenience of acquisition and comparison, each differential information object is identified by a unique hash value. Each differential information vertex is associated with a certificate chain pool, which stores one or more certificate chains. The differential information generated by each certificate chain in the certificate chain pool is the same as the differential information represented by this vertex.

[0101] (2) E represents the set of differential information transfer edges. If the certificate corresponding to a differential information vertex u can mutate into a test case with differential information v, then a differential information transfer edge u → v is added, indicating that it can be transferred from the differential information vertex u to the differential information vertex v.

[0102] (3) W represents the set of edge weights. The weight of the differential information transfer edge u → v is defined by the following formula:

[0103]

[0104] Among them, Certs(u → v) represents the number of mutated certificate chains obtained by mutating the certificate chains in the differential information vertex u to obtain differential information v, and Certs(v) represents the number of all certificate chains in the certificate chain pool corresponding to the differential information vertex v. For example, if a certain certificate chain in the certificate chain pool corresponding to vertex u mutates to obtain some mutated certificate chains that trigger new differential information (assuming this differential information corresponds to vertex v), then a transfer edge is established from vertex u to vertex v, and its weight is the proportion of the mutated certificate chains in all the certificate chains in the certificate chain pool corresponding to v.

[0105] 2 Certificate Generation Phase

[0106] Traverse the differential information transfer graph, select differential information vertices and seed certificate chains according to the potential value, and generate test cases through fuzzification. Specifically, it includes the following contents:

[0107] 2.1 Selection of Differential Information Vertices

[0108] Vertices are selected by assigning a potential value to each vertex in the differential information transfer graph. The higher the potential value of a vertex, the easier it is to be selected for testing. Specifically, the calculation formula of the potential value is as follows:

[0109]

[0110] Among them, trans_num(v) represents the number of transfer paths starting from vertex v, p(trans(v)) represents the weight of each transfer path starting from vertex v, and choose(v) represents the number of times vertex v is selected for testing. From the formula, the potential value of each differential information vertex depends on the number of transfer paths starting from this vertex, the sum of the weights of each transfer path starting from this vertex, and the proportion of this vertex being selected. Therefore, it can be found that if the number of transfer paths of a vertex's differential information or the easier it is to transfer to other vertices, and the fewer times it is selected, then its potential value is higher and it is more likely to be selected for testing.

[0111] Normalize the potential values of all vertices and convert them into a probability distribution in [0, 1], and then select vertices by means of random numbers.

[0112] 2.2 Certificate Chain Selection

[0113] All the certificate chains in the certificate chain pool corresponding to each vertex have the same differential information. Therefore, after selecting a vertex, randomly select a certificate chain from its corresponding certificate chain pool as a seed for testing.

[0114] The pseudocode of the above process is as follows:

[0115]

[0116]

[0117] 2.3 Test Case Generation

[0118] After selecting the certificate chain as a seed, it is necessary to mutate it to generate test cases. As Figure 4 shown, first mutate the certificate chain, then select the certificates on the chain, convert them into a tree structure, and mutate its leaf nodes, and then perform a bottom-up repair on the mutated certificate through backtracking repair to ensure that the mutated certificate still conforms to the DER structure. The specific operations are as follows:

[0119] 2.3.1 Certificate Chain Mutation

[0120] In the certificate chain mutation stage, only perform chain-level mutations on the certificate chain, such as inserting or deleting a certificate in the certificate chain, or replacing a certain certificate in the chain with another certificate, without mutating the certificate fields. When performing chain mutations on the certificate chain, usually the issuer and subject fields of the certificates on the chain also need to be updated to ensure that each certificate is issued by the subsequent certificate. The present invention implements the chain mutation operator in Mucert for the mutation of the certificate chain.

[0121] 2.3.2 Certificate Field Mutation

[0122] The certificate field mutation phase mainly mutates a single certificate on the chain.

[0123] First, randomly select one or more certificates on the chain and add them to the queue of certificates to be mutated. Next, parse each certificate to be mutated in the queue into a tree structure representation. After that, perform a depth-first traversal of the tree and randomly select leaf nodes for mutation. Mutation strategies include modifying the signature algorithm or field of the certificate, deleting certificate fields, adding certificate fields, etc.

[0124] 2.3.3 Backtracking repair

[0125] In the backtracking repair phase, the DER structure damaged by the mutation is repaired recursively from the mutated leaf nodes from the bottom up. The specific repair steps are as follows:

[0126] (1) Determine whether the current mutation operation changes the length of the Value field. If not, it means that the mutation will not damage the DER structure and the repair is complete. Otherwise, proceed to the next step.

[0127] (2) Correct the Length field of the current leaf node so that its value is consistent with the length of the Value field of the current node;

[0128] (3) Recursively correct the Length field of the parent node of the current node to make its value consistent with the length of the Value field of the current node, until the current node is the root node and the repair is completed.

[0129] Restore the repaired certificate from the tree structure to a certificate in DER format.

[0130] The pseudo code of the above process is as follows:

[0131]

[0132]

[0133] 3. Certificate Verification Phase

[0134] The mutated certificate chain is sent as a test case to all SSL / TLS protocol implementations to be tested, the difference information generated after they execute the test case is collected, and the difference information transfer graph is updated.

[0135] Specifically include the following:

[0136] 3.1 Test case verification and difference information collection

[0137] Send the completed certificate chain after mutation as a test case to all SSL / TLS protocol implementations to be tested, and collect the difference information generated after they execute this test case. Then, add the current test case to the certificate corpus for storage.

[0138] 3.2 Update of the difference information transfer graph

[0139] Perform a breadth-first traversal of the entire difference information transfer graph to determine whether there is a vertex of difference information that is the same as the difference information generated in this test. If the same vertex of difference information is found, synchronize the current test case to the certificate chain pool corresponding to this vertex. If not found, update the difference information generated in this round of testing as a new vertex to the difference information transfer graph, synchronize the current test case to the certificate chain pool corresponding to this new vertex, and add a transfer path from the vertex of difference information of the original certificate to this new vertex. Finally, update the edge weights of the difference information transfer graph.

[0140] The pseudocode of the above process is as follows:

[0141]

[0142]

[0143] Another object of the present invention is to provide an X.509 certificate verification differential fuzz testing system based on a difference information transfer graph, mainly including:

[0144] Certificate collection module: The fuzz testing system uses the Zmap tool to perform network scanning on port 443 and establish an SSL connection, and obtains a corpus containing 1005 DER certificates provided by Frankencert developers as the initial certificate seeds.

[0145] Difference information transfer graph construction module: The fuzz testing system uses the script exec.sh provided by transcert to send the seed certificate chains in the certificate corpus to all SSL / TLS protocol implementation programs to be tested in turn, and collect all the difference information generated after verification. Use the improved seed_init method of the present invention to construct the initial difference information transfer graph, which will judge whether each difference information exists in the difference information transfer graph. If it exists, add the current seed certificate chain to the certificate chain pool corresponding to this vertex, otherwise create a new vertex of difference information and add this certificate chain to its certificate chain pool.

[0146] Test case generation module: The fuzz testing system traverses the difference information transfer graph, selects vertices of difference information and seed certificate chains according to the potential value, generates test cases through fuzzification, and sends them to all SSL / TLS protocol implementations to be tested for certificate verification.

[0147] Difference information recording module: The fuzz testing system collects the difference information generated after verifying all the test cases of the SSL / TLS protocol implementation to be tested, and adds the current test case to the certificate corpus for storage. Then, it traverses the entire difference information transfer graph, adds the current test case to the certificate chain pool corresponding to the vertex in the graph, and updates the weights of the edges in the graph.

[0148] Embodiment

[0149] To verify the effectiveness of the method of the present invention, the present invention implements a differential fuzz testing tool Diffcert based on this method by adjusting and modifying transcert. The present invention applies it to the SSL / TLS protocol implementation programs OpenSSL, MatrixSSL, and mbedTLS, and conducts experimental comparisons with the differential fuzz testing tools Frankencert, transcert, and NEZHA respectively. The experimental design and experimental results will be introduced below.

[0150] Experimental scheme design: The differential fuzz testing tool Diffcert implemented based on the method of the present invention is used as the experimental group. The differential fuzz testing tool Frankencert is selected as the control group 1. The differential fuzz testing tool NEZHA is selected as the control group 2. The differential fuzz testing tool transcert is selected as the control group 3.

[0151] To minimize the influence of initial conditions on the experimental results, the same initial certificate is selected as the seed for all four tools. The present invention uses a corpus containing 1005 certificates provided by the Frankencert developers as the initial seed, and these certificates are collected by scanning the network on port 443 through Zmap and establishing SSL connections.

[0152] The number of iterations of the three control group tools and the experimental group tool are all set to 50,000 times. To avoid the influence of randomness on the experiment, each group of experiments is repeated three times, and the subsequent chart data uses the average value of the results of the three experiments.

[0153] Test target selection: The present invention selects the SSL / TLS protocol implementation programs OpenSSL (version 1.1.1c), MatrixSSL (version 4.2.1), and mbedTLS (version 2.16.2) as the test targets. Each test target obtains code coverage information, access paths, and output information through SanitizerCoverage and AdressSanitizer.

[0154] Experimental evaluation metrics: To evaluate the effectiveness of the Diffcert method, the present invention adopts the following three evaluation metrics:

[0155] (1) Number of triggered verification differences. The number of certificate verification differences triggered within the same time. A certificate verification difference indicates that the target program under test produces non-identical verification results for the same test case. That is, when a program under test rejects a test case, at least one other program under test accepts the same test case. This is one of the most intuitive indicators for judging the effectiveness of a differential fuzzing tool.

[0156] (2) Test case diversity. Test case diversity is defined as the ratio of the number of unique difference messages triggered to the number of test cases generated. For TransCert and DiffCert, the unique difference messages triggered can be represented by the number of vertices in the transition graph. For NeZha, the unique difference messages triggered can be represented by the total number of tuples of its three metrics. Since FrankenCert is a black-box test without guidance, the unique difference messages it triggers are represented as the unique coverage differences generated when all programs under test execute its test cases. The higher the test case diversity, the higher the quality of the generated test cases, which can cover more verification behaviors of the SSL / TLS protocol implementation and trigger more differences.

[0157] (3) Line coverage. The ratio of the number of lines of code in the program covered by the test cases generated by the fuzzer within the same time to the total number of lines of code. The higher the line coverage, the more lines of code in the protocol implementation under test can be covered by the test cases generated by the fuzzer. Therefore, it can directly reflect the ability of the fuzzer to explore the depth of the code.

[0158] Experimental environment settings: The system environment for all experiments is the Ubuntu 20.04.4 LTS operating system, with 4GB of memory and a 2-core Intel(R) Core(TM) i5-9300HF CPU@2.40GHz 2.40GHz.

[0159] Table 1 shows the comparison of the experimental results of the number of verification differences triggered by four differential fuzz testing tools after the same number of iterations. From the data in the table, it can be seen that after the same number of iterations, Diffcert triggered more verification differences. Specifically, the number of verification differences generated per iteration increased by 6.8% compared to Transcert, 30.3% compared to NEZHA, and 466.2% compared to Frankencert. This is because, compared with the mutation-based NEZHA and Frankencert, the mutation strategies based on generation used by Transcert and Diffcert are more capable of generating test cases that conform to the protocol specification format, thus passing the format check and reducing invalid tests. At the same time, due to the asymmetric test guidance of NEZHA's behavior, more differences can be found compared to Frankencert. In addition, since Diffcert is more capable of generating test cases with correct formats than Transcert, it is slightly superior to Transcert in triggering the number of verification differences. Through the following Figure 5 A more intuitive comparison can be made.

[0160] Table 1 Experimental comparison of the number of verification differences triggered by four testing tools after the same number of iterations

[0161]

[0162] Table 2 shows the comparison of the experimental results of the number of unique difference information generated by four testing tools under the same number of test cases. From the data in the table, it can be seen that when the number of test cases is small, NEZHA and Frankencert can trigger more unique difference information compared to Transcert and Diffcert. This may be because the random mutation strategies adopted by NEZHA and Frankencert can cover more verification behaviors in the SSL / TLS protocol implementation than the mutation strategies based on generation when the test scale is small. At the same time, since Diffcert uses more fine-grained difference information for test guidance, it can generate more difference information compared to Transcert that uses only single coverage information. Figure 6 It shows the variation of the test case diversity of the four testing tools with the number of test cases. It can be found that as the number of test cases increases, the ability of the four tools to discover new difference information gradually decreases, but the diversity of Diffcert decreases more slowly compared to the other three tools. In fact, after 22,000 test cases, the diversity of Diffcert has exceeded that of the other three tools, which indicates that as the test scale increases, Diffcert can cover more code and trigger more difference information than the other three tools.

[0163] Comparison of the number of unique difference information generated by four test tools under the same number of test cases

[0164]

[0165] Table 3 shows the comparison of the line coverage and branch coverage experimental results generated by four test tools in OpenSSL after 20,000 iterations. From the data in the table, it can be seen that although the differences among the four tools in covering the OpenSSL source code are not obvious, Diffcert is slightly better than the other three tools in line coverage and is second only to NEZHA in branch coverage. This is because both tools use a more fine-grained difference-guided approach, and the random mutation strategy of NEZHA may be able to cover some additional branch paths. As follows Figure 7 Shows the changes in line coverage of the four tools during iteration. It can be found that after about 1,700 iterations, Diffcert has covered more code than Frankencert and NEZHA, and exceeded transcert after 10,000 iterations.

[0166] Table 3 Comparison of coverage generated by four test tools in OpenSSL after 20,000 iterations

[0167]

[0168]

[0169] The above are only the preferred embodiments of the present invention. It should be pointed out that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. X.509 certificate verification differential test method based on differential information transfer graph, Features It includes four stages: difference information transfer graph construction stage, certificate generation stage and certificate verification stage; In the differential information transfer graph construction phase, the real DER format certificate chains captured from the Internet are collected as initial seeds and stored in the corpus, and the differential information transfer graph is constructed based on them. The differential information transfer graph is defined as a weighted directed graph. The difference information transfer graph construction phase specifically includes the following steps: 1-2-1. Seed collection; Collect DER real certificate chains from the Internet and store them in the seed certificate chain corpus as initial seeds; 1-2-2. Seed verification and difference information collection Send each unmutated seed certificate chain in the certificate corpus to all SSL / TLS protocol implementation programs to be tested for certificate verification, and collect all difference information generated after all SSL / TLS protocol implementation programs to be tested verify the certificate chain; 1-2-3. Construction of difference information transfer graph Determine whether the vertex corresponding to the difference information exists in the difference information transfer graph. If so, add the current seed certificate chain to the certificate chain pool corresponding to the vertex. Otherwise, create a new difference information vertex and add the certificate chain to its certificate chain pool. In the certificate generation phase, the difference information transfer graph is traversed, the difference information vertices and seed certificate chains are selected according to the potential value, and test cases are generated through fuzzification; During the certificate verification phase, the test cases are sent to all SSL / TLS protocol implementations to be tested, the verification differences found are recorded, and the corpus and difference information transfer graph are updated.

2. The X.509 certificate verification differential test method based on the difference information transfer graph according to claim 1, Features The difference information transfer graph in the difference information transfer graph construction phase is defined as follows; 1-1. The difference information transfer graph is defined as a weighted directed graph G = (V, E, W), where: (1) V represents a set of difference information vertices; each difference information vertex represents a unique difference information; in order to facilitate acquisition and comparison, each difference information vertex is identified by a unique hash value; each difference information vertex is associated with a certificate chain pool, which stores one or more certificate chains; the difference information that can be generated by each certificate chain in the certificate chain pool is the same as the difference information represented by the vertex; (2) E represents the set of difference information transfer edges. If the certificate chain corresponding to a difference information vertex u mutates into a test case of a difference information vertex v, a difference information transfer edge u→v is added, indicating the transfer from difference information vertex u to difference information vertex v. (3)W represents the set of edge weights; the weight of the difference information transfer edge u→v is defined by the following formula: Among them, Certs(u→v) represents the number of mutated certificate chains of difference information vertex v obtained by mutating the certificate chain in difference information vertex u, and Certs(v) represents the number of all certificate chains in the certificate chain pool corresponding to difference information vertex v.

3. The X.509 certificate verification differential testing method based on the differential information transfer graph according to claim 2, characterized in that the differential information described in step 1-2-2 refers to the differences between the responses, coverage rates, and path information generated by all protocol implementation programs when performing certificate verification on the same test case, and is quantified by the following three indicators: (1) Response Index: Given a set of target protocol implementers SUT = [sut_1, sut_2, …, sut_n] and a test case Certs, each target protocol implementer sut_i in the SUT set will produce a verification result r after performing certificate verification on the test case Certs. sut_i ; The tuple <r sut_1 , r sut_2 , …, r sut_n > composed of the above verification results is defined as a response index of the test case Certs. (2) Coverage metric: Given a set of target protocol implementation programs SUT = [sut 1 , sut_2, …, sut_n] and a test case Certs, collect the coverage information cov sut_i when each target protocol implementation program sut_i in SUT performs certificate verification on the test case Certs; define the tuple <cov sut_1 , cov sut_2 , …, cov sut_n > composed of the above coverages as a coverage metric for the test case Certs; (3) Path metric: Given a set of target protocol implementers SUT = [sut_1, sut_2, …, sut_n] and a test case Certs, collect the path information path of the control flow graph accessed by each target protocol implementer sut_i in SUT when performing certificate verification on the test case Certs. sut_i ; Define the tuple <|path sut_1 |, |path sut_2 |, …, |path sut_n |> composed of the number of edges of the above paths as a path metric of the test case Certs. The difference information generated by testing a set of target protocol implementation programs SUT with a test case is represented by response metrics, coverage metric rates, and path metrics as: {<r sut_1 ,r sut_2 ,…,r sut_n >,<cov sut_1 ,cov sut_2 ,…,cov sut_n >,<|path sut_1 |,|path sut_2 |,…,|path sut_n |>}, and the response metrics, coverage metric rates, and path metric contents are converted into a hash value through a hash operation to uniquely represent a difference information vertex.

4. The X.509 certificate verification differential testing method based on the differential information transfer graph according to claim 3, characterized in that the certificate generates test cases by traversing the differential information transfer graph during the generation phase, selecting differential information vertices and seed certificate chains according to the potential value, and generating test cases through fuzzification. The specific implementation is as follows: 2-1. Selection of differential information vertices Vertex selection is performed by assigning a potential value to each vertex in the differential information transfer graph; the higher the potential value of a vertex, the more likely it is to be selected for testing; specifically, the calculation formula for the potential value is as follows: where trans_num(v) represents the number of transfer paths starting from the differential information vertex v, p(trans(v)) represents the weight of each transfer path starting from the differential information vertex v, and choose(v) represents the number of times the differential information vertex v is selected for testing; From the formula, the potential value of each differential information vertex depends on the number of transfer paths starting from this differential information vertex, the sum of the weights of each transfer path starting from this differential information vertex, and the proportion of this differential information vertex being selected; therefore, it is found that if a differential information vertex has more differential information transfer paths and fewer times of being selected for testing, then the potential value of this differential information vertex is higher and it is more likely to be selected for testing; Normalize the potential values of all differential information vertices to a probability distribution in [0,1], and select differential information vertices by randomly selecting numbers in [0,1]; 2-2. Selection of certificate chains Since the differential information generated by all certificate chains in the certificate chain pool corresponding to each differential information vertex is the same, after selecting the differential information vertex, randomly select a certificate chain in its corresponding certificate chain pool as a seed for testing; 2-3. Generation of test cases After selecting the certificate chain as a seed, it is necessary to mutate it to generate test cases; first, mutate the certificate chain, then randomly select multiple certificates on the certificate chain, convert each selected certificate into a tree structure, then mutate the leaf nodes of the tree structure, and then perform a bottom-up repair on the mutated certificate through backtracking repair to ensure that the mutated certificate still conforms to the DER structure.

5. The X.509 certificate verification differential testing method based on the differential information transfer graph according to claim 4, characterized in that the specific operation of generating test cases in step 2-3 is as follows: 2-3-1 Mutation of certificate chains In the certificate chain mutation phase, only the chain-level mutation is performed on the certificate chain, and the certificate fields are not mutated. Specific certificate chain mutation operations: a random number of mutation operations within the range of [1, Q] are performed on the certificate chain, including inserting a certificate in the certificate chain, deleting a certificate, and replacing a certificate in the certificate chain with another certificate. When a certificate chain is mutated, the issuer and subject fields of the certificates on the certificate chain must also be updated to ensure that each certificate is issued by a subsequent certificate. 2-3-2 Certificate Field Mutation The certificate field mutation phase mainly mutates a single certificate in the certificate chain; First, randomly select one or more certificates in the certificate chain and add them to the queue of certificates to be mutated. Then, parse each certificate in the queue into a tree structure. After that, perform a depth-first traversal of the tree structure and randomly select leaf nodes for mutation. The mutation strategy includes modifying the signature algorithm or field of the certificate, deleting the certificate field, and adding the certificate field. 2-3-3 Backtracking Repair In the backtracking repair phase, the DER structure damaged by the mutation is repaired recursively from the mutated leaf nodes from the bottom up. The specific repair steps are as follows: (1) Determine whether the current mutation operation changes the length of the Value field. If not, it means that the mutation will not damage the DER structure and the repair is complete. Otherwise, proceed to the next step. (2) Correct the Length field of the current leaf node so that the Length value is consistent with the length of the Value field of the current node; (3) Recursively correct the Length field of the parent node of the current node to make the Length value consistent with the length of the Value field of the current node, until the current node is the root node, and the repair is complete; (4) Restore the repaired certificate from the tree structure to the mutated certificate in DER format.

6. The X.509 certificate verification differential test method based on the differential information transfer graph according to claim 5, Features In the certificate verification phase, the mutated certificate chain is sent as a test case to all SSL / TLS protocol implementations to be tested, the difference information generated after they execute the test case is collected, and the difference information transfer graph is updated; the specific implementation is as follows: 4.1 Test case verification and difference information collection Send the mutated certificate chain as a test case to all SSL / TLS protocol implementations to be tested, and collect the difference information generated after they execute the test case I; Then add the current test case to the certificate corpus for saving; 4.2 Update of difference information transfer diagram Traverse the entire difference information transfer graph in a breadth-first manner to determine whether there is a difference information vertex that is the same as the difference information I generated by this test; If the same difference information vertex is found, the test case is synchronized to the certificate chain pool corresponding to the difference information vertex; If not found, the difference information generated in this round of testing is updated as a new difference information vertex in the difference information transfer graph. The current test case is synchronized to the certificate chain pool corresponding to this new difference information vertex, and a transfer path is added from the difference information vertex of the original certificate to this new difference information vertex. Finally, update the edge weights of the difference information transfer graph.

7. An X.509 certificate verification differential testing system based on a difference information transfer graph characterized in that it includes: Certificate collection module: The fuzz testing system uses the Zmap tool to perform network scanning on port 443 and establish an SSL connection, and obtains a corpus containing 1005 DER certificates provided by Frankencert developers as the initial certificate seeds. Difference information transfer graph construction module: The fuzz testing system uses the script file exec.sh provided by transcert to sequentially send the seed certificate chains in the certificate corpus to all SSL / TLS protocol implementation programs to be tested, and collects all the difference information generated after verification; uses an improved seed_init method to construct an initial difference information transfer graph, which will judge whether each difference information exists in the difference information transfer graph. If it exists, the current seed certificate chain is added to the certificate chain pool corresponding to this difference information vertex. Otherwise, a new difference information vertex is created and the certificate chain is added to its certificate chain pool. Test case generation module: The fuzz testing system traverses the difference information transfer graph, selects difference information vertices and seed certificate chains according to the potential value, generates test cases through fuzzification, and sends them to all SSL / TLS protocols to be tested for certificate verification. Difference information recording module: The fuzz testing system collects the difference information generated after all SSL / TLS protocol implementation programs to be tested verify the test cases, and adds the current test case to the certificate corpus for storage; then, traverses the entire difference information transfer graph, adds the current test case to the certificate chain pool corresponding to the vertex in the graph, and updates the weights of the edges in the graph.

Images