Database fuzzy testing method based on coverage rate gradient rollback optimization

Abstract

The invention discloses a database testing method and system. By performing fuzzy testing on the database, SQL statements are continuously input, and the change of the coverage rate is observed. And recording the rising gradient of the coverage rate, and when the SQL continuously varies but the coverage rate is kept stable, indicating that the variation of the SQL cannot further go deep into code logic. And at the moment, returning to the statement with the highest rising gradient of the coverage rate and the current database state, restarting variation, and continuing to try to explore a higher code coverage rate. According to the method, the database testing efficiency and the code coverage rate can be effectively improved, and a powerful guarantee is provided for the stability and reliability of the database.

Description

Technical Field

[0001] The present invention relates to the technical field of database testing, and in particular to a database fuzzy testing method capable of improving testing efficiency and code coverage. Technical Background

[0002] In today's digital age, databases, as core components of information storage and management, are widely used in various fields. From business operations of enterprises to data processing of scientific research institutions, from Internet services to financial transaction systems, their importance is self-evident.

[0003] SQL (Structured Query Language) is a standard language for managing relational databases. It allows users to perform various operations, such as querying data, inserting new records, updating existing data, and deleting unnecessary data. A database is a warehouse that organizes, stores, and manages data according to data structures. Relational databases store data in tables. The rows in the table represent records, and the columns represent attributes. These data can be easily operated and managed through SQL statements. Although different database systems (such as MySQL, Oracle, SQL Server, etc.) differ in function and performance, they all follow the SQL standard to implement data processing.

[0004] Fuzz testing is a software testing technique that discovers potential vulnerabilities and errors by inputting a large amount of random or semi-random data into the target system or application. The basic principle is to use automated tools to generate various possible input data and then observe the target system's response to these inputs. If the target system exhibits abnormal behavior, such as crashes, incorrect outputs, or produces unexpected results, it may indicate the existence of security vulnerabilities or functional defects. In database testing, fuzz testing can help discover stability and reliability issues of the database when processing various complex and abnormal data.

[0005] In software testing, coverage is a key indicator to measure test completeness. For database testing, coverage usually refers to the degree of execution coverage of database code logic by test cases. Specifically, it includes statement coverage, branch coverage, condition coverage and other aspects. High code coverage means that test cases can touch more parts of the program and are more likely to discover potential vulnerabilities and errors.

[0006] As database application scenarios become increasingly complex and diverse, databases face various potential risks and challenges. On the one hand, databases need to process data from different sources and in different formats, which may contain some data that does not conform to conventional norms or expectations. For example, in network applications, the data entered by users may have problems such as format errors and malicious injections. If the database cannot properly handle these abnormal data, it may lead to serious consequences such as data leakage and system crashes. On the other hand, there may be some hidden vulnerabilities in the code logic of the database. These vulnerabilities may be difficult to detect in normal usage scenarios, but may be exposed when faced with special or abnormal data. Through fuzz testing, various complex and abnormal situations can be simulated to comprehensively test the code logic of the database, thereby improving the quality and security of the database.

[0007] Improving coverage when fuzz testing a database means that the test cases can more fully execute the code paths in the database, making it more likely to discover potential problems. The benefits of improving coverage are multifaceted. First, it can detect errors and defects in the database more comprehensively, reducing the risk of failures in actual applications. For example, if a code branch is not covered in the test, then the errors that may exist in the branch cannot be discovered, and if the corresponding situation is encountered during actual operation, it may cause the system to be abnormal. Secondly, high coverage can enhance confidence in the stability and reliability of the database. When the coverage reaches a high level, it means that the operation of the database in various situations has been fully tested and verified, and it can better cope with various scenarios and requirements in actual use. Finally, improving coverage helps optimize the performance of the database. Through comprehensive testing, the performance bottlenecks of the database in certain situations can be discovered, so that targeted optimization and improvement can be carried out to improve the overall operation efficiency of the database.

[0008] There are many methods for fuzz testing of database systems. For example, Squirrel determines the direction of mutation based on coverage guidance, converts and abstracts the original SQL statements into intermediate representations, so that mutation operations can be performed in a more structured and standardized form, improving the accuracy of mutation-generated SQL statements, thereby improving the effectiveness of the test; Griffin, as a mutation-based syntax-free database fuzz testing tool, summarizes the state of the database system as a metadata graph, which is a lightweight data structure used to describe the dependency relationship between metadata and statements. Based on these graphs, it rearranges statements and uses metadata-guided replacement to correct semantic errors, which can be adapted to more database systems; SQLRight is improved on the basis of Squirrel, combining coverage-based guidance, validity-oriented mutations, and oracles to focus on logical error detection. These methods have enhanced the effectiveness of database fuzz testing to a certain extent. However, they often get into trouble after continuously mutating SQL statements for a period of time. Specifically, as the mutation proceeds, it is inevitable to get stuck in a bottleneck in a certain branch or to have explored the entire branch but find it difficult to jump to a new branch, which shows that the coverage cannot be further improved, limiting more comprehensive and in-depth vulnerability detection and security assessment of database systems. In order to explore new code branches, it is often necessary to restart the testing process, which also results in a waste of time and resources.

[0009] The database testing method proposed in the present invention improves the shortcomings of traditional methods. During the test process, SQL statements are input into the database based on fuzzy testing and coverage changes are paid attention to. However, when SQL continues to mutate and the coverage remains stable, the method does not simply stop and restart, but falls back to the statement with the highest coverage gradient and the database state at that time. There are two important advantages in doing so: on the one hand, the previous mutation process is fully utilized, rather than completely abandoning the previous work and starting over, reducing the waste of time and resources. On the other hand, because the point with the highest coverage gradient often means that further mutation on this basis is more likely to explore new code branches, discover new problems and paths, and fall back from the place where the coverage gradient is the largest to continue testing, increasing the possibility of exploring deeper code logic. Compared with the blind restart of the traditional method, the present method is more targeted and efficient, and can more effectively improve the test efficiency and code coverage, thereby providing a stronger guarantee for the stability and reliability of the database. Summary of the invention

[0010] The present invention application provides a database fuzz testing method using coverage gradient for fallback optimization, which is used to improve the code coverage of fuzz testing on the database and provide a stronger guarantee for the stability and reliability of the database. The specific technical solution is as follows:

[0011] 1. Test preparation stage

[0012] In the preparation stage, the database system needs to be compiled with stub tags. After the compilation is completed, some basic SQL statements are prepared as seed statements for the test, and the fuzz test engine is connected to the database system to prepare to start the test.

[0013] (1) Perform instrumentation compilation on the database;

[0014] Stub compilation is a technique that inserts additional code during the program compilation process. These stub codes can be used for a variety of purposes, such as collecting information when the program is running, including but not limited to the number of function calls, changes in variable values, program execution paths, etc. During the compilation phase, the compiler inserts stub codes at specific locations in the original program code according to predefined rules and requirements. These locations can be function entry points, exit points, key branch statement locations, etc. For database systems, stub compilation can insert code into database operation-related code (such as SQL statement execution functions, data storage and reading functions, etc.). These stub codes can record which code blocks are executed and which are not executed when the program is running. For fuzz testing, when running fuzz test cases, the stub-inserted program can collect information about the code execution path to determine code coverage.

[0015] (2) Prepare test seed SQL;

[0016] Prepare some SQL statements related to insertion, query, deletion, etc. in advance and use them as seed SQL for a fuzz test. The fuzz test system will mutate based on this.

[0017] The characteristics and structure of the seed will affect the direction and results of the mutation, thus guiding the test in a more meaningful and more likely direction to discover vulnerabilities. By selecting a specific seed, the functional modules related to the seed in the target system can be tested in depth.

[0018] (3) Establish database connection;

[0019] Connect the fuzz testing engine to the database to ensure that the fuzz testing system can communicate and interact with the target database. Prepare to start fuzz testing.

[0020] 2. Statement mutation testing phase

[0021] After the test starts, the fuzz testing engine will begin to mutate the seed SQL statements provided in advance, input the mutated SQL statements into the database system, detect their response and coverage, and guide the direction of statement mutation based on this information, continuously mutating the statements and inputting them into the database system.

[0022] (1) Sentence variation;

[0023] The mutations of fuzz testing include but are not limited to numerical values, character conditions, connection conditions, sorting rules, grouping conditions, etc. During the test, it is necessary to additionally record the properties of tables and columns that may be created by the mutated statements, to ensure the grammatical and semantic correctness of the generated SQL statements as much as possible, to minimize the generation of invalid statements, and to improve the efficiency of the test.

[0024] (2) Input and collect response data;

[0025] When the mutated SQL statements are input into the database system, the fuzz testing system will closely detect the database's response to these SQL statements and information such as code coverage. The database's response may include normal execution and return of result sets, return of syntax error prompts, runtime errors, or abnormal situations.

[0026] 3. Code coverage gradient recording stage

[0027] During the testing process, it is necessary to always record the code coverage gradient, the corresponding database status and other information, and determine whether to enter the rollback phase based on this information.

[0028] (1) Code coverage gradient;

[0029] As you continue to input various SQL statements into the database, the code coverage will change dynamically. When you find that the code coverage is rising, you should accurately record the gradient of the coverage increase. The gradient is calculated by comparing the difference between the current coverage and the previous coverage.

[0030] (2) Database status;

[0031] While recording the coverage increase gradient, you must also record the database status at that time. The database status includes but is not limited to the data content of each table in the current database, index structure, execution status of stored procedures and triggers, etc. Detailed records of database status can provide status guidance for subsequent rollback.

[0032] (3) Determine whether to roll back;

[0033] When continuously monitoring the increase in code coverage during the test, it is detected that the code coverage remains stable or does not increase significantly within a period of time or a test process. This means that under the current test statements, it is difficult to further trigger new code branches to explore deeper code logic, determine whether the rollback conditions are met, and start entering the rollback phase.

[0034] 4. Fallback phase

[0035] During the test process, when it is detected that the coverage remains stable and meets the rollback conditions, the fuzzy testing system suspends the test and enters the rollback phase; locates the record point with the highest coverage gradient and rolls back the database status; when the database status rollback is completed, returns to the statement mutation test phase and restarts the test from the corresponding SQL statement.

[0036] (1) Database status rollback;

[0037] The rolled-back database state covers many important aspects, including the data content, data structure, and transaction status in the database. For data content, the data of each table in the database needs to be restored to the state at the recording point to ensure that subsequent tests can be performed on the same data basis. For data structure, the table structure, indexes, constraints, etc. must be restored to the state at the recording point to ensure the structural consistency of the database.

[0038] (2) SQL statement rollback;

[0039] Roll back the SQL statement being tested to the SQL statement recorded at the rollback point. Subsequent fuzz testing will restart based on this. On the one hand, the SQL statement at this recorded point represents that during the test process, the statement mutated by this SQL has entered a deeper level of code logic, indicating that this SQL is at the critical point of the code branch. Mutating this SQL in other directions is more likely to explore new code branches, thereby continuing testing, improving test code coverage, and increasing the ability to troubleshoot database problems. On the other hand, through such a rollback operation, the mutation logic of the original part is reused, and there is no need to restart the test from the seed statement, which can avoid wasting time and resources on invalid test paths, and instead restart statement mutation and testing from a more promising starting point.

[0040] Based on the above method, you can make full use of the previous mutation process instead of abandoning the previous work and starting over. In addition, the point with the highest coverage gradient often means that further mutation on this basis is more likely to go deep into the code logic, discover new problems and paths, and fall back to this point to continue testing, which increases the possibility of exploring deeper code logic, thereby making it more likely to test potential database problems and provide support for the stability and reliability of the database system. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 The present invention is a flowchart of a database fuzz testing method that uses coverage gradient for fallback optimization. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0043] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0044] A best embodiment is given below:

[0045] Reference Figure 1 As shown, it is a flow chart of a database fuzzy testing method for fallback optimization using coverage gradient provided in an embodiment of the present application, including:

[0046] (1) Perform stub compilation on the database to obtain the database under test;

[0047] For example, use afl-g++ in AFL++ as an instrumentation compilation tool to perform instrumentation compilation on the database. AFL++ has made many improvements and optimizations based on the famous fuzz testing tool AFL. It has a higher detection rate and faster execution speed, and can more effectively discover vulnerabilities in software. This tool inserts code for collecting execution information at key locations of the target program before executing compilation. Each time the program executes to the inserted code location, it records relevant execution information, such as the number of the current basic block. Based on this information, the corresponding code coverage data is calculated during the test.

[0048] (2) Start the test process, continuously mutate the SQL and input it;

[0049] Based on the given SQL statement seed, use the afl-fuzz tool to start fuzz testing, generate mutated SQL statements as test cases and input them into the database system, continue this process and collect corresponding data.

[0050] (3) Record code coverage gradient;

[0051] During the test, the code coverage rate reported by the fuzz test engine is continuously detected, and the coverage rate gradient is calculated within a period of time or within the test number range. The status of the database system and the test SQL statement when the gradient is the largest are recorded.

[0052] For example, during the test, SQL statements of test input are continuously added to the record array. The fuzzy test system reads the coverage data once a minute and calculates the coverage difference as the coverage gradient. Compared with the previous maximum coverage gradient, the calculation formula is: maxGradient=max(maxGradient,newCoverage–oldCoverage). If the maximum gradient value changes, it means that a more representative variation is detected. This coverage gradient and the subscript of the corresponding starting point SQL statement in the record array are recorded.

[0053] When the rollback condition is determined to be met, the SQL statement before this subscript is used to restore the database state, and the SQL statement corresponding to the subscript is used as the initial test case for a new round of testing.

[0054] (4) Go back to the location where the code coverage gradient is the largest and continue testing;

[0055] When the rollback conditions are met, the fuzzy testing system starts the rollback operation. According to the database status information saved in the test process, it rolls back to the position with the largest coverage gradient, and uses the test SQL statement at this position to restart the test.

[0056] For example, when using the method of saving and executing SQL to restore the database, the rollback operation will first suspend the test process, reinitialize a database system, and then read the record array and input all the saved SQL statements one by one until the SQL statement corresponding to the maximum position of the coverage gradient is reached. At this time, the rollback and restoration of the database state is completed. The corresponding SQL statement is used as the seed SQL statement for the new round of testing and the test is restarted.

[0057] After restarting the test process, some of the previous test data, such as coverage information, will be reused, and the coverage gradient will continue to be recorded. When the rollback conditions are met again, a new round of rollback operations will be started and retested. This will continue until a problem occurs in the test database system, or the test time and other dimensional data reach the required threshold.

[0058] The algorithm flow of the database fuzz testing method using coverage gradient for fallback optimization:

[0059] 1. Algorithm Description

[0060] Algorithm input: oldSql;

[0061] Algorithm output: status;

[0062] Note: oldSql is the seed SQL, and status is the related test result.

[0063] 2. Algorithm steps

[0064] 1) Initialize the SQL array to store all executed SQL statements for rollback or backtracking after an error occurs;

[0065] 2) Start testing until the time or coverage threshold is reached;

[0066] 3) Mutate and execute SQL to obtain code coverage;

[0067] 4) Update the state according to the coverage gradient;

[0068] 5) When the rollback conditions are met, execute rollback.

[0069] The pseudo code is as follows:

[0070]

[0071] Obviously, it is understandable to those skilled in the art that various changes, modifications, substitutions and variations may be made to these embodiments without departing from the principle and spirit of the invention, and the present application is intended to include these changes and variations.

Claims

1. A database fuzzy testing method based on coverage gradient backoff optimization, characterized in that: include: Input SQL statements into the database based on fuzz testing, monitor coverage changes, and record coverage gradients and corresponding status information; When the rollback conditions are met, the database status and test cases are rolled back to the specified location, and the fuzz test is restarted, thereby reusing the original test data, being able to explore deeper code logic, and further improving the code coverage of the database fuzz test.

2. The method according to claim 1, characterized in that The monitoring of coverage changes and recording of coverage gradients and corresponding status information include: During fuzz testing, the code coverage of the target database affected by different test cases is read, the coverage growth is calculated at the same interval, and the maximum value of the coverage gradient and the corresponding status information are updated.

3. The method according to claim 3, characterized in that: The maximum value of the updated coverage gradient includes: The coverage gradient is the growth value of the code coverage brought by the test case at the same time interval. The update calculation formula for the maximum value of the coverage gradient is maxGradient=max(maxGradient,newCoverage-oldCoverage).

4. The method according to claim 3, characterized in that The corresponding status information includes: The status information includes database status and test cases. The database status includes but is not limited to the data content of each table in the current database, index structure, execution status of stored procedures and triggers, etc. The test cases are all executed SQL statements. The corresponding status information is the database status when the SQL statement with the largest coverage gradient is executed and all SQL statements executed before this statement.

5. The method according to claim 1, characterized in that The fallback conditions include: When the number of executions of a period of time or some test cases during the fuzz test reaches the threshold, and the coverage rate does not increase or the increase is less than expected, it means that the mutation of the current SQL statement cannot further deepen the code logic and improve the code coverage rate, and the possibility of detecting code problems is low, and a rollback operation is required, that is, the rollback condition is met.

6. The method according to claim 1, characterized in that The database status and test cases are rolled back to the specified location, including: The specified position is the database state corresponding to the maximum value of the recorded coverage gradient and the SQL statement that has been executed. The rollback operation is to initialize a new database system and re-execute the recorded SQL statements until the SQL statement corresponding to the maximum value of the coverage gradient is reached, thus completing the rollback of the database state.

7. The method according to claim 1, characterized in that The restarting of the fuzz test comprises: In the database state after rollback, mutation starts based on the test cases after rollback, and the fuzzy testing process continues until an error occurs in the database or the test time reaches a threshold.

Images