Code auditing method and tool for large-scale project code library

Abstract

The invention is suitable for the field of code auditing, and discloses a large-scale project code library code auditing method and tool, and the method comprises the steps: screening out a first vulnerability rule used for auditing operation from a preset basic vulnerability rule library, and screening out a second vulnerability rule used for auditing operation from a preset user-defined vulnerability rule library; combining the screened first vulnerability rule and the second vulnerability rule into an audit rule set; searching codes in the large-scale project code library by using the auditing rule set to obtain potential vulnerability codes; the potential vulnerability codes are compared with audited codes in a preset auditing historical database, if the potential vulnerability codes are repeated with the audited codes, the potential vulnerability codes are skipped, and if the potential vulnerability codes are not repeated with the audited codes, the potential vulnerability codes are marked as to-be-audited codes; and after the audit operation of the to-be-audited code is completed, the audit result is stored in the audit historical database, so that the audit efficiency and the audit depth can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of code auditing, and in particular to a code auditing method and tool for a large-scale project code base. Background Art

[0002] As the scale of software projects continues to expand, the security and stability of the code have become important issues that must be paid attention to during the development process. In the code base of large projects, the number of lines of code has exceeded one million or even tens of millions. On the other hand, the continuous iteration and update of the code makes the vulnerability detection and audit of the code particularly complicated. Code audits are usually divided into two categories: manual audits and automated tool audits. Manual code audits are conducted by security experts or developers to check the code line by line to find possible security vulnerabilities or logical defects. When faced with a large-scale code base, manual audits become particularly difficult and are prone to missing details. Automated tool audits use technical means such as static code analysis and dynamic code analysis to review the code through predefined rules and algorithms. At this stage, although automated tools can cover a large number of common vulnerabilities, false positives and missed reports are still a problem in complex business logic audits. Moreover, as the scale of software projects continues to expand, how to maintain audit depth while improving efficiency is an urgent problem to be solved in the industry.

[0003] Therefore, the existing technology still needs to be improved and developed. Summary of the invention

[0004] The first object of the present invention is to provide a code auditing method for a large project code base, which can effectively improve the auditing efficiency and auditing depth.

[0005] To achieve the above object, the solution provided by the present invention is: A code auditing method for a large project code base, the code auditing method for a large project code base comprising: screening out a first vulnerability rule for an audit operation from a preset basic vulnerability rule base, and screening out a second vulnerability rule for an audit operation from a preset custom vulnerability rule base; merging the screened out first vulnerability rule and second vulnerability rule into an audit rule set; using the audit rule set to search for codes in the large project code base to obtain potential vulnerability codes; comparing the potential vulnerability codes with audited codes in a preset audit history database, if the potential vulnerability codes are repeated with the audited codes, skipping the potential vulnerability codes, if the potential vulnerability codes are not repeated with the audited codes, marking the potential vulnerability codes as codes to be audited; when the audit operation of the codes to be audited is completed, saving the audit results to the audit history database.

[0006] Preferably, the code audit method for a large project code base also includes: obtaining new vulnerability information, the vulnerability information including vulnerability type, vulnerability mode, and vulnerability call interface; summarizing and organizing the obtained vulnerability information and forming a structured rule file; and updating the first vulnerability rule in a preset basic vulnerability rule base based on the structured rule file.

[0007] Preferably, the potential vulnerability code is compared with the audited code in a preset audit history database. If the potential vulnerability code is repeated with the audited code, the potential vulnerability code is skipped; if the potential vulnerability code is not repeated with the audited code, the potential vulnerability code is marked as code to be audited. The process also includes: obtaining a regular expression for performing an audit job, and using the obtained regular expression to search for codes in a large project code base to obtain the potential vulnerability code.

[0008] Preferably, the audit result includes the audited code and the vulnerability type, code line number, first vulnerability rule, second vulnerability rule and version information of the code file corresponding to the audited code.

[0009] The second object of the present invention is to provide a code audit tool for a large project code base, which is used to implement the code audit method for a large project code base as described above. The code audit tool for a large project code base includes an audit history database, a basic vulnerability rule base, a custom vulnerability rule base and a management module. The audit history database, the basic vulnerability rule base and the custom vulnerability rule base are respectively connected to the management module. The audit history database stores audited code and corresponding audit results. The basic vulnerability rule base stores a number of first vulnerability rules. The custom vulnerability rule base stores a number of second vulnerability rules. The management module includes a custom rule management unit, and the custom rule management unit is used to add the second vulnerability rule to the custom vulnerability rule base.

[0010] Preferably, the custom rule management unit is also used to view, modify or delete the second vulnerability rule in the custom vulnerability rule library.

[0011] Preferably, the custom rule management unit is further used to export the second vulnerability rule in the custom vulnerability rule library into a file in a preset format, and to import the vulnerability rule of a preset project as the second vulnerability rule.

[0012] Preferably, the management module further includes an updating unit, and the updating unit is used to obtain new vulnerability information and update the first vulnerability rule in the basic vulnerability rule library according to the obtained vulnerability information.

[0013] In this solution, the first vulnerability rules for auditing are selected from the preset basic vulnerability rule library. These rules are usually based on extensive security research and experience accumulation, covering common types of security vulnerabilities. At the same time, the second vulnerability rules for auditing are selected from the preset custom vulnerability rule library. These rules can be customized according to the needs, business logic or security policies of specific projects, thereby enhancing the pertinence and accuracy of the audit. By integrating these two rules, a comprehensive and flexible audit rule set can be built, which can accurately and quickly identify potential vulnerability codes; at the same time, by comparing the potential vulnerability code with the audited code, it is possible to quickly determine which codes have been audited and the audit results recorded, effectively avoiding repeated audits, improving audit efficiency, and ensuring that all newly discovered potential vulnerabilities can be audited and recorded in a timely manner, enhancing the security and maintainability of the code base. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0015] Figure 1 It is a flow chart of a code audit method for a large project code base provided by an embodiment of the present invention; Figure 2 It is a structural diagram of a code audit tool for a large project code base provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0016] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0017] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 In an embodiment of the present invention, a code audit method for a large project code base includes: Step S101, filtering out a first vulnerability rule for an audit operation from a preset basic vulnerability rule library, and filtering out a second vulnerability rule for an audit operation from a preset custom vulnerability rule library.

[0018] Step S102: merging the screened first vulnerability rule and the second vulnerability rule into an audit rule set.

[0019] Step S103, using the audit rule set to search the code in the large project code base to obtain potential vulnerability code.

[0020] Step S104, compare the potential vulnerability code with the audited code in the preset audit history database. If the potential vulnerability code is repeated with the audited code, the potential vulnerability code is skipped. If the potential vulnerability code is not repeated with the audited code, the potential vulnerability code is marked as a code to be audited.

[0021] Step S105: When the audit code completes the audit operation, the audit result is saved in the audit history database.

[0022] In this embodiment, in step S101, the preset basic vulnerability rule library stores the first vulnerability rule. The custom vulnerability rule library stores the second vulnerability rule. The first vulnerability rule can adapt to multiple programming languages ​​and cover common vulnerability types, including SQL injection, cross-site scripting attacks, file path traversal, etc. The second vulnerability rule is used to supplement the first vulnerability rule. The second vulnerability rule includes a global vulnerability matching rule and a vulnerability matching rule for a specific project. The global vulnerability matching rule is a general rule suitable for various projects and programming languages, and the vulnerability matching rule for a specific project is suitable for a specific project.

[0023] In this embodiment, according to the user's input, the first vulnerability rule and the second vulnerability rule for performing the audit operation are screened out from the first vulnerability rule and the second vulnerability rule.

[0024] In this embodiment, in step S102, rules with similar descriptions or the same triggering conditions are merged to avoid repeated audits. During the merging process, it is ensured that no duplicate rules exist. If duplicate rules are found, only one is retained and the other duplicates are deleted.

[0025] In this embodiment, in step S103, the audit rule set is used to search the code in the large project code base to obtain the potential vulnerability code, which specifically includes: traversing the code files in the large project code base, using the audit rule set to search and match each code file, and marking the potential vulnerability code that meets the rule conditions.

[0026] In this embodiment, in step S104, the potential vulnerability code is traversed, and for each potential vulnerability code, a search is performed in the audit history database to determine whether the same code already exists; if the same code is found and the audit result is known, the potential vulnerability code is skipped; if the same code is not found or the audit result is unknown, the potential vulnerability code is marked as a code to be audited.

[0027] In this embodiment, in step S105, the audit result includes the audited code and the vulnerability type, code line number, first vulnerability rule, second vulnerability rule and version information of the code file corresponding to the audited code.

[0028] In this embodiment, the code audit method for a large project code base also includes: obtaining new vulnerability information, the vulnerability information including vulnerability type, vulnerability mode, and vulnerability call interface; summarizing and organizing the obtained vulnerability information and forming a structured rule file; and updating the first vulnerability rule in a preset basic vulnerability rule base based on the structured rule file.

[0029] Specifically, the acquired vulnerability information is classified according to vulnerability type, severity, impact scope, etc. Then, the classified information is sorted, including vulnerability description, trigger conditions, attack examples, repair suggestions, etc. Next, the sorted information is converted into a structured rule file to facilitate the use of subsequent audit tools and the update of the rule base. The rule file should contain key information such as the unique identifier, description, trigger conditions, and repair suggestions of the vulnerability. Finally, the new structured rule file is imported into the preset basic vulnerability rule base to update or expand the existing first vulnerability rule.

[0030] In this embodiment, before comparing the potential vulnerability code with the audited code in the preset audit history database, if the potential vulnerability code is repeated with the audited code, the potential vulnerability code is skipped, and if the potential vulnerability code is not repeated with the audited code, the potential vulnerability code is marked as the code to be audited, and the following steps are also included: obtaining a regular expression for performing an audit operation, and using the obtained regular expression to search the code in the large project code base to obtain the potential vulnerability code. That is, in addition to using the filtered first vulnerability rule and the second vulnerability rule to search the code in the large project code base, the regular expression input by the user can also be obtained to search the code in the large project code base to obtain the potential vulnerability code.

[0031] In this embodiment, first vulnerability rules for auditing operations are screened out from a preset basic vulnerability rule library. These rules are usually based on extensive security research and experience accumulation, covering common types of security vulnerabilities. At the same time, second vulnerability rules for auditing operations are screened out from a preset custom vulnerability rule library. These rules can be customized according to the needs, business logic or security policies of a specific project, thereby enhancing the pertinence and accuracy of the audit. By integrating these two rules, a comprehensive and flexible audit rule set can be constructed, which can accurately and quickly identify potential vulnerability codes; at the same time, by comparing the potential vulnerability code with the audited code, it is possible to quickly determine which codes have been audited and the audit results recorded, effectively avoiding repeated audits, improving audit efficiency, and ensuring that all newly discovered potential vulnerabilities are audited and recorded in a timely manner, thereby enhancing the security and maintainability of the code base.

[0032] An embodiment of the present invention also provides a code audit tool for a large project code base, which is used to implement the code audit method for a large project code base as described above. The code audit tool for a large project code base includes an audit history database 201, a basic vulnerability rule base 202, a custom vulnerability rule base 203 and a management module 204. The audit history database 201, the basic vulnerability rule base 202, and the custom vulnerability rule base 203 are respectively connected to the management module 204. The audit history database 201 stores audited codes and corresponding audit results. The basic vulnerability rule base 202 stores a number of first vulnerability rules. The custom vulnerability rule base 203 stores a number of second vulnerability rules 203. The management module 204 includes a custom rule management unit 2041. The custom rule management unit 2041 is used to add second vulnerability rules to the custom vulnerability rule base 203.

[0033] In this embodiment, the first vulnerability rules are divided into multiple types according to the programming language, including but not limited to the first vulnerability rules based on the SQL language, the first vulnerability rules based on the HTML language, and the first vulnerability rules based on the Python language.

[0034] Similarly, the second vulnerability rules are divided into multiple types according to the programming language, including but not limited to the second vulnerability rules based on the SQL language, the second vulnerability rules based on the HTML language and the second vulnerability rules based on the Python language, so as to facilitate auditors to select the first vulnerability rules and the second vulnerability rules for auditing work.

[0035] For example, SQL injection attacks usually occur when an application splices user input directly into an SQL query without proper filtering or validation. This attack takes advantage of the way SQL statements are constructed, allowing malicious users to manipulate the behavior of database queries by entering specific characters or statements. Common SQL statement types, such as SELECT, INSERT INTO, UPDATE, etc., may all become targets of SQL injection attacks. In these statements, if the user's input is not properly processed, it may be interpreted as part of the SQL code, resulting in unauthorized data access or modification. The first vulnerability rule based on the SQL language is used for the identification of SQL statements and interpolation.

[0036] Exemplarily, XSS cross-site scripting attacks are common in both front-end and back-end codes. In front-end codes, XSS cross-site scripting attacks usually occur when user input is directly inserted into HTML pages without proper escaping or validation. Based on the first vulnerability rule of HTML language, it is used to identify statements such as innerHTML in Js, v-html in Vue framework, dangerouslySetInnerHTML in React framework, etc. In back-end codes, XSS cross-site scripting attacks may occur in the process of HTML template interpolation or direct generation of HTML code. If the back-end code does not properly filter or validate user input, malicious scripts may be injected into HTML pages. Based on the first vulnerability rule of HTML language, it is used to simultaneously identify the logic of HTML template interpolation and direct generation of HTML code in the back-end.

[0037] For example, in web applications, file path traversal attacks usually occur when user input is directly concatenated into the file path without proper filtering or validation. For example, in web frameworks, the backend often obtains user input variables from the URL path and query parameters, and performs file path concatenation operations. For example, in the process of obtaining static resources, Python's Flask web framework often obtains<path:file_path> Get the file path. If the file_path is not filtered and directly spliced ​​into the file path, it will cause serious arbitrary file read and write vulnerabilities. The first vulnerability rule based on Python language is used to check the variables that get user input from the URL path and query parameters in the Flask framework, and check whether these variables are directly spliced ​​into the file path.

[0038] In summary, proprietary vulnerabilities of different programming languages: Different programming languages ​​often have specific types of vulnerabilities due to their characteristics and usage. For example, Python's built-in tarfile.extractall() function will fully decompress compressed files by default, including compressed files with absolute paths and relative paths, which may lead to arbitrary file write vulnerabilities. Another example is Java's built-in serialization function readObject(), which may lead to arbitrary code execution vulnerabilities when deserializing the serialized stream provided by the user. The basic vulnerability rule library 202 can provide corresponding first vulnerability rules based on each language type and their unique vulnerabilities.

[0039] In this embodiment, the history database stores audited codes and corresponding audit results, including audited codes and vulnerability types, code line numbers, first vulnerability rules, second vulnerability rules, and version information of code files corresponding to the audited codes.

[0040] In this embodiment, the second vulnerability rule includes a global vulnerability matching rule and a specific project vulnerability matching rule. The global vulnerability matching rule is a general rule applicable to various projects and programming language designs, and the specific project vulnerability matching rule is applicable to a specific project. These two types of rules each have different characteristics and implementation methods.

[0041] The global vulnerability matching rules are general rules designed for various projects and programming languages ​​to detect and identify common security vulnerabilities. The global vulnerability matching rules are based on known security vulnerability types and are intended to improve the efficiency and accuracy of code auditing. The global vulnerability matching rules will supplement the vulnerability types not considered by the vulnerability matching rule library and expand the basic vulnerability rule library 202. For example, to check SQL injection vulnerabilities, it is necessary to define general matching rules applicable to most languages, rather than just considering the coding habits of a specific project.

[0042] Project-specific vulnerability matching rules are designed for the code features and potential vulnerabilities of a specific project. These rules are usually customized based on the specific needs of the project, programming language, code structure, and identified security vulnerabilities. For example, when auditing a large Ruby on Rails project code base, using the html_safe function can match potential XSS injection vulnerabilities, but vulnerability matching only through this function will result in a large number of false positive results. In addition, the project also uses rails' sprintf() function to interpolate ERB templates, which may be more likely to cause XSS cross-site scripting injection vulnerabilities in some cases. Therefore, in order to reduce false positive results, it is necessary to develop project-specific vulnerability matching rules that can distinguish between safe html_safe uses and potential XSS injection points. For example, using the sprintf().html_safe keyword can provide more precise vulnerability matching rules for the project.

[0043] In this embodiment, one of the core functions of the custom rule management unit 2041 is to allow the user to add a new second vulnerability rule to the custom vulnerability rule library 203. This usually involves defining key information such as the matching conditions, vulnerability types, risk levels, and repair suggestions of the rule. By adding rules, the user can customize more accurate and effective vulnerability matching rules based on the code characteristics and potential vulnerabilities of a specific project.

[0044] Further, the custom rule management unit 2041 is also used to view, modify or delete the second vulnerability rules in the custom vulnerability rule base 203. That is, for the viewing function, the user can browse all the second vulnerability rules in the custom vulnerability rule base 203 through this module to understand the detailed information of each rule, such as the rule name, description, matching conditions, etc. For the modification function, the user can modify the existing rules as needed. This may include updating the matching conditions, adjusting the risk level, or providing new repair suggestions. For the deletion function, if a rule is no longer applicable or has been replaced by other more precise rules, the user can choose to delete it from the custom vulnerability rule base 203 to keep the rule base neat and efficient.

[0045] Furthermore, in order to facilitate users to share rules between different projects or different teams, or to back up the rules to a safe location, the custom rule management unit 2041 is also used to export the second vulnerability rule in the custom vulnerability rule library 203 into a file in a preset format. The exported file format can be a text file, XML, JSON, etc., depending on user needs and system support.

[0046] In addition, the custom rule management unit 2041 is also used to import the vulnerability rules of the preset project as the second vulnerability rules. This allows the user to quickly apply the rules that have been verified to be effective in other projects to the current project, thereby improving the efficiency of vulnerability scanning and repair.

[0047] In this embodiment, the large-scale project code base code audit tool also includes an update unit 2042, that is, the large-scale project code base code audit tool establishes a dynamic update mechanism for the basic vulnerability rule base 202, which can regularly obtain the latest vulnerability information from the cloud background.

[0048] Specifically, the updating unit 2042 can obtain new vulnerability information from the cloud backend, and update the first vulnerability rule in the basic vulnerability rule library 202 according to the obtained vulnerability information.

[0049] The large-scale project code base code audit tool of the embodiment of the present invention integrates preset basic vulnerability rules and custom vulnerability rules, so as to accurately and quickly identify potential vulnerability codes; at the same time, the scheme uses the audit history database 201 to effectively avoid repeated audits, improve audit efficiency, and ensure that all newly discovered potential vulnerabilities can be audited and recorded in a timely manner, thereby enhancing the security and maintainability of the code base.

[0050] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A code audit method for a large project code base, characterized in that: The code audit method for large project code bases includes: Filter out a first vulnerability rule for the audit operation from a preset basic vulnerability rule library, and filter out a second vulnerability rule for the audit operation from a preset custom vulnerability rule library; The screened first vulnerability rule and the second vulnerability rule are combined into an audit rule set; Use the audit rule set to search the code in the code base of a large project to obtain potential vulnerability code; Compare the potential vulnerability code with the audited code in a preset audit history database, and if the potential vulnerability code is repeated with the audited code, skip the potential vulnerability code; if the potential vulnerability code is not repeated with the audited code, mark the potential vulnerability code as a code to be audited; When the auditing operation of the code to be audited is completed, the auditing result is saved in the auditing history database.

2. The code audit method for a large project code base according to claim 1 is characterized in that: Also includes: Obtain new vulnerability information, including vulnerability type, vulnerability mode, and vulnerability calling interface; Summarize and organize the acquired vulnerability information and form a structured rule file; The first vulnerability rule in the preset basic vulnerability rule library is updated based on the structural rule file.

3. The code audit method for a large project code base according to claim 1 is characterized in that: The method further comprises comparing the potential vulnerability code with the audited code in a preset audit history database, and if the potential vulnerability code is repeated with the audited code, skipping the potential vulnerability code; if the potential vulnerability code is not repeated with the audited code, marking the potential vulnerability code as a code to be audited. Obtain a regular expression for auditing, and use the obtained regular expression to search for codes in a large project code base to obtain potential vulnerability codes.

4. The code audit method for a large project code base according to claim 1 is characterized in that: The audit result includes the audited code and the vulnerability type, code line number, first vulnerability rule, second vulnerability rule and version information of the code file corresponding to the audited code.

5. A code audit tool for a large project code base, used to implement the code audit method for a large project code base as claimed in claim 1, characterized in that: The code audit tool for large-scale project code bases includes an audit history database, a basic vulnerability rule base, a custom vulnerability rule base and a management module. The audit history database, the basic vulnerability rule base and the custom vulnerability rule base are respectively connected to the management module. The audit history database stores audited codes and corresponding audit results. The basic vulnerability rule base stores a number of first vulnerability rules. The custom vulnerability rule base stores a number of second vulnerability rules. The management module includes a custom rule management unit, and the custom rule management unit is used to add the second vulnerability rules to the custom vulnerability rule base.

6. The large-scale project code base code audit tool according to claim 5, characterized in that: The custom rule management unit is also used to view, modify or delete the second vulnerability rule in the custom vulnerability rule library.

7. The large-scale project code base code audit tool according to claim 5, characterized in that: The custom rule management unit is also used to export the second vulnerability rule in the custom vulnerability rule library into a file in a preset format, and to import the vulnerability rule of a preset project as the second vulnerability rule.

8. The large-scale project code base code audit tool according to claim 5, characterized in that: The management module further includes an updating unit, which is used to obtain new vulnerability information and update the first vulnerability rule in the basic vulnerability rule library according to the obtained vulnerability information.

Images