Code security assessment method and system
By obtaining code from online code sharing web pages and ChatLLM platform, programming language classification and feature analysis, identifying and calculating security vulnerabilities, the problem of poor applicability of code security assessment in the existing technology is solved, and a more comprehensive and accurate security assessment is achieved.
Patent Information
- Application Number
- CN202411200219.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-29
- Publication Date
- 2024-12-27
AI Technical Summary
The prior art has problems with poor applicability and insufficient evaluation capabilities in code security evaluation, especially the inability to comprehensively evaluate the code security generated by multiple Chat LLMs, and lacks a security interface that reflects dynamic changes in code quality and is available to developers.
By obtaining multiple segments of code from online code sharing web pages and ChatLLM platform, performing programming language classification and feature analysis, selecting appropriate code analysis methods to identify security vulnerabilities, performing CWE classification and severity weight calculation, calculating the security score of each segment of code, providing development The user interface querys the security evaluation results.
Improve the applicability and accuracy of code evaluation, provide a more comprehensive and accurate security assessment, considers various attributes of the code, and enhances the security assessment capabilities of multiple Chat LLM codes.
Smart Images
Figure CN119203145A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of network security technology, and in particular to a code security assessment method and system thereof. Background Art
[0002] With the rapid development of artificial intelligence technology, large language model (LLM) generative artificial intelligence (AI) tools such as ChatGPT have demonstrated strong capabilities in the field of code generation. Taking ChatGPT as an example, according to the Sonatype 2023 Supply Chain Report, 97% of developers and security managers integrate generative AI into their software processes. At the same time, traditional programming question-and-answer platforms such as GitHub and StackOverflow are still important sources for programmers to obtain code solutions. However, whether it is dynamic and open LLM code generation or static code platform sources, the code snippets provided may have security vulnerabilities, posing a potential threat to the security of software development. Based on this, a conceptual analysis of specific LLM chat robots such as ChatGPT ("2024IEEE Security and Privacy Workshops") provides some representative differentiated vulnerability types, aiming to list some security vulnerabilities that have not been taken seriously on both platforms, so as to improve software developers' awareness of security vulnerabilities when selecting code snippets. The main method is to collect code samples from a specific common weakness enumeration CWE source and send the corresponding problems to ChatGPT to obtain code snippets, then use the static code analysis tool CodeQL to detect code security vulnerabilities, and calculate the frequency and proportion of fixed type vulnerabilities. Finally, provide a descriptive text document to remind software developers of the current analysis results. However, this method has the following disadvantages: limited scope, usually only for a specific code base or platform, lack of comprehensive evaluation staticity of multiple Chat LLMs; unable to reflect the dynamic changes in the quality of AI model-generated code; fails to provide developers with a directly available interface or benchmark to select the safest code reference source; the type comparison is single, and the code security vulnerabilities studied are limited to the Java programming language. Therefore, a more comprehensive and efficient code security assessment method is needed. Summary of the invention
[0003] In view of the deficiencies in the prior art, the present invention provides a code security assessment method and system thereof, which solves the problems of poor applicability and insufficient assessment capability of code assessment in the prior art.
[0004] According to an embodiment of the present invention, a code security assessment method is provided, comprising the following steps: S1, extracting multiple code segments from a code database, wherein the code database includes multiple code segments regularly obtained from an online code sharing webpage and a ChatLLM platform, each code segment is attached with a timestamp, and some of the multiple code segments have the same function; S2, performing programming language classification and feature analysis on the extracted code, wherein the feature analysis includes determining the degree of structuring of the code, determining whether the code contains dynamically generated code segments, and determining the complexity of the code; S3, according to the programming language used by the code and the result of the feature analysis, selecting a pair of codes from a preset code analysis method for analysis to identify security vulnerabilities in the code; S4, summarizing the identified security vulnerabilities, performing common weakness enumeration CWE classification, dividing the severity weight coefficient according to the type according to the CWE classification result, and calculating the frequency of occurrence of a specific CWE type; S5, calculating the security score of each code segment based on the severity weight coefficient, frequency of occurrence, timestamp, and total code quantity.
[0005] According to an embodiment of the present invention, the code obtained from the online code sharing webpage is the code obtained from the online code sharing webpage by using code cloning or a web crawler, wherein the obtained code also includes a corresponding question sequence.
[0006] According to an embodiment of the present invention, the code obtained from the ChatLLM platform is inputting the corresponding question sequence into the ChatLLM platform and copying the code result returned by the ChatLLM platform.
[0007] According to an embodiment of the present invention, the ChatLLM platform includes ChatGPT, Claude and Qwen.
[0008] According to an embodiment of the present invention, the preset code analysis method includes a static analysis method, a dynamic analysis method and a security vulnerability classification model.
[0009] According to an embodiment of the present invention, based on the programming language used by the code and the results of the feature analysis, selecting a pair of codes from preset code analysis methods for analysis includes the following steps: S31. When the degree of structuring of the code is greater than a first threshold, a static analysis method is adopted; S32. When the code contains dynamically generated code segments, a dynamic analysis method is adopted; S33. In other cases, a pre-trained security vulnerability classification model is used to predict the security vulnerabilities of the code.
[0010] According to an embodiment of the present invention, types of security vulnerabilities include buffer overflow, structured query language SQL injection, and input validation.
[0011] According to an embodiment of the present invention, the security score is:
[0012] score=100-Σ(CWE_severity*CWE_frequency*time_factor) / total_code_samples*100,
[0013] Among them, CWE_severity represents the severity of each CWE type, CWE_frequency represents the frequency of a specific CWE type in the code sample, time_factor is the time decay factor representing the newness of the code, and total_code_samples represents the total number of codes used for normalization.
[0014] According to an embodiment of the present invention, the code security assessment method further includes storing the security score in a database.
[0015] In a second aspect, according to another embodiment of the present invention, there is provided a code security assessment system, comprising: a code extraction module, configured to extract multiple code segments from a code database, wherein the code database comprises multiple code segments regularly obtained from an online code sharing webpage and a ChatLLM platform, each code segment is attached with a timestamp, and some of the multiple code segments have the same function; a security vulnerability analysis module, configured to classify the extracted code into programming languages and perform feature analysis, and select a pair of codes from a preset code analysis method for analysis based on the programming language used by the code and the result of the feature analysis to identify security vulnerabilities in the code, wherein the feature analysis module Point analysis includes determining the degree of code structuring, determining whether the code contains dynamically generated code segments, and determining the complexity of the code; the dynamic scoring module is configured to summarize the identified security vulnerabilities, perform common weakness enumeration CWE classification, divide the severity weight coefficient by type according to the CWE classification results, and calculate the frequency of occurrence of specific CWE types, as well as calculate the security score of each code segment based on the severity weight coefficient, frequency of occurrence, timestamp and total code quantity; the data storage and management module is used to store code, analysis results and security scores; the developer interface module is used for developers to query and compare the security scores of code snippets from different sources.
[0016] The technical principle of the present invention is as follows: multiple code segments are obtained from an online code sharing webpage, and AI-generated code is obtained from the ChatLLM platform according to the corresponding problems solved by the code, and then the code is analyzed according to the code characteristics to find security vulnerabilities, and then the identified security vulnerabilities are summarized and classified into common weakness enumeration CWE. According to the CWE classification results, the severity weight coefficient is divided according to the type, and the occurrence frequency of specific CWE types is calculated, and finally the score is calculated based on the above attributes.
[0017] Compared with the prior art, the present invention has the following beneficial effects: by regularly obtaining multiple code segments from online code sharing web pages and ChatLLM platforms, the applicability of code evaluation is made stronger; in addition, targeted analysis is performed according to the characteristics of the code, thereby enhancing the accuracy of code analysis; further, score calculation is performed based on various attributes of the code, making security assessment more comprehensive and accurate. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 The figure is a flowchart of a code evaluation method according to an embodiment of the present invention. DETAILED DESCRIPTION
[0019] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.
[0020] As described in the background technology, the existing technology has the problems of poor applicability and insufficient evaluation capability of code evaluation. Figure 1 As shown, an embodiment of the present invention proposes a code security assessment method, comprising the following steps: S1, extracting multiple code segments from a code database, wherein the code database includes multiple code segments regularly obtained from an online code sharing webpage and a ChatLLM platform, each code segment is attached with a timestamp, and some of the multiple code segments have the same function; S2, performing programming language classification and feature analysis on the extracted code, wherein the feature analysis includes determining the degree of structuring of the code, determining whether the code contains dynamically generated code segments, and determining the complexity of the code; S3, according to the programming language used by the code and the result of the feature analysis, selecting a pair of codes from a preset code analysis method for analysis to identify security vulnerabilities in the code; S4, summarizing the identified security vulnerabilities, performing common weakness enumeration CWE classification, dividing the severity weight coefficient according to the type according to the CWE classification result, and calculating the frequency of occurrence of a specific CWE type; S5, calculating the security score of each code segment based on the severity weight coefficient, frequency of occurrence, timestamp, and total code quantity. By regularly obtaining multiple pieces of code from online code sharing web pages and ChatLLM platforms, the applicability of code evaluation is made stronger. In addition, targeted analysis is performed based on the characteristics of the code, which enhances the accuracy of code analysis. Furthermore, score calculations are performed based on various attributes of the code, making security assessments more comprehensive and accurate.
[0021] According to an embodiment of the present invention, the code obtained from the online code sharing webpage is obtained from the online code sharing webpage using code cloning or web crawlers, wherein the obtained code also includes a corresponding question sequence. The online code sharing webpage includes StackOverflow or GitHub, etc.
[0022] According to an embodiment of the present invention, the code obtained from the ChatLLM platform is to input the corresponding problem sequence into the ChatLLM platform and copy the code result returned by the ChatLLM platform. In detail, the corresponding problem sequence can be sent to each ChatLLM platform (such as ChatGPT, Claude and Qwen, etc.) through an API call interface. In addition, it also includes collating code samples to form a combination set of orthogonal code fragments. The collating includes preliminary filtering of the combination set (mainly some validity filtering) and classification of programming languages, sending it to the decider, and further outputting the supported programming languages. The output processing mainly includes code standardization (proofreading), data encryption, and then sending it to the data management system for the next step of operation. In addition, unsupported programming languages are marked and recorded. Supported programming language set: mainstream languages such as Java, Python, JavaScript, C++ and Go.
[0023] In addition, by expanding the analysis methods that support corresponding languages, it is possible to support the evaluation of different types of codes, including not only code snippets of object-oriented languages, but also codes covering multiple programming paradigms such as scripting languages and functional languages.
[0024] According to an embodiment of the present invention, the preset code analysis method includes a static analysis method, a dynamic analysis method and a security vulnerability classification model. According to the programming language and feature analysis results adopted by the code, selecting a pair of codes from the preset code analysis method for analysis includes the following steps: S31, when the structured degree of the code is greater than the first threshold, a static analysis method is adopted; S32, when the code contains dynamically generated code segments, a dynamic analysis method is adopted; S33, in other cases, a pre-trained security vulnerability classification model is adopted to predict the security vulnerabilities of the code. The first threshold can be 0.5, indicating the structured degree of the code. In addition, when the static analysis method is adopted, the code structure is analyzed using an abstract syntax tree (AST) to identify common security issues; when dynamic analysis is adopted: the code is executed in a sandbox environment to monitor runtime security issues; in addition, when a machine learning model is adopted, a pre-trained security vulnerability classification model is adopted to predict the security vulnerabilities of the code. The security vulnerability classification model is based on Transformer. The original code snippet is used as input, and the multi-label classification result (indicating the probability of different types of security vulnerabilities) and the attention weight (indicating which code parts the model believes may have problems) are output. In addition, the security vulnerability classification model also conducts continuous learning, collects new vulnerability samples and false positive information, and regularly retrains the model to adapt to new vulnerability patterns. In addition to CWE-based security vulnerability analysis, other evaluation dimensions such as code complexity, readability, and maintainability can also be introduced to provide developers with a more comprehensive code quality reference.
[0025] According to an embodiment of the present invention, after code analysis, the identified vulnerabilities are summarized, CWE classification (industry standard evaluation certification) and severity assessment are performed, and severity weight coefficients are divided according to the classification results. Types of security vulnerabilities include buffer overflow (CWE-25), structured query language SQL injection (CWE-89) and input validation (CWE-20). The severity weight coefficient indicates the severity of each CWE type, and the possible value range is 0 to 1. For example, a serious remote code execution vulnerability may be assigned a severity of 0.9, while a minor style problem may only have a severity of 0.1. In addition, the frequency of occurrence of a specific CWE type in a code sample is also counted. The calculation method is the number of occurrences of the CWE type divided by the total number of codes.
[0026] According to an embodiment of the present invention, the security score is:
[0027] score=100-Σ(CWE_severity*CWE_frequency*time_factor) / total_code_samples*100,
[0028] Among them, CWE_severity represents the severity of each CWE type, CWE_frequency represents the frequency of occurrence of a specific CWE type in a code sample, time_factor is a time decay factor representing the newness of the code, and total_code_samples represents the total number of codes used for normalization. Among them, the most recent samples will have a higher weight. For example, an exponential decay function can be used so that the weight of samples one week ago is 1, and the weight of samples one month ago may drop to 0.5. In addition, the score can be calculated separately for codes in different programming languages. Thereafter, the security score is stored in a database. The database is a distributed database or a relational database. Different database solutions are used according to the data type. For example, a NoSQL database (distributed database) such as MongoDB is used when storing unstructured code data; a database (relational database) such as PostgreSQL is used when storing structured analysis results and scoring data. Data encryption is adopted in the database to ensure the security of stored code snippets and analysis results. Data compression logic is generally implemented by a combination of multiple methods such as page compression, dictionary encoding, and prefix compression. In addition, a backup strategy is used to ensure data security. In addition, database access control mainly regulates user access to data resources through user authentication and permission management.
[0029] According to another embodiment of the present invention, a code security assessment system is provided, comprising: a code extraction module, configured to extract multiple code segments from a code database, wherein the code database includes multiple code segments regularly obtained from online code sharing web pages and ChatLLM platforms, each code segment is attached with a timestamp, and some of the multiple code segments have the same function; a security vulnerability analysis module, configured to perform programming language classification and feature analysis on the extracted code, and select a pair of codes from a preset code analysis method according to the programming language used by the code and the result of the feature analysis to analyze the code to identify security vulnerabilities in the code, wherein the feature analysis includes determining the degree of structuring of the code, determining whether the code contains dynamically generated code segments, and determining the complexity of the code; a dynamic scoring module, configured to summarize the identified security vulnerabilities, perform common weakness enumeration CWE classification, divide the severity weight coefficient according to the type according to the CWE classification result, and calculate the frequency of occurrence of a specific CWE type, and calculate the security score of each code segment based on the severity weight coefficient, the frequency of occurrence, the timestamp and the total code quantity; a data storage and management module, used to store the code, the analysis results and the security score; a developer interface module, used for developers to query and compare the security scores of code fragments from different sources.
[0030] According to another embodiment of the present invention, popular multi-channel developer interface options are supported for parameter acquisition, and example interfaces include:
[0031] 1. Web application: responsive design, support for multi-device access;
[0032] 2. IDE plug-in: integrated into mainstream IDEs, such as VSCode, pyCharm, etc.;
[0033] 3. API interface: allows developers to integrate this function into their own tools.
[0034] Through these developer interfaces, the parameters available to developers come from upstream scoring data and analysis data, such as vulnerability assessment scoring tables and code data sets. You can also query the data management module to output query results. In addition, according to the developer's secondary development needs, the visual report can be uploaded, optimized, prompted, and other operations based on the interface type, so that developers can make full use of the report and feedback the results.
[0035] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.
Claims
1. A code security assessment method, characterized in that: The following steps are involved: S1. extracting multiple code segments from a code database, wherein the code database includes multiple code segments regularly obtained from online code sharing web pages and ChatLLM platforms, each code segment is attached with a timestamp, and some of the multiple code segments have the same function; S2. Classify the extracted codes by programming language and analyze their characteristics, wherein the characteristic analysis includes determining the degree of structure of the codes, determining whether the codes contain dynamically generated code segments, and determining the complexity of the codes; S3. According to the programming language and feature analysis results adopted by the code, select a pair of codes from the preset code analysis methods for analysis to identify security vulnerabilities in the code; S4. Summarize the identified security vulnerabilities, perform common weakness enumeration (CWE) classification, divide the severity weight coefficient by type based on the CWE classification results, and calculate the occurrence frequency of specific CWE types; S5. Calculate the security score of each code segment based on the severity weight coefficient, occurrence frequency, timestamp, and total code quantity.
2. A code security assessment method as claimed in claim 1, characterized in that: The code obtained from the online code sharing webpage is the code obtained from the online code sharing webpage by using code cloning or web crawler, wherein the obtained code also includes the corresponding question sequence.
3. A code security assessment method as claimed in claim 2, characterized in that: The code obtained from the ChatLLM platform is to input the corresponding question sequence into the ChatLLM platform and copy the code result returned by the ChatLLM platform.
4. A code security assessment method as claimed in claim 1, characterized in that: The ChatLLM platform includes ChatGPT, Claude and Qwen.
5. A code security assessment method as claimed in claim 1, characterized in that: The preset code analysis methods include static analysis methods, dynamic analysis methods and security vulnerability classification models.
6. A code security assessment method as claimed in claim 5, characterized in that: According to the programming language used by the code and the result of the characteristic analysis, selecting a pair of codes from the preset code analysis method for analysis includes the following steps: S31. When the structured degree of the code is greater than a first threshold, a static analysis method is used; S32. When the code contains dynamically generated code segments, a dynamic analysis method is used; S33. In other cases, a pre-trained security vulnerability classification model is used to predict the security vulnerabilities of the code.
7. A code security assessment method as claimed in claim 1, characterized in that: Types of security vulnerabilities include buffer overflow, SQL injection, and input validation.
8. A code security assessment method as claimed in claim 1, characterized in that: The safety scores are: score=100-Σ(CWE_severity*CWE_frequency*time_factor) / total_code_samples*100, Among them, CWE_severity represents the severity of each CWE type, CWE_frequency represents the frequency of a specific CWE type in the code sample, time_factor is the time decay factor representing the newness of the code, and total_code_samples represents the total number of codes used for normalization.
9. A code security assessment method as claimed in claim 1, characterized in that: It also includes storing the security score in a database.
10. A code security assessment system, characterized in that: include: A code extraction module is configured to extract multiple code segments from a code database, wherein the code database includes multiple code segments regularly obtained from an online code sharing webpage and a ChatLLM platform, each code segment is attached with a timestamp, and some of the multiple code segments have the same function; The security vulnerability analysis module is configured to classify the extracted code into programming languages and perform feature analysis, and select a pair of codes from preset code analysis methods to analyze the codes according to the programming languages used by the codes and the results of the feature analysis, so as to identify security vulnerabilities in the codes, wherein the feature analysis includes determining the degree of structure of the codes, determining whether the codes contain dynamically generated code segments, and determining the complexity of the codes; A dynamic scoring module is configured to aggregate the identified security vulnerabilities, perform Common Weakness Enumeration (CWE) classification, classify the severity weight coefficients by type based on the CWE classification results, calculate the frequency of occurrence of specific CWE types, and calculate the security score of each code segment based on the severity weight coefficient, frequency of occurrence, timestamp, and total code quantity; Data storage and management module, used to store code, analysis results, and security scores; The developer interface module is used by developers to query and compare the security scores of code snippets from different sources.