Source code detection method

Abstract

The invention relates to the technical field of software engineering, in particular to a source code detection method, which comprises the steps of source code preprocessing, code dependency analysis, semantic understanding and analysis, code quality evaluation, dynamic behavior simulation, intelligent vulnerability detection and the like to realize comprehensive detection of potential vulnerabilities and quality problems in source codes. A support vector machine and a K-means clustering algorithm are combined, known and unknown vulnerabilities can be effectively recognized, in addition, a continuous optimization feedback mechanism is introduced, a mixed vulnerability detection model can be continuously optimized according to repair operation and feedback information of a developer, the detection accuracy is improved, and the coverage range is expanded. The generated comprehensive detection report covers code structures, vulnerability positioning, quality problems and optimization suggestions, supports integration of various output formats and development tool chains, helps developers to efficiently repair problems, and improves code quality and project progress; according to the method, the comprehensiveness, the self-adaptability and the user experience of code detection are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of software engineering, and in particular to a method for detecting source code. Background Art

[0002] As the complexity and scale of software development continue to increase, code quality and security issues have become increasingly prominent. Modern software projects often involve multiple programming languages ​​and complex library dependencies, and various potential vulnerabilities and quality issues are easily introduced during the development process, such as unhandled boundary conditions, redundant code, inefficient code structure, and irregular comments. These problems not only reduce the performance and maintainability of the software, but may also lead to serious security risks and affect the reliability of the entire project.

[0003] However, existing source code detection technologies have certain limitations. First, most detection tools rely on a single analysis method and can only detect known security vulnerabilities, but cannot effectively identify unknown vulnerabilities and code defects. In addition, existing technologies are often unable to cope with projects with multi-language mixed programming and complex dependencies, and cannot fully cover all possible dependency paths and logical vulnerabilities. Second, existing detection tools usually lack adaptive capabilities and cannot be optimized and iteratively updated based on feedback information from actual projects, resulting in a gradual decline in detection results as the project evolves. Summary of the invention

[0004] Based on the above purpose, the present invention provides a method for source code detection.

[0005] A method for source code detection comprises the following steps:

[0006] S1, source code preprocessing: preprocess the source code to be tested, including code formatting, comment cleaning, redundant code deletion and code block identification;

[0007] S2, code dependency analysis: Based on the code preprocessing results, static analysis tools are used to analyze the dependencies in the source code. For projects involving multiple programming languages ​​and complex library dependencies, semantic analysis and data flow analysis techniques are used to build a complete code dependency graph;

[0008] S3, semantic understanding and analysis: Introducing natural language processing technology to understand the functions, variable names and comments in the source code, and combining the code dependency graph to analyze the code functions at the semantic level to identify potential logic vulnerabilities and security risks;

[0009] S4, code quality assessment: Based on the preset code quality standards, the source code is comprehensively evaluated. The quality standards include code complexity, duplicate code ratio, comment coverage, function length and module coupling, and each evaluation result is scored;

[0010] S5, dynamic behavior simulation: through dynamic analysis technology, simulate the running behavior of source code under different input conditions, detect the response of source code under abnormal input and extreme input conditions, and combine with static analysis results to further confirm potential vulnerabilities in the code;

[0011] S6, Intelligent Vulnerability Detection: Use machine learning algorithms to train source code vulnerability detection models, input the detection results from code quality assessment and dynamic behavior simulation into the source code vulnerability detection model, and perform further vulnerability detection on the source code;

[0012] S7, comprehensive test report generation: Based on the test results of each step, a comprehensive test report including code quality, vulnerability status, and optimization suggestions is generated. The report content includes source code structure analysis, location of potential problems, repair suggestions and their priority ranking;

[0013] S8, continuous optimization feedback: The generated detection report will be fed back to the developer, and based on the developer's repair results and feedback information, the source code detection method will be continuously optimized, the source code vulnerability detection model and rule library will be updated, and the accuracy and coverage of detection will be improved.

[0014] Optionally, the S1 specifically includes:

[0015] S11, code formatting: using regular expressions to identify and unify indentation, blank lines, and trailing spaces in the code;

[0016] S12, comment cleaning: Use natural language processing technology to identify the comment part in the code, separate the comments from the code logic, analyze the comment content, delete the comments that are repeated or meaningless with the code logic, retain the valuable explanatory comments, format the retained comments, unify the comment style, and ensure the readability and consistency of the comments;

[0017] S13, redundant code deletion: Use static analysis tools to scan the repeated code snippets in the code, identify redundant code by comparing the functional similarity of the code snippets, merge the redundant code with similar functions into an independent function to reduce code redundancy and improve code maintainability, mark and delete completely redundant code snippets, and keep backups for subsequent inspection;

[0018] S14, code block identification: Use the abstract syntax tree tool to parse the source code and generate a syntax tree representation of the code to facilitate structured analysis of the code blocks. Based on the structure of the abstract syntax tree, identify independent functional modules or code blocks in the code and mark the boundaries of the code blocks. According to the functions and dependencies of the code blocks, classify the code blocks into different modules.

[0019] Optionally, the code dependency analysis in S2 specifically includes:

[0020] S21, function call analysis: use static analysis tools to scan the source code, identify the function definitions and call relationships in the code, and generate a preliminary function call relationship diagram;

[0021] S22, cross-language dependency analysis: For multi-language projects, a cross-language syntax parser is used to parse the code of each language separately, identify function calls and data dependencies between different languages, use cross-language semantic mapping technology to match similar functions or modules in different languages, generate cross-language dependencies, and integrate the dependency graphs of each language into a unified cross-language dependency graph;

[0022] S23, Data Flow Analysis: Through data flow analysis technology, the definition, assignment and transfer path of variables in the code are tracked to generate a variable dependency graph;

[0023] S24, dependency graph construction: Integrate the function call relationship graph, cross-language dependency relationship graph and variable dependency relationship graph into a complete code dependency graph, optimize the dependency graph through graph theory algorithms (such as minimum spanning tree, strongly connected component analysis, etc.), eliminate redundant dependencies and circular dependencies, and output the optimized complete dependency graph for use in subsequent analysis steps.

[0024] Optionally, the S3 specifically includes:

[0025] S31, function name, variable name and comment extraction: identify and extract all function names from the source code, record the name of each function and its corresponding location, scan the source code, extract all variable names, record the variable names and their definition and usage locations, and extract the comment content, especially the comments associated with functions and variables, by parsing the comment part in the source code;

[0026] S32, noun and key phrase extraction: The extracted function names, variable names and comments are segmented, and nouns and phrases are extracted. These words or phrases reflect the main concepts and functions in the code. The extracted nouns and phrases are standardized (such as converting to lowercase, removing special characters, etc.) to ensure consistency;

[0027] S33, word embedding conversion: input the extracted nouns and phrases into a pre-trained word vector model (such as Word2Vec, GloVe, etc.), convert them into corresponding word vector representations, and store the generated word vectors in a vector database for subsequent steps;

[0028] S34, semantic similarity calculation: calculate the semantic similarity between each function name and its parameter variable name, determine the semantic consistency between the function naming and the variable usage, calculate the semantic similarity between each function name and its related comments, and ensure that the comment content accurately reflects the function's function;

[0029] S35, semantic anomaly marking: according to the preset similarity threshold, detect the function name, variable name and comment combination below the semantic similarity threshold, mark it as potential semantic anomaly, classify the semantic anomaly into categories such as inconsistent naming and inaccurate annotation, and generate a detailed anomaly report for developers to correct.

[0030] Optionally, the code quality assessment in S4 specifically includes:

[0031] S41, code complexity assessment: use McCabe's method to calculate the cyclomatic complexity of each function, evaluate the cognitive complexity of the code, quantify the difficulty of understanding the code by accumulating the nesting depth of conditional branches, and calculate the comprehensive complexity score by combining cyclomatic complexity and cognitive complexity according to preset standards;

[0032] S42, duplicate code detection: Use text matching algorithm and abstract syntax tree analysis to extract similar fragments in the code, calculate the similarity between code fragments, calculate the code duplication ratio and convert it into a score;

[0033] S43, comment coverage evaluation: count the number of comment lines and the total number of code lines in the source code and calculate the comment coverage;

[0034] S44, function length evaluation: count the number of lines of code for each function, identify functions that are too long or too short, analyze the rationality of function length based on the complexity and functional requirements of the function, avoid maintainability issues caused by too long functions, and calculate the function length score;

[0035] S45, module coupling evaluation: Based on the code dependency graph generated by S2, analyze the dependencies between modules and calculate the module coupling.

[0036] Optionally, the dynamic behavior simulation in S5 specifically includes:

[0037] S51, symbolic execution path generation: symbolize the input condition I, replace the actual value with the symbol σ(I), so that the code can process different inputs in symbolic form, and construct the path condition P for each branch statement in the code (such as 'if statement, loop statement) k , each condition represents the premise of path execution, and each path condition P is recorded step by step when executing the symbolic path k The execution path set P = {P1, P2, ..., P n};

[0038] S52, Taint analysis and data flow tracing: Identify possible tainted data in the code (such as user input, external data), trace the propagation path of tainted data in the program, identify how data affects other variables through assignment, parameter passing, etc., identify all taint propagation paths that may cause security vulnerabilities, and record them in the path collection;

[0039] S53, path coverage analysis: statistics the explored symbol paths, calculate the symbol path coverage, and generate optimization suggestions based on the analysis results of path coverage and taint propagation paths, prompting developers to improve the code to increase coverage or fix potential vulnerabilities.

[0040] Optionally, the intelligent vulnerability detection in S6 uses a support vector machine as a supervised learning algorithm and K-means as an unsupervised learning algorithm, specifically including:

[0041] S61, supervised learning model training: collect code samples containing known vulnerabilities and build training dataset D s , where each sample contains the feature vector X s and the corresponding vulnerability label Y s , extract the code features related to the vulnerability from the training dataset, including code patterns, function call structures, and variable usage, and generate a feature vector X s , use the support vector machine algorithm to train the feature vector X s Conduct training and establish a supervised learning model M S , the supervised learning model is expressed as: M S (X s )=SVM(X s );

[0042] Among them, M S (X s ) is the support vector machine model, X s is the input feature vector, SVM(X s ) represents the classification result of the support vector machine for the input features;

[0043] S62, Unsupervised Learning Model Training: Collecting Large-Scale Unlabeled Code Samples D u These samples do not contain clear vulnerability labels. We use the K-means algorithm to cluster the unlabeled data, identify potential patterns or anomalies in the code, and determine K cluster centers C. u , for each cluster center C u Extract feature vector X from samples in u , establish an unsupervised learning model M U , used to identify possible unknown vulnerabilities, the unsupervised learning model is expressed as: M U (Xu )=K-means(X u , C u )

[0044] Among them, M U (X u ) is an unsupervised learning model, X u is the input feature vector, K-means(X u , C u ) represents the clustering result of K-means algorithm based on cluster centers;

[0045] S63, Supervised and Unsupervised Model Fusion: The supervised learning model M S and the unsupervised learning model M U The output results of are fused to build a hybrid vulnerability detection model M, which is expressed as:

[0046] M(X)=α·M S (X)+β·M U (X);

[0047] Among them, M(X) is the output of the hybrid model, X is the input code feature vector, and M S (X) is the prediction result of the supervised learning model, M U (X) is the result of the unsupervised learning model, α and β are fusion weight parameters, and α+β=1;

[0048] S64, vulnerability detection and report generation: Use the fused hybrid model to comprehensively scan the source code to detect known and unknown vulnerabilities, classify and organize the detected vulnerabilities according to the severity and type of the vulnerabilities, and generate a vulnerability report.

[0049] Optionally, the generation of the comprehensive detection report in S7 specifically includes:

[0050] S71, Code Structure Analysis: Use static analysis tools to extract the structural information of the source code, including module division, function call hierarchy and dependency, and generate a code structure diagram;

[0051] S72, problem location: Based on the vulnerabilities identified by the intelligent vulnerability detection model in step S6, locate the location of the vulnerability in the code and generate a vulnerability location report. For the quality problems identified in step S4, such as excessive complexity and duplicated code, mark the location and generate a quality problem report. According to the severity of the vulnerability, the impact of the quality problem and the difficulty of repair, prioritize all problems and form a problem priority list to facilitate developers to arrange repair plans;

[0052] S73, Optimization Suggestions: Generate repair suggestions based on vulnerability location reports and quality problem reports, including code refactoring, annotation improvement, and redundant deletion. According to the problem priority list and repair suggestions, develop a repair roadmap to help developers arrange repair work;

[0053] S74, multi-format report generation: provides report output options in multiple formats, such as PDF, HTML, JSON, etc., to meet the needs of different development teams.

[0054] Optionally, the continuous optimization feedback in S8 specifically includes:

[0055] S81, Repair operation collection: capture and record the developer's operations when repairing vulnerabilities or optimizing codes, including the modified code segment, operation time, and code differences before and after the repair, form a repair operation log, re-test the repaired code, verify whether the repair is successful, confirm whether the problem has been completely solved, and record the verification results. The repair operation log and verification results are stored in the preset central database for subsequent analysis and model optimization;

[0056] S82, feedback information processing and model update: Provide a developer feedback interface to allow developers to submit problems encountered during the repair process, improvement suggestions or additional repair information to form feedback data. Analyze the collected feedback data, identify common problems and improvement suggestions, and convert them into inputs for model optimization. Update the hybrid vulnerability detection model based on the analysis results.

[0057] S83, model iteration and optimization: Use the latest repair data and feedback information to iteratively train the hybrid vulnerability detection model, generate an optimized hybrid vulnerability detection model, deploy the optimized hybrid vulnerability detection model to the detection system, start a new round of detection tasks, and continuously improve the system's vulnerability detection and code optimization capabilities.

[0058] Beneficial effects of the present invention:

[0059] The present invention, by combining multiple analysis methods, including static analysis of source code, dynamic behavior simulation, semantic understanding, code quality assessment, etc., can comprehensively and accurately detect potential vulnerabilities and quality issues in source code. In particular, through the combination of support vector machine and K-means clustering algorithm, it can not only identify known security vulnerabilities, but also discover unknown code defects, significantly improving the coverage and accuracy of detection. In addition, through symbolic execution and taint analysis technology, the system can simulate the behavior of code under different input conditions and effectively detect security risks under complex conditions.

[0060] The present invention introduces a continuous optimization feedback mechanism. By automatically capturing and analyzing the developer's repair operations and feedback information, the system can continuously optimize the hybrid vulnerability detection model. This closed-loop feedback mechanism enables the detection system to have adaptive capabilities, and it can be iteratively updated according to new problems in actual projects and the developer's repair methods, ensuring that the system always maintains efficient and accurate detection capabilities and can adapt to the ever-changing development environment in the long term. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only for the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0062] Figure 1 A schematic diagram of a method flow chart of an embodiment of the present invention;

[0063] Figure 2 Schematic diagram of the S3 step flow chart of an embodiment of the present invention. DETAILED DESCRIPTION

[0064] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. At the same time, it is explained here that in order to make the embodiments more detailed, the following embodiments are the best and preferred embodiments, and those skilled in the art may also adopt other alternatives to implement some known technologies; and the accompanying drawings are only for more specific description of the embodiments, and are not intended to specifically limit the present invention.

[0065] It should be noted that the references to "one embodiment", "an embodiment", "an exemplary embodiment", "some embodiments" and the like in the specification indicate that the embodiments described may include specific features, structures or characteristics, but not every embodiment may include the specific features, structures or characteristics. In addition, when a specific feature, structure or characteristic is described in conjunction with an embodiment, it should be within the knowledge of a person skilled in the art to implement such feature, structure or characteristic in conjunction with other embodiments (whether or not explicitly described).

[0066] In general, a term can be understood, at least in part, from its use in context. For example, depending, at least in part, on the context, the term "one or more" as used herein can be used to describe any feature, structure, or characteristic in the singular sense, or can be used to describe a combination of features, structures, or characteristics in the plural sense. Additionally, the term "based on" can be understood as not necessarily intended to convey an exclusive set of factors, but can instead, depending, at least in part, on the context, allow for the presence of other factors that are not necessarily explicitly described.

[0067] like Figure 1-Figure 2 As shown, a method for source code detection includes the following steps:

[0068] S1, source code preprocessing: preprocess the source code to be tested, including code formatting, comment cleaning, redundant code deletion and code block identification, to ensure the standardization and unification of the code structure;

[0069] S2, code dependency analysis: Based on the code preprocessing results, static analysis tools are used to analyze the dependencies in the source code. For projects involving multiple programming languages ​​and complex library dependencies, semantic analysis and data flow analysis techniques are used to build a complete code dependency graph;

[0070] S3, semantic understanding and analysis: Introducing natural language processing technology to understand the functions, variable names and comments in the source code, and combining the code dependency graph to analyze the code functions at the semantic level to identify potential logic vulnerabilities and security risks;

[0071] S4, code quality assessment: Based on the preset code quality standards, the source code is comprehensively evaluated. The quality standards include code complexity, duplicate code ratio, comment coverage, function length and module coupling, and each evaluation result is scored;

[0072] S5, dynamic behavior simulation: through dynamic analysis technology, simulate the running behavior of source code under different input conditions, detect the response of source code under abnormal input and extreme input conditions, and combine with static analysis results to further confirm potential vulnerabilities in the code;

[0073] S6, Intelligent Vulnerability Detection: Use machine learning algorithms to train source code vulnerability detection models, input the detection results from code quality assessment and dynamic behavior simulation into the source code vulnerability detection model, and perform further vulnerability detection on the source code;

[0074] S7, comprehensive test report generation: Based on the test results of each step, a comprehensive test report including code quality, vulnerability status, and optimization suggestions is generated. The report content includes source code structure analysis, location of potential problems, repair suggestions and their priority ranking;

[0075] S8, continuous optimization feedback: The generated detection report will be fed back to the developer, and based on the developer's repair results and feedback information, the source code detection method will be continuously optimized, the source code vulnerability detection model and rule library will be updated, and the accuracy and coverage of detection will be improved.

[0076] S1 specifically includes:

[0077] S11, code formatting: Use regular expressions to identify and unify indentation, blank lines, and end-of-line spaces in the code to ensure consistency in code format;

[0078] S12, comment cleaning: Use natural language processing technology to identify the comment part in the code, separate the comments from the code logic, analyze the comment content, delete the comments that are repeated or meaningless with the code logic, retain the valuable explanatory comments, format the retained comments, unify the comment style, and ensure the readability and consistency of the comments;

[0079] S13, redundant code deletion: Use static analysis tools to scan the repeated code snippets in the code, identify redundant code by comparing the functional similarity of the code snippets, merge the redundant code with similar functions into an independent function to reduce code redundancy and improve code maintainability, mark and delete completely redundant code snippets, and keep backups for subsequent inspection;

[0080] S14, code block identification: Use the abstract syntax tree tool to parse the source code and generate a syntax tree representation of the code to facilitate the structured analysis of the code block. Based on the structure of the abstract syntax tree, identify the independent functional modules or code blocks in the code and mark the boundaries of the code blocks. According to the functions and dependencies of the code blocks, classify the code blocks into different modules for subsequent dependency parsing and analysis;

[0081] Through the preprocessing steps, the source code is formatted, cleaned and structured, so that the standardization and consistency of the code are guaranteed, laying the foundation for the subsequent analysis process.

[0082] The code dependency analysis in S2 specifically includes:

[0083] S21, function call analysis: Use static analysis tools to scan the source code, identify the function definitions and call relationships in the code, and generate a preliminary function call relationship graph. For recursive calls and chain calls, the tool will further trace these call chains to ensure the integrity of the function call relationship. The identified function call relationship is represented in the form of a directed graph, with nodes representing functions and edges representing call relationships.

[0084] S22, cross-language dependency analysis: For multi-language projects, a cross-language syntax parser is used to parse the code of each language separately, identify the function calls and data dependencies between different languages, use cross-language semantic mapping technology to match similar functions or modules in different languages, generate cross-language dependencies, and integrate the dependency graphs of each language into a unified cross-language dependency graph to ensure that the call relationships between different language codes in the project are accurately parsed;

[0085] S23, Data Flow Analysis: Through data flow analysis technology, the definition, assignment and transfer path of variables in the code are tracked to generate a variable dependency graph. For key variables (such as global variables, transfer parameters, etc.), their flow paths between different functions or modules are further analyzed to identify possible dependency chains. By simplifying and optimizing the data flow tracing results, unnecessary redundant dependencies are removed to ensure the simplicity and effectiveness of the dependency graph.

[0086] S24, dependency graph construction: Integrate the function call relationship graph, cross-language dependency relationship graph and variable dependency relationship graph into a complete code dependency graph, optimize the dependency graph through graph theory algorithms (such as minimum spanning tree, strongly connected component analysis, etc.), eliminate redundant dependencies and circular dependencies, and output the optimized complete dependency graph for subsequent analysis steps;

[0087] Through detailed dependency parsing steps, a complete and accurate code dependency graph is generated, ensuring comprehensive coverage of code dependencies, especially when dealing with multi-language projects and complex library dependencies.

[0088] S3 specifically includes:

[0089] S31, function name, variable name and comment extraction: identify and extract all function names from the source code, record the name of each function and its corresponding location, scan the source code, extract all variable names, record the variable names and their definition and usage locations, and extract the comment content, especially the comments associated with functions and variables, by parsing the comment part in the source code;

[0090] S32, noun and key phrase extraction: The extracted function names, variable names and comments are segmented, and nouns and phrases are extracted. These words or phrases reflect the main concepts and functions in the code. The extracted nouns and phrases are standardized (such as converting to lowercase, removing special characters, etc.) to ensure consistency;

[0091] S33, word embedding conversion: input the extracted nouns and phrases into a pre-trained word vector model (such as Word2Vec, GloVe, etc.), convert them into corresponding word vector representations, and store the generated word vectors in a vector database for subsequent steps;

[0092] S34, semantic similarity calculation: Calculate the semantic similarity between each function name and its parameter variable name, determine the semantic consistency between the function naming and the variable usage, calculate the semantic similarity between each function name and its related comments, and ensure that the comment content accurately reflects the function's function. The semantic similarity S(f,v,c) is expressed as:

[0093]

[0094] Where, W(f) is the word vector of the function name, W(v) is the word vector of the variable name, W(c) is the word vector of the comment content, and cos(W(x), W(y)) is the cosine similarity between word vectors;

[0095] S35, semantic anomaly marking: according to the preset similarity threshold, function names, variable names and annotation combinations below the semantic similarity threshold are detected and marked as potential semantic anomalies. The semantic anomalies are classified into categories such as inconsistent naming and inaccurate annotations, and a detailed anomaly report is generated for developers to correct;

[0096] Through the semantic understanding and analysis steps, the semantic relationship between naming and annotations in the source code can be effectively analyzed, and potential semantic inconsistency problems can be detected and marked, thereby improving the readability and maintainability of the code.

[0097] The code quality assessment in S4 specifically includes:

[0098] S41, code complexity evaluation: Use McCabe's method to calculate the cyclomatic complexity of each function, expressed as:

[0099] C c =E-N+2P;

[0100] Among them, C c is the cyclomatic complexity, E is the number of edges in the control flow graph, N is the number of nodes in the control flow graph, and P is the number of connected components in the control flow graph. It evaluates the cognitive complexity of the code and quantifies the difficulty of understanding the code by accumulating the nesting depth of conditional branches. According to the preset criteria, the cyclomatic complexity and cognitive complexity are combined to calculate the comprehensive complexity score S C , expressed as:

[0101] S C =w c ·C c +w n ·C n ;

[0102] Among them, C c is the cyclomatic complexity score, C n is the cognitive complexity score, w c and w n is the corresponding weight coefficient, and w c +w n =1;

[0103] S42, duplicate code detection: Use text matching algorithm and abstract syntax tree analysis to extract similar fragments in the code and calculate the similarity Sim (C i , C j ), expressed as:

[0104]

[0105] Among them, C i and C j Represents two code snippets, |C i | and |C j | respectively represent the length of the code snippet, |C i ∩C j | represents the number of similar lines of code between two code snippets, and calculates the code duplication ratio S D And converted into a score, expressed as:

[0106]

[0107] Where N is the total number of code snippets;

[0108] S43, comment coverage evaluation: count the number of comment lines in the source code L c and the total number of lines of code L t , calculate the annotation coverage S CC , expressed as:

[0109]

[0110] S44, function length evaluation: count the number of lines of code L(fi) for each function, identify functions that are too long or too short, analyze the rationality of function length based on the complexity and functional requirements of the function, avoid maintainability issues caused by too long functions, and calculate the function length score S FL , expressed as:

[0111]

[0112] Among them, N f is the total number of functions;

[0113] S45, module coupling evaluation: Based on the code dependency graph generated by S2, analyze the dependencies between modules and calculate the module coupling S MC , expressed as:

[0114]

[0115] Among them, M i and M j It is a module, Calls(M i , M j ) indicates module M i and M j The number of calls between m is the total number of modules;

[0116] Through the code quality assessment steps, the code is quantitatively evaluated from multiple dimensions (including complexity, repetition, comment coverage, function length and module coupling), generating a comprehensive quality score and providing developers with clear optimization directions.

[0117] Dynamic behavior simulation in S5 specifically includes:

[0118] S51, symbolic execution path generation: symbolize the input condition I, replace the actual value with the symbol σ(I), so that the code can process different inputs in symbolic form, and construct the path condition P for each branch statement in the code (such as 'if statement, loop statement) k , each condition represents the premise of path execution, and each path condition P is recorded step by step when executing the symbolic path k The execution path set P = {P1, P2, ..., P n};

[0119] S52, Taint analysis and data flow tracing: Identify possible tainted data in the code (such as user input, external data), trace the propagation path of tainted data in the program, identify how data affects other variables through assignment, parameter passing, etc., identify all taint propagation paths that may cause security vulnerabilities, and record them in the path collection;

[0120] S53, Path Coverage Analysis: For the explored symbolic path P explored Perform statistics and calculate the symbol path coverage Cov(P), which is expressed as:

[0121]

[0122] Among them, |P explored | represents the number of symbolic paths explored, |P total | represents the total number of symbol paths. Based on the analysis results of path coverage and taint propagation paths, optimization suggestions are generated to prompt developers to improve the code to increase coverage or fix potential vulnerabilities.

[0123] Through symbolic execution and taint analysis, the system can simulate the dynamic behavior of the code under different input conditions, identify potential security risks and logical loopholes, and provide developers with targeted optimization suggestions through path coverage analysis. This method enhances the reliability and security of the code, especially when dealing with complex conditional branches and input verification.

[0124] The intelligent vulnerability detection in S6 uses support vector machine as the supervised learning algorithm and K-means as the unsupervised learning algorithm, including:

[0125] S61, supervised learning model training: collect code samples containing known vulnerabilities and build training dataset D s , where each sample contains the feature vector X s and the corresponding vulnerability label Y s , extract the code features related to the vulnerability from the training dataset, including code patterns, function call structures, and variable usage, and generate a feature vector X s , use the support vector machine algorithm to train the feature vector X s Conduct training and establish supervised learning model M S , the supervised learning model is expressed as: M S (X s )=SVM(X s );

[0126] Among them, M S (X s ) is the support vector machine model, X s is the input feature vector, SVM(X s ) represents the classification result of the support vector machine for the input features;

[0127] S62, Unsupervised Learning Model Training: Collecting Large-Scale Unlabeled Code Samples D u These samples do not contain clear vulnerability labels. We use the K-means algorithm to cluster the unlabeled data, identify potential patterns or anomalies in the code, and determine K cluster centers C. u , for each cluster center C u Extract feature vector X from samples in u , establish an unsupervised learning model M U , used to identify possible unknown vulnerabilities, the unsupervised learning model is expressed as: M U (X u )=K-means(X u , C u )

[0128] Among them, M U (X u ) is an unsupervised learning model, X u is the input feature vector, K-means(X u , C u ) represents the clustering result of K-means algorithm based on cluster centers;

[0129] S63, Supervised and Unsupervised Model Fusion: The supervised learning model M S and the unsupervised learning model M U The output results of are fused to build a hybrid vulnerability detection model M, which is expressed as:

[0130] M(X)=α·MS (X)+β·M U (X);

[0131] Among them, M(X) is the output of the hybrid model, X is the input code feature vector, and M S (X) is the prediction result of the supervised learning model, M U (X) is the result of the unsupervised learning model, α and β are fusion weight parameters, and α+α=1;

[0132] S64, vulnerability detection and report generation: Use the fused hybrid model to comprehensively scan the source code, detect known and unknown vulnerabilities, classify and organize the detected vulnerabilities according to the severity and type of the vulnerabilities, generate vulnerability reports, and generate repair suggestions based on the detection results to help developers fix vulnerabilities and improve the security of the code;

[0133] By adopting the intelligent vulnerability detection method of support vector machine and K-means algorithm, it can effectively identify known and unknown vulnerabilities in the source code and generate detailed vulnerability reports and repair suggestions. It combines the hybrid model method of supervised and unsupervised learning to significantly improve the comprehensiveness and accuracy of code security detection, which is particularly suitable for vulnerability detection in complex codes.

[0134] The comprehensive test report generation in S7 specifically includes:

[0135] S71, Code Structure Analysis: Use static analysis tools to extract the structural information of the source code, including module division, function call hierarchy and dependency, and generate a code structure diagram;

[0136] S72, problem location: Based on the vulnerabilities identified by the intelligent vulnerability detection model in step S6, locate the location of the vulnerability in the code and generate a vulnerability location report. For the quality problems identified in step S4, such as excessive complexity and duplicated code, mark the location and generate a quality problem report. According to the severity of the vulnerability, the impact of the quality problem and the difficulty of repair, prioritize all problems and form a problem priority list to facilitate developers to arrange repair plans;

[0137] S73, Optimization Suggestions: Generate repair suggestions based on vulnerability location reports and quality problem reports, including code refactoring, annotation improvement, and redundant deletion. According to the problem priority list and repair suggestions, develop a repair roadmap to help developers arrange repair work;

[0138] S74, multi-format report generation: provides report output options in multiple formats, such as PDF, HTML, JSON, etc., to meet the needs of different development teams;

[0139] Through the steps of generating a comprehensive detection report, comprehensive code analysis and optimization suggestion functions are provided. The generated report not only includes code structure, vulnerability location, quality issues and repair suggestions, but can also be output in multiple formats, ensuring that developers can efficiently understand and solve problems in the code and improve the overall quality and security of the code.

[0140] The continuous optimization feedback in S8 specifically includes:

[0141] S81, Repair operation collection: capture and record the developer's operations when repairing vulnerabilities or optimizing codes, including the modified code segment, operation time, and code differences before and after the repair, form a repair operation log, re-test the repaired code, verify whether the repair is successful, confirm whether the problem has been completely solved, and record the verification results. The repair operation log and verification results are stored in the preset central database for subsequent analysis and model optimization;

[0142] S82, feedback information processing and model update: Provide a developer feedback interface to allow developers to submit problems encountered during the repair process, improvement suggestions or additional repair information to form feedback data. Analyze the collected feedback data, identify common problems and improvement suggestions, and convert them into inputs for model optimization. Update the hybrid vulnerability detection model based on the analysis results.

[0143] S83, model iteration and optimization: Use the latest repair data and feedback information to iteratively train the hybrid vulnerability detection model, generate an optimized hybrid vulnerability detection model, deploy the optimized hybrid vulnerability detection model to the detection system, start a new round of detection tasks, and continuously improve the system's vulnerability detection and code optimization capabilities;

[0144] Through detailed continuous optimization feedback steps, the present invention provides a closed-loop feedback mechanism that can continuously collect and analyze data and optimize the hybrid vulnerability detection model while developers are repairing code. This method ensures that the detection system can continue to improve with the development of the project, enhances the detection capabilities of new vulnerabilities and code quality issues, and ensures the stability and scalability of the system.

[0145] The present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention. In order to make the public have a thorough understanding of the present invention, specific details are described in detail in the following preferred embodiments of the present invention, but those skilled in the art can fully understand the present invention without the description of these details. In addition, in order to avoid unnecessary confusion about the essence of the present invention, well-known methods, processes, procedures, components and circuits are not described in detail.

[0146] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for source code detection, characterized in that: The following steps are involved: S1, source code Preprocessing: Preprocess the source code to be tested, including code formatting, comment cleaning, redundant code deletion, and code block identification; S2, code dependency analysis: Based on the code preprocessing results, static analysis tools are used to analyze the dependencies in the source code. For projects involving multiple programming languages ​​and complex library dependencies, semantic analysis and data flow analysis techniques are used to build a complete code dependency graph; S3, semantic understanding and analysis: Introducing natural language processing technology to understand the functions, variable names and comments in the source code, and combining the code dependency graph to analyze the code functions at the semantic level to identify potential logic vulnerabilities and security risks; S4, code quality assessment: Based on the preset code quality standards, the source code is comprehensively evaluated. The quality standards include code complexity, duplicate code ratio, comment coverage, function length and module coupling, and each evaluation result is scored; S5, dynamic behavior simulation: Through dynamic analysis technology, simulate the running behavior of source code under different input conditions, detect the response of source code under abnormal input and extreme input conditions, and combine with static analysis results to further confirm potential vulnerabilities in the code; S6, Intelligent Vulnerability Detection: Use machine learning algorithms to train source code vulnerability detection models, input the detection results from code quality assessment and dynamic behavior simulation into the source code vulnerability detection model, and perform further vulnerability detection on the source code; S7, comprehensive test report generation: Based on the test results of each step, a comprehensive test report including code quality, vulnerability status, and optimization suggestions is generated. The report content includes source code structure analysis, location of potential problems, repair suggestions and their priority ranking; S8, continuous optimization feedback: The generated detection report will be fed back to the developer, and based on the developer's repair results and feedback information, the source code detection method will be continuously optimized, the source code vulnerability detection model and rule library will be updated, and the accuracy and coverage of detection will be improved.

2. A source code detection method according to claim 1, characterized in that: The S1 specifically includes: S11, code formatting: using regular expressions to identify and unify indentation, blank lines, and trailing spaces in the code; S12, comment cleaning: Use natural language processing technology to identify the comment part in the code, separate the comments from the code logic, and delete the comments that are repeated with the code logic by analyzing the comment content; S13, redundant code deletion: Use static analysis tools to scan repeated code snippets in the code, identify redundant code by comparing the functional similarity of the code snippets, merge redundant code with similar functions into an independent function, and mark and delete completely redundant code snippets; S14, code block identification: Use the abstract syntax tree tool to parse the source code and generate a syntax tree representation of the code. Based on the structure of the abstract syntax tree, identify the independent functional modules or code blocks in the code and mark the boundaries of the code blocks. According to the functions and dependencies of the code blocks, classify the code blocks into different modules.

3. A method for source code detection according to claim 2, characterized in that: The code dependency analysis in S2 specifically includes: S21, function call analysis: use static analysis tools to scan the source code, identify the function definitions and call relationships in the code, and generate a preliminary function call relationship diagram; S22, cross-language dependency analysis: For multi-language projects, a cross-language syntax parser is used to parse the code of each language separately, identify function calls and data dependencies between different languages, use cross-language semantic mapping technology to match similar functions or modules in different languages, generate cross-language dependencies, and integrate the dependency graphs of each language into a unified cross-language dependency graph; S23, Data Flow Analysis: Through data flow analysis technology, the definition, assignment and transfer path of variables in the code are tracked to generate a variable dependency graph; S24, dependency graph construction: Integrate the function call relationship graph, cross-language dependency relationship graph and variable dependency relationship graph into a complete code dependency graph, optimize the dependency graph through graph theory algorithms (such as minimum spanning tree, strongly connected component analysis, etc.), eliminate redundant dependencies and circular dependencies, and output the optimized complete dependency graph for use in subsequent analysis steps.

4. A method for source code detection according to claim 3, characterized in that: The S3 specifically includes: S31, function name, variable name and comment extraction: identify and extract all function names from the source code, scan the source code, extract all variable names, and extract the comment content by parsing the comment part in the source code; S32, noun and key phrase extraction: perform word segmentation on the extracted function names, variable names and comments, extract the nouns and phrases therein, and perform standardization on the extracted nouns and phrases; S33, word embedding conversion: input the extracted nouns and phrases into the pre-trained word vector model, convert them into corresponding word vector representations, and store the generated word vectors in the vector database; S34, semantic similarity calculation: calculate the semantic similarity between each function name and its parameter variable name, determine the semantic consistency between the function naming and the variable usage, calculate the semantic similarity between each function name and its related comments, and ensure that the comment content accurately reflects the function's function; S35, semantic anomaly marking: according to a preset similarity threshold, function names, variable names and comment combinations below the semantic similarity threshold are detected, marked as potential semantic anomalies, and an anomaly report is generated for developers to correct.

5. A method for source code detection according to claim 4, characterized in that: The code quality assessment in S4 specifically includes: S41, code complexity assessment: use McCabe's method to calculate the cyclomatic complexity of each function, evaluate the cognitive complexity of the code, quantify the difficulty of understanding the code by accumulating the nesting depth of conditional branches, and calculate the comprehensive complexity score by combining cyclomatic complexity and cognitive complexity according to preset standards; S42, duplicate code detection: Use text matching algorithm and abstract syntax tree analysis to extract similar fragments in the code, calculate the similarity between code fragments, calculate the code duplication ratio and convert it into a score; S43, comment coverage evaluation: count the number of comment lines and the total number of code lines in the source code and calculate the comment coverage; S44, function length evaluation: count the number of lines of code for each function, analyze the rationality of the function length based on the complexity and functional requirements of the function, and calculate the function length score; S45, module coupling evaluation: Based on the code dependency graph generated by S2, analyze the dependencies between modules and calculate the module coupling.

6. A method for source code detection according to claim 5, characterized in that: The dynamic behavior simulation in S5 specifically includes: S51, symbolic execution path generation: symbolize the input condition I, replace the actual value with the symbol σ(I), so that the code can process different inputs in symbolic form, and construct the path condition P for each branch statement in the code. k , each condition represents the premise of path execution, and each path condition P is recorded step by step when executing the symbolic path k The execution path set P = {P1, P2, ..., P n }; S52, taint analysis and data flow tracing: Identify tainted data in the code, trace the propagation path of tainted data in the program, identify all taint propagation paths that lead to security vulnerabilities, and record them in the path set; S53, path coverage analysis: statistics the explored symbol paths, calculate the symbol path coverage, and generate optimization suggestions based on the analysis results of path coverage and taint propagation paths, prompting developers to improve the code to increase coverage or fix potential vulnerabilities.

7. A method for source code detection according to claim 6, characterized in that: The intelligent vulnerability detection in S6 uses support vector machine as a supervised learning algorithm and K-means as an unsupervised learning algorithm, specifically including: S61, supervised learning model training: collect code samples containing known vulnerabilities and build training dataset D s , where each sample contains the feature vector X s and the corresponding vulnerability label Y s , extract the code features related to the vulnerability from the training dataset, including code patterns, function call structures, and variable usage, and generate a feature vector X s , use the support vector machine algorithm to train the feature vector X s Conduct training and establish supervised learning model M S , the supervised learning model is expressed as: M S (X s )=SVM(X s ); Among them, M S (X s ) is the support vector machine model, X s is the input feature vector, SVM(X s ) represents the classification result of the support vector machine for the input features; S62, Unsupervised Learning Model Training: Collecting Unlabeled Code Samples D u , use the K-means algorithm to cluster the unlabeled data, identify potential patterns or anomalies in the code, and determine K cluster centers C u , for each cluster center C u Extract feature vector X from samples in u , establish an unsupervised learning model M U , used to identify the existence of unknown vulnerabilities, the unsupervised learning model is expressed as: M U (X u )=K-means(X u , C u ) Among them, M U (X u ) is an unsupervised learning model, X u is the input feature vector, K-means(X u ,C u ) represents the clustering result of K-means algorithm based on cluster centers; S63, Supervised and Unsupervised Model Fusion: The supervised learning model M S and the unsupervised learning model M U The output results of are fused to build a hybrid vulnerability detection model M, which is expressed as: M(X)=α·M S (X)+β·M U (X); Among them, M(X) is the output of the hybrid model, X is the input code feature vector, and M S (X) is the prediction result of the supervised learning model, M U (X) is the result of the unsupervised learning model, α and β are fusion weight parameters, and α+β=1; S64, vulnerability detection and report generation: Use the fused hybrid model to comprehensively scan the source code to detect known and unknown vulnerabilities, classify and organize the detected vulnerabilities according to the severity and type of the vulnerabilities, and generate a vulnerability report.

8. A source code detection method according to claim 7, characterized in that: The generation of the comprehensive test report in S7 specifically includes: S71, Code Structure Analysis: Use static analysis tools to extract the structural information of the source code, including module division, function call hierarchy and dependency, and generate a code structure diagram; S72, problem location: Based on the vulnerabilities identified by the intelligent vulnerability detection model in step S6, locate the location of the vulnerability in the code and generate a vulnerability location report. For the quality problems identified in step S4, mark the location and generate a quality problem report. According to the severity of the vulnerability, the impact of the quality problem and the difficulty of repair, prioritize all problems and form a problem priority list; S73, Optimization Suggestions: Generate repair suggestions based on vulnerability location reports and quality problem reports, including code refactoring, annotation improvement, and redundant deletion. According to the problem priority list and repair suggestions, develop a repair roadmap to help developers arrange repair work; S74, multi-format report generation: provides report output options in multiple formats to meet the needs of different development teams.

9. A method for source code detection according to claim 8, characterized in that: The continuous optimization feedback in S8 specifically includes: S81, repair operation collection: capture and record the developer's operations when repairing vulnerabilities or optimizing codes, including the modified code segment, operation time, and code differences before and after the repair, form a repair operation log, re-test the repaired code, verify whether the repair is successful, and record the verification results, and store the repair operation log and verification results in a preset central database; S82, feedback information processing and model update: Provide a developer feedback interface to allow developers to submit problems encountered during the repair process, improvement suggestions or additional repair information to form feedback data. Analyze the collected feedback data, identify common problems and improvement suggestions, and convert them into inputs for model optimization. Update the hybrid vulnerability detection model based on the analysis results. S83, model iteration and optimization: Use the latest repair data and feedback information to iteratively train the hybrid vulnerability detection model, generate an optimized hybrid vulnerability detection model, deploy the optimized hybrid vulnerability detection model to the detection system, start a new round of detection tasks, and continuously improve the system's vulnerability detection and code optimization capabilities.

Images