AI code review quality control method, device and medium based on multi-level verification

Abstract

This invention discloses a multi-level verification-based AI code review quality control method, device, and medium, relating to the field of artificial intelligence. The method includes: receiving a set of code changes to be reviewed and inputting it into a pre-trained AI code review model to obtain an initial list of issues, which is then processed sequentially using a concatenated three-layer verification architecture; verifying the factual accuracy of the issue items in the first verification layer and outputting the passed issue items to the second verification layer; verifying the logical rationality of the issue items from the first verification layer in the second verification layer and outputting the passed issue items to the third verification layer; conducting a comprehensive final quality review of the issue items from the second verification layer in the third verification layer to generate a high-credibility issue list; and generating a final review report based on this list. This invention achieves systematic and multi-level verification of AI code review results, and can output review reports with high factual accuracy, low false positive rate, and stable reliability.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, device and medium for quality control of AI code review based on multi-level verification. Background Technology

[0002] In recent years, using large language models for automated code review has gradually become a trend. A typical approach is to directly input code changes into the large language model and obtain the generated review report. However, this method lacks a verification mechanism for the model's output, which may lead to the output of fictitious problems that do not exist in the code, or misjudgments of reasonable code implementations. Furthermore, the results fluctuate due to the randomness of the model, resulting in insufficient reliability.

[0003] Furthermore, to improve reliability, existing technologies have proposed a hybrid approach combining static analysis tools and AI, executing both in parallel and merging the results. The inventors of this application have found that in this approach, the AI's output is not deeply verified or constrained by the rule system of the static analysis tool; the two are merely a simple superposition of results. Therefore, the problems of fictitious and false alarms generated by AI still exist. At the same time, the inherent limitations of the static analysis tool's rule coverage make it difficult to handle complex logical problems beyond the scope of the rules.

[0004] It is evident that existing technical solutions have failed to effectively address the fundamental deficiencies of AI code review in terms of factual accuracy, logical rationality, and result stability. Summary of the Invention

[0005] This invention provides a method, device, and medium for quality control of AI code review based on multi-level verification. The technical problem it aims to solve is: how to provide an effective solution that can systematically and multi-level verify the results of AI code review, thereby outputting a review report with high factual accuracy, low false positive rate, and stable reliability.

[0006] In a first aspect, embodiments of the present invention provide an AI code review quality control method based on multi-level verification, comprising: Receive code change sets pending review; The code change set is input into a pre-trained AI code review model to obtain the original problem list generated by the AI ​​code review model; The original question list is processed by a three-layer verification architecture, which includes a first verification layer, a second verification layer, and a third verification layer connected in sequence. In the first verification layer, the factual accuracy of the question entries in the original question list is verified, and the question entries that pass the verification are output to the second verification layer. In the second verification layer, the logical rationality of the question entries from the first verification layer is verified, and the question entries that pass the verification are output to the third verification layer. In the third verification layer, a comprehensive quality review is conducted on the issue items from the second verification layer to generate a list of highly credible issues; Based on the list of highly credible issues, a final review report is generated.

[0007] Optionally, in the first verification layer, the factual accuracy verification of the question entries in the original question list includes: Extract one problem entry from the original problem list. The problem entry contains the line number and code snippet content of the suspected problem code. Based on the original files in the code repository, obtain the actual code snippet corresponding to the line number; Compare the code snippet content in the problem entry with the actual code snippet; If the code snippet content is inconsistent with the actual code snippet, the problem entry will be marked as verification failed.

[0008] Optionally, the factual accuracy verification of the question entries in the original question list in the first verification layer further includes: When the code snippet content matches the actual code snippet, determine the software development framework used by the code changeset. The code snippet is input into a predefined framework rule base, which contains syntax rules corresponding to the software development framework. The code snippet content is parsed and validated using the aforementioned syntax rules; If the code snippet violates the syntax rules, the issue entry will be marked as failing verification.

[0009] Optionally, in the second verification layer, the logical rationality verification of the question entries from the first verification layer includes: From the issue entries from the first verification layer, filter out issue entries whose remediation suggestion is to delete a specific code segment; Perform dependency analysis on the complete codebase containing the code change set to identify other code locations that directly or indirectly reference the specific code segment. When other code locations that reference the specific code segment are identified, the issue entry is marked as failing verification.

[0010] Optionally, the method further includes: In the first verification layer, all issues marked as failing verification are removed from the processing flow.

[0011] Optionally, the method further includes: For each issue item that passes the logical rationality verification, the issue type identified in the issue item is obtained. The issue type includes one of the following: security vulnerability, performance defect, code style violation, or logical error. The logical role of the code segment corresponding to the issue item in the business system is analyzed, and the positional importance level is determined based on the logical role. Based on the issue type and the positional importance level, the original severity label in the issue item is adjusted. The adjusted severity label is associated with the issue item and output to the third verification layer. For issues that fail the logical rationality verification, add explanatory notes explaining the failure to the issue and then discard it from the processing flow.

[0012] Optionally, in the third verification layer, a comprehensive quality review is performed on the issue entries from the second verification layer to generate a high-credibility issue list, including: The issue entries from the second verification layer are reviewed, including the consistency between the issue description and the actual code, as well as the reasonableness of the suggested fixes. After the review is completed, all the problem entries that failed the review are removed from the problem entries from the second verification layer to form the high credibility problem list.

[0013] Optionally, generating a final review report based on the list of high-credibility issues includes: During the analysis of code change sets, identify excellent code snippets that meet preset best practice criteria; The descriptions and location information of the excellent code snippets are integrated with the list of high-credibility issues to form the final review report.

[0014] Secondly, embodiments of the present invention also provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0015] Thirdly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the above-described method.

[0016] This invention provides a method, device, and medium for AI code review quality control based on multi-level verification. The method includes: receiving a set of code changes to be reviewed; inputting the set of code changes into a pre-trained AI code review model to obtain an initial problem list generated by the AI ​​code review model; inputting the initial problem list into a concatenated three-layer verification architecture for processing, the three-layer verification architecture including a first verification layer, a second verification layer, and a third verification layer connected in sequence; in the first verification layer, verifying the factual accuracy of problem items in the initial problem list, and outputting the verified problem items to the second verification layer; in the second verification layer, verifying the logical rationality of problem items from the first verification layer, and outputting the verified problem items to the third verification layer; in the third verification layer, performing a comprehensive final quality review of problem items from the second verification layer to generate a high-credibility problem list; and generating a final review report based on the high-credibility problem list. This invention transforms the single output of an artificial intelligence model into a systematic verification process by constructing a mandatory concatenated multi-level verification architecture. This architecture first verifies the factual accuracy of issues identified by the model, ensuring that all identified issues are based on objectively existing code content, thereby eliminating fictitious issues caused by model illusions. Furthermore, it verifies the logical rationality of confirmed issues, assessing the necessity and constructiveness of their fixes in conjunction with the code context, effectively filtering out a large number of false alarms that are reasonable implementations but misjudged. Finally, a comprehensive final quality audit reviews and balances the aforementioned results, ensuring the rigor and stability of the output conclusions. This invention overcomes the shortcomings of existing AI code review technologies, such as frequent factual errors, high false positive rates, and unreliable quality. Through a process-oriented, layer-by-layer screening and verification, it produces review reports with high factual accuracy, low false positive rates, and stable verifiability, meeting the credibility requirements of code quality tools in enterprise-level production environments. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating an AI code review quality control method based on multi-level verification, provided for an embodiment of the present invention; Figure 2 This is a schematic block diagram of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0021] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0022] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0023] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0024] Please see Figure 1 This invention provides a quality control method for AI code review based on multi-level verification. This invention constructs a mandatory, serial verification pipeline to systematically process the raw output of the AI ​​model, thereby improving the credibility of the review results. Specifically, this invention includes the following steps: S1 receives the code change set to be reviewed.

[0025] In practice, the system receives the code change set to be reviewed through an application programming interface (API) integrated with a version control system (such as Git). This code change set is typically represented as a diff file, which contains the added, modified, or deleted lines of code and their context.

[0026] S2, input the code change set into the pre-trained AI code review model to obtain the original problem list generated by the AI ​​code review model.

[0027] In practice, the system takes the received set of code changes as input and passes it to a pre-trained AI code review model. This AI code review model can be built on a large language model, for example, by fine-tuning a pre-trained language model based on the Transformer architecture for code review tasks.

[0028] Furthermore, after analyzing the input set of code changes, the AI ​​code review model outputs a raw list of issues. Each entry in this list, i.e., an issue entry, includes at least an issue description generated by the AI ​​model, the line number of the code suspected of having an issue, a fix suggestion given by the AI ​​code review model, and optionally an initial severity label (such as blocking, severe, minor).

[0029] S3, the original problem list is input into a three-layer verification architecture for processing, the three-layer verification architecture includes a first verification layer, a second verification layer and a third verification layer connected in sequence.

[0030] In practice, the system sends the original list of questions into a serialized three-layer verification architecture for processing. This three-layer verification architecture logically consists of a first verification layer, a second verification layer, and a third verification layer connected in sequence, with data flowing unidirectionally from the first layer to the third layer.

[0031] S4, in the first verification layer, verify the factual accuracy of the question entries in the original question list, and output the verified question entries to the second verification layer.

[0032] In practice, in the first verification layer, the system performs factual accuracy verification on each question entry in the original question list. The core purpose of this factual accuracy verification is to confirm whether the questions reported by the AI ​​are based on objectively existing code facts.

[0033] During implementation, the system iterates through the list, extracts the suspected problematic code line number for each problem entry, locates the corresponding original file in the code repository based on the line number, and reads the actual code snippet corresponding to that line number.

[0034] Furthermore, the system compares the code snippet content recorded in the problem entry with the actual code snippet at the string or semantic level. If the comparison matches, the problem entry passes the factual basis verification of this step and is allowed to be output to the second verification layer for further processing.

[0035] Furthermore, if there is a discrepancy, it indicates that the AI ​​may be hallucinating or mislocating, and the system marks the problematic entry as failing verification.

[0036] In some preferred embodiments, the factual accuracy verification of the problem entries in the original problem list in the first verification layer includes: extracting a problem entry from the original problem list, the problem entry containing the line number and code snippet content of the suspected problem code; obtaining the actual code snippet corresponding to the line number based on the original file in the code repository; comparing the code snippet content in the problem entry with the actual code snippet; and marking the problem entry as failing verification if the code snippet content is inconsistent with the actual code snippet.

[0037] In practice, the system extracts one question entry from the original question list, either sequentially or in parallel. This question entry includes the suspected question code line number generated by the AI ​​code review model, as well as the code snippet referenced by the AI ​​when generating the question. This code snippet may be a single line of code or a short, multi-line block of code; this invention does not specifically limit its scope.

[0038] Furthermore, the system performs code location operations based on the original source code files in the code repository. The system navigates to the corresponding source file based on the line number provided in the issue entry and precisely reads the line of code pointed to by that line number. If the issue entry references a multi-line code snippet, the system reads all lines of code within the corresponding line number range to obtain the actual code snippet.

[0039] Further, the system performs a comparison operation. It compares the string of code snippets parsed from the problem entry with the actual string of code snippets read from the source code file. The comparison can be an exact string match or a semantic match that ignores non-essential differences such as whitespace and comments. In one implementation, the system compares the two strings character by character to see if they are completely identical.

[0040] Furthermore, the system makes a judgment based on the comparison results. If the comparison finds that the code snippet content is inconsistent with the actual code snippet, such as the presence of extra characters, missing key symbols, or completely incorrect lines of code, the system determines that the problem item lacks factual basis and marks its status as verification failed.

[0041] Furthermore, the marking can be done by setting a flag in the data structure or by moving it to a separate list of failed entries. Only problematic entries that pass this comparison will be retained and passed to subsequent verification steps or a second verification layer.

[0042] This embodiment establishes a fundamental fact-checking defense for AI code review through a mandatory code fact comparison mechanism, effectively intercepting and eliminating the most basic factual errors caused by AI model illusions or output errors. For example, when an AI model fabricates a non-existent function call and reports it as a problem, this step can immediately detect the fabrication by comparing it with the actual code. Similarly, when an AI model incorrectly associates line numbers or references irrelevant code snippets due to misunderstanding, this step can also identify it through precise matching.

[0043] In some preferred embodiments, the factual accuracy verification of the problem entries in the original problem list in the first verification layer further includes: when the code snippet content is consistent with the actual code snippet, determining the software development framework used by the code change set; inputting the code snippet content into a predefined framework rule base, the framework rule base containing syntax rules corresponding to the software development framework; parsing and verifying the code snippet content using the syntax rules; and marking the problem entry as failing verification if the code snippet content violates the syntax rules.

[0044] In practice, after comparing the code snippet content with the actual code snippet, if the comparison result is consistent, the system continues to execute the following steps: The system determines the software development framework used by the current set of code changes to be reviewed. This can be achieved in several ways, such as analyzing project configuration files, parsing specific import statements or annotations in the code, or obtaining it through pre-defined project metadata information.

[0045] Furthermore, the system inputs the code snippets that have passed the consistency comparison into a predefined framework rule base. This framework rule base is a knowledge base that stores specific syntax rules and best practice patterns for various popular frameworks (such as Spring, MyBatis, Django, and React). Based on the software development framework determined in the previous step, the system loads the syntax rule set corresponding to that framework from the rule base.

[0046] Furthermore, the system uses the loaded syntax rules to parse and validate the input code snippets. For example, for the Java Spring framework, the rules might include checking whether the use of the `@Autowired` annotation conforms to best practices for constructor injection; for MyBatis, the rules might include checking whether the parameter placeholders `#{}` in the SQL statements of the XML mapping file are formatted correctly. The validation process can be completed by calling a dedicated static analysis engine or rule matching algorithm.

[0047] Furthermore, the system makes decisions based on the verification results. If the parsing and verification process finds that the code snippet violates the loaded syntax rules corresponding to the framework, it indicates that the AI ​​model may have failed to correctly understand the framework constraints, or that the framework rule base has identified a deeper problem that the AI ​​has not discovered. In this case, the system marks the problem entry as a failed verification because the problem it points out may be based on a misunderstanding, or its description may not accurately reflect the actual framework specification violation.

[0048] It should be noted that the framework rule base can be constructed by parsing the official documentation of the open-source framework, extracting common code patterns, and combining rules from static analysis tools; this invention does not specifically limit this.

[0049] This embodiment addresses the knowledge blind spots and comprehension biases inherent in general AI models when reviewing code for specific technology stacks by introducing framework-aware, specialized verification. While general AI models possess extensive code knowledge, their understanding of complex conventions, annotation meanings, or configuration syntax specific to certain frameworks may be imprecise, leading to two consequences: first, misjudging correct code conforming to framework specifications as incorrect (generating false positives); and second, failing to identify genuine framework usage anti-patterns (generating false negatives). The technical advantage of this embodiment lies in its secondary verification through a predefined, high-precision framework rule base, effectively correcting AI model judgment errors in framework-specific domains. It utilizes domain knowledge to calibrate and supplement the AI's initial judgments, significantly improving the accuracy and professionalism of code review in complex framework environments. This allows the entire system to better adapt to diverse technology stacks in enterprise-level projects, providing more targeted and reliable review opinions.

[0050] In some preferred embodiments, the method further includes: in the first verification layer, removing all problem entries marked as failing verification from the processing flow.

[0051] In practice, this embodiment clarifies the handling method for problematic items that fail verification in the first verification layer, ensuring the cleanliness and efficiency of the verification pipeline.

[0052] Specifically, in the first verification layer, after verifying the accuracy of the aforementioned facts for all entries in the original problem list, the system performs a cleanup step. The system checks the status flags of all problem entries and identifies those marked as failing verification.

[0053] Furthermore, the system removes these entries marked as failing validation from the currently active processing flow. In implementation, this can be accomplished through various data structure operations. For example, the system can maintain two lists: one for storing entries that pass validation (output list), and another for storing entries that fail (discard list). During validation, entries that meet the criteria are added to the output list, and those that fail are added to the discard list. After validation, only the output list is passed to the subsequent second validation layer. Another implementation is to operate directly on the original list, removing failed entries from the list or setting their status to "filtered," so that subsequent processing layers only iterate over and process entries with a status of "passed."

[0054] Furthermore, this removal operation means that these problematic entries that failed the first verification layer will not proceed to the second verification layer for logical rationality verification, nor will they appear in the final review report. They may be logged for auditing purposes, but will no longer participate in the main process.

[0055] This embodiment establishes the principle of filtering out invalid issues early in the verification pipeline by implementing an explicit removal mechanism at the end of the first verification layer. This can promptly clean up issue entries that are proven to be based on fictitious facts or violate the basic framework specifications, preventing these invalid information from continuing to occupy subsequent verification resources that are more complex and computationally more expensive (such as logical analysis and global dependency analysis in the second verification layer).

[0056] S5, in the second verification layer, the logical rationality of the question items from the first verification layer is verified, and the question items that pass the verification are output to the third verification layer.

[0057] In practice, in the second verification layer, the system performs logical rationality verification on each issue item from the first verification layer. This stage is no longer limited to whether the code exists, but rather assesses whether the issues pointed out by the AI ​​are logically necessary to fix, and whether the suggested fixes are reasonable. The system analyzes the context of the issue item, such as considering the intent of the code change, the project's historical code patterns, and common business logic. For example, for AI-suggested code optimizations, the system assesses whether they will substantially improve performance or introduce unnecessary complexity. Issue items that pass this logical verification are output to the third verification layer.

[0058] In some preferred embodiments, the second verification layer performs logical rationality verification on the problem entries from the first verification layer, including: filtering problem entries from the problem entries from the first verification layer whose repair suggestion is to delete a specific code segment; performing dependency analysis on the complete codebase where the code change set is located to identify other code locations that directly or indirectly reference the specific code segment; and marking the problem entry as failing verification when other code locations that reference the specific code segment are identified.

[0059] In practice, the system first iterates through the list of issue entries from the first verification layer, filtering them based on the repair suggestion field within each entry. The system identifies entries whose core repair suggestion involves deleting a specific code segment. For example, the repair suggestion might be described as removing an unused variable, deleting a piece of logic considered redundant, or cleaning up a print statement.

[0060] Furthermore, for each selected issue item suggesting the deletion of a specific code segment, the system initiates dependency analysis. The analysis scope is not limited to the current code changeset but extends to the entire codebase or project containing that changeset. The system uses code analysis tools (such as an abstract syntax tree-based analyzer) to scan the entire codebase, identifying all other code locations that directly or indirectly reference the specific code segment to be deleted. Direct references include calls to functions within that code segment, and the use of variables or classes defined within that code segment; indirect references may include connections established through reflection, configuration file associations, or dependency injection.

[0061] Furthermore, the system makes a judgment based on the results of dependency analysis. When the analysis engine identifies that at least one other code location references the specific code segment to be deleted, the system determines that the deletion recommendation is high-risk. This is because performing this deletion operation may break existing functionality, leading to runtime errors or compilation failures.

[0062] Furthermore, the system will mark entries deemed high-risk for deletion recommendations as failing verification. Once marked, the entry can be discarded or downgraded with a warning indicating a dependency was found, but it will not be output as a direct, safe remediation recommendation.

[0063] It should be noted that dependency analysis can be performed using cross-file call chain analysis techniques based on abstract syntax trees; best practice identification can be achieved based on preset coding standard templates and pattern matching algorithms.

[0064] This embodiment introduces global dependency analysis during logical rationality verification, providing a security mechanism specifically for the high-risk suggestion type of code deletion. AI code review models often rely on static analysis of local code snippets (such as assuming variables are unused or methods can be inlined) when suggesting code deletion, lacking a global perspective on the overall dependency structure of the codebase. This limitation easily leads to destructive suggestions, such as deleting a critical function that appears not to be called by the current module but is actually depended upon by other modules through interfaces or event mechanisms. This embodiment analyzes the dependency graph of the complete codebase to proactively identify and intercept such potentially destructive deletion suggestions, effectively preventing the introduction of new defects or system crashes due to adopting AI suggestions.

[0065] In some preferred embodiments, the method further includes: for problem entries that pass the logical rationality verification, obtaining the problem type identified in the problem entry, the problem type including one of security vulnerability, performance defect, code style violation, or logical error; analyzing the logical role of the code segment corresponding to the problem entry in the business system, and determining the positional importance level based on the logical role; adjusting the original severity label in the problem entry based on the problem type and the positional importance level; associating the adjusted severity label with the problem entry and outputting it to the third verification layer; for problem entries that fail the logical rationality verification, adding explanatory comments indicating that the logical verification failed to the problem entry and discarding it from the processing flow.

[0066] In practice, after verifying the logical rationality of the problem item in the second verification layer, the system executes the following branch processing: For each issue that passes the logical plausibility verification, the system first retrieves the issue type identified in that issue. The issue type can be a predefined category, such as security vulnerabilities, performance defects, code style violations, or logical errors. This information may be provided by the AI ​​model when generating the issue, or it may be automatically categorized by the system during the verification process based on the issue description.

[0067] Furthermore, the system analyzes the logical role of the code segment corresponding to the problem item within the overall business system. For example, by analyzing the code's call chain, its module (such as the user authentication module, payment core module, or log tool module), and the sensitivity of the data it processes (such as personal identification information or transaction amounts), the system determines the importance level of the code's location. One implementation approach is to establish a code module importance mapping table, or to quantify its importance by analyzing the number of callers and the frequency of calls.

[0068] Furthermore, based on the acquired problem type and the analyzed importance level of the location, the system adjusts the original severity labels in the problem entries. The adjustment logic could be as follows: for the same problem type (such as code style violation), if it appears in the code of the core payment module, its severity might be upgraded from "minor" to "serious"; while for a performance defect, if it appears in a rarely executed utility function, its severity might be downgraded. The adjustment algorithm can be based on a preset rule matrix or a weighted calculation model.

[0069] Furthermore, the system associates the adjusted new severity labels with the issue entries, overwriting or supplementing the original labels, and outputs the updated issue entries to the third verification layer.

[0070] Furthermore, for entries that fail the logical rationality verification, the system adds a step before discarding them: adding an explanatory comment explaining the failure to the logical verification. The comment can briefly explain the reason for the failure, such as the deletion suggestion being rejected due to external dependencies. After adding the comment, the system discards the entry from the main processing flow, but the commented entry may be stored in the history or feedback log for subsequent analysis of AI model shortcomings.

[0071] This embodiment achieves personalized calibration of review comments and effective retention of process information through refined processing of the output results of the second verification layer. For issues that pass verification, the severity is adjusted by combining the importance of issue type and code location, making the priority ranking in the final report more in line with actual business risks and providing developers with more accurate remediation guidance. For issues that fail verification, they are discarded after adding explanatory comments, which not only filters out invalid information but also generates valuable feedback data.

[0072] S6, in the third verification layer, a comprehensive quality review is conducted on the issue items from the second verification layer to generate a list of highly credible issues.

[0073] In practice, the third verification layer performs a comprehensive final quality review of all issue items from the second verification layer. This layer serves as the final quality checkpoint, aiming to comprehensively control and confirm the results of the first two verification layers, ensuring the overall reliability and stability of the output. This final review may include checking the standardization of issue descriptions, merging similar issues, and finalizing the format of the output report structure. After completing the final review, the system generates a high-credibility issue list, which includes issues that have passed all the aforementioned verification stages.

[0074] This final review stage places particular emphasis on zero tolerance for fabricated issues. Even if an issue has passed the first two layers of verification, if any uncertainty or contradiction is found in its connection to the final anchoring of the code facts during this review, the issue will be forcibly excluded, ensuring that every item in the high-credibility issue list has an indisputable factual basis. Simultaneously, this layer also controls the overall balance of the report, preventing it from presenting only criticisms.

[0075] For example, in some preferred embodiments, the step of performing a comprehensive final quality review on the issue entries from the second verification layer in the third verification layer to generate a high-confidence issue list includes: reviewing the issue entries from the second verification layer, the review including the consistency between the issue description and the code facts and the reasonableness of the repair suggestions; after the review is completed, removing all issue entries that failed the review from the issue entries from the second verification layer to form the high-confidence issue list.

[0076] In practice, the system first reviews the set of issue entries from the second verification layer. The review focuses on the consistency between the issue description and the underlying code facts, as well as the reasonableness of the suggested fixes. This review is particularly focused on identifying any remaining, fictitious issues based on AI illusions. This can be seen as a quality check of the first two layers of verification work. For example, the consistency review between the issue description and the code facts specifically includes: re-acquiring the actual code segment corresponding to the line number pointed to by the issue entry, comparing the code content corresponding to the issue description recorded in the issue entry with the actual code segment; if the code content, line number, or logical condition referenced in the issue description does not match the actual code segment, the review is deemed unsuccessful. Furthermore, the rationality review of the repair suggestions specifically includes: analyzing the technical feasibility of the repair suggestions provided in the problem item, including whether the repair suggestions conform to the syntax rules of the selected programming language, whether they are compatible with the software development framework used in the project, and whether they are operable in the existing code context; if the repair suggestions have syntax errors, framework conflicts, or are not operable in the context, the review is deemed to have failed. Furthermore, after all review work (whether it's sampling review or full review) is completed, the system performs a final screening based on the review results. After the review is completed, all problem entries that failed the review are removed from the problem entries from the second verification layer, forming the high-confidence problem list. In this invention, the system removes, one by one, entries from the problem entry set from the second verification layer that were deemed to have unclear factual basis or be fabricated in this review. This operation is a mandatory step, designed to achieve zero tolerance for fabricated problems. Criteria for judging unclear factual basis include, but are not limited to: inaccurate code references, discrepancies between the problem description and the actual code performance, and technically unfeasible repair suggestions.

[0077] Furthermore, the system will integrate the remaining problematic items, after removing invalid entries and undergoing multiple rounds of verification and final review, to form a final list of highly credible issues. This list represents the review conclusions that the system believes can be confidently presented to users.

[0078] In addition, as part of the comprehensive final quality review, this layer is also responsible for ensuring the balance of the final review report structure, that is, the report not only lists the issues to be improved, but also clearly identifies and affirms good practices in the code by integrating the results of the best practice identification module.

[0079] S7. Based on the list of high-credibility issues, generate the final review report.

[0080] In practice, the system organizes and generates a final review report based on the generated list of high-credibility issues, following a predefined report template. This report may include an issue summary, a detailed list of issues (including line numbers, descriptions, and recommendations), and is delivered to the developers.

[0081] This embodiment transforms the originally one-off, black-box AI code review process into a multi-stage, traceable, and standardized quality control process by constructing a three-layered, interconnected verification architecture. The first verification layer, through mandatory comparison of line numbers and code content, fundamentally anchors the factual basis of review opinions, effectively filtering out false issues without code basis caused by model illusions, thereby directly improving the accuracy of the output results at the factual level. The second verification layer introduces logical rationality judgment, enabling the system to go beyond simple pattern matching and assess the substantial impact of problems and the value of remedial suggestions in conjunction with code context. This filters or downgrades a large number of false positives that are reasonable implementations but misjudged by AI, significantly reducing the false positive rate of reports. The third verification layer, as the final quality checkpoint, ensures the uniformity of output format and the final confirmation of conclusions through comprehensive final review, enhancing the stability and reliability of the entire process. In summary, this invention, through a hierarchical and progressive verification mechanism, systematically solves the problems of factual inaccuracies, logical misjudgments, and quality fluctuations in AI code review, transforming AI from an unreliable auxiliary tool into an automated quality assurance system capable of producing stable and credible review results.

[0082] In some preferred embodiments, generating a final review report based on the high-confidence issue list includes: identifying excellent code snippets that meet preset best practice standards during the analysis of the code change set; and integrating the description and location information of the excellent code snippets with the high-confidence issue list to form the final review report.

[0083] In practice, throughout the entire process of analyzing code change sets (e.g., during the initial analysis phase of the AI ​​model, or during a certain layer of verification), best practice identification is performed simultaneously. The system scans the code change set according to a pre-defined library of best practice standards. Best practice standards may include: using efficient algorithms, implementing clear error handling, writing expressive variable names, following specific design patterns, or including valuable unit tests, etc., but this invention does not specifically limit these aspects.

[0084] Furthermore, the system will identify excellent code snippets that meet these preset best practice criteria and record the description of the snippet (e.g., "uses a thread-safe concurrent collection class") and its location information in the code (e.g., file path and line number range).

[0085] Furthermore, during the report generation phase, the system integrates the descriptions and location information of the identified excellent code snippets with the aforementioned list of high-credibility issues. Integration means organizing these two types of information into the same report document or data structure. The report is typically divided into different chapters or sections, such as "Issues Found" and "Good Practices." The good practices section, like the issue list, provides specific code locations and brief positive evaluations.

[0086] Furthermore, the system will integrate reports that contain descriptions of high-credibility issues and best practices as a complete final review report output.

[0087] This embodiment transforms the review output from simply "finding errors" to "comprehensive evaluation" by incorporating the identification of best practices and positive evaluation into the final review report. Traditional automated review tools and pure AI solutions typically focus only on discovering defects and violations, resulting in reports that are emotionally one-way criticisms, easily triggering developer resistance and hindering tool adoption and team collaboration. This embodiment, while ensuring the accuracy of issue review, adds the identification and encouragement of good coding behavior. This balanced output makes the review report more constructive and instructive, not only helping developers avoid errors but also reinforcing correct coding patterns.

[0088] Please see Figure 2 , Figure 2 This is a schematic block diagram of a computer device provided in an embodiment of this application. The computer device 500 can be a terminal or a server, wherein the server can be a standalone server or a server cluster composed of multiple servers.

[0089] The computer device 500 includes a processor 502, a memory, and a network interface 505 connected via a system bus 501. The memory may include a non-volatile storage medium 503 and internal memory 504.

[0090] The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. When the computer program 5032 is executed, it causes the processor 502 to execute an AI code review quality control method based on multi-level verification.

[0091] The processor 502 provides computing and control capabilities to support the operation of the entire computer device 500.

[0092] The internal memory 504 provides an environment for the execution of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute an AI code review quality control method based on multi-level verification.

[0093] The network interface 505 is used for network communication with other devices. Those skilled in the art will understand that the above structure is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device 500 to which the present application is applied. A specific computer device 500 may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements.

[0094] The processor 502 is used to run a computer program 5032 stored in the memory to perform the following steps: Receive code change sets pending review; The code change set is input into a pre-trained AI code review model to obtain the original problem list generated by the AI ​​code review model; The original question list is processed by a three-layer verification architecture, which includes a first verification layer, a second verification layer, and a third verification layer connected in sequence. In the first verification layer, the factual accuracy of the question entries in the original question list is verified, and the question entries that pass the verification are output to the second verification layer. In the second verification layer, the logical rationality of the question entries from the first verification layer is verified, and the question entries that pass the verification are output to the third verification layer. In the third verification layer, a comprehensive quality review is conducted on the issue items from the second verification layer to generate a list of highly credible issues; Based on the list of highly credible issues, a final review report is generated.

[0095] Optionally, in the first verification layer, the factual accuracy verification of the question entries in the original question list includes: Extract one problem entry from the original problem list. The problem entry contains the line number and code snippet content of the suspected problem code. Based on the original files in the code repository, obtain the actual code snippet corresponding to the line number; Compare the code snippet content in the problem entry with the actual code snippet; If the code snippet content is inconsistent with the actual code snippet, the problem entry will be marked as verification failed.

[0096] Optionally, the factual accuracy verification of the question entries in the original question list in the first verification layer further includes: When the code snippet content matches the actual code snippet, determine the software development framework used by the code changeset. The code snippet is input into a predefined framework rule base, which contains syntax rules corresponding to the software development framework. The code snippet content is parsed and validated using the aforementioned syntax rules; If the code snippet violates the syntax rules, the issue entry will be marked as failing verification.

[0097] Optionally, in the second verification layer, the logical rationality verification of the question entries from the first verification layer includes: From the issue entries from the first verification layer, filter out issue entries whose remediation suggestion is to delete a specific code segment; Perform dependency analysis on the complete codebase containing the code change set to identify other code locations that directly or indirectly reference the specific code segment. When other code locations that reference the specific code segment are identified, the issue entry is marked as failing verification.

[0098] Optionally, the method further includes: In the first verification layer, all issues marked as failing verification are removed from the processing flow.

[0099] Optionally, the method further includes: For each issue item that passes the logical rationality verification, the issue type identified in the issue item is obtained. The issue type includes one of the following: security vulnerability, performance defect, code style violation, or logical error. The logical role of the code segment corresponding to the issue item in the business system is analyzed, and the positional importance level is determined based on the logical role. Based on the issue type and the positional importance level, the original severity label in the issue item is adjusted. The adjusted severity label is associated with the issue item and output to the third verification layer. For issues that fail the logical rationality verification, add explanatory notes explaining the failure to the issue and then discard it from the processing flow.

[0100] Optionally, in the third verification layer, a comprehensive quality review is performed on the issue entries from the second verification layer to generate a high-credibility issue list, including: The issue entries from the second verification layer are reviewed, including the consistency between the issue description and the actual code, as well as the reasonableness of the suggested fixes. After the review is completed, all the problem entries that failed the review are removed from the problem entries from the second verification layer to form the high credibility problem list.

[0101] Optionally, generating a final review report based on the list of high-credibility issues includes: During the analysis of code change sets, identify excellent code snippets that meet preset best practice criteria; The descriptions and location information of the excellent code snippets are integrated with the list of high-credibility issues to form the final review report.

[0102] It should be understood that in the embodiments of this application, the processor 502 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0103] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program may be stored in a storage medium, which is a computer-readable storage medium. The computer program is executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0104] Therefore, the present invention also provides a storage medium. This storage medium can be a computer-readable storage medium. The storage medium stores a computer program. When executed by a processor, the computer program causes the processor to perform the following steps: Receive code change sets pending review; The code change set is input into a pre-trained AI code review model to obtain the original problem list generated by the AI ​​code review model; The original question list is processed by a three-layer verification architecture, which includes a first verification layer, a second verification layer, and a third verification layer connected in sequence. In the first verification layer, the factual accuracy of the question entries in the original question list is verified, and the question entries that pass the verification are output to the second verification layer. In the second verification layer, the logical rationality of the question entries from the first verification layer is verified, and the question entries that pass the verification are output to the third verification layer. In the third verification layer, a comprehensive quality review is conducted on the issue items from the second verification layer to generate a list of highly credible issues; Based on the list of highly credible issues, a final review report is generated.

[0105] Optionally, in the first verification layer, the factual accuracy verification of the question entries in the original question list includes: Extract one problem entry from the original problem list. The problem entry contains the line number and code snippet content of the suspected problem code. Based on the original files in the code repository, obtain the actual code snippet corresponding to the line number; Compare the code snippet content in the problem entry with the actual code snippet; If the code snippet content is inconsistent with the actual code snippet, the problem entry will be marked as verification failed.

[0106] Optionally, the factual accuracy verification of the question entries in the original question list in the first verification layer further includes: When the code snippet content matches the actual code snippet, determine the software development framework used by the code changeset. The code snippet is input into a predefined framework rule base, which contains syntax rules corresponding to the software development framework. The code snippet content is parsed and validated using the aforementioned syntax rules; If the code snippet violates the syntax rules, the issue entry will be marked as failing verification.

[0107] Optionally, in the second verification layer, the logical rationality verification of the question entries from the first verification layer includes: From the issue entries from the first verification layer, filter out issue entries whose remediation suggestion is to delete a specific code segment; Perform dependency analysis on the complete codebase containing the code change set to identify other code locations that directly or indirectly reference the specific code segment. When other code locations that reference the specific code segment are identified, the issue entry is marked as failing verification.

[0108] Optionally, the method further includes: In the first verification layer, all issues marked as failing verification are removed from the processing flow.

[0109] Optionally, the method further includes: For each issue item that passes the logical rationality verification, the issue type identified in the issue item is obtained. The issue type includes one of the following: security vulnerability, performance defect, code style violation, or logical error. The logical role of the code segment corresponding to the issue item in the business system is analyzed, and the positional importance level is determined based on the logical role. Based on the issue type and the positional importance level, the original severity label in the issue item is adjusted. The adjusted severity label is associated with the issue item and output to the third verification layer. For issues that fail the logical rationality verification, add explanatory notes explaining the failure to the issue and then discard it from the processing flow.

[0110] Optionally, in the third verification layer, a comprehensive quality review is performed on the issue entries from the second verification layer to generate a high-credibility issue list, including: The issue entries from the second verification layer are reviewed, including the consistency between the issue description and the actual code, as well as the reasonableness of the suggested fixes. After the review is completed, all the problem entries that failed the review are removed from the problem entries from the second verification layer to form the high credibility problem list.

[0111] Optionally, generating a final review report based on the list of high-credibility issues includes: During the analysis of code change sets, identify excellent code snippets that meet preset best practice criteria; The descriptions and location information of the excellent code snippets are integrated with the list of high-credibility issues to form the final review report.

[0112] The storage medium is a physical, non-transient storage medium, such as a USB flash drive, external hard drive, read-only memory (ROM), magnetic disk, or optical disk, or any other physical storage medium capable of storing program code. The computer-readable storage medium can be non-volatile or volatile.

[0113] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0114] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0115] The steps in the method of this invention can be adjusted, merged, or reduced in order according to actual needs. The units in the device of this invention can be merged, divided, or reduced according to actual needs. Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0116] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0117] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0118] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Since these modifications and variations fall within the scope of the claims and their equivalents, this invention also intends to include these modifications and variations.

[0119] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A quality control method for AI code review based on multi-level verification, characterized in that, include: Receive code change sets pending review; The code change set is input into a pre-trained AI code review model to obtain the original problem list generated by the AI ​​code review model; The original question list is processed by a three-layer verification architecture, which includes a first verification layer, a second verification layer, and a third verification layer connected in sequence. In the first verification layer, the factual accuracy of the question entries in the original question list is verified, and the question entries that pass the verification are output to the second verification layer. In the second verification layer, the logical rationality of the question entries from the first verification layer is verified, and the question entries that pass the verification are output to the third verification layer. In the third verification layer, a comprehensive quality review is conducted on the issue items from the second verification layer to generate a list of highly credible issues; Based on the list of highly credible issues, a final review report is generated.

2. The AI ​​code review quality control method based on multi-level verification according to claim 1, characterized in that, In the first verification layer, the factual accuracy of the question entries in the original question list is verified, including: Extract one problem entry from the original problem list. The problem entry contains the line number and code snippet content of the suspected problem code. Based on the original files in the code repository, obtain the actual code snippet corresponding to the line number; Compare the code snippet content in the problem entry with the actual code snippet; If the code snippet content is inconsistent with the actual code snippet, the problem entry will be marked as verification failed.

3. The AI ​​code review quality control method based on multi-level verification according to claim 2, characterized in that, The factual accuracy verification of the question entries in the original question list in the first verification layer also includes: When the code snippet content matches the actual code snippet, determine the software development framework used by the code changeset. The code snippet is input into a predefined framework rule base, which contains syntax rules corresponding to the software development framework. The code snippet content is parsed and validated using the aforementioned syntax rules; If the code snippet violates the syntax rules, the issue entry will be marked as failing verification.

4. The AI ​​code review quality control method based on multi-level verification according to claim 1, characterized in that, In the second verification layer, the logical rationality of the question items from the first verification layer is verified, including: From the issue entries from the first verification layer, filter out issue entries whose remediation suggestion is to delete a specific code segment; Perform dependency analysis on the complete codebase containing the code change set to identify other code locations that directly or indirectly reference the specific code segment. When other code locations that reference the specific code segment are identified, the issue entry is marked as failing verification.

5. The AI ​​code review quality control method based on multi-level verification according to claim 1, characterized in that, The method further includes: In the first verification layer, all issues marked as failing verification are removed from the processing flow.

6. The AI ​​code review quality control method based on multi-level verification according to claim 1, characterized in that, The method further includes: For each issue item that passes the logical rationality verification, the issue type identified in the issue item is obtained. The issue type includes one of the following: security vulnerability, performance defect, code style violation, or logical error. The logical role of the code segment corresponding to the issue item in the business system is analyzed, and the positional importance level is determined based on the logical role. Based on the issue type and the positional importance level, the original severity label in the issue item is adjusted. The adjusted severity label is associated with the issue item and output to the third verification layer. For issues that fail the logical rationality verification, add explanatory notes explaining the failure to the issue and then discard it from the processing flow.

7. The AI ​​code review quality control method based on multi-level verification according to claim 1, characterized in that, In the third verification layer, a comprehensive quality review is conducted on the issue items from the second verification layer to generate a list of highly credible issues, including: The issue entries from the second verification layer are reviewed, including the consistency between the issue description and the actual code, as well as the reasonableness of the suggested fixes. After the review is completed, all the problem entries that failed the review are removed from the problem entries from the second verification layer to form the high credibility problem list.

8. The AI ​​code review quality control method based on multi-level verification according to claim 1, characterized in that, The final review report, generated based on the list of high-credibility issues, includes: During the analysis of code change sets, identify excellent code snippets that meet preset best practice criteria; The descriptions and location information of the excellent code snippets are integrated with the list of high-credibility issues to form the final review report.

9. A computer device, characterized in that, The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, can implement the method as described in any one of claims 1-8.

Images