Code defect reasoning instruction template generation method and system for large language model

Abstract

The application provides a code defect reasoning instruction template generation method and system for a large language model. According to the method of the application, a code defect is constructed into a defect event triple. A defect rule thinking chain is established based on the defect event triple. A structured defect reasoning instruction template is defined according to the defect rule thinking chain. The method further includes passing the structured defect reasoning instruction template as input to a large language model and guiding the large language model to perform code defect detection. The application formalizes and structures the defect detection task, making the input of the large language model more explicit and more reasoning-oriented, significantly reducing Prompt ambiguity, improving the understanding and judgment ability of the large language model in complex semantic scenarios, and constructing a general and expandable input standard that can be used with different detection tools. It is suitable for high-precision tasks in the defect detection field, reduces false positives, and improves controllability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of code defect detection, in particular to a code defect reasoning instruction template generation method and system for a large language model. BACKGROUND

[0002] Current static code analysis tools (such as Coverity, Fortify) are widely used in the industry, and based on abstract syntax tree matching and data flow analysis, code defect detection is completed, which has high accuracy, but poor scalability and flexibility, and it is difficult to cover complex semantic scenarios; at the same time, with the development of open source large language models (such as DeepSeek, Qwen), they have shown great ability in code generation, unit testing, defect repair, and annotation generation, etc. Tasks, triggering a new trend of applying them to code defect detection.

[0003] However, large models lack professional structured input guidance for defect detection tasks, and usually use natural language prompts to directly input code and problem descriptions, resulting in unstable model output and poor context capture ability, especially in handling defects with strong logic and complex patterns.

[0004] Existing research attempts to guide the model by adding a small amount of templated Prompt, such as: "Please find the bug in the following code and explain the reason" and "Point out the potential null pointer problem in the following function", but these prompts are ambiguous, the information structure is not clear, and there is a lack of systematic description of defect types, context, program execution process, etc. Elements, which cannot fully mobilize the reasoning ability of the model; in addition, existing solutions lack a unified event format and reasoning chain, which is not conducive to training stable large models. SUMMARY

[0005] The purpose of the present application is to provide a code defect reasoning instruction template generation method and system for a large language model, aiming to solve the problems of non-standard input, insufficient context understanding, and weak semantic reasoning ability of current large language models in code defect detection.

[0006] The code defect reasoning instruction template generation method for a large language model provided by the embodiment of the present application comprises:

[0007] Constructing a code defect as a defect event triple;

[0008] Establishing a defect rule thinking chain based on the defect event triple;

[0009] Defining a structured defect reasoning instruction template according to the defect rule thinking chain.

[0010] The embodiment of the present application provides a code defect reasoning instruction template generation system for a large language model, comprising:

[0011] A triple construction module is configured to construct a code defect into a defect event triple;

[0012] A thought chain establishment module is configured to establish a defect rule thought chain based on the defect event triple;

[0013] A template generation module is configured to define a structured defect reasoning instruction template according to the defect rule thought chain.

[0014] The embodiment of the present application also provides an electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the code defect reasoning instruction template generation method for a large language model when executed by the processor.

[0015] The embodiment of the present application also provides a computer readable storage medium, wherein the computer readable storage medium stores an information transmission implementation program, and the program is configured to implement the steps of the code defect reasoning instruction template generation method for a large language model when executed by a processor.

[0016] The embodiment of the present application can have the following beneficial effects: the embodiment of the present application formalizes and structures a defect detection task, makes the input of a large language model more explicit and more reasoning-oriented, significantly reduces the ambiguity of Prompt, improves the understanding and judgment ability of the model for complex semantic scenarios, and constructs an input standard with strong universality and scalability, which can be used with different detection tools, is suitable for high-precision tasks in the defect detection field, reduces the false alarm rate and improves controllability. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the one or more embodiments of the present application or the prior art, the drawings needed in the embodiment or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments described in the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0018] Figure 1 is a flow chart of the code defect reasoning instruction template generation method for a large language model of the embodiment of the present application;

[0019] Figure 2 is a schematic diagram of the code defect reasoning instruction template generation system for a large language model of the embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to make the person skilled in the art better understand the technical solutions in one or more embodiments of the present specification, the technical solutions in one or more embodiments of the present specification will be clearly and completely described below in conjunction with the drawings in one or more embodiments of the present specification. Obviously, the described embodiments are only a part of the embodiments of the present specification, not all. Based on one or more embodiments of the present specification, all other embodiments obtained by those skilled in the art without creative labor should belong to the protection scope of the present document.

[0021] Method embodiment

[0022] According to the embodiments of the present application, a code defect reasoning instruction template generation method for a large language model is provided, Figure 1 is the flow chart of the code defect reasoning instruction template generation method for a large language model of the embodiments of the present application, as Figure 1 shown, the code defect reasoning instruction template generation method for a large language model according to the embodiments of the present application specifically includes:

[0023] Step S101, the code defect is constructed into a defect event triple;

[0024] Among them, the defect event triple includes position, type and context;

[0025] Step S102, establishing a defect rule thinking chain based on the defect event triple;

[0026] Among them, the defect rule thinking chain is a hierarchical rule chain, including defect main type, defect sub-type and specific example;

[0027] Step S103, defining a structured defect reasoning instruction template according to the defect rule thinking chain;

[0028] Among them, the defect reasoning instruction template is a language-independent event template;

[0029] The defect reasoning instruction template includes task index, task, output requirement and output format;

[0030] The output requirement and the output format are self-defined according to actual needs;

[0031] The output format includes defect authenticity judgment, cause analysis and repair suggestion;

[0032] The method further includes:

[0033] The structured defect reasoning instruction template is inputted into a large language model, and the large language model is guided to perform code defect detection.

[0034] The technical solutions of the above-described embodiments of the present application will be described in detail in combination with the specific cases of a code defect reasoning instruction template generation method for a large language model of the embodiments of the present application.

[0035] Existing large language models mainly rely on natural language prompts for input, which has limited expression ability and lacks structured semantic information, resulting in low accuracy of model positioning and reasoning for code defects, especially in complex cross-function and semantic dependency scenarios, with high false positive rate and large false negative rate. The embodiments of the present application propose a structured instruction format, abstracts defect events into triples with unified semantic structure, and introduces hierarchical defect rule thinking chains to realize formal guidance of large language models for defect detection tasks, significantly improving the context perception ability and reasoning accuracy of the model. Specifically, it includes:

[0036] I. Defect event triple construction

[0037] Abstract code defects into formal triples (position, type, context):

[0038] A. Position: specific function, line number, code snippet.

[0039] B. Type: the defect rule it belongs to, such as null pointer dereference, resource leak, array out-of-bounds, etc.

[0040] C. Context: including variable dependency, control flow path, function call relationship, etc.

[0041] For example:

[0042] {

[0043] "Position": "function foo, line 27",

[0044] "Type": "null pointer dereference",

[0045] "Context": "variable ptr comes from external call, not checked for NULL"

[0046] }

[0047] II. Defect rule thinking chain construction

[0048] Establish a hierarchical rule chain from the main type of defect to the sub-type and then to the specific use case:

[0049] A. Main type: resource management defect

[0050] B. Sub-type: file not closed

[0051] Example chain:

[0052] Opening a file → manipulating the file (read / write) → failing to close the file → resource leak;

[0053] The thought chain uses natural language and structured entries to describe and embeds them in the model Prompt as reasoning guidance information.

[0054] III. Command Template Definition and Generation

[0055] At the beginning of the instructions, specify to the large model that the task is to determine whether the problem of the static tool is a positive or a false alarm; specify to the large model the output response specification (outputConstraint) and the example response format (outputFormat); insert the defect rule thinking chain described in step two above.

[0056] The instruction template example is as follows:

[0057] [Task Index]: Task Number

[0058] [Task]: This code has been detected by the tool and contains an issue (quality defect or security vulnerability). Below is the rule description and detection results. Please determine whether the defect / vulnerability exists and generate a remediation suggestion. Rule Name: Null Pointer Dereference; Rule Description: The pointer is checked to be null, but it is still dereferenced.

[0059] [Output Requirements] Note:

[0060] 1. When dealing with code file paths, use relative paths whenever possible to make it easier for other developers to view them;

[0061] 2. Please fix the issue of blank lines before and after the main code;

[0062] 3. Even if the problematic code contains multiple defects or issues, only the issues specified in the problem description and the process of defect generation must be fixed;

[0063] 4. Do not change anything other than fixing the issues specified in the problem description and defect occurrence process, including spaces and blank lines;

[0064] 5. Comments added after code modifications are descriptions of the changes and must be in Chinese.

[0065] 6. Please only return the complete, repaired code. Do not reply with any extra content or omissions.

[0066] 7. List the "Analysis of the True Causes of the Defect" in one go, without dividing it into multiple subheadings;

[0067] 8. "Repair suggestion" gives the pre-repair and post-repair code for the same code file at one time, not in multiple subheadings;

[0068] 9. Forced to use Chinese to reply.

[0069] [Output format] The following are two examples of replies, one is suspected to be a false positive, and the other is a real defect. For reference. When the authenticity judgment result is suspected to be a false positive, there is no need to provide repair suggestions; when the authenticity judgment result is that the defect is real, please provide repair suggestions.

[0070] [Example 1 of reply] (when the defect is suspected to be a false positive):

[0071] ##Defect authenticity judgment suspected false positive

[0072] ##Reason analysis of suspected false positive

[0073] 1. xxx 2. xxx

[0074] [Example 2 of reply] (when the defect is real):

[0075] ##Defect authenticity judgment defect real

[0076] ##Reason analysis of real defect

[0077] 1. xxx 2. xxx

[0078] ##Repair suggestion According to the rule description and the code segment of the defect generation process, the following is the repair suggestion:

[0079] ###Code file path: / path / to / file1.java

[0080] **Repair idea**: 1. xxx 2. xxx\n\n

[0081] **Code before modification**: <code content with line number>

[0082] **Modified Code**: <fixed code content with line number>

[0083] [Defect Type]: Null Pointer Dereference

[0084] [Defect Location]: Line 27 in function foo

[0085] [Context Information]: Variable ptr comes from calling function bar without checking for NULL

[0086] [Defect Rule Chain]:

[0087] 1. Function receives external pointer → 2. No null check → 3. Dereference operation → Possible program crash

[0088] Please determine whether the following code has the above defect and explain the reason:

[0089] <Code Snippet>

[0090] IV. An example of actual Instruction Prompt is as follows (json format)

[0091] {

[0092] "taskId": "261051605619843072",

[0093] "task": "This code has been detected by the tool, there is a problem (quality defect or security vulnerability), the following is the rule description and detection result, please judge whether the defect / vulnerability exists and generate repair suggestions.\n\nRule name: NULL_DEREFERENCE\nRule type: Null Pointer Dereference\nCWE: CWE-476\nRule description: The pointer is checked to be empty or is dereferenced.\n\n",

[0094] "outputConstraint" : "Note:\n1. Involve code file path, try to use relative path, convenient for other developers to view;\n2. Please fix the code to keep the blank line before and after the overall code;\n3. Even if the problem code has multiple defects or problems, you must only fix the problem described and the problem specified in the defect generation process;\n4. Do not change any other content, including spaces and blank lines, except for fixing the problem described and the problem specified in the defect generation process;\n5. Add comments to the modified code as change notes, and use Chinese uniformly;\n6. Be sure to return only the complete code after fixing, prohibit replying to any extra content, and prohibit any omission;\n7. "Defect real reason analysis" is listed once, do not divide into multiple subheadings;\n8. "Repair suggestion" for the same code file, give the code before and after the repair at one time, do not divide into multiple subheadings;\n9. Forced to reply in Chinese\n",

[0095] "outputFormat" : "The following are two examples of replies, one is suspected of false positives, and one is a real defect, for reference. When the authenticity judgment result is suspected of false positives, do not provide repair suggestions; when the authenticity judgment result is a real defect, please provide repair suggestions.\n\nReply Example 1 (when the defect is suspected of false positives):\n\n## Defect authenticity judgment\nSuspected false positive\n\n## Suspected false positive reason analysis\n1. xxx\n2. xxx\n\nReply Example 2 (when the defect is real):\n\n## Defect authenticity judgment\nDefect real existence\n\n## Defect real existence reason analysis\n1. xxx\n2. xxx\n\n## Repair suggestion\nAccording to the rule description and the code segment of the defect generation process, the following is the repair suggestion:\n\n### Code file path: / path / to / file1.java\n\n**Repair idea**:\n1. xxx\n2. xxx\n\n**Modified code**:\n <code content with line number>\n**Modified code**:\n<fixed code content with line number>\n",

[0096] "rule_name":"NULL_DEREFERENCE",

[0097] "event":"Related code snippet and defect generation process\n```\n / / / LOCAL / public / evaluation-ws / temp / OpenHarmony-v4.1-Release / OpenHarmony / third_party / skia / third_party / externals / icu / source / common / utrie2.cpp\n

[0098] \n455:}

[0099] \n456:if(enumValue==NULL) {

[0100] \n457:enumValue=enumSameValue;

[0101] \n458:}

[0102] \n459:\n[Event 1] In the condition trie->newTrie == NULL, trie->newTrie is checked for null.

[0103] \n460:if(trie->newTrie==NULL) {

[0104] \n461: / * frozen trie * /

[0105] \n462:idx=trie->index;

[0106] \n463:U_ASSERT(idx!=NULL); / *the following code assumes trie->newTrieis not NULL when idx is NULL * /

[0107] \n464: data32=trie->data32;

[0108] \n...\n552:}

[0109] \n553:for(;i2<i2Limit;++i2) {

[0110] \n554: if (idx!= NULL) {

[0111] \n555: block = (int32_t)idx[i2Block + i2] << UTRIE2_INDEX_SHIFT;

[0112] \n556:} else { \n[Event 2] Dereference null pointer trie->newTrie.

[0113] \n557: block = trie->newTrie->index2[i2Block + i2];

[0114] \n558:}

[0115] \n559:

[0116] if (block == prevBlock && (c - prev) >= UTRIE2_DATA_BLOCK_LENGTH) {

[0117] \n560: / * the block is the same as the previous one, and filled with prevValue * /

[0118] \n561: c += UTRIE2_DATA_BLOCK_LENGTH;

[0119] \n... \n```\n",

[0120] "think": "",

[0121] }

[0122] V. Key content of actual large model reply (json format)

[0123] {

[0124] "judgement": "Defect exists in reality",

[0125] "reason": "1. In line 460 of the code, `trie->newTrie` is checked for a null pointer.\n

[0126] 2. In line 557 of the code, `trie->newTrie` is dereferenced, although it has been confirmed to be a null pointer in the previous check.\n

[0127] 3. Such dereferencing of a null pointer can cause a program crash or undefined behavior, which is a serious code defect.

[0128] }

[0129] Six, multi-language adaptation and universality support

[0130] For C, C++, Java, Python, etc. Mainstream languages, build language-independent event templates, abstract syntax layer into a unified structure, realize reuse and promotion in multi-language scenarios.

[0131] Seven, tool support

[0132] Develop a set of instruction generator scripts to support automatic parsing of static code analysis tool detection results, combined with syntax analyzer to generate standardized defect triples and thought chains, and finally splice into the format of Instruction Prompt input to Qwen or DeepSeek large model.

[0133] In summary, the key points of the embodiment of the application are:

[0134] 1. Defect event triple construction method and its embedding method in Prompt;

[0135] 2. Hierarchical defect rule thought chain construction mechanism;

[0136] 3. Unified Instruction Prompt template structure design;

[0137] 4. Multi-language compatible format adaptation mechanism.

[0138] System embodiment

[0139] According to the embodiment of the application, a code defect reasoning instruction template generation system for large language model is provided, Figure 2 is the schematic diagram of the code defect reasoning instruction template generation system for large language model of the embodiment of the application, as Figure 2 shown, the code defect reasoning instruction template generation system for large language model according to the embodiment of the application specifically comprises:

[0140] The triple construction module 20 is used to construct code defects into defect event triples;

[0141] The thought chain establishment module 22 is used to establish a defect rule thought chain based on the defect event triples;

[0142] The template generation module 24 is used to define a structured defect reasoning instruction template according to the defect rule thought chain;

[0143] The system further comprises:

[0144] a template application module configured to pass the structured defect reasoning instruction template as input to a large language model and guide the large language model to perform code defect detection.

[0145] The embodiment of the present application is a system embodiment corresponding to the above-mentioned method embodiment, and the specific operation of each module can be understood with reference to the description of the method embodiment, which will not be repeated here.

[0146] To sum up, the beneficial effects of the embodiment of the present application include:

[0147] 1. Formalize and structure the defect detection task, so that the large language model input is more explicit and more reasoning-oriented;

[0148] 2. Significantly reduce Prompt ambiguity and improve the understanding and judgment ability of the model for complex semantic scenarios;

[0149] 3. Build a general and extensible input standard that can be used with different detection tools;

[0150] 4. Suitable for high-precision tasks in the defect detection field, reduce false positive rate and improve controllability.

[0151] Device embodiment one

[0152] The embodiment of the present application provides an electronic device, comprising: a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the computer program is executed by the processor to implement the steps as described in the method embodiment.

[0153] Device embodiment two

[0154] The embodiment of the present application provides a computer readable storage medium, and the computer readable storage medium stores an implementation program of information transmission, and the program is executed by a processor to implement the steps as described in the method embodiment.

[0155] The computer readable storage medium described in the embodiment includes but is not limited to ROM, RAM, magnetic disk or optical disk, etc.

[0156] Finally, it should be pointed out that: the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solution deviate from the scope of the technical solutions of the embodiments of the present application.< / code> < / code>

Claims

1. A method for generating instruction templates for code defect reasoning in large language models, characterized in that... include: Construct code defects into defect event triples; The defect event triple includes location, type, and context; wherein, the type is the defect rule to which it belongs; the context includes variable dependencies, control flow execution paths, and function call relationships related to the code defect, and the logical connections before and after the defect is generated must be clearly defined; A defect rule thinking chain is established based on the defect event triple; the defect rule thinking chain is a hierarchical rule chain, including defect main type, subtype and specific example; wherein, the specific example is a complete execution flow description of the defect generation, including code operation steps, logical judgment nodes and defect triggering conditions; A structured defect reasoning instruction template is defined based on the defect rule thinking chain; the defect reasoning instruction template is a language-independent event template; the defect reasoning instruction template includes a task index, task, output requirements, and output format; The output requirements include using relative paths for code files, retaining blank lines and formatting of the original code in the repaired code, only repairing the defect types specified in the template without changing other code content, and using Chinese characters for comments added after modification, which are only used to explain the changes.

2. The method according to claim 1, characterized in that, The method further includes: The structured defect reasoning instruction template is passed as input to the large language model, which then guides the large language model to perform code defect detection.

3. The method according to claim 1, characterized in that, The output requirements and output format can be customized according to actual needs; The output format includes a determination of the authenticity of the defect, a cause analysis, and repair suggestions.

4. A code defect reasoning instruction template generation system for large language models, characterized in that... include: The triplet building module is used to construct code defects into defect event triplets. The defect event triple includes location, type, and context; wherein, the type is the defect rule to which it belongs; the context includes variable dependencies, control flow execution paths, and function call relationships related to the code defect, and the logical connections before and after the defect is generated must be clearly defined; The thought chain building module is used to build a defect rule thought chain based on the defect event triple; the defect rule thought chain is a hierarchical rule chain, including defect main type, subtype and specific example; wherein, the specific example is a complete execution flow description of the defect generation, including code operation steps, logical judgment nodes and defect triggering conditions; The template generation module is used to define a structured defect reasoning instruction template based on the defect rule thinking chain; the defect reasoning instruction template is a language-independent event template; the defect reasoning instruction template includes a task index, task, output requirements, and output format; The output requirements include using relative paths for code files, retaining blank lines and formatting of the original code in the repaired code, only repairing the defect types specified in the template without changing other code content, and using Chinese characters for comments added after modification, which are only used to explain the changes.

5. The system according to claim 4, characterized in that, The system further includes: The template application module is used to pass the structured defect reasoning instruction template as input to the large language model and guide the large language model to perform code defect detection.

6. An electronic device, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the code defect reasoning instruction template generation method for a large language model as described in any one of claims 1-3.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an implementation program for information transmission, which, when executed by a processor, implements the steps of the code defect reasoning instruction template generation method for large language models as described in any one of claims 1-3.

Images