Cross-model collaborative scheduling method for code generation based on large language model

Abstract

The invention relates to a cross-model collaborative scheduling method for code generation based on large language models (LLMs), and belongs to the field of artificial intelligence and software engineering, and the method comprises the following steps: S1, obtaining a demand set R, and dividing the demand set R into N subsets; s2, sequentially extracting requirements from the subsets and constructing codes to generate prompts; s3, loading LLMs, transmitting code generation prompts into the quantized LLMs, and generating codes in parallel; s4, testing the correctness of the generated code, and if an error is detected, constructing a diagnosis correction prompt; s5, using LLMs to diagnose and correct errors in the code according to the diagnosis correction prompt to obtain a diagnosed and corrected code; s6, if the codes do not pass the test, performing cross-model generation; s7, circularly executing the steps S2-S6 until all the requirements are processed, and finally obtaining a code set which successfully passes through the test case and a code set which does not pass through the test case; according to the method, the code generation performance of different LLMs is fully mined, and the automatic code generation capability of cross-model collaboration is improved.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and software engineering, and in particular to a code generation method based on Large Language Models (LLMs), which enhances the performance of code generation by LLMs through cross-model collaboration. Background Art

[0002] Automatic code generation technology has many advantages, such as shortening the software development cycle, improving software quality and consistency, and reducing human errors and repetitive tasks. Therefore, it has attracted great attention in academia and industry. In the past few decades, a variety of code generation techniques have been developed, including deductive synthesis, inductive synthesis, and neural network-based synthesis. These methods have shown broad prospects in accurately capturing user requirements and improving overall efficiency.

[0003] Large Language Models (LLMs) based on the Transformer architecture have performed well in various generation tasks. By pre-training on public code datasets such as GitHub, LLMs learn rich contextual representations that can be effectively transferred to code generation tasks. To improve code generation performance, existing studies have explored effective strategies to ensure that the generated code conforms to pseudocode or natural language descriptions and passes test cases. Although these studies have enhanced the code generation capabilities of LLMs, LLMs still have difficulty generating suitable code snippets in complex practical application scenarios. Therefore, recent studies have attempted to improve performance through self-healing mechanisms, especially guiding LLMs to introspect and correct errors in generated code to mimic the writing, compiling, testing, and debugging processes of software engineers. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a cross-model collaborative scheduling method for code generation based on LLMs. Through cross-model collaboration, the code generation capability of the cross-model collaborative scheduling method is not limited to the code generation capability of a single LLMs, but can be combined with different LLMs for code generation, thereby realizing cross-model collaborative code generation. By tracking errors in the compilation, execution and testing stages of the code, generating diagnostic correction prompts, LLMs can autonomously try to correct errors in the generated code, thereby improving the code generation capability of LLMs.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] 1. A cross-model collaborative scheduling method for code generation based on LLMs, characterized in that the method includes the following detailed steps:

[0007] S1: Get a requirement set R, where the requirement includes a natural language description or pseudo code, and then divide the requirement set R into N subsets {R A ,R B ,…,R N}, where there is no intersection between demand subsets, and the demand is represented by R Xy Represents; R A = {R A1 ,R A2 ,R A3 ,…,R Ai ,…,R Aa}, R B = {R B1 ,R B2 ,R B3 ,…,R Bj ,…,R Bb}, …, R N = {R N1 ,R N2 ,R N3 ,…,R Nr ,…,R Nn},|R A |=a,|R B |=b,…,|R N | = n, and |R| = a + b + ... + n;

[0008] S2: In the subset {R A ,R B ,…,R N} extract one requirement at a time to obtain the requirement {R Ai ,R Bj ,…,R Nr}, where 1≤i≤a, 1≤j≤b, …, 1≤r≤n, the demand {R Ai ,R Bj ,…,R Nr}Build code generation prompts to get code generation prompts that include requirements Some task requirements require corresponding test cases to construct prompts;

[0009] S3: Load LLMs {M A ,M B ,…,M N} and can be quantified into {M according to actual usage. QA ,M QB ,…,M QN}To reduce computing resources, the code generation prompts The incoming quantized LLMs {M QA ,M QB ,…,MQN}, generate code in parallel The present invention takes quantized LLMs as an example, but is not limited to quantized LLMs. The LLMs hereinafter refer to quantized LLMs;

[0010] S4: Compile, execute and use test cases to test the correctness of the generated code. If the test passes, add the code to code set C. S ,If an error is detected, the error information obtained during the compilation, execution and testing phases of the code are used to build diagnostic ,correction hints;

[0011] S5: Different diagnostic correction prompts are passed to the LLMs used in the code generation phase to enable them to self-diagnose and correct errors in the code, obtain the code after diagnosis correction, compile, execute and test the code after diagnosis correction, and add the code that passes the test to the code set C S ;

[0012] S6: If the code fails the test, cross-model generation is performed, that is, the requirements corresponding to the code are tried to be processed using other LLMs, and the code generation and diagnosis correction process is continued. This process will try all available LLMs in sequence until the test passes or all LLMs are tried and the final code is obtained. The code that passes the test is added to the code set C S , the code that failed the test is added to code set C F ;

[0013] S7: Loop through steps S2-S6 until all requirements R are processed, and finally obtain the code set C that successfully passes the test case S and code set C that fails the test case F , and |R|=|C S |+|C F |.

[0014] Furthermore, in step S1, the requirement set R is divided into a plurality of subsets by:

[0015] Method 1: When multiple LLMs are deployed on one GPU or multiple GPUs, that is, when the number of GPUs is not equal to the number of LLMs, the demand set R is divided as evenly as possible according to the number of actually callable LLMs;

[0016] Method 2: When multiple LLMs are deployed on different GPUs, that is, the number of GPUs is equal to the number of LLMs, they are allocated according to the number of GPU cores and the base frequency of each core:

[0017] a / b / … / n=(N a ×f a ) / (N b ×fb ) / … / (N n ×f n )

[0018] Where a, b, ..., n represent the specific demand quantity of different subsets, N a ,N b ,…,N n Respectively represent the number of cores of different GPUs, f a ,f b ,…f n Represents the base frequency of each core of different GPUs.

[0019] Furthermore, the template of the code generation prompt contains four key parts: roles, requirements, requirements, and outputs: the role part aims to assign specific identities to LLMs, and through this setting, provides key contextual information, which is essential for LLMs to accurately understand the requirements; the requirement part aims to indicate the specific types of tasks that LLMs need to perform and provide clear action guidance; the requirement part lists the specific requirements for code generation in detail to ensure that the generated code meets specific quality standards, among which T C Refers to the test case of the code; the output part specifies the output format requirements of LLMs to ensure the standardization and availability of the results so as to identify the generated code; the prompt templates designed by the present invention include but are not limited to Chinese and English, and it is only necessary to prepare the template according to the natural language supported by the LLMs used, the same below; the prompt template for code generation is as follows:

[0020]

[0021] Here, R Xy is a given specific requirement, R Xy ∈{R Ai ,R Bj ,…,R Nr}, T Xy Represents a given demand R Xy Corresponding test cases; some task requirements require corresponding test cases to construct prompts. If test cases are not required to construct prompts, skip the first item in the requirements part of the above template.

[0022] Further, in step S3, LLMs work in parallel to generate codes according to their respective requirements. The code generation process is represented by the following general formula:

[0023]

[0024] Where R Xy Indicates a specific requirement that the code is to solve. Xy Indicates that for R XyFollowing the prompt template above, M Q represents the quantized LLMs, G(·) represents the process of code generation using LLMs, Represents the generated code.

[0025] Further, in step S4, the functionality and correctness of the code are verified by automatically compiling and executing test cases. The detected code errors are divided into the following three types:

[0026] Compile error: An error that occurs when the code does not conform to the syntax rules or language specifications, causing the program to fail to compile;

[0027] Execution error: an error caused by improper program logic, which results in timeout or abnormal termination during testing;

[0028] Test Error: The program runs fine, but the output does not match the given answer, resulting in a test failure or error.

[0029] During the compilation, execution, and running of test cases, error information is recorded based on the error messages reported by the compiler and interpreter, as well as the differences from the expected output of the test cases. The method is as follows:

[0030]

[0031] Among them C C (·), C E (·), C T (·) respectively represent the generated code The module collects error information during compilation, execution, and testing. The error information collected at each stage is used and It is important to note that some programming languages, such as Python, are interpreted languages ​​and do not involve the traditional compilation process. Therefore, when dealing with Python-related requirements, the compilation step needs to be skipped and the diagnostic correction process related to compilation errors will not be performed.

[0032] Furthermore, the diagnosis and correction prompt template is as follows:

[0033]

[0034] Further, in step S5, the self-diagnosis of LLMs is used to perform error analysis and repair:

[0035]

[0036] Where L C , L E , and L TThey represent the diagnostic correction loops in the code compilation, execution, and testing phases. The loop ends when the maximum number of diagnostic corrections is reached or the test is passed. represents the diagnostically corrected codes generated in these stages, and DC(·) refers to the diagnostic correction process.

[0037] Further, take corresponding strategies according to different situations:

[0038] Case 1: If all codes pass the test; corresponding strategy 1: No further action is required, directly add the correct code to code set C S ;

[0039] Case 2: If some code fails the test; corresponding strategy 2: Try to use all available models to generate code for each requirement corresponding to each code that fails the test, until a model successfully passes the test or all models are tried. Add the code that passed the test to code set C S , the code that failed the test is added to code set C F ;

[0040] Among them C S Indicates the code set that successfully passes the test case, C F Represents the set of codes that did not successfully pass the test case.

[0041] The beneficial effects of the present invention are:

[0042] (1) The present invention proposes a cross-model collaborative scheduling method based on LLMs code generation, which generates code through cross-model code generation and diagnosis and correction mechanism. It can combine the code generation capabilities of different LLMs to achieve cross-model collaborative code generation, and improve the automatic code generation capability under the cross-model method by fully exploring the code generation performance of different LLMs.

[0043] (2) In the process of multi-model collaborative code generation, the present invention utilizes multiple high-performance LLMs to collaboratively generate code, and divides the requirement set into multiple subsets, so that multiple LLMs can generate code in parallel. This method improves the efficiency of LLMs and eliminates idle time during task execution, ensuring that different LLMs process their respective parts in parallel. This arrangement not only shortens the processing time, but also improves resource utilization efficiency.

[0044] (3) In the process of diagnosis and correction, the present invention generates diagnosis and correction prompts by tracking the errors in the compilation, execution and testing stages of the code, and enables LLMs to automatically repair the errors in the code based on the diagnosis and correction prompts. By utilizing the self-correction ability of the model, the accuracy and quality of the generated code are improved, ensuring that the final code meets the expected requirements.

[0045] (4) In the cross-model code generation process of the present invention, when the current LLMs encounters an error when processing a specific requirement, the system will switch to the next LLMs to diagnose and correct the code. In this way, even if a single LLM cannot solve a given requirement, the collaboration between multiple LLMs can significantly improve the overall performance of the cross-model collaborative scheduling method.

[0046] Other advantages, objectives and features of the present invention will be described in the following description, and to some extent, may be guided and helped by the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0048] Figure 1 It is a schematic diagram of the overall process of the cross-model collaborative scheduling method of the present invention;

[0049] Figure 2 This is a schematic diagram of code generation based on multi-model collaboration according to the present invention;

[0050] Figure 3 This is a schematic diagram of the code diagnosis and correction process of the present invention;

[0051] Figure 4 This is a schematic diagram of the cross-model collaborative code generation process described in the present invention. DETAILED DESCRIPTION

[0052] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0053] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0054] The present invention proposes a cross-model collaborative scheduling method for code generation based on LLMs, such as Figure 1 The flowchart specifically includes the following steps:

[0055] S1: Get a requirement set R, where the requirement includes a natural language description or pseudo code, and then divide the requirement set R into N subsets {R A , R B , …, R N}, where there is no intersection between demand subsets, and the demand is represented by R Xy Represents; R = {R A1 , R A2 , R A3 , …, R Ai …, R Aa}, R B = {R B1 , R B2 , R B3 , …, R Bj , …, R Bb}, …, R N = {R N1 , R N2 , R N3 , …, R Nr , …, R Nn},|R A |=a,|R B |=b,…,|R N | = n, and |R| = a + b + ... + n; the method of dividing the requirement set R into multiple subsets:

[0056] Method 1: When multiple LLMs are deployed on one GPU or multiple GPUs, that is, when the number of GPUs is not equal to the number of LLMs, the demand set R is divided as evenly as possible according to the number of actually callable LLMs;

[0057] Method 2: When multiple LLMs are deployed on different GPUs, that is, the number of GPUs is equal to the number of LLMs, they are allocated according to the number of GPU cores and the base frequency of each core:

[0058] a / b / … / n=(N a ×f a ) / (N b ×f b ) / … / (N n ×f n )

[0059] Where a, b, ..., n represent the specific demand quantity of different subsets, N a , N b ,…,N n Respectively represent the number of cores of different GPUs, f a, f b , …f n Represents the base frequency of each core of different GPUs.

[0060] S2: In the subset {R A , R B , …, R N} extract one requirement at a time to obtain the requirement {R Ai , R B j,…,R Nr}, where 1≤i≤a, 1≤j≤b, …, 1≤r≤n, the demand {R Ai , R Bj , …, R Nr}Build code generation prompts to get code generation prompts that include requirements Some task requirements require corresponding test cases to construct prompts;

[0061] The template of the code generation prompt contains four key parts: roles, requirements, requirements, and outputs: The role part is intended to assign specific identities to LLMs, and this setting provides key contextual information, which is essential for LLMs to accurately understand the requirements; the requirement part is intended to indicate the specific types of tasks that LLMs need to perform and provide clear action guidance; the requirement part lists the specific requirements for code generation in detail to ensure that the generated code meets specific quality standards, among which T C Refers to the test case of the code; the output part specifies the output format requirements of LLMs to ensure the standardization and availability of the results so as to identify the generated code; the prompt templates designed by the present invention include but are not limited to Chinese and English, and it is only necessary to prepare the template according to the natural language supported by the LLMs used, the same below; the prompt template for code generation is as follows:

[0062]

[0063]

[0064] Here, R Xy is a given specific requirement, R Xy ∈{R Ai ,R Bj ,…,R Nr}, T Xy Represents a given demand R Xy Corresponding test cases; some task requirements require corresponding test cases to construct prompts. If test cases are not required to construct prompts, skip the first item in the requirements part of the above template.

[0065] S3: Load LLMs {M A ,M B ,…,MN} and can be quantified into {M according to actual usage. QA ,M QB ,…,M QN}To reduce computing resources, the code generation prompts The incoming quantized LLMs {M QA ,M QB ,…,M QN}, generate code in parallel The present invention takes quantized LLMs as an example, but is not limited to quantized LLMs. The LLMs hereinafter represent quantized LLMs. The code generation process is represented by the following general formula:

[0066]

[0067] Where R Xy Indicates a specific requirement that the code is to solve. Xy Indicates that for R Xy Following the prompt template above, M Q represents the quantized LLMs, G(·) represents the process of code generation using LLMs, Represents the generated code. The above multi-model collaborative generation process is as follows Figure 2 shown.

[0068] S4: Compile, execute and use test cases to test the correctness of the generated code. If the test passes, add the code to code set C. S ; If an error is detected, use the error information obtained during the compilation, execution, and testing phases of the code to build diagnostic correction hints. We get feedback from the errors and classify them into the following three types:

[0069] Compile error: An error that occurs when the code does not conform to the syntax rules or language specifications, causing the program to fail to compile;

[0070] Execution error: an error caused by improper program logic, which results in timeout or abnormal termination during testing;

[0071] Test Error: The program runs fine, but the output does not match the given answer, resulting in a test failure or error.

[0072] During the compilation, execution, and running of test cases, error information is recorded based on the error messages reported by the compiler and interpreter, as well as the differences from the expected output of the test cases. The method is as follows:

[0073]

[0074] Among them C C (·), CE (·), C T (·) respectively represent the generated code The module collects error information during compilation, execution, and testing. The error information collected at each stage is used and It is important to note that some programming languages, such as Python, are interpreted languages ​​and do not involve the traditional compilation process. Therefore, when dealing with Python-related requirements, the compilation step needs to be skipped and the diagnostic correction process related to compilation errors will not be performed.

[0075] If an error is detected, a diagnostic correction hint is constructed using the error information obtained during the compilation, execution, and testing phases of the code. Diagnostic correction hints can be used in the compilation, execution, and testing phases, and there are corresponding templates for each phase. The diagnostic correction template is used to guide LLMs to analyze three types of error information and improve the generated code by reasoning about the analysis results. These design templates are versatile and can incorporate other useful information to effectively organize error messages. For compilation errors, the collected data is used, including the error line number, line context, and complete error information. For execution errors, information indicating abnormal program operation is used. For test errors, they are also encapsulated in the designed template. These error messages provide additional context and clarity for the generated code, and are intended to guide LLMs to perform diagnostic corrections based on the error information provided. The diagnostic correction hint template is as follows:

[0076]

[0077] S5: Different diagnostic correction hints are passed to the LLMs used in the code generation stage, so that they can self-diagnose and correct the errors in the code to obtain the code after diagnosis correction. Specifically, in each stage of compilation, execution and testing, the corresponding diagnostic correction strategy will be implemented, and the formula is as follows:

[0078]

[0079] Where L C , L E , and L T They represent the diagnostic correction loops in the code compilation, execution, and testing phases. The loop ends when the maximum number of diagnostic corrections is reached or the test is passed. represents the diagnostically corrected codes generated in these stages, and DC(·) refers to the diagnostic correction process.

[0080] Finally, the code is obtained after diagnosis and correction. The above diagnosis and correction process is as follows: Figure 3 Compile, execute and test the code after diagnosis and correction, and add the code that passes the test to code set C S.

[0081] S6: If the code fails the test, cross-model generation is performed, that is, the requirements corresponding to the code are tried to be processed using other LLMs, and the code generation and diagnosis and correction process is continued. This process will try all available LLMs in sequence until the test passes or all LLMs are tried, and the final code is obtained.

[0082] Adopt corresponding strategies according to different situations:

[0083] Case 1: If all codes pass the test; corresponding strategy 1: No further action is required, directly add the correct code to code set C S ;

[0084] Case 2: If some code fails the test; corresponding strategy 2: Try to use all available models to generate code for each requirement corresponding to each code that fails the test, until a model successfully passes the test or all models are tried. Add the code that passed the test to code set C S , the code that failed the test is added to code set C F ;

[0085] Among them C S Indicates the code set that successfully passes the test case, C F Represents the set of codes that did not successfully pass the test case.

[0086] Example 1

[0087] S1: Get the requirement set R, and divide the requirement set R into N parts according to the N (N≥2) LLMs actually called, and get the requirement subsets {R A ,R B ,…,R N}.

[0088] S2: R A For example, in the subset R A Extract a requirement to get requirement R Ai . The demand R Ai And the corresponding test cases are used to generate code prompts to obtain the requirements R Ai Code generation tips

[0089]

[0090] S3: Load LLMs {M A ,M B ,…,M N} and quantized as {M QA ,M QB ,…,M QN}, where the code generation prompt Incoming M QA Conduct demand R Ai The code generated by

[0091]

[0092] S4: The generated codes Compile, execute, and test using test cases to determine the correctness of the code.

[0093] Test errors occur in the code, and test error diagnosis and correction prompts are built

[0094]

[0095]

[0096] S5: Compile error diagnosis and correction prompts The big model is input, and the big model is used for self-diagnosis and automatic correction technology to analyze and repair errors, and the corrected diagnosis is obtained. Code:

[0097]

[0098] Use the test case again to generate The code is compiled, executed and tested. The compilation passes but it is found that The code has a test error but the number of diagnostic corrections has been exceeded, so the diagnostic correction has failed.

[0099] S6: When encountering situation 2, the error code and its demand shift to M QB After code generation and diagnostic correction, code C is generated Ai :

[0100]

[0101] C Ai The code passes the test, and the C Ai Code added to code set C that passed the test case S .

[0102] S7: Loop S2-S6 until all requirements are processed, and finally get the code set C that passes the test case S and code set C that fails the test case F .

[0103] The above embodiments further illustrate the purpose, technical solutions and advantages of the present invention in detail. It should be understood that the above embodiments are only preferred implementation modes of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made to the present invention within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A cross-model collaborative scheduling method for code generation based on Large Language Models (LLMs), characterized in that: The method includes the following detailed steps: S1: Get a requirement set R, where the requirement includes a natural language description or pseudo code, and then divide the requirement set R into N subsets {R A ,R B ,…,R N }, where there is no intersection between demand subsets, and the demand is represented by R Xy express; S2: In the subset {R A ,R B ,…,R N } extract one requirement at a time to obtain the requirement {R Ai ,R Bj ,…,R Nr }, the requirement {R Ai ,R Bj ,…,R Nr }Build code generation prompts to get code generation prompts that include requirements Some task requirements require corresponding test cases to construct prompts; S3: Load LLMs {M A ,M B ,…,M N } and can be quantified into {M according to actual usage. QA ,M QB ,…,M QN }To reduce computing resources, the code generation prompts The incoming quantized LLMs {M QA ,M QB ,…,M QN }, generate code in parallel The present invention takes quantized LLMs as an example, but is not limited to quantized LLMs. The LLMs hereinafter refer to quantized LLMs; S4: Compile, execute and use test cases to test the correctness of the generated code. If the test passes, add the code to code set C. S ,If an error is detected, the error information obtained during the compilation, execution and testing phases of the code are used to build diagnostic ,correction hints; S5: Different diagnostic correction prompts are passed to the LLMs used in the code generation phase to enable them to self-diagnose and correct errors in the code, obtain the code after diagnosis correction, compile, execute and test the code after diagnosis correction, and add the code that passes the test to the code set C S ; S6: If the code fails the test, cross-model generation is performed, that is, the requirements corresponding to the code are tried to be processed using other LLMs, and the code generation and diagnosis correction process is continued. This process will try all available LLMs in sequence until the test passes or all LLMs are tried and the final code is obtained. The code that passes the test is added to the code set C S , the code that failed the test is added to code set C F ; S7: Loop through steps S2-S6 until all requirements R are processed, and finally obtain the code set C that successfully passes the test case S and code set C that fails the test case F , and |R|=|C S |+|C F |.

2. The cross-model collaborative scheduling method for code generation based on LLMs according to claim 1, characterized in that: In step S1, the requirement set R is divided into multiple subsets in the following manner: Method 1: When multiple LLMs are deployed on one GPU or multiple GPUs, that is, when the number of GPUs is not equal to the number of LLMs, the demand set R is divided as evenly as possible according to the number of actually callable LLMs; Method 2: When multiple LLMs are deployed on different GPUs, that is, when the number of GPUs is equal to the number of LLMs, they are allocated according to the number of GPU cores and the base frequency of each core.

3. The cross-model collaborative scheduling method for code generation based on LLMs according to claim 1, characterized in that: In step S2, the template of the code generation prompt contains four key parts: roles, requirements, requirements, and outputs: the role part aims to assign specific identities to LLMs, and through this setting, provides key contextual information, which is essential for LLMs to accurately understand the requirements; the requirement part aims to indicate the specific types of tasks that LLMs need to perform and provide clear action guidance; the requirement part lists the specific requirements for code generation in detail to ensure that the generated code meets specific quality standards, among which T C Refers to the test cases of the code; The output part specifies the output format requirements of LLMs to ensure the standardization and usability of the results so as to identify the generated code; the prompt templates designed by the present invention include but are not limited to Chinese and English, and it is only necessary to prepare the templates according to the natural language supported by the LLMs used, the same below.

4. The cross-model collaborative scheduling method for code generation based on LLMs according to claim 1, characterized in that: In step S4, the functionality and correctness of the code are verified by automatically compiling and executing test cases. The detected code errors are divided into the following three types: Compile error: An error that occurs when the code does not conform to the syntax rules or language specifications, causing the program to fail to compile; Execution error: an error caused by improper program logic, which results in timeout or abnormal termination during testing; Test error: The program runs normally, but the output does not match the given answer, resulting in test failure or error; During the process of compiling, executing, and running test cases, error information is recorded based on the error messages reported by the compiler and interpreter, as well as the differences from the expected output of the test cases.

5. The cross-model collaborative scheduling method for code generation based on LLMs according to claim 1, characterized in that: In step S4, the diagnosis and correction template provides three key information: the error details generated by the compiler, the current code part that causes the error and the requirement reference; the execution error diagnosis and correction template provides three aspects of information: the error information generated during program execution, the current code fragment that causes the error and the requirement reference; the test error diagnosis and correction template provides four key information: the provided test case, the difference between the expected output and the actual output, and the current generated code and requirements; Its purpose is to help LLMs find the cause of failure in the code and perform targeted optimization to make the program achieve the expected results.

6. The cross-model collaborative scheduling method for code generation based on LLMs according to claim 1, characterized in that: In step S5, the self-diagnosis of LLMs is used to perform error analysis and repair: Where L C , L E , and L T They represent the diagnostic correction loops in the code compilation, execution, and testing phases. The loop ends when the maximum number of diagnostic corrections is reached or the test is passed. represents the diagnostically corrected codes generated in these stages, and DC(·) refers to the diagnostic correction process.

7. The cross-model collaborative scheduling method for code generation based on LLMs according to claim 1, characterized in that: In step S6, corresponding strategies are adopted according to different situations: Case 1: If all codes pass the test; corresponding strategy 1: No further action is required, directly add the correct code to code set C S ; Case 2: If some code fails the test; corresponding strategy 2: Try to use all available models to generate code for each requirement corresponding to each code that fails the test, until a model successfully passes the test or all models are tried. Add the code that passed the test to code set C S , the code that failed the test is added to code set C F ; Among them C S Indicates the code set that successfully passes the test case, C F Represents the set of codes that did not successfully pass the test case.

Images