Code generation method based on demand refinement and consistency verification

Through the method of demand refinement and consistency verification, the error problem caused by demand refinement in automatic generation of PLC codes is solved, high-quality code generation is achieved, and the accuracy and consistency of the code is improved.

CN119045831APending Publication Date: 2024-11-29BEIHANG UNIV
View PDF 0 Cites 4 Cited by

Patent Information

Application Number
CN202411232859.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-04
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

The existing PLC code automatic generation method based on large language models has not been processed with requirements, resulting in many errors in the generated code, functions cannot meet the actual needs of users, and the inability to effectively verify the natural language requirements documents, resulting in syntax and logic errors.

Method used

Using a code generation method based on demand refinement and consistency verification, by building a use case design knowledge base and property specification knowledge base, generating refined requirements documents and performing syntax verification, using a large language model to generate and repair code, decomposing the code repair task as Bug analysis, bug fixes and code generation ensures the correctness of the code through multiple rounds of verification-modification.

Benefits of technology

It improves the accuracy and robustness of PLC code, reduces syntax and functional errors, enhances the functional consistency and correctness of the code, and reduces the "disastrous forgetting" impact of large language models.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119045831A_ABST
    Figure CN119045831A_ABST
Patent Text Reader

Abstract

The invention relates to a code generation method based on demand refinement and consistency verification, and belongs to the field of software engineering. Comprising the steps of generating refined demands based on original demands, a knowledge base and a cue word template; generating a mode design based on the refined demand and the cue word template, generating a refined property specification based on the mode design and the knowledge base, and performing grammar and consistency verification on the refined property specification; and generating an original code based on the verified refinement property specification and the corresponding refinement demand, performing verification-modification in-loop operation on the original code based on the large language model, and iteratively generating a verified code. According to the method, refining and property specification consistency verification are carried out on original requirements, verified refined requirements and refined property specifications are generated, then PLC codes are generated, code grammar and functional errors are repaired through verification-modification in-loop operation, and the problems that according to an existing PLC code automatic generation method, requirement refining processing is not carried out, and the verification efficiency is high are solved. The generated code errors are more, and the function cannot meet the user requirement.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of software engineering, and in particular to a code generation method based on requirement refinement and consistency verification. Background Art

[0002] Manufacturing Automation Systems (MAS) play a key role in modern industry. Programmable Logic Controllers (PLCs) are the core of MAS, and their code writing is crucial to actual industrial production. With the rapid development of industrial automation, developers are faced with the challenge of developing high-quality PLC programs within a tight time frame.

[0003] PLC can be programmed using Structured Text (ST) language, which is similar to traditional programming languages ​​in syntax and structure and relatively easy to learn, but proficient use of it still requires developers to have rich professional knowledge and experience. As one of the standardized programming paradigm languages ​​defined by IEC 61131-3, ST language makes it possible to automatically generate PLC code based on ST language using Large Language Model (LLM). Therefore, in the face of increasingly complex manufacturing automation systems, how to automatically generate PLC code accurately and efficiently has become a hot topic in recent research.

[0004] However, the existing LLM-based automatic PLC code generation method directly generates code from the requirement document in natural language form without refining the requirements, resulting in the following problems in the generated code: first, since the requirement description is too abstract, the requirement document may be ambiguous, resulting in the generated code being unable to meet the actual needs of users; second, the requirement document in natural language form cannot be directly verified, resulting in the generated code being prone to grammatical errors; finally, the requirement document in natural language form may have inconsistent descriptions, resulting in conflicting constraints in the document, leading to logical errors in the generated code. Summary of the invention

[0005] In view of the above analysis, the present invention aims to provide a code generation method based on requirement refinement and consistency verification, so as to solve the problem that the current PLC code automatic generation method does not perform requirement refinement processing, resulting in many errors in the generated code and the functions cannot meet the actual needs of users.

[0006] The present invention provides a code generation method based on requirement refinement and consistency verification, the method comprising the following steps:

[0007] Build a use case design knowledge base and a use case design prompt word template, and generate a refined requirement document based on the original requirement document, the use case design knowledge base, and the use case design prompt word template;

[0008] Construct a property specification knowledge base and a pattern design prompt word template, generate an original property specification document based on the original requirement document, generate a pattern design document based on the refined requirement document and the pattern design prompt word template, generate a refined property specification document based on the pattern design document and the property specification knowledge base and perform syntax verification on it and consistency verification between the original property specification document and the refined property specification document, and obtain a verified refined property specification document and its corresponding refined requirement document;

[0009] The original code is generated based on the verified refined property specification document and its corresponding refined requirement document, and the correctness of the original code syntax and function is judged. If it is incorrect, the original code is modified based on the large language model until its code syntax and function are correct or the maximum number of modifications is reached to obtain the verified code.

[0010] Furthermore, a PLC code robot is constructed to generate original code; the verified refined requirement document, refined property specification document, and PLC syntax error example set are sent to the PLC code robot to generate original code.

[0011] Furthermore, a PLC code repair group chat is constructed to analyze and repair code defects; syntax and functional error information is sent to the PLC code repair group chat, and the defects are repaired by analyzing the error information; and the modified code is generated by the PLC code robot.

[0012] Furthermore, the PLC code repair group chat includes: a bug analysis robot, a bug repair robot, and a user agent robot; wherein the bug analysis robot is used to analyze the received syntax and functional error information and send the inferred error cause to the bug repair robot; the bug repair robot is used to modify the code bug according to the received error cause and send the modified code to the PLC code robot; the user agent robot is used to act as an agent for the user to interact with other robots.

[0013] Furthermore, a second keyword related to pattern design is established, and a pattern design prompt word template is constructed according to the second keyword, the number of variables, the use case in the first keyword, and the event flow; the second keyword includes a pattern and parameters to be filled in.

[0014] Furthermore, the refined requirement document and the pattern design prompt word template are input into the large language model, so that the large language model extracts the content corresponding to the first keyword from the refined requirement document according to the pattern design prompt word template, and generates several corresponding patterns for each use case in the refined requirement document. Each pattern is filled with corresponding content based on the content corresponding to the extracted first keyword according to the pattern design prompt word template, thereby generating a pattern design document.

[0015] Furthermore, the pattern design document is segmented according to the pattern to obtain a number of pattern design document fragments, and for each pattern design document fragment, a third vector matching it is obtained in the property specification knowledge base, and the matching third vector is converted into a sentence to obtain a third sentence matching it.

[0016] Furthermore, for each pattern design document fragment: convert the pattern design document fragment into a fourth vector; calculate the vector similarity between the fourth vector and all third vectors, and select the top k third vectors with the largest vector similarity; set the sentence window size, and establish the third sentence windows corresponding to the top k third vectors according to the sentence window size; calculate the semantic similarity between the fourth vector and all corresponding third sentence windows, select all third vectors corresponding to the third sentence windows with the largest semantic similarity as the third vector matching the pattern design document fragment, and convert the matched third vectors into sentences in context order to obtain the third sentence matching the pattern design document fragment.

[0017] Furthermore, all pattern design document fragments and their matching third sentences are input into the large language model, so that the large language model extracts the variable expressions corresponding to the parameters to be filled in the pattern design document fragment from the third sentence according to the pattern design document fragment, thereby generating a refined property specification document.

[0018] Furthermore, second knowledge documents related to property specifications are collected, and the second knowledge documents are segmented and stored in a vector index library to form a property specification knowledge base; the second knowledge documents are parsed and cleaned, and the cleaned second knowledge documents are segmented by sentences, and all the segmented sentences are converted into third vectors and stored in the vector index library, and all the stored third vectors constitute the property specification knowledge base.

[0019] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0020] 1. The present invention converts a large amount of effective information into vectors for storage by establishing a vector index library, so that LLM can retrieve relevant vectors as context information through the vector index library without having to input this information into LLM as prompts, thereby improving the accuracy and efficiency of the answers output by LLM; and, by persisting the information in the database, the information can be retrieved and provided to LLM at any time as needed, which is equivalent to providing LLM with long-term memory capability and improving the correctness of LLM's answers.

[0021] 2. The present invention first searches for vectors with high similarity to the original requirement document vectors in the constructed vector index library, then searches for matching vectors with the highest semantic similarity to the original requirement document through the sentence windows of these vectors, and finally converts the matching vectors into sentences in their context order as part of the LLM context input, so that the LLM can obtain knowledge that is not mentioned in the training and is truly relevant to the input content without retraining before answering, thereby improving its accuracy in identifying relevant content in the original requirement document and providing a basis for improving the correctness of refined requirements and refined property specifications.

[0022] 3. The present invention sends the pattern design document, the current refined property specification document, the SPEC statement error information, and the refined requirement document to the auxiliary modification robot, so that the robot can check and modify the refined requirement document and the refined property specification document based on sufficient external information prompts; and through multiple rounds of verification-modification-in-the-loop operations, it iteratively generates the refined property specification document and the refined requirement document after consistency verification, thereby further improving the correctness of the refined requirement document and providing a basis for improving the correctness of the automatically generated code.

[0023] 4. The present invention splits the task of repairing the code and generating the modified code into three independent tasks: bug analysis, bug repair, and code generation. The tasks are respectively assigned to different robots for processing, so that each robot can perform its own duties and avoid requiring a single robot to have a long memory due to excessively long context information. This not only improves the efficiency and correctness of defect repair, but also reduces the impact of "catastrophic forgetting" caused by LLM. The present invention iteratively generates verified code through multiple rounds of verification-modification-in-the-loop operations, thereby improving the robustness of the PLC code and the correctness of its syntax and functions.

[0024] In the present invention, the above-mentioned technical solutions can also be combined with each other to achieve more preferred combination solutions. Other features and advantages of the present invention will be described in the subsequent description, and some advantages can become obvious from the description, or can be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained through the contents particularly pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like components throughout the drawings.

[0026] Figure 1 A flowchart of a code generation method based on requirement refinement and consistency verification according to an embodiment of the present invention;

[0027] Figure 2 A schematic diagram of a first vector in a knowledge base designed for a use case of an embodiment of the present invention;

[0028] Figure 3 A schematic diagram of designing a prompt word template for a use case of an embodiment of the present invention;

[0029] Figure 4 A schematic diagram of a requirement document described in natural language according to an embodiment of the present invention;

[0030] Figure 5 A schematic diagram of a refined requirements document according to an embodiment of the present invention;

[0031] Figure 6 A schematic diagram of a prompt word template designed for an embodiment mode of the present invention;

[0032] Figure 7 A schematic diagram of a design document for an embodiment mode of the present invention;

[0033] Figure 8 A schematic diagram of a property specification document according to an embodiment of the present invention;

[0034] Fig. 9 A schematic diagram showing that a counterexample of a variable expression in an embodiment of the present invention can be correctly output;

[0035] Fig.10 A schematic diagram showing that a counterexample of a variable expression in an embodiment of the present invention cannot be correctly output;

[0036] Fig.11 A schematic diagram of an auxiliary modification robot making modification suggestions according to an embodiment of the present invention;

[0037] Fig.12 A schematic diagram of a modification result returned by an auxiliary modification robot according to an embodiment of the present invention;

[0038] Fig.13 A schematic diagram of a SPEC statement according to an embodiment of the present invention;

[0039] Fig.14 This is an example of a refined property specification document after consistency verification according to an embodiment of the present invention;

[0040] Fig.15 Code examples generated for embodiments of the present invention;

[0041] Fig.16 This is a code example after the bug repair robot in the embodiment of the present invention repairs the syntax error;

[0042] Fig.17 The bug repair robot of the embodiment of the present invention repairs the functional error and the modified code example;

[0043] Fig.18 This is a code example verified by the embodiment of the present invention;

[0044] Fig.19 This is a schematic diagram of an original requirement document of a case in an embodiment of the present invention after manual processing. DETAILED DESCRIPTION

[0045] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0046] Embodiment 1:

[0047] A specific embodiment of the present invention discloses a code generation method based on requirement refinement and consistency verification. Figure 1 As shown, the method comprises the following steps:

[0048] Step S1, constructing a use case design knowledge base and a use case design prompt word template, and generating a refined requirement document based on the original requirement document, the use case design knowledge base and the use case design prompt word template;

[0049] Step S2, constructing a property specification knowledge base and a pattern design prompt word template, generating an original property specification document based on the original requirement document, generating a pattern design document based on the refined requirement document and the pattern design prompt word template, generating a refined property specification document based on the pattern design document and the property specification knowledge base, and performing syntax verification on the original property specification document and consistency verification on the refined property specification document, to obtain a verified refined property specification document and its corresponding refined requirement document;

[0050] Step S3: Generate original code based on the verified refined property specification document and its corresponding refined requirement document, determine the correctness of the original code syntax and function, and if incorrect, modify the original code based on the large language model and verify the correctness of the modified code syntax and function until the code syntax and function are all correct or the maximum number of modifications is reached, and obtain the verified code.

[0051] Specifically, in step S1, first knowledge documents related to use case diagram design are collected, and the first knowledge documents are segmented and stored in a vector index library to form a use case design knowledge library.

[0052] Specifically, the first knowledge documents related to the use case diagram design include electronic books, articles, patents and other documents related to the use case diagram design principles, specifications and examples. The first knowledge documents are parsed and cleaned, and the cleaned first knowledge documents are segmented by sentences. All the segmented sentences are converted into first vectors and stored in a vector index library. All the stored first vectors constitute a use case design knowledge library.

[0053] It should be noted that parsing the first knowledge document refers to parsing the first knowledge document of different formats (such as PDF, Word, HTML, etc.) into a unified data format (such as a string). Cleaning the first knowledge document includes removing irrelevant symbols, removing repeated parts, standardizing, and removing irrelevant information from the parsed first knowledge document; the removal of irrelevant symbols refers to removing symbols that have no impact on semantics, such as line breaks, HTML tags, page numbers, etc.; the removal of repeated parts refers to removing semantically repeated parts to avoid redundant information in subsequent retrievals; the standardization refers to unifying terms with the same semantics, such as unifying "LLM" and "Large Language Model"; the removal of irrelevant information refers to removing information without substantive content, such as headers, footers, signatures, etc. All the segmented sentences are converted into first vectors through the embedding model. For each first knowledge document: all the corresponding first vectors are stored in the vector index library in the order of the sentences in the first knowledge document; the vector index library supports real-time data updates. Figure 2 As shown, the first vector is a high-dimensional data vector, which can represent the semantic information of the document.

[0054] Exemplarily, the BAAI / bge-large-en-v1.5 model is used as the embedding model.

[0055] It can be understood that the present invention converts a large amount of effective information into vectors for storage by establishing a vector index library, that is, a knowledge base, so that LLM can retrieve related vectors as context information through the vector index library without having to input this information as prompts into LLM, thereby improving the accuracy and efficiency of the answers output by LLM; and, by persisting the information in the database, it can be retrieved and provided to LLM at any time as needed, which is equivalent to providing LLM with long-term memory capabilities and improving the correctness of LLM's answers.

[0056] Furthermore, a first keyword related to the use case diagram design is established, and a use case design prompt word template is constructed according to the first keyword.

[0057] Specifically, the first keywords include use cases, participants, system boundaries, relationships between participants, preconditions, postconditions, use case scenarios, event flows, brief descriptions, and special requirements. Figure 3 As shown, the use case design prompt word template includes the first keyword and its corresponding description.

[0058] It should be noted that the use case includes a detailed description of the name and function of the use case; the participant refers to an external user, organization or other system that interacts with the system; the system boundary refers to the boundary between systems, defining which use cases are internal to the system and which are external to the system; the relationship between the participants refers to the relationship between the use cases, including inclusion, extension and generalization. The inclusion relationship represents that the execution of one use case involves the execution of another use case, the extension relationship represents that a use case can be extended to another use case, and the generalization relationship represents that there is an inheritance relationship between use cases or participants; the precondition refers to the state that the system should be in before the use case is executed; the postcondition refers to the state that the system should reach after the use case is executed; the use case scenario includes success scenario and abnormal scenario, representing the execution instance of the use case; the event flow includes basic flow and alternative flow, the basic flow represents the basic process of successful execution of the use case, and the alternative flow represents other processes that may occur during the execution of the use case; the brief description refers to a brief description of the role and quantitative information of the use case; the special requirements refer to a detailed description of the non-functional requirements (such as reliability, performance, availability) and design constraints of the use case.

[0059] Furthermore, the original requirement document is segmented into sentences to obtain a number of original requirement document fragments, and for each original requirement document fragment, a first vector matching it is obtained in the use case design knowledge base, and the matching first vector is converted into a sentence to obtain a first sentence matching it.

[0060] Specifically, for each original requirement document fragment: convert the original requirement document fragment into a second vector; calculate the vector similarity between the second vector and all first vectors, and select the first k first vectors with the largest vector similarity; set the sentence window size, and establish the first sentence windows corresponding to the first k first vectors according to the sentence window size; calculate the semantic similarity between the second vector and all corresponding first sentence windows, select all first vectors corresponding to the first sentence windows with the largest semantic similarity as the first vector matching the original requirement document fragment, and convert the matched first vectors into sentences in context order to obtain the first sentence matching the original requirement document fragment.

[0061] Specifically, the original requirement document fragment is converted into a second vector through the embedding model, the cosine similarity between the second vector and all the first vectors is calculated, the cosine similarities are arranged from large to small, and the first vectors corresponding to the first k cosine similarities are selected.

[0062] Specifically, the size of the first sentence window is set, the number of context sentences is determined according to the size of the first sentence window, the first vector and the first vector corresponding to the corresponding number of context sentences are used as the first sentence window corresponding to the first vector, and the semantic similarity between the second vector and all corresponding first sentence windows is calculated through the semantic analysis model.

[0063] It should be noted that since all first vectors corresponding to each first knowledge document are stored in the vector index library in the order of sentences in the first knowledge document, the first vectors corresponding to the adjacent preceding and following sentences can be obtained through a first vector.

[0064] Exemplarily, the size of the first sentence window is set to 3, then the first sentence window corresponding to a certain first vector is the first vector corresponding to 3 sentences above + the first vector + the first vector corresponding to 3 sentences below, and the first sentence window contains the first vectors corresponding to 7 sentences. Assuming that the semantic similarity between the first sentence window and the second vector corresponding to a certain original demand document fragment is the largest, the 7 first vectors corresponding to the first sentence window are used as the first vectors matching the original demand document fragment, and the 7 first vectors are converted into sentences according to the context order of the first sentence window, and 7 first sentences matching the original demand document fragment are obtained.

[0065] It is understandable that in order to improve the accuracy of automatically generating PLC code, it is necessary to refine the original requirement document in natural language form through a large language model. However, as manufacturing automation systems become increasingly complex, their requirement documents are also increasingly complex. It is difficult to achieve a good requirement refinement effect by only training and fine-tuning the large language model. Therefore, the present invention first searches for vectors with high similarity to the original requirement document vectors in the constructed vector index library, and then searches for matching vectors with the highest semantic similarity to the original requirement document through the sentence windows of these vectors, and finally converts the matching vectors into sentences as part of the LLM context input according to their context order, so that the LLM can obtain knowledge that is not mentioned in the training and is truly related to the input content without retraining before answering, thereby improving its accuracy in identifying relevant content in the original requirement document and providing a basis for improving the correctness of refined requirements and refined property specifications.

[0066] Furthermore, all original requirement document fragments and their matching first sentences and use case design prompt word templates are input into the large language model, so that the large language model extracts the content corresponding to the first keyword from the original requirement document fragments according to the use case design prompt word template and fills it into the use case design prompt word template, thereby generating a refined requirement document.

[0067] For example, Figure 4As shown in FIG. 1 , it is an original requirement document for “obtaining the absolute value of input data”. It can be seen that it is only roughly described in natural language. Through the above method, based on the original requirement document for “obtaining the absolute value of input data”, the use case design knowledge base, Figure 3 The use case design prompt word template shown in the figure generates the following by calling the GPT-4 API: Figure 5 As shown in the refined requirement document, it can be seen that the refined requirement document is a use case design document with engineering specifications.

[0068] It should be noted that the refined requirement document contains all use cases related to the requirement, and each use case is generated in the format of the use case design prompt template. If necessary, you can manually check whether the event flow, use case scenario, preconditions, postconditions and other information in the refined requirement document meet the requirements of the original requirement document.

[0069] It can be understood that the present invention provides LLM with specifications for refined requirement documents by constructing a use case design prompt word template, so that LLM can fill in the objects, event flows, preconditions, postconditions and other information accurately identified from the requirement document into the use case design prompt word template according to the standard format, thereby generating a refined requirement document with engineering specifications and use case design format.

[0070] Specifically, in step S2, the grammatical correctness of the refined property specification document is verified. If the grammar is incorrect, the refined property specification document is modified based on the large language model and the grammatical correctness of the modified refined property specification document is verified until the grammar is completely correct or the maximum number of modifications is reached; the consistency of the original property specification document and the refined property specification document after grammar verification is verified. If they are inconsistent, the refined property specification document after grammar verification and its corresponding refined requirement document are modified based on the large language model and the consistency of the modified refined property specification document and the original property specification document is verified until the two are consistent or the maximum number of modifications is reached, thereby obtaining the refined property specification document after consistency verification and its corresponding refined requirement document.

[0071] Specifically, second knowledge documents related to property specifications are collected, and the second knowledge documents are segmented and stored in a vector index library to form a property specification knowledge library.

[0072] Specifically, the second knowledge documents related to property specification include design principles of model checking tools, CTL and LTL syntax specifications, conversion rules between patterns and CTL or LTL, e-books, articles, patents and other documents related to CTL and LTL design examples. The second knowledge documents are parsed and cleaned, and the cleaned second knowledge documents are segmented by sentences. All the segmented sentences are converted into third vectors and stored in a vector index library. All the stored third vectors constitute a property specification knowledge base.

[0073] It should be noted that CTL (Computation Tree Logic) and LTL (Linear Temporal Logic) represent two types of temporal logic, and the property specification documents in CTL and LTL formats can be verified for consistency through model tools.

[0074] Furthermore, a second keyword related to pattern design is established, and a pattern design prompt word template is constructed according to the second keyword, the number of variables included in the pattern, and the use cases and event flows in the first keyword.

[0075] Specifically, the second keyword includes a pattern and a parameter to be filled in. The pattern design prompt word template includes the second keyword, a use case, a basic flow and an alternative flow and their corresponding descriptions. If the pattern corresponding to the use case contains only one variable, such as a Boolean variable, the number of parameters to be filled in the basic flow and the alternative flow is 1, such as Figure 6 As shown in the first use case property specification in ; if the pattern corresponding to the use case contains 2 variables, the number of parameters to be filled in the basic flow and the alternative flow is 2, such as Figure 6 This is shown in the second use case property specification.

[0076] It should be noted that the modes include implication mode, invariant mode, forbidden state mode, state change during a cycle, state change between cycles, reachability mode, repeatability mode, and leads to mode; the implication mode is used to verify conditional relationships; the invariant mode is used to verify that certain conditions must always be true during program execution; the forbidden state mode is used to verify that certain states are not allowed to be entered; the state change during a cycle mode is used to verify the change of state within a single cycle; the state change between cycles mode is used to verify whether the state is converted as expected between consecutive cycles; the reachability mode is used to verify whether a certain state can be reached from the current state; the repeatability mode is used to verify whether the same output can be repeatedly generated under a given input; and the leads to mode is used to verify causal relationships. The parameters to be filled in represent the corresponding variable states that need to be filled in.

[0077] Furthermore, the refined requirement document and the pattern design prompt word template are input into the large language model, so that the large language model extracts the content corresponding to the first keyword from the refined requirement document according to the pattern design prompt word template, and generates several corresponding patterns for each use case in the refined requirement document. Each pattern is filled with corresponding content based on the content corresponding to the extracted first keyword according to the pattern design prompt word template, thereby generating a pattern design document.

[0078] Exemplarily, through the above method, based on Figure 5 The refined requirements document shown, Figure 6 The pattern design prompt word template shown in the figure is generated by calling the GPT-4 API. Figure 7 As shown in the pattern design document, it can be seen that the pattern design document defines the status of its variables according to different patterns, providing a basis for the subsequent generation of property specification files in CTL or LTL format.

[0079] It should be noted that the pattern design document contains all patterns corresponding to all use cases in the refined requirements. Each use case corresponds to one or more patterns, and each pattern is generated according to the format of the pattern design prompt word template.

[0080] Furthermore, the pattern design document is segmented according to the pattern to obtain a number of pattern design document fragments, and for each pattern design document fragment, a third vector matching it is obtained in the property specification knowledge base, and the matching third vector is converted into a sentence to obtain a third sentence matching it.

[0081] Specifically, for each pattern design document fragment: convert the pattern design document fragment into a fourth vector; calculate the vector similarity between the fourth vector and all third vectors, and select the top k third vectors with the largest vector similarity; set the sentence window size, and establish the third sentence windows corresponding to the top k third vectors according to the sentence window size; calculate the semantic similarity between the fourth vector and all corresponding third sentence windows, select all third vectors corresponding to the third sentence windows with the largest semantic similarity as the third vector matching the pattern design document fragment, and convert the matched third vectors into sentences in context order to obtain the third sentence matching the pattern design document fragment.

[0082] It should be noted that, according to the method in step S1, the first k third vectors with the largest vector similarity are selected, a third sentence window is established, and the semantic similarity between the fourth vector and all corresponding third sentence windows is calculated.

[0083] Furthermore, all pattern design document fragments and their matching third sentences are input into the large language model, so that the large language model extracts the variable expressions corresponding to the parameters to be filled in the pattern design document fragment from the third sentence according to the pattern design document fragment, thereby generating a refined property specification document.

[0084] Exemplarily, through the above method, based on Figure 7 The model design document and property specification knowledge base shown in the figure are generated by calling the GPT-4 API. Figure 8 The refined property specification document shown in FIG. 1 corresponds to Figure 7 Each pattern in the pattern design document is shown and is in CTL or LTL format.

[0085] It can be understood that the present invention provides LLM with rules for converting from schema design to CTL or LTL property specifications by constructing a property specification knowledge base, so that LLM can convert schema design documents into refined property specification documents in CTL or LTL format.

[0086] It should be noted that in order to perform consistency check on the refined property specification document, the original requirement document needs to be manually converted into an original property specification document in CTL or LTL format.

[0087] Furthermore, the verification of the grammatical correctness of the refined property specification document includes: constructing a counterexample for each variable expression in the refined property specification document; if the constructed counterexample can obtain a correct output, then the variable expression corresponding to the counterexample is judged to be grammatically correct; if the grammars of all variable expressions are correct, then the grammar of the refined property specification document is judged to be correct; otherwise, the grammar is judged to be incorrect.

[0088] For example, suppose a variable expression is: input<0->AF result=-input, and the variable expression is set to false as its counterexample. If the counterexample can get the correct output, such as Fig. 9 As shown, the variable expression syntax is determined to be correct; otherwise, Fig.10 As shown, it is determined that the variable expression syntax is incorrect.

[0089] Furthermore, an auxiliary modification robot is constructed to check and modify the refined requirement documents and refined property specification documents; a maximum number of modifications is set; if the syntax is incorrect, the pattern design document, the refined property specification document, the counterexample of incorrect syntax and its error information are sent to the auxiliary modification robot, so that it modifies the refined property specification document; the grammatical correctness of the modified refined property specification document is verified until its syntax is completely correct or the maximum number of modifications is reached.

[0090] Specifically, the auxiliary modification robot is constructed through the agent development and management framework based on LLM, and the robot is given the ability of multi-round dialogue, so that the auxiliary modification robot can check and modify the syntax errors of variable expressions according to the received information. If the refined property specification document still has syntax errors when the maximum number of modifications is reached, it will be modified manually.

[0091] Exemplarily, the first assistive robot is for Fig.10 Suggested corrections for syntax errors such as Fig.11 As shown; the first auxiliary robot is for Fig.10 The syntax error shown in the following example returns the modified result. Fig.12 shown.

[0092] It is understandable that although the auxiliary modification robot may not be able to correct all grammatical errors in the refined property specification within the set maximum number of modifications, it has also made considerable modifications to its grammatical errors, and then manually corrected the remaining grammatical errors on this basis, greatly reducing the workload of manual correction.

[0093] Furthermore, the logical relationship between the original property specification document and the corresponding variable expressions in the refined property specification document after grammatical verification is verified. If the variable expression in the refined property specification document after grammatical verification implies or is equivalent to the corresponding variable expression in the original property specification document, the two are determined to be consistent; otherwise, the two are determined to be inconsistent.

[0094] Specifically, a SPEC statement is constructed, wherein the front part of the SPEC statement is a variable expression in a refined property specification document after syntax verification, the middle part of the SPEC statement is a specific symbol, and the back part of the SPEC statement is a corresponding variable expression in the original property specification document; the specific symbol represents the logical relationship between the front part and the back part of the SPEC statement, including "->", "=", wherein "->" represents implication and "=" represents equivalent to; if the test result of the SPEC statement is true, it means that the logical relationship represented by the specific symbol holds, that is, the two are consistent.

[0095] For example, Fig.13 As shown, the front part of the SPEC statement is "(input<0->AF(result=-input))&(input=0->AF(result=0))&(input>0->AF(result=input))", the middle part of the SPEC statement is "->", and the back part of the SPEC statement is "result>=0".

[0096] It can be understood that if the original property specification document is consistent with the refined property specification document, the variable expressions in the two do not conflict, and the corresponding original requirement documents and refined requirement documents are also consistent, so that the integrity of the original requirements can be preserved in the refined requirements.

[0097] Furthermore, a maximum number of modifications is set; if there is any inconsistency, the pattern design document, the current refined property specification document and its corresponding refined requirement document, and the SPEC statement error information are sent to the auxiliary modification robot, so that it can modify the refined property specification document and its corresponding refined requirement document after syntax verification; verify the consistency of the modified refined property specification document with the original property specification document until the two are consistent or the maximum number of modifications is reached.

[0098] Specifically, if the maximum number of modifications is reached and the refined property specification document after syntax verification still has variable expressions inconsistent with the original property specification, it is manually modified.

[0099] For example, the refined property specification document after consistency verification is as follows: Fig.14 As shown, the corresponding original property specification document contains only one variable expression: "result>=0", which means "the return value is always greater than or equal to 0"; and the refined property specification document after consistency verification contains three variable expressions: "(input<0->AF(result=-input)", "input=0->AF(result=0)", "input>0->AF(result=input)", which means "for the three cases where the input value is positive, negative, and zero, the corresponding value of the return value". It can be seen that the refined property specification after consistency verification has expanded and improved the original property specification.

[0100] It can be understood that the present invention sends the pattern design document, the current refined property specification document and its corresponding refined requirement document, and the SPEC statement error information to the auxiliary modification robot, so that the robot can check and modify the refined requirement document and the refined property specification document on the basis of sufficient external information prompts; and through multiple rounds of verification-modification-in-the-loop operations, iteratively generates refined property specification documents and refined requirement documents after consistency verification, thereby further improving the correctness of the refined requirement document and providing a basis for improving the correctness of the automatically generated code.

[0101] Specifically, in step S3, a PLC code robot is constructed to generate original code; the verified refined property specification document and its corresponding refined requirement document, and a set of PLC syntax error examples are sent to the PLC code robot to generate original code.

[0102] Specifically, a single-agent PLC code robot is constructed by using the AutoGen framework based on LLM, and the robot is endowed with the ability of multi-round dialogue, so that the PLC code robot can generate PLC code according to the received information by playing the role of a programmer familiar with PLC programming. The generated PLC code uses ST language, such as Fig.15 shown.

[0103] It should be noted that the PLC syntax error example set is a statistically obtained set of high-frequency syntax errors that occur when LLM generates PLC code, which are sent to the PLC code robot as counterexamples to help it reduce syntax errors in the generated code.

[0104] Furthermore, a PLC code repair group chat is constructed to analyze and repair code defects; syntax and functional error information is sent to the PLC code repair group chat, and the defects are repaired by analyzing the error information; and the modified code is generated by the PLC code robot.

[0105] Specifically, the PLC code repair group chat with multi-agent collaboration is constructed through the LLM-based AutoGen framework. The PLC code repair group chat includes three robots: Bug analysis robot PLCBug_analyse, Bug repair robot PLCDebug, and user agent robot user_proxy. The above three robots have independent agents and all have the ability of multi-round dialogue; wherein the Bug analysis robot plays the role of an expert familiar with PLC code bug analysis, and is used to analyze the received syntax and function error information and send the inferred error cause to the Bug repair robot; the Bug repair robot plays the role of an expert familiar with PLC code bug repair, and is used to modify the code bug according to the received error cause and send the modified code to the PLC code robot; the user agent robot is used to act as an agent for the user to interact with other robots.

[0106] Specifically, a maximum number of modifications is set, and the number of modifications for syntax errors and functional errors shall not exceed the maximum number of modifications; the grammatical correctness of the original code is verified, and if the syntax is incorrect, the original code is modified based on the PLC code repair group chat and the grammatical correctness of the modified code is verified until the syntax is completely correct or the maximum number of modifications is reached; the functional correctness of the code after syntax verification is verified, and if the function is incorrect, the code after syntax verification is modified based on the PLC code repair group chat and the functional correctness of the modified code is verified until the function is completely correct or the maximum number of modifications is reached, so as to obtain the code after functional verification.

[0107] It should be noted that if the modified code still has syntax or functional errors when the maximum number of modifications is reached, it will be repaired manually.

[0108] It can be understood that the present invention splits the task of repairing code and generating modified code into three independent tasks: bug analysis, bug repair, and code generation, and assigns them to different robots for processing, so that each robot can perform its own duties, avoiding the requirement for a single robot to have long memory due to excessively long context information. This not only improves the efficiency and correctness of defect repair, but also reduces the impact of "catastrophic forgetting" caused by LLM.

[0109] For example, for a syntax error message, the bug analysis robot finds that the syntax error is caused by the use of a non-existent keyword "END". The bug repair robot modifies the code bug according to the above error cause. The modified code is as follows: Fig.16 As shown in the figure, for a function error message, the bug analysis robot found that the function error was caused by an int value overflow. The bug repair robot modified the code bug according to the above error cause. The modified code is as follows Fig.17 As shown; finally the verified code is obtained, such as Fig.18 shown.

[0110] It can be understood that the present invention iteratively generates verified code through multiple rounds of verification-modification-in-the-loop operations, thereby improving the robustness of the PLC code and the correctness of its syntax and functions.

[0111] Embodiment 2:

[0112] The syntax and functional correctness of the PLC code generated by the method of the present invention are evaluated in conjunction with specific examples.

[0113] 20 PLC classic programming cases were selected, including 1 function type, 4 PID control types and 15 practical application types. Fig.19 As shown, the original requirement document of each case is preprocessed, and the original model design corresponding to the original requirement document and the original property specification document in CTL / LTL format are obtained manually.

[0114] The syntax and functional correctness of the PLC codes generated by the two are compared using the following indicators:

[0115] 1. Property specification pass rate: indicates the pass rate of the generated refined property specification and the original property specification for verification; the property specification pass rate includes: the pass rate of the property specification generated directly from the refined requirements, the pass rate of the property specification generated from the refined requirements after verification-modification in-loop operation, and the pass rate of the refined property specification for consistency verification; the property specification pass rate of the pattern design directly generated from the refined requirements indicates that the pattern design is directly generated according to the use case in the refined requirements and the pattern design is checked whether it completely covers the original property specification, if it is completely covered, it means it has passed; the property specification pass rate of the pattern design generated from the refined requirements after verification-modification in-loop operation indicates that after the auxiliary modification robot verifies the pattern design-modifies in-loop operation, it checks whether the pattern design completely covers the original property specification, if it is completely covered, it means it has passed; the refined property specification consistency verification pass rate indicates the proportion of the refined property specification that passes the syntax check and consistency verification through the verification tool after the auxiliary modification robot verifies the pattern design-modifies in-loop operation and generates the refined property specification in CTL / LTL format;

[0116] 2. Compilation pass rate: indicates the pass rate of the generated PLC code during compilation;

[0117] 3. Average number of syntax errors: Count the average number of syntax errors in the PLC code generated for the first time;

[0118] 4. Functional property specification pass rate: It indicates the pass rate of the generated PLC code when the functional property specification is verified by the verification tool, so as to measure whether the generated code function meets the requirements.

[0119] It should be noted that in the method of directly generating PLC code through LLM, in order to detect the functional correctness of the generated code, the measure of manually correcting grammatical errors is adopted, that is, when taking statistics, only the codes that can be verified by functional property specifications after manual correction of grammatical errors are considered, and the pass rate of the verification is recorded; for the codes that still cannot be verified by functional property specifications after manual correction of grammatical errors, they are recorded as "failed".

[0120] For the above 20 cases, the method of directly generating PLC code through LLM and the method of generating PLC code provided by the present invention are respectively adopted, wherein "gpt-4-1106-preview" is selected as the LLM used by both, PLCVerif is selected as the verification tool, and the number of modifications of the auxiliary modification robot in the present invention is limited to 3 times, and the following results are obtained:

[0121] Regarding the pass rate of property specifications: Among them, the method provided by the present invention generates PLC code, and the pass rate of property specifications for pattern design directly generated from refined requirements is 70.5%, the pass rate of property specifications for pattern design generated from refined requirements after verification-modification in-loop operation is 100%, and the pass rate of consistency verification of refined property specifications is 49.2%, of which the syntax check pass rate is 100%, and the consistency verification pass rate is 49.2%. It should be noted that the above results are obtained by limiting the number of modifications of the auxiliary modification robot to less than 3 times, and in actual applications, the number of modifications can be set as needed to improve the pass rate of property specifications. Since the PLC code is directly generated by LLM, no refined requirements and refined property specifications are generated, and it is impossible to evaluate this indicator.

[0122] Regarding the compile pass rate: the compile pass rate of PLC code generated directly through LLM is only 5%, and only one code can pass the functional property specification verification. The compile pass rate of PLC code generated by the method provided by the present invention is 95% after verification-modification in-loop operation, which is an increase of 90%.

[0123] Regarding the average number of grammatical errors: the average number of grammatical errors when the PLC code is directly generated by LLM is 2.75, while the average number of grammatical errors when the PLC code is generated by the method provided by the present invention is 1.15. It should be noted that when counting the grammatical errors of the PLC code generated for the first time by the two methods, no manual or LLM-involved grammatical error modification is performed on the generated code.

[0124] Regarding the pass rate of functional property specification: after directly generating PLC code through LLM and manually correcting syntax errors, its pass rate of functional property specification is 23.41%, while when generating PLC code through the method provided by the present invention, its pass rate of functional property specification is as high as 75.4%, which is improved by 51.99%.

[0125] In summary, the PLC code generation method provided by the present invention has significant improvements in the above indicators compared with the method of directly generating PLC code through LLM. Therefore, the PLC code generation method provided by the present invention has advantages in improving the quality of generated code, compilation pass rate and property specification compliance.

[0126] Compared with the prior art, the code generation method based on requirement refinement and consistency verification provided by the present invention has the following beneficial effects:

[0127] 1. The present invention converts a large amount of effective information into vectors for storage by establishing a vector index library, so that LLM can retrieve relevant vectors as context information through the vector index library without having to input this information into LLM as prompts, thereby improving the accuracy and efficiency of the answers output by LLM; and, by persisting the information in the database, the information can be retrieved and provided to LLM at any time as needed, which is equivalent to providing LLM with long-term memory capability and improving the correctness of LLM's answers.

[0128] 2. The present invention first searches for vectors with high similarity to the original requirement document vectors in the constructed vector index library, then searches for matching vectors with the highest semantic similarity to the original requirement document through the sentence windows of these vectors, and finally converts the matching vectors into sentences in their context order as part of the LLM context input, so that the LLM can obtain knowledge that is not mentioned in the training and is truly relevant to the input content without retraining before answering, thereby improving its accuracy in identifying relevant content in the original requirement document and providing a basis for improving the correctness of refined requirements and refined property specifications.

[0129] 3. The present invention sends the pattern design document, the current refined property specification document, the SPEC statement error information, and the refined requirement document to the auxiliary modification robot, so that the robot can check and modify the refined requirement document and the refined property specification document based on sufficient external information prompts; and through multiple rounds of verification-modification-in-the-loop operations, it iteratively generates the refined property specification document and the refined requirement document after consistency verification, thereby further improving the correctness of the refined requirement document and providing a basis for improving the correctness of the automatically generated code.

[0130] 4. The present invention splits the task of repairing the code and generating the modified code into three independent tasks: bug analysis, bug repair, and code generation. The tasks are respectively assigned to different robots for processing, so that each robot can perform its own duties and avoid requiring a single robot to have a long memory due to excessively long context information. This not only improves the efficiency and correctness of defect repair, but also reduces the impact of "catastrophic forgetting" caused by LLM. The present invention iteratively generates verified code through multiple rounds of verification-modification-in-the-loop operations, thereby improving the robustness of the PLC code and the correctness of its syntax and functions.

[0131] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0132] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A code generation method based on requirements refinement and consistency verification, It is characterized in that The method comprises the following steps: Build a use case design knowledge base and a use case design prompt word template, and generate a refined requirement document based on the original requirement document, the use case design knowledge base, and the use case design prompt word template; Construct a property specification knowledge base and a pattern design prompt word template, generate an original property specification document based on the original requirement document, generate a pattern design document based on the refined requirement document and the pattern design prompt word template, generate a refined property specification document based on the pattern design document and the property specification knowledge base and perform syntax verification on it and consistency verification between the original property specification document and the refined property specification document, and obtain a verified refined property specification document and its corresponding refined requirement document; The original code is generated based on the verified refined property specification document and its corresponding refined requirement document, and the correctness of the original code syntax and function is judged. If it is incorrect, the original code is modified based on the large language model until its code syntax and function are correct or the maximum number of modifications is reached to obtain the verified code.

2. The code generation method based on requirement refinement and consistency verification according to claim 1, It is characterized in that A PLC code robot is constructed to generate original code; the verified refined requirement document, refined property specification document, and PLC syntax error example set are sent to the PLC code robot to generate original code.

3. The code generation method based on requirement refinement and consistency verification according to claim 2, It is characterized in that A PLC code repair group chat is constructed to analyze and repair code defects; syntax and function error information is sent to the PLC code repair group chat, and the defects are repaired by analyzing the error information; and the modified code is generated by the PLC code robot.

4. The code generation method based on requirement refinement and consistency verification according to claim 3, It is characterized in that The PLC code repair group chat includes: a bug analysis robot, a bug repair robot, and a user agent robot; wherein the bug analysis robot is used to analyze the received syntax and functional error information and send the inferred error cause to the bug repair robot; the bug repair robot is used to modify the code bug according to the received error cause and send the modified code to the PLC code robot; the user agent robot is used to act as an agent for the user to interact with other robots.

5. The code generation method based on requirement refinement and consistency verification according to claim 1, It is characterized in that A second keyword related to pattern design is established, and a pattern design prompt word template is constructed according to the second keyword, the number of variables, the use case in the first keyword, and the event flow; the second keyword includes a pattern and parameters to be filled in.

6. The code generation method based on requirement refinement and consistency verification according to claim 5, It is characterized in that The refined requirement document and the pattern design prompt word template are input into the large language model, so that the large language model extracts the content corresponding to the first keyword from the refined requirement document according to the pattern design prompt word template, and generates several corresponding patterns for each use case in the refined requirement document. Each pattern is filled with corresponding content based on the content corresponding to the extracted first keyword according to the pattern design prompt word template, thereby generating a pattern design document.

7. The code generation method based on requirement refinement and consistency verification according to claim 5, It is characterized in that The pattern design document is segmented according to the pattern to obtain a number of pattern design document fragments, and for each pattern design document fragment, a third vector matching it is obtained in the property specification knowledge base, and the matching third vector is converted into a sentence to obtain a third sentence matching it.

8. The code generation method based on requirement refinement and consistency verification according to claim 7, It is characterized in that For each pattern design document fragment: convert the pattern design document fragment into a fourth vector; calculate the vector similarity between the fourth vector and all third vectors, and select the first k third vectors with the largest vector similarity; set the sentence window size, and establish the third sentence windows corresponding to the first k third vectors according to the sentence window size; calculate the semantic similarity between the fourth vector and all corresponding third sentence windows, select all third vectors corresponding to the third sentence windows with the largest semantic similarity as the third vector matching the pattern design document fragment, and convert the matched third vectors into sentences in context order to obtain the third sentence matching the pattern design document fragment.

9. The code generation method based on requirement refinement and consistency verification according to claim 7, It is characterized in that All pattern design document fragments and their matching third sentences are input into the large language model, so that the large language model extracts the variable expression corresponding to the parameter to be filled in the pattern design document fragment from the third sentence according to the pattern design document fragment, thereby generating a refined property specification document.

10. The code generation method based on requirement refinement and consistency verification according to claim 1, It is characterized in that Collect second knowledge documents related to property specifications, segment the second knowledge documents and store them in a vector index library to form a property specification knowledge base; parse and clean the second knowledge documents, segment the cleaned second knowledge documents by sentences, convert all the segmented sentences into third vectors and store them in the vector index library, and all the stored third vectors constitute the property specification knowledge base.

Citation Information

Cited By

  • Multi-agent-based programmable logic controller structured text generation method

    CN120450033A

  • Computing device and method for synthesizing code to generate training data

    CN120524999A

  • Code generation method and system based on large model

    CN120763072A

  • A large model-based code generation method and system

    CN120763072B