Task execution method and device based on pseudo code, storage medium and equipment

Abstract

The embodiment of the invention discloses a task execution method based on a pseudo code, which is characterized in that a task needing to be executed is changed into a corresponding pseudo code, the overall logic accuracy of the task execution process is constrained by means of the semantic accuracy of the pseudo code, and specific steps which can only be executed by adopting knowledge can be executed by adopting knowledge. And if the knowledge exists, the knowledge is expressed by a special operator defined by a natural language, and the special operator is executed on the basis of a knowledge base containing the knowledge through LLM, so that the accuracy and reliability of executing the whole task are improved.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a pseudocode-based task execution method, apparatus, storage medium, and device. Background Art

[0002] With the development of artificial intelligence (AI) technology, large language models (LLM) have been widely used in various fields.

[0003] However, due to the limitations of training data, LLM may suffer from problems such as lack of knowledge, knowledge errors, or outdated knowledge. These problems may cause LLM to experience "hallucinations" when performing tasks, thereby reducing the accuracy and reliability of LLM's task execution.

[0004] Therefore, how to accurately and reliably execute tasks based on LLM has become an urgent problem to be solved. Summary of the Invention

[0005] The embodiments of this specification provide a task execution method, device, storage medium, and electronic device based on pseudocode to partially solve the problems existing in the above-mentioned prior art.

[0006] The embodiments of this specification adopt the following technical solutions: This specification provides a pseudo-code-based task execution method, which includes: Obtain pre-written pseudocode corresponding to the business logic process, wherein the pseudocode includes the operator name of a special operator, and the calculation logic of the special operator is defined in natural language; Inputting the pseudocode into a compiler corresponding to a preset type of computer program code, so as to compile the pseudocode into the preset type of computer program code as a target code through the compiler; Running the operating environment of the preset type of computer program code and loading business data; Based on the business data, running the target code through the operating environment; When the target code reaches a statement containing the operator name, the pre-trained large language model LLM is called; Input the business data and / or the intermediate result obtained by currently running the target code as input parameters of the special operator corresponding to the operator name into the LLM, so that the LLM executes the calculation logic of the special operator according to the input parameters and a preset knowledge base to determine the execution result of the special operator; Continue executing the statement according to the execution result of the special operator.

[0007] This specification provides a pseudo-code-based task execution device, the device comprising: An acquisition module is used to acquire pre-written pseudocode corresponding to the business logic process, wherein the pseudocode includes the operator name of a special operator, and the calculation logic of the special operator is defined in natural language; a compiling module, configured to input the pseudocode into a compiler corresponding to a preset type of computer program code, so as to compile the pseudocode into the preset type of computer program code as a target code through the compiler; A loading module, configured to run an operating environment of the preset type of computer program code and load business data; An operating module, configured to operate the target code through the operating environment based on the business data; A calling module, configured to call a pre-trained large language model LLM when running to a statement containing the operator name in the target code; An operator execution module, configured to input the business data and / or the intermediate result obtained by currently running the target code as input parameters of the special operator corresponding to the operator name into the LLM, so that the LLM executes the calculation logic of the special operator according to the input parameters and a preset knowledge base to determine the execution result of the special operator; The execution module is further configured to continue executing the statement according to the execution result of the special operator.

[0008] This specification provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned pseudo-code-based task execution method is implemented.

[0009] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the above-mentioned pseudocode-based task execution method is implemented.

[0010] At least one of the above technical solutions adopted in the embodiments of this specification can achieve the following beneficial effects: The embodiment of this specification discloses a task execution method based on pseudocode, which pre-writes pseudocode corresponding to a business logic process, wherein the pseudocode contains the operator name of a special operator, and the calculation logic of the special operator is defined by natural language. When executing the task corresponding to the business logic process, the pseudocode is compiled into a preset type of computer program code as a target code, and then the target code is run through the running environment of the preset type of computer program code. When a statement containing the operator name of the special operator is run, a pre-trained LLM is called to execute the calculation logic of the special operator to determine the execution result of the special operator, and the statement is continued to be executed according to the execution result. This method constrains the overall logical accuracy of the task execution process by converting the task to be executed into the corresponding pseudocode, and for the specific steps that require knowledge to be executed, they are expressed by special operators defined by natural language, and the special operators are executed by LLM based on the knowledge base containing the knowledge, thereby improving the accuracy and reliability of the execution of the entire task. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The exemplary embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation of this specification. In the drawings: Figure 1 A flowchart of a task execution method based on pseudocode provided in an embodiment of this specification; Figure 2 A schematic diagram of a task execution device based on pseudocode provided in an embodiment of this specification; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0012] To make the objectives, technical solutions, and advantages of this specification more clear, the following will clearly and completely describe the technical solutions of this specification in conjunction with the specific embodiments of this specification and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this specification.

[0013] The technical solutions provided by the embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0014] Figure 1 The following is a flowchart of a task execution method based on pseudocode provided in an embodiment of this specification, which specifically includes the following steps: S100: Obtain pre-written pseudo code corresponding to the business logic process, wherein the pseudo code includes the operator name of a special operator, and the calculation logic of the special operator is defined in a natural language.

[0015] In the embodiments of this specification, Figure 1 The device that performs the task in the method shown can be any electronic device, such as a computer, a server, or a server cluster composed of multiple servers, etc. For the convenience of description, the following description is made using a server as an example.

[0016] The server may first obtain pre-written pseudocode corresponding to the business logic process. The pseudocode may be manually written based on the business logic process, or generated based on the business logic process by a pre-trained LLM.

[0017] The business logic processes described in this specification include Standard Operating Procedures (SOPs). An SOP is a document describing a task's standard operating procedures in natural language. It details the work, workflow, and steps involved in the task. It provides standard internal process specifications for the business corresponding to the task, for internal personnel to reference and follow during operations. In addition to the standardized process for the task, the SOP also includes clear responsibilities and authorities for each role within the task, emergency measures, quality standards and assessments, and document management.

[0018] The embodiments of this specification are described below using the above-mentioned tasks as risk control tasks as an example.

[0019] When the above task is to perform risk control tasks based on recorded risk logs, the business logic process of the risk control task (that is, the risk control logic process) can be described in the following natural language: The first step is to analyze the cmd field to determine whether the cmd field grants the file executable permission. If the cmd command line does not grant the file executable permission, the conclusion is that no action is required. The second step is to determine whether the installation package is a commonly used software installation package. If it is a commonly used installation software, the conclusion is that no action is required; The third step is to determine whether the cmd field has been manually analyzed and judged by security operations personnel within the past N days and determined to require no action. For example, if the chmod command line is judged to require no action, then no action is required for this risk log. The fourth step is to determine whether the risk log does not need to be handled or generates an attack alarm based on the results of the above three steps.

[0020] The risk log is a risk log of other businesses recorded by the user when performing other businesses, and the risk control task is a task that requires risk control of the other businesses based on the risk log.

[0021] Pseudocode is an informal, computer-program-like description used to express the logic of an algorithm or program without strictly adhering to the grammatical rules of a specific computer program. It lies somewhere between natural language and actual code, aiming to make it easier to understand the core ideas of an algorithm without getting bogged down in the grammatical details.

[0022] When using a pre-trained LLM to generate the pseudocode above, the SOP described in natural language can be pre-entered into the pre-trained large language model (LLM). This LLM can be pre-deployed on a server, or alternatively, on other devices. If deployed on other devices, the server can send the business logic process to the other devices, allowing them to input the received business logic process into the LLM. The following description uses the LLM deployed on the server as an example.

[0023] After the SOP is input into the LLM, the LLM can be used to identify the steps contained in the SOP and the logical relationships between the steps.

[0024] Generally, a SOP consists of several steps arranged according to certain logical relationships. For example, in the example above, the first step is executed first. If the first step evaluates to "no," the output is "no action required." If the evaluation is "yes," the second step is executed. If the second step evaluates to "yes," the output is "no action required." Otherwise, the third step is executed. If the third step evaluates to "yes," the output is "no action required." Otherwise, an attack alert is generated. Therefore, the logical relationships between the different steps in the SOP constitute the basic framework of the business logic process. Specifically, each step requires the use of knowledge specific to the SOP, including the concepts and attributes of the entities required by the SOP, as well as the relationships between them. For example, in the example above, the first step requires determining whether the "cmd" field grants executable permissions to the file. The knowledge required to execute this first step includes at least the concept of the "cmd" field, the concept of file executable permissions, and the relationship between the "cmd" field and file executable permissions (i.e., which "cmd" field indicates that the file has been granted executable permissions, and which "cmd" field indicates that the file has not been granted executable permissions).

[0025] It can be seen that an SOP is essentially composed of the logical relationship between each step and the knowledge required to execute each step. Therefore, to generate pseudocode for executing the SOP, LLM needs to identify the steps contained in the SOP and the logical relationship between the steps based on the SOP described in natural language.

[0026] Specifically, since the identification of the steps contained in the SOP and the logical relationships between the steps may also require a knowledge base corresponding to the SOP, in the embodiment of this specification, the knowledge base corresponding to the SOP may be pre-injected into the pre-trained LLM. The knowledge base may include at least one of the following: The knowledge graph corresponding to the SOP includes the concept description of each entity required in the SOP (used to describe what the entity is, such as the concept of cmd command line in the above example), the attributes of each entity (used to describe all possible attributes of the entity in the SOP), and the relationship between each entity; The calling interface of the tool required in the SOP includes API or SDK, etc. For example, in the second step of the above example, it is necessary to determine whether the installation package is an installation package of commonly used software. In this case, it is necessary to call the tool to obtain a list of installation packages of commonly used software. This requires calling the tool through the calling interface of the tool.

[0027] After injecting the above-mentioned knowledge base containing the knowledge graph corresponding to the SOP and / or the calling interface of the required tools into the pre-trained LLM, when the server inputs the SOP described in natural language to the LLM, the LLM can identify the semantics of the SOP described in the natural language based on its own reasoning ability, and map the concepts in the SOP into entities in the knowledge graph. Then, based on the semantics, the entire SOP is split into several steps, and finally the logical relationship between the steps is inferred.

[0028] Then, LLM can generate pseudo codes for representing the steps according to the logical relationship.

[0029] Since natural language itself has a certain degree of fuzziness and ambiguity, which is an inherent defect of any natural language, directly using natural language to express the logical relationship between the steps in the business logic process is likely to lead to logical confusion of each step. Especially when the SOP is read by people other than the person who wrote it, the fuzziness and ambiguity of natural language may lead to inaccurate understanding, which will affect the task itself.

[0030] Compared with natural language, pseudocode has clearer semantics and stricter logical structure, and can effectively avoid the above-mentioned problems of natural language. Therefore, in the embodiment of this specification, for the logical relationship between each step in the SOP, the server can generate pseudocode for representing the logical relationship between each step as a framework code based on the identified logical relationship between each step through LLM.

[0031] While generating the framework code, the server may generate, for each step, at least one special operator for executing the step based on the natural language used to describe the step in the SOP and a preset knowledge base corresponding to the SOP.

[0032] As for the knowledge required for each step in the SOP, natural language is more natural and more understandable than computer programming language. Therefore, in the embodiments of this specification, a step is represented by a special operator defined in natural language.

[0033] The computational logic of the special operators described in the embodiments of this specification is defined in natural language, but the operator names of these special operators are still function names in pseudocode. Therefore, the special operators defined in natural language in this specification are essentially custom functions in pseudocode, which are high-level abstract expressions of the logic that implements the steps.

[0034] Specifically, LLM can determine the knowledge required to execute each step identified from the SOP based on the semantics of the natural language used to describe the step in the identified SOP and the knowledge base corresponding to the SOP, such as one or more entities in the knowledge graph, one or more attributes of the entity, and the relationship between the entity and other entities, and / or the calling interface of the tool required for the step. After determining the knowledge required to execute the step, at least one special operator for executing the step can be generated based on it. The calculation logic of the special operator is defined in natural language, and the operator name is still the function name of the pseudocode.

[0035] Continuing with the previous example, the first step is to determine whether the cmd field grants the file executable permission. Therefore, LLM can generate the operator name hop_judge and define the special operator hop_judge corresponding to this operator name as follows: def hop_judge Determine whether {cmd} grants file executable permissions As can be seen, the definition of the special operator hop_judge is completely defined using natural language "judge whether {cmd} grants file executable permissions", making the operator more understandable.

[0036] The above is just an example of expressing a step using a special operator. Those skilled in the art should understand that in actual applications, for more complex steps, more special operators (such as more than two operators) can be used to express the step.

[0037] After the framework code and the special operators are generated, the server may generate pseudocode for executing the SOP based on the framework code and the operator name of the special operator generated for each step.

[0038] As explained above, an SOP is essentially composed of the logical relationship between each step and the knowledge required to execute each step. The logical relationship has been expressed using the generated framework code, and the knowledge required to execute each step has also been expressed using special operators defined in natural language. Therefore, LLM can combine the framework code and the special operators corresponding to each step to generate pseudocode for executing the business logic process.

[0039] Of course, since the pseudocode does not require strict syntax, the pseudocode including the framework code and special operators can also be directly written manually, and the embodiments of this specification do not limit this.

[0040] After the pseudo code corresponding to the SOP is written in advance, the server can obtain the pseudo code when executing the task corresponding to the SOP.

[0041] S102: Input the pseudocode into a preset pseudocode compiler, so that the pseudocode is compiled into a preset type of computer program code as a target code by the pseudocode compiler.

[0042] In an embodiment of the present specification, after the server obtains the pseudocode corresponding to the SOP through the above step S100, it can input the pseudocode into the LLM and call a preset pseudocode compiler through the LLM to compile the pseudocode into a preset type of computer program code through the pseudocode compiler.

[0043] The computer program code of the preset type described in this specification includes Python code, or may be any other type of computer program code. For ease of description, the following description will only take Python code as an example.

[0044] The server can compile the pseudocode obtained in step S100 into Python code as the target code using a pre-deployed pseudocode compiler that can be called by the LLM. The pseudocode compiler is used to convert human-readable pseudocode into executable, actual, preset computer program code. In other words, the pseudocode compiler can convert non-strict pseudocode syntax into legal code that conforms to the preset computer program code syntax, helping developers quickly generate executable computer program code without having to manually process syntax details.

[0045] Specifically, when the pseudocode compiler compiles the pseudocode into Python code, for special operators contained in the pseudocode, the operator name and calculation logic of the special operator can be kept unchanged, without worrying about errors when the compiled Python code is subsequently executed.

[0046] Continuing with the above example, LLM calls the pseudocode compiler to compile the pseudocode corresponding to the above SOP into the following Python code: def evaluate(cmd): if hop_judge_a("Judge (cmd) whether to grant file executable permissions") #High-level judgment pkg = hop_get("Get the installation package used by (cmd)") if not pkg in hop_knowledge_retrieve("Common Installation Package") #High-level knowledge concept matching if not hop_judge_b("Check the history to see if (cmd) has been manually judged as not requiring action") return "Attack Alarm" return "No need to dispose" Among them, in the above Python code (i.e., the target code), the content in ("") is the definition of the corresponding special operator using natural language. These special operators include hop_judge_a, hop_get, hop_knowledge_retrieve, and hop_judge_b. The framework code expressed in the above Python shows that the entire risk control logic process is as follows: first, execute the operator hop_judge_a to determine whether (cmd) has been granted executable permissions. If so, continue to execute the operator hop_get to obtain the installation package used by (cmd) and assign it to pkg. If not, directly output "No action required". After obtaining the installation package used by (cmd) and assigning it to pkg, execute the operator hop_knowledge_retrieve to obtain the commonly used installation package and determine whether pkg is not among the commonly used installation packages. If so, continue to execute the operator hop_judge_b. If not, directly output "No action required". Execute the operator hop_judge_b to determine whether (cmd) has not been manually judged as not requiring action in history. If so, output "Attack alarm"; otherwise, output "No action required".

[0047] It can be seen that the logical process expressed through the Python code is completely consistent with the original risk control logic process described in natural language. Therefore, whether through the above Python code or uncompiled pseudocode, even if the personnel in the department performing the risk control business change, when the Python code or pseudocode is read by other personnel (other than the person who wrote the original risk control logic process), the logical confusion of each step caused by the ambiguity and ambiguity inherent in natural language can be avoided. Moreover, although the special operators in the above Python code are defined in natural language, the operator names are still function names in Python code format. Therefore, the above target code itself is composed of Python framework code referencing various Python-formatted operator names. From a formal perspective, it is still Python code. Therefore, the above target code is not only readable by humans, but can also be directly executed in the Python execution environment to implement the above risk control logic process.

[0048] In addition, since the above target code is formally Python code, but contains several special operators whose calculation logic is defined by natural language, the above target code can actually be called a high-order program (HOP). The above target code will be referred to as HOP below.

[0049] S104: Running the operating environment of the preset type of computer program code and loading business data.

[0050] After obtaining the HOP in step S102, the server can run the runtime environment for the pre-set computer program code type and then load the business data required by the HOP. For example, if the HOP is Python code, the server can launch the Python runtime environment and load the business data. If the task corresponding to the SOP is a risk control task, the business data may include, but is not limited to, risk logs requiring risk control.

[0051] S106: Based on the business data, run the target code through the running environment.

[0052] In the embodiments of the present specification, the framework code of the HOP is written by the above-mentioned preset type of computer program code. Even if the code contains special operators, the operator name of the special operator also conforms to the function name format requirements of the preset type of computer program code. Therefore, the server can run the HOP based on the loaded business data, using the business data as the input parameter of the entire HOP, by pulling up the running environment of the preset type of computer program code.

[0053] S108: When the target code reaches a statement containing the operator name, the pre-trained large language model LLM is called.

[0054] pass Figure 1 After the execution environment of the preset type of computer program code is pulled up and the service data is loaded in step S108, the HOP can be executed through the execution environment based on the service data.

[0055] When running to the special operator defined by natural language in the HOP, it is necessary to call the above-mentioned LLM. Among them, since the framework of the HOP is still a preset type of computer program code, the operating environment using the preset type of computer program code can directly run the HOP. As for the special operator defined by natural language contained in the HOP, since the special operator is essentially a custom function in the preset type of computer program code, the HOP references the function name, which means executing the function corresponding to the function name, obtaining the execution result, and then continuing to execute the subsequent code according to the execution result. Therefore, when running to the operator name of the special operator defined by natural language in the HOP, it is necessary to call the pre-defined calculation logic of the special operator. When the calculation logic of the special operator is parsed by the operating environment, if it is parsed that the calculation logic is defined by natural language, the operating environment can directly call the above-mentioned LLM.

[0056] S110: Input the business data and / or the intermediate result obtained by currently running the target code into the LLM as the input parameters of the special operator corresponding to the operator name, so that the LLM executes the calculation logic of the special operator corresponding to the operator name according to the input parameters and the preset knowledge base to determine the execution result of the special operator.

[0057] After calling the above LLM, you can Figure 1 In step S11 shown, the business data and / or the intermediate results obtained by currently running the HOP are input into the LLM as the input parameters of the operator, so that the LLM executes the calculation logic of the operator according to the input parameters and the knowledge base corresponding to the above-mentioned business logic process to determine the execution result of the operator.

[0058] Since the calculation logic of the special operator is defined by natural language, the running environment of the preset type of computer program code cannot execute the special operator normally. Therefore, the server can input the business data and the intermediate results obtained by currently running the HOP as the input parameters of the special operator into the LLM, so that the LLM can execute the calculation logic of the special operator according to the input parameters and the knowledge in the preset knowledge base to obtain the execution result of the special operator.

[0059] Continuing with the above example, for the special operator hop_judge_a in the above HOP, its natural language defined calculation logic is "Judge whether (cmd) grants file executable permissions." In this case, business data, that is, the pre-recorded risk log, can be used as the input parameter of the special operator. The natural language defined calculation logic of the special operator and the input parameter are both input into the LLM. Prompt information can also be input into the LLM. Under the guidance of the prompt information, the LLM first identifies the cmd command line in the input parameter (i.e., the entity in the knowledge graph) based on the knowledge base corresponding to the risk control logic process. Then, based on the knowledge base and the identified cmd command line, the LLM determines the attributes of the cmd command line in the input parameter (i.e., the attributes of the entity in the knowledge graph). Based on the attributes, it determines whether the cmd command line grants file executable permissions (i.e., the relationship between different entities in the knowledge graph). The judgment result is used as the output parameter of the special operator hop_judge_a and returned to the Python runtime environment, allowing the Python runtime environment to continue executing subsequent code in the HOP.

[0060] For another example, for the special operator hop_knowledge_retrieve in the above HOP, the calculation logic defined in natural language is "common installation packages". The calculation logic defined in natural language of the special operator can be input into LLM, and prompt information can be input to LLM, so that the LLM, under the guidance of the prompt information, first determines the tool for obtaining the list of commonly used installation packages based on the knowledge base corresponding to the risk control logic process, and then calls the tool through the calling interface of the tool to obtain the commonly used installation packages through the called tool. Finally, the obtained list of commonly used installation packages is used as the output parameter of the special operator hop_knowledge_retrieve and returned to the Python runtime environment, so that the Python runtime environment continues to execute subsequent code in the HOP to determine whether the intermediate result pkg of the current running of the HOP is not included in the list of commonly used installation packages.

[0061] S112: Continue executing the statement according to the execution result of the special operator.

[0062] After obtaining the execution result of the special operator, the server can continue to execute the statement containing the special operator according to the execution result of the special operator until the entire HOP is executed.

[0063] exist Figure 1In step S108 shown, since the calculation logic of the special operator is defined through natural language and the LLM may have "hallucinations", in order to ensure the accuracy of the LLM's execution of the special operator, in the embodiment of this specification, after each time the LLM executes the calculation logic of a special operator and obtains the execution result, it is also necessary to perform a correctness check on the execution result obtained by the LLM executing the special operator. If the correctness check passes, the LLM can return the execution result to the operating environment of the above-mentioned preset type of computer program code, so that the operating environment continues to execute the HOP until the entire HOP is executed. If the correctness check fails, the calculation logic of the special operator can be re-executed by the LLM to re-determine the execution result of the special operator until the obtained execution result passes the correctness check, or the number of times the LLM executes the calculation logic of the special operator reaches a preset number.

[0064] If the number of times that the LLM executes the calculation logic of the special operator reaches a preset number, and the execution result of the special operator determined for the last time still fails to pass the correctness check, an assistance execution request can be generated by the LLM and output. The assistance execution request is used to request manual assistance to obtain the execution result of the special operator. The assistance execution request can be sent to a user device, and the user can input the execution result of the special operator to the user device according to the calculation logic and input parameters of the special operator carried in the assistance execution request, and the user device returns the execution result input by the user to the LLM. The LLM receives the execution result returned by the user device and uses the received execution result as an execution result that has passed the correctness check. It is also possible to perform supervised fine-tuning training on at least part of the model parameters in the LLM based on the calculation logic of the special operator and the execution result received from the user, such as only performing supervised fine-tuning training on the model parameters of LoRA in the LLM.

[0065] If the correctness check is passed, the LLM returns the execution result of the special operator to the execution environment of the preset type of computer program code, and the execution environment continues to execute the statement until the HOP is completely executed.

[0066] In the embodiment of this specification, when the server performs a correctness check on the execution result obtained by executing the calculation logic of a special operator on the LLM, the server may also perform a correctness check based on the knowledge base corresponding to the above-mentioned SOP.

[0067] Specifically, after executing the calculation logic of the special operator, LLM outputs the execution result and can also output the reasoning process of LLM obtaining the execution result, or the reason for obtaining the execution result. The server can obtain the reasoning process of LLM obtaining the execution result, and verify the correctness of the execution result obtained by LLM based on the reasoning process and the above-mentioned preset knowledge base corresponding to SOP.

[0068] Among them, the server can use another LLM (hereinafter referred to as the verification LLM) to pre-inject the above-mentioned preset knowledge base corresponding to the SOP into the verification LLM, and through the LLM, according to the reasoning process and the above-mentioned preset knowledge base corresponding to the SOP, perform correctness verification on the execution result. It can also still use the LLM that executes the special operator to perform correctness verification on the execution result.

[0069] Regardless of the LLM used to perform the correctness check on the execution result, the LLM used for correctness check can identify the entities and relationships between entities contained in the above-mentioned reasoning process and the execution result obtained by executing the special operator. Based on the identified entities and relationships between entities, each triple to be checked is determined, wherein each triple to be checked contains two entities and the relationship between the two entities. For each triple to be checked, based on the knowledge graph in the preset knowledge base, it is determined whether the triple to be checked matches at least one triple contained in the knowledge graph. If so, the triple to be checked is determined to have passed the check; otherwise, the triple to be checked is determined to have failed the check. If all triples to be checked pass the check, the correctness check of the execution result obtained by executing the special operator by the LLM is determined to have passed. If at least one triple to be checked fails the check, the correctness check of the execution result obtained by executing the special operator by the LLM is determined to have failed.

[0070] When determining whether a triple to be verified matches at least one triple contained in the knowledge graph, it can be specifically determined whether the semantics of the triple to be verified matches the semantics of at least one triple contained in the knowledge graph. If so, it is determined that the triple to be verified matches at least one triple contained in the knowledge graph; otherwise, it is determined that the triple to be verified does not match any triple contained in the knowledge graph.

[0071] Still taking the execution of risk control tasks as an example, assume that the business data that needs to be loaded includes emails, and the calculation logic of a special operator is to determine whether the email has phishing risks based on the content of the email. According to the knowledge graph contained in the preset knowledge base, it can be seen that there are two triples. One triple is the entity "salary" and the entity "wage". The relationship between the two entities is an inclusion relationship, that is, "salary" conceptually includes "wage". The other triple is the entity "salary" and the entity "phishing risk". The relationship between the two entities is a belonging relationship, that is, if the email content contains the entity "salary", then the email has phishing risks.

[0072] If, when LLM executes the special operator in the HOP, the loaded business data (i.e., the email content to be risk-controlled) contains the "salary" entity, and the result of executing this special operator is "no phishing risk," the reasoning process is "the email content only contains "salary" and not "salary," so it does not pose a phishing risk." From this reasoning process and execution result, LLM extracts two triples to be verified. One triple is the "salary" entity and the "salary" entity, and the relationship between the two entities is a non-inclusion relationship. In other words, the reasoning process indicates that "salary" conceptually does not contain "salary." The other triple is the "salary" entity and the "phishing risk" entity, and the relationship between the two entities is a belongs relationship. The semantics of the first triple do not match those of any triple in the knowledge graph, so it fails verification. Therefore, the LLM's execution result of this special operator, "no phishing risk," is incorrect and fails the correctness check.

[0073] In addition, in the embodiments of this specification, if a pseudocode corresponding to an SOP is generated by LLM, in order to enable the LLM to accurately convert the input natural language description of the SOP into the above-mentioned pseudocode, it is not enough to simply inject the knowledge base corresponding to the SOP into the LLM. The LLM also needs to be fine-tuned and trained in advance in the business scenario corresponding to the SOP.

[0074] Specifically, when fine-tuning the LLM, you can first obtain a sample logical process described in natural language. This sample logical process also includes several steps, hereinafter referred to as sample steps. This sample logical process and the aforementioned SOP are based on the same business scenario. That is, the knowledge base corresponding to the sample logical process is exactly the same as the knowledge base corresponding to the aforementioned SOP.

[0075] After obtaining the sample logical process, the sample logical process can also be input into the LLM to be trained. The LLM to be trained can identify the sample steps contained in the sample logical process and the logical relationship between the sample steps. According to the logical relationship between the sample steps, a pseudo code for representing the logical relationship between the sample steps is generated as the framework code to be optimized. Moreover, for each sample step, according to the natural language used to describe the sample step in the sample logical process and the knowledge base corresponding to the preset sample logical process, at least one special operator for executing the sample step is generated as the operator to be optimized. The calculation logic of the operator to be optimized is also defined in natural language, and the operator name of the operator to be optimized is also a function name in pseudo code form. Finally, according to the framework code to be optimized and the operator name of the operator to be optimized generated for each sample step, the pseudo code to be optimized for executing the sample logical process is generated.

[0076] After the LLM to be trained generates the pseudocode to be optimized, the optimized pseudocode after adjusting the pseudocode to be optimized can be determined. Among them, the pseudocode to be optimized can be adjusted manually so that the logical relationship between the sample steps expressed by the framework code to be optimized is completely consistent with the original sample logical process, and the natural language corresponding to the operator to be optimized defined by natural language contained therein is more in line with the knowledge in the knowledge base corresponding to the sample logical process, and the calculation logic of the operator to be optimized expressed is clearer and more accurate. Of course, the pseudocode to be optimized can also be adjusted by other trained LLMs, and this embodiment of the specification does not limit this.

[0077] After obtaining the optimized pseudocode, the LLM to be trained can be subjected to supervised fine-tuning training (SFT) based on the optimized pseudocode, that is, the optimized pseudocode is used as the annotation corresponding to the sample logic process, and used as the supervisory signal to adjust the model parameters in the LLM to be trained, so that the model parameters of the LLM to be trained are adjusted to make the LLM suitable for generating various SOPs in the business scenario. Figure 1 The devices used in the methods shown can be the same device or different devices. Furthermore, because the embodiments of this specification require that the original reasoning capabilities of the LLM be maintained as much as possible and adapted only to the service scenario, when performing SFT on the LLM, not all model parameters of the LLM are adjusted; instead, only some of the model parameters within the LLM are adjusted. The adjusted model parameters may only include the LoRA model parameters within the LLM.

[0078] After obtaining the LLM through the aforementioned SFT training, the server can use the LLM to generate the pseudocode corresponding to the SOP for the business scenario. If the business scenario is a risk control business scenario, that is, the SOP is the risk control logic process for the risk control business scenario, the loaded business data can be a pre-recorded risk log. This risk log is the risk log of other businesses recorded by the user when executing other businesses. The risk control business is the business that requires risk control of the aforementioned other business based on this risk log.

[0079] Those skilled in the art will appreciate that the above description uses SOP as an example of a risk control logic process. In reality, the methods provided in the embodiments of this specification can be used to execute the corresponding pseudocode for business logic processes in any business scenario. Furthermore, the aforementioned preset types of computer program code are described using Python code as an example. The preset types of computer program code described in the embodiments of this specification may also be other types of computer program code.

[0080] The above is a pseudo-code-based task execution method provided in an embodiment of this specification. Based on the same idea, this specification also provides corresponding devices, storage media, and electronic devices.

[0081] Figure 2 A schematic diagram of a task execution device based on pseudocode provided in an embodiment of this specification, the device comprising: An acquisition module 201 is configured to acquire pre-written pseudocode corresponding to a business logic process, wherein the pseudocode includes the operator name of a special operator whose calculation logic is defined in a natural language; A compiling module 202 is configured to input the pseudocode into a compiler corresponding to a preset type of computer program code, so as to compile the pseudocode into the preset type of computer program code as a target code through the compiler; A loading module 203 is configured to run an operating environment of the preset type of computer program code and load business data; An operating module 204 is configured to operate the target code through the operating environment based on the business data; A calling module 205 is configured to call a pre-trained large language model LLM when a statement containing the operator name is run in the target code; An operator execution module 206 is configured to input the business data and / or the intermediate result obtained by currently executing the target code as input parameters of the special operator corresponding to the operator name into the LLM, so that the LLM executes the calculation logic of the special operator based on the input parameters and a preset knowledge base to determine the execution result of the special operator; The execution module 204 is further configured to continue executing the statement according to the execution result of the special operator.

[0082] Optionally, the operator name conforms to the function name format of the preset type of computer program code; The calling module 205 is specifically used to call the calculation logic of the special operator corresponding to the operator name when running to a statement containing the operator name in the target code; and call the LLM when it is parsed through the running environment that the calculation logic is defined by a natural language.

[0083] Optionally, the running module 204 is specifically configured to perform a correctness check on the execution result; when the correctness check passes, continue to execute the statement according to the execution result of the special operator.

[0084] Optionally, the operator execution module 206 is further configured to, when the correctness check fails, re-execute the calculation logic of the special operator through the LLM to re-determine the execution result of the operator.

[0085] Optionally, the running module 204 is also used to generate and output an assisted execution request when the LLM executes the calculation logic of the special operator for a preset number of times and the execution result of the special operator determined for the last time still fails to pass the correctness check; and receive the execution result of the special operator input by the user device in response to the assisted execution request as an execution result that has passed the correctness check.

[0086] Optionally, the running module 204 is specifically configured to obtain the reasoning process by which the LLM obtains the execution result; and perform correctness verification on the execution result based on the reasoning process and the preset knowledge base.

[0087] Optionally, the operation module 204 is specifically used to identify the entities and the relationships between the entities contained in the reasoning process and the execution result; determine the triples to be verified based on the identified entities and the relationships between the entities, wherein each triple to be verified contains two entities and the relationships between the two entities; for each triple to be verified, based on the knowledge graph in the preset knowledge base, determine whether the triple to be verified matches at least one triple contained in the knowledge graph; if so, determine that the triple to be verified passes the verification; otherwise, determine that the triple to be verified fails the verification; if all triples to be verified pass the verification, determine that the correctness verification of the execution result passes; if at least one triple to be verified fails the verification, determine that the correctness verification of the execution result fails.

[0088] This specification also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it can be used to execute the pseudo-code-based task execution method provided above.

[0089] based on Figure 1 The pseudo-code-based task execution method shown in the embodiment of this specification also provides Figure 3 The structural diagram of the electronic device shown in FIG. Figure 3 At the hardware level, the electronic device includes a processor, an internal bus, a network interface, memory, and non-volatile storage, and may also include other hardware required for its operations. The processor reads the corresponding computer program from the non-volatile storage into the memory and then runs it, thereby implementing the aforementioned pseudocode-based task execution method.

[0090] The foregoing is merely an example of the present invention and is not intended to limit the present invention. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be included within the scope of the claims of the present invention.

Claims

1. A task execution method based on pseudocode, the method comprising: Obtain pre-written pseudocode corresponding to the business logic process, wherein the pseudocode includes the operator name of a special operator, and the calculation logic of the special operator is defined in natural language; Inputting the pseudocode into a preset pseudocode compiler so as to compile the pseudocode into a preset type of computer program code as a target code through the pseudocode compiler; Running the operating environment of the preset type of computer program code and loading business data; Based on the business data, running the target code through the operating environment; When the target code reaches a statement containing the operator name, the pre-trained large language model LLM is called; Input the business data and / or the intermediate result obtained by currently running the target code as input parameters of the special operator corresponding to the operator name into the LLM, so that the LLM executes the calculation logic of the special operator according to the input parameters and a preset knowledge base to determine the execution result of the special operator; Continue executing the statement according to the execution result of the special operator.

2. The method according to claim 1, wherein the operator name conforms to the function name format of the computer program code of the preset type; When the target code reaches a statement containing the operator name, the pre-trained LLM is called, specifically including: When running to a statement containing the operator name in the target code, calling the calculation logic of the special operator corresponding to the operator name; When it is found through the operating environment that the computing logic is defined in a natural language, the LLM is called.

3. The method according to claim 1, further comprising: Performing correctness check on the execution result; When the correctness check passes, the statement continues to be executed according to the execution result of the special operator.

4. The method of claim 3, further comprising: When the correctness check fails, the calculation logic of the special operator is re-executed through the LLM to re-determine the execution result of the operator.

5. The method of claim 4, further comprising: When the number of times the LLM executes the calculation logic of the special operator reaches a preset number, and the last determined execution result of the special operator still fails the correctness check, an assistance execution request is generated and output; An execution result of the special operator input by the user equipment in response to the execution assistance request is received as an execution result that has passed the correctness check.

6. The method according to claim 3, wherein correctness checking of the execution result specifically comprises: Obtaining the reasoning process of the LLM to obtain the execution result; The correctness of the execution result is verified based on the reasoning process and the preset knowledge base.

7. The method according to claim 6, further comprising: performing a correctness check on the execution result based on the reasoning process and the preset knowledge base, specifically comprising: Identifying entities and relationships between entities included in the reasoning process and the execution result; Determine triples to be verified based on the identified entities and the relationships between the entities, wherein each triple to be verified includes two entities and the relationship between the two entities; For each triple to be verified, judging whether the triple to be verified matches at least one triple contained in the knowledge graph according to the knowledge graph in the preset knowledge base; if so, determining that the triple to be verified passes verification; otherwise, determining that the triple to be verified fails verification; If all the triples to be verified pass the verification, it is determined that the correctness verification of the execution result passes; If at least one triplet to be verified fails the verification, it is determined that the correctness verification performed on the execution result fails.

8. A task execution device based on pseudocode, the device comprising: An acquisition module is used to acquire pre-written pseudocode corresponding to the business logic process, wherein the pseudocode includes the operator name of a special operator, and the calculation logic of the special operator is defined in natural language; a compiling module, configured to input the pseudocode into a compiler corresponding to a preset type of computer program code, so as to compile the pseudocode into the preset type of computer program code as a target code through the compiler; A loading module, configured to run an operating environment of the preset type of computer program code and load business data; An operating module, configured to operate the target code through the operating environment based on the business data; A calling module, configured to call a pre-trained large language model LLM when running to a statement containing the operator name in the target code; An operator execution module, configured to input the business data and / or the intermediate result obtained by currently running the target code as input parameters of the special operator corresponding to the operator name into the LLM, so that the LLM executes the calculation logic of the special operator according to the input parameters and a preset knowledge base to determine the execution result of the special operator; The execution module is further configured to continue executing the statement according to the execution result of the special operator.

9. A computer-readable storage medium storing a computer program, wherein the computer program implements the method according to any one of claims 1 to 7 when executed by a processor.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 7 when executing the program.

Images