A task execution method, device, storage medium and equipment based on pseudocode

Abstract

This specification discloses a pseudocode-based task execution method. This method transforms the task to be executed into corresponding pseudocode, and uses the semantic precision of the pseudocode to constrain the overall logical accuracy of the task execution process. For specific steps that require knowledge to execute, they are expressed by special operators defined by natural language. The LLM is used to execute the special operators based on a knowledge base containing the knowledge, thereby improving the accuracy and reliability of the entire task execution.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a task execution method, apparatus, storage medium and device based on pseudocode. Background Technology

[0002] With the development of artificial intelligence (AI) technology, large language models (LLM) have been widely applied in various fields.

[0003] However, LLM may suffer from problems such as lack of knowledge, knowledge errors, or outdated knowledge due to limitations in training data. These problems may cause LLM to experience "illusions" when performing tasks, thereby reducing the accuracy and reliability of LLM in performing tasks.

[0004] Therefore, how to perform tasks accurately and reliably based on LLM has become an urgent problem to be solved. Summary of the Invention

[0005] This specification provides a pseudocode-based task execution method, apparatus, storage medium, and electronic device to partially solve the problems existing in the prior art.

[0006] The embodiments in this specification adopt the following technical solutions:

[0007] This specification provides a pseudocode-based task execution method, which includes:

[0008] Obtain pre-written pseudocode corresponding to the business logic process. The pseudocode contains operator names of special operators, and the calculation logic of the special operators is defined by natural language.

[0009] The pseudocode is input into a compiler corresponding to a preset type of computer program code, so that the compiler can compile the pseudocode into the preset type of computer program code, which is then used as the target code.

[0010] The runtime environment for running the preset type of computer program code is configured to load business data.

[0011] Based on the business data, the target code is run in the aforementioned runtime environment;

[0012] When the execution reaches a statement in the target code that contains the operator name, the pre-trained Large Language Model (LLM) is invoked;

[0013] The business data and / or the intermediate results obtained from running the target code are used as input parameters to the LLM for the special operator corresponding to the operator name. The LLM then executes the calculation logic of the special operator based on the input parameters and a preset knowledge base to determine the execution result of the special operator.

[0014] The statement continues to be executed based on the result of the execution of the special operator.

[0015] This specification provides a pseudocode-based task execution device, the device comprising:

[0016] The acquisition module is used to acquire pre-written pseudocode corresponding to the business logic process. The pseudocode contains the operator name of a special operator, and the calculation logic of the special operator is defined by natural language.

[0017] A compilation module is used to input the pseudocode into a compiler corresponding to a preset type of computer program code, so that the compiler can compile the pseudocode into the preset type of computer program code as object code.

[0018] The loading module is used to run the computer program code of the preset type in the runtime environment and load business data;

[0019] The runtime module is used to run the target code based on the business data through the runtime environment.

[0020] The calling module is used to invoke the pre-trained Large Language Model (LLM) when the execution reaches a statement in the target code that contains the operator name;

[0021] The operator execution module is used to input the business data and / or the intermediate results obtained by currently running the target code as input parameters of the special operator corresponding to the operator name into the LLM, so that the LLM executes the calculation logic of the special operator according to the input parameters and the preset knowledge base to determine the execution result of the special operator;

[0022] The running module is also used to continue executing the statement based on the execution result of the special operator.

[0023] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described pseudocode-based task execution method.

[0024] This specification provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the above-described pseudocode-based task execution method.

[0025] The above-described at least one technical solution adopted in the embodiments of this specification can achieve the following beneficial effects:

[0026] This specification discloses a pseudocode-based task execution method. The method pre-writes pseudocode corresponding to a business logic process, containing operator names for specific operators. The computational logic of these operators is defined using natural language. When executing the task corresponding to the business logic process, the pseudocode is compiled into a preset type of computer program code as the target code. This target code is then run in the runtime environment of the preset type of computer program code. When a statement containing the operator name of a specific operator is executed, a pre-trained LLM is invoked to execute the computational logic of that specific operator, determining the execution result. Based on this result, the statement continues to be executed. This method transforms the task to be executed into corresponding pseudocode, using the semantic precision of the pseudocode to constrain the overall logical accuracy of the task execution process. Specific steps requiring knowledge are expressed using special operators defined in natural language. The LLM executes these special operators based on a knowledge base containing this knowledge, thereby improving the accuracy and reliability of the entire task execution. Attached Figure Description

[0027] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings:

[0028] Figure 1 A flowchart of a pseudocode-based task execution method provided in the embodiments of this specification;

[0029] Figure 2 A schematic diagram of a pseudocode-based task execution device provided in the embodiments of this specification;

[0030] Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this specification. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0032] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0033] Figure 1 A flowchart of a pseudocode-based task execution method provided in this specification includes the following steps:

[0034] S100: Obtain pre-written pseudocode corresponding to the business logic process. The pseudocode contains the operator name of a special operator, and the calculation logic of the special operator is defined by natural language.

[0035] In the embodiments described in this specification, the following are employed: Figure 1 The device used to perform the task as illustrated can be any electronic device, such as a computer, server, or server cluster consisting of multiple servers. For ease of description, a server will be used as an example below.

[0036] The server can first obtain pre-written pseudocode corresponding to the business logic process. This pseudocode can be manually written in advance based on the business logic process, or it can be generated based on the business logic process through a pre-trained LLM.

[0037] The business logic processes described in this specification include Standard Operating Procedures (SOPs). An SOP is a standard operating procedure for a task, described in natural language. It is a document that details the work, workflow, and steps involved in that task. It provides the organization's internal standard process specifications for the business corresponding to that task, for internal personnel to refer to and follow during operation. In addition to the standardized process for the task, an SOP also includes clearly defined responsibilities and authorities for each role in the task, emergency measures, quality standards and assessments, document management, and other content.

[0038] The embodiments in this manual will be described below using the above-mentioned task as an example of a risk control task.

[0039] When the above task is a risk control task based on the recorded risk logs, the business logic process of the risk control task (i.e., the risk control logic process) can be described in the following natural language:

[0040] The first step is to analyze the cmd field to determine whether the cmd field has been granted executable permissions. If the cmd command line has not been granted executable permissions, the conclusion is that no action is required.

[0041] The second step is to determine whether the installation package is for commonly used software. If it is, the conclusion is that no action is required.

[0042] The third step is to determine if the cmd field has been manually reviewed by security operations personnel within the past N days and has been determined to require no action. For example, if the chmod command line is determined to require no action, then the risk log does not need to be processed.

[0043] The fourth step is to determine, based on the results of the above three steps, whether the risk log requires no action or generates an attack alert.

[0044] The aforementioned risk log is a risk log of other businesses recorded by the user when performing other business operations, and the aforementioned risk control task is a task that needs to perform risk control on the other businesses based on the risk log.

[0045] Pseudocode, on the other hand, is an informal, computer program-like descriptive method used to express the logic of algorithms or programs without strictly adhering to the syntax rules of any specific computer program code. It lies between natural language and actual code, aiming to make the core ideas of algorithms easier to understand without getting bogged down in syntactic details.

[0046] When generating the above pseudocode using a pre-trained LLM, the SOP described in natural language can be pre-input into the pre-trained large language model LLM. This LLM can be pre-deployed on a server, or it can be deployed on other devices. If deployed on other devices, the server can send the business logic process to those devices, allowing them to input the received business logic process into the LLM. The following explanation uses the deployment of the LLM on a server as an example.

[0047] After inputting the SOP into the LLM, the LLM can identify the steps contained in the SOP and the logical relationships between the steps.

[0048] Generally, a Standard Operating Procedure (SOP) consists of several steps arranged according to a certain logical relationship. In the example above, the first step is executed. If the result of the first step is negative, no action is required. If the result is positive, the second step is executed. If the result of the second step is positive, no action is required. Otherwise, the third step is executed. If the result of the third step is positive, no action is required. Otherwise, an attack alert is generated. Therefore, the logical relationship between different steps in an SOP constitutes the basic framework of the business logic process. Specifically, the execution of each step requires the knowledge corresponding to the SOP, including the concepts, attributes, and relationships between the entities required in the SOP. For example, in the first step of the example above, it is necessary to determine whether the `cmd` field has been granted executable permissions. The knowledge required to execute this first step includes at least: the concept of the `cmd` field, the concept of file executable permissions, and the relationship between the `cmd` field and file executable permissions (i.e., which `cmd` fields indicate that file executable permissions have been granted, and which indicate that file executable permissions have not been granted).

[0049] Therefore, it can be seen that a Standard Operating Procedure (SOP) is essentially composed of the logical relationships between the steps and the knowledge required to execute each step. Thus, in order to generate pseudocode for executing the SOP, LLM needs to identify the steps contained in the SOP and the logical relationships between them based on the SOP described in natural language.

[0050] Specifically, since identifying the steps included in the Standard Operating Procedure (SOP) and the logical relationships between them may require a knowledge base corresponding to the SOP, in this embodiment of the specification, the knowledge base corresponding to the SOP can be pre-injected into the pre-trained LLM. This knowledge base may include at least one of the following:

[0051] The knowledge graph corresponding to this SOP includes the conceptual descriptions of each entity required in the SOP (used to describe what the entity is, such as the concept of the cmd command line in the example above), the attributes of each entity (used to describe all possible attributes of the entity in this SOP), and the relationships between the entities.

[0052] The SOP requires the calling interface of the tools, including APIs or SDKs. For example, in the second step of the above example, it is necessary to determine whether the installation package is a common software installation package. This requires calling the tool to obtain the list of common software installation packages, which requires calling the tool through the calling interface of the tool.

[0053] After injecting the knowledge base containing the knowledge graph corresponding to the SOP and / or the calling interface of the required tools into the pre-trained LLM, when the server inputs an SOP described in natural language into the LLM, the LLM can identify the semantics of the SOP described in natural language based on its own reasoning ability, map the concepts in the SOP to entities in the knowledge graph, and then break down the entire SOP into several steps according to the semantics, and finally reason out the logical relationship between each step.

[0054] Then, the LLM can generate pseudocode to represent the relationships between the steps based on the logical relationships.

[0055] Because natural language inherently possesses a certain degree of ambiguity and vagueness—a flaw inherent in any natural language—directly using natural language to express the logical relationships between steps in a business process is highly likely to lead to logical confusion in those steps. In particular, when others other than the person who wrote the SOP read it, their understanding may be inaccurate due to the ambiguity and vagueness of natural language, thus affecting the task itself.

[0056] Compared to natural language, pseudocode has clearer semantics and a more rigorous logical structure, which can effectively avoid the aforementioned problems of natural language. Therefore, in the embodiments of this specification, for the logical relationships between the steps in the SOP, the server can generate pseudocode to represent the logical relationships between the steps based on the identified logical relationships between the steps, as the framework code.

[0057] While generating the framework code, the server can generate at least one special operator for each step, based on the natural language used to describe the step in the SOP and the preset knowledge base corresponding to the SOP.

[0058] Since natural language is more natural and more understandable than computer programming languages, the knowledge required for each step in the SOP is represented by a special operator defined in natural language in the embodiments of this specification.

[0059] In this specification, the computational logic of the special operators described in the embodiments is defined using natural language, but the operator name is still a function name in pseudocode form. Therefore, the special operators defined in natural language in this specification are essentially functions defined in pseudocode, which are high-order abstract expressions of the logic that implements the step.

[0060] Specifically, for each step identified from the Standard Operating Procedure (SOP), the LLM can determine the knowledge required to execute the step based on the semantics of the natural language used to describe the step in the identified SOP, and the knowledge base corresponding to the SOP. This knowledge includes one or more entities in the knowledge graph, one or more attributes of that entity, the relationship between that entity and other entities, and / or the calling interface of the tool required for the step. Once the knowledge required to execute the step is determined, at least one special operator for executing the step can be generated. The computational logic of this special operator is defined in natural language, and the operator name is still a pseudocode function name.

[0061] Continuing with the previous example, the first step was to determine whether the cmd field had executable permissions. Therefore, LLM can generate the operator name hop_judge and define the special operator hop_judge corresponding to this operator name as follows:

[0062] def hop_judge

[0063] Determine if {cmd} has been granted executable permissions.

[0064] Therefore, the definition of the special operator hop_judge is entirely based on the natural language concept of "determining whether {cmd} has been granted executable permissions", making the operator more understandable.

[0065] The above is just an example of using one special operator to express a step. Those skilled in the art should understand that in practical applications, for more complex steps, more special operators (such as two or more operators) can be used to express the step.

[0066] After generating the framework code and special operators, the server can generate pseudocode for executing the SOP based on the framework code and the operator name of the special operator generated for each step.

[0067] As explained above, an SOP is essentially composed of the logical relationships between the steps and the knowledge required to execute each step. The logical relationships have been expressed using generated framework code, and the knowledge required to execute each step has been expressed using special operators defined in natural language. Therefore, LLM can combine this framework code and the special operators corresponding to each step to generate pseudocode for executing the business logic process.

[0068] Of course, since pseudocode does not require strict syntax, the pseudocode, including the framework code and special operators, can also be written manually. This specification does not limit this.

[0069] After the pseudocode corresponding to the SOP is written in advance, the server can obtain the pseudocode when the task corresponding to the SOP is executed.

[0070] S102: Input the pseudocode into a preset pseudocode compiler, so that the pseudocode is compiled into computer program code of a preset type by the pseudocode compiler, which is used as target code.

[0071] In the embodiments of this specification, after the server obtains the pseudocode corresponding to the SOP through the above step S100, it can input the pseudocode into the LLM and call the preset pseudocode compiler through the LLM to compile the pseudocode into computer program code of the preset type.

[0072] The computer program code of the preset type described in this specification includes Python code, but can also be any other type of computer program code. For ease of description, the following explanation uses Python code as an example only.

[0073] The server can compile the pseudocode obtained in step S100 into Python code, which can be called by the aforementioned LLM, through a pre-deployed pseudocode compiler. This pseudocode compiler converts human-readable pseudocode into executable, actual computer program code of a preset type. In other words, the pseudocode compiler can convert non-strict pseudocode syntax into legal code that conforms to the syntax of a preset type of computer program code, helping developers quickly generate runnable computer program code without manually handling syntax details.

[0074] Specifically, when compiling pseudocode into Python code, the pseudocode compiler can keep the operator name and calculation logic of special operators contained in the pseudocode unchanged, without worrying about errors when executing the compiled Python code later.

[0075] Continuing with the previous example, the LLM calls the pseudocode compiler to compile the pseudocode corresponding to the above SOP into the following Python code:

[0076] def evaluate(cmd):

[0077] if hop_judge_a("Check if (cmd) has been granted executable permissions") #Higher-order condition

[0078] pkg = hop_get("Get the installation package used by (cmd)")

[0079] If not pkg in hop_knowledge_retrieve("common installation packages") #Advanced knowledge concept matching

[0080] If not hop_judge_b("Check history, (cmd) whether it was manually judged as requiring no action in history")

[0081] Return "Attack Alert"

[0082] Return "No action required"

[0083] In the Python code above (i.e., the target code), the content within ("") is the definition of the corresponding special operator in natural language. These special operators include hop_judge_a, hop_get, hop_knowledge_retrieve, and hop_judge_b. Based on the framework code expressed in Python above, the entire risk control logic process is as follows: First, operator hop_judge_a is executed to determine if (cmd) has executable permissions. If yes, operator hop_get is executed to retrieve the installation package used by (cmd) and assign it to pkg. Otherwise, "No action required" is output. After retrieving the installation package used by (cmd) and assigning it to pkg, operator hop_knowledge_retrieve is executed to retrieve commonly used installation packages and determine if pkg is not among them. If yes, operator hop_judge_b is executed. Otherwise, "No action required" is output. Finally, operator hop_judge_b is executed to determine if (cmd) has not been manually judged as requiring no action in the history. If so, "Attack Alert" is output; otherwise, "No action required" is output.

[0084] As can be seen, the logical process represented by the Python code is completely consistent with the original risk control logic process described in natural language. Therefore, regardless of whether it is the Python code or the uncompiled pseudocode, even if the personnel in the department performing risk control operations change, the Python code or pseudocode can be read by other personnel (other than those who wrote the original risk control logic process) without causing logical confusion in each step due to the ambiguity and vagueness inherent in natural language. Moreover, although the special operators in the Python code are defined by natural language, the operator names are still function names in Python code format. Therefore, the target code itself is composed of Python framework code referencing various Python-formatted operator names. From a formal perspective, it is still Python code. Therefore, the target code is not only human-readable but can also be directly executed in the Python execution environment to implement the aforementioned risk control logic process.

[0085] Furthermore, since the target code described above appears to be Python code in form, but contains several special operators whose computational logic is defined by natural language, the target code described above can essentially be called a knowledge higher-order program (HOP). The target code described above will be referred to as HOP below.

[0086] S104: Run the computer program code of the preset type in the runtime environment and load the business data.

[0087] After obtaining the HOP through step S102, the server can run the runtime environment of the computer program code of that preset type, and then load the business data required by the HOP. For example, when the HOP is Python code, the server can launch the Python runtime environment and load the business data. Wherein, when the task corresponding to the aforementioned SOP is a risk control task, the business data may include, but is not limited to, risk logs that require risk control.

[0088] S106: Based on the business data, run the target code through the operating environment.

[0089] In the embodiments of this specification, the framework code of the HOP is written by the aforementioned preset type of computer program code. Even if the code contains special operators, the operator names of the special operators conform to the function name format requirements of the preset type of computer program code. Therefore, the server can run the HOP based on the loaded business data, using the business data as the input parameter of the entire HOP, and run the HOP through the runtime environment of the preset type of computer program code.

[0090] S108: When the execution reaches a statement in the target code that contains the operator name, the pre-trained Large Language Model (LLM) is invoked.

[0091] pass Figure 1 After step S108, which involves launching the runtime environment of the computer program code of the preset type and loading the business data, the HOP can be run through the runtime environment based on the business data.

[0092] When the execution reaches a special operator defined in natural language within the HOP, the aforementioned LLM needs to be invoked. Since the framework of this HOP is still predefined type computer program code, the runtime environment using this predefined type of computer program code can directly run the HOP. As for the special operators defined in natural language within the HOP, these special operators are essentially user-defined functions within the predefined type of computer program code. The HOP references these function names, meaning it executes the function corresponding to that name, obtains the execution result, and then continues executing subsequent code based on that result. Therefore, when the execution reaches the operator name of a special operator defined in natural language within the HOP, the predefined computational logic of that special operator needs to be invoked. When the runtime environment parses the computational logic of this special operator, if it determines that the computational logic is defined in natural language, the runtime environment can directly invoke the aforementioned LLM.

[0093] S110: Input the business data and / or the intermediate results obtained by currently running the target code as input parameters of the special operator corresponding to the operator name into the LLM, so that the LLM executes the calculation logic of the special operator corresponding to the operator name according to the input parameters and the preset knowledge base, so as to determine the execution result of the special operator.

[0094] After calling the LLM mentioned above, it can be accessed through Figure 1 In step S11, the business data and / or the intermediate results obtained from the current running of the HOP are used as input parameters to the LLM, so that the LLM executes the calculation logic of the operator according to the input parameters and the knowledge base corresponding to the above business logic process to determine the execution result of the operator.

[0095] Since the computational logic of this special operator is defined by natural language, the runtime environment of the computer program code of this preset type cannot execute the special operator normally. Therefore, the server can use the business data and the intermediate results obtained by running the HOP as input parameters of the special operator into the LLM, so that the LLM can execute the computational logic of the special operator according to the input parameters and the knowledge in the preset knowledge base, and obtain the execution result of the special operator.

[0096] Continuing with the previous example, for the special operator hop_judge_a in the HOP, its natural language-defined computational logic is "to determine whether (cmd) has been granted executable permissions." Business data, i.e., pre-recorded risk logs, can be used as the input parameter for this special operator. Both the natural language-defined computational logic of this special operator and the input parameter are input into the LLM. Prompt information can also be input to the LLM, guiding it to first identify the cmd command line (i.e., an entity in the knowledge graph) in the input parameter based on the knowledge base corresponding to the risk control logic process. Then, based on the knowledge base and the identified cmd command line, it determines the attributes of the cmd command line in the input parameter (i.e., the attributes of the entity in the knowledge graph). Based on these attributes, it determines whether the cmd command line has been granted executable permissions (i.e., the relationship between different entities in the knowledge graph). The judgment result is used as the output parameter of the special operator hop_judge_a, returning to the Python runtime environment, allowing the Python runtime environment to continue executing subsequent code in the HOP.

[0097] For example, for the special operator hop_knowledge_retrieve in the above HOP, its natural language definition of computational logic is "common installation packages". The computational logic of this special operator can be input into the LLM. Prompt information can also be input into the LLM. Under the guidance of the prompt information, the LLM will first determine the tool for obtaining the list of common installation packages based on the knowledge base corresponding to the risk control logic process. Then, it will call the tool through the calling interface to obtain the common installation packages. Finally, the obtained list of common installation packages will be used as the output parameter of the special operator hop_knowledge_retrieve and returned to the Python runtime environment. The Python runtime environment will then continue to execute the subsequent code in the HOP to determine whether the intermediate result pkg of the current HOP is not included in the list of common installation packages.

[0098] S112: Continue executing the statement based on the execution result of the special operator.

[0099] Once the execution result of the special operator is obtained, the server can continue to execute the statements containing the special operator based on the execution result, until the entire HOP is executed.

[0100] exist Figure 1 In step S108, since the computational logic of the special operator is defined by natural language and the LLM may experience "illusions," in order to ensure the accuracy of the LLM's execution of the special operator, in this embodiment of the specification, after the LLM executes the computational logic of a special operator and obtains the execution result, it is also necessary to perform a correctness check on the execution result obtained by the LLM in executing the special operator. If the correctness check passes, the LLM can return the execution result to the runtime environment of the computer program code of the preset type mentioned above, so that the runtime environment can continue to execute the HOP until the entire HOP is executed. If the correctness check fails, the LLM can re-execute the computational logic of the special operator to redetermine the execution result of the special operator until the obtained execution result passes the correctness check, or the number of times the LLM executes the computational logic of the special operator reaches the preset number.

[0101] If the LLM executes the computational logic of the special operator a preset number of times, and the execution result of the last determined special operator still fails the correctness check, an assistance execution request can be generated and output by the LLM. This assistance execution request is used to request human assistance in obtaining the execution result of the special operator. This assistance execution request can be sent to a user device, whereby the user can input the execution result of the special operator according to the computational logic and input parameters carried in the assistance execution request. The user device then returns the user-input execution result to the LLM. The LLM receives the execution result returned by the user device and considers the received execution result as the execution result that has passed the correctness check. Furthermore, based on the computational logic of the special operator and the received user-input execution result, supervised fine-tuning training can be performed on at least some of the model parameters in the LLM, such as supervised fine-tuning training only on the LoRA model parameters in the LLM.

[0102] If the correctness check passes, the LLM returns the execution result of the special operator to the execution environment of the computer program code of the preset type, which then continues to execute the statement until the entire HOP has been executed.

[0103] In the embodiments of this specification, when the server performs correctness verification on the execution result obtained by performing a special operator calculation logic on the LLM, it can also perform correctness verification based on the knowledge base corresponding to the above SOP.

[0104] Specifically, after executing the computational logic of this special operator, the LLM can output the execution result along with the reasoning process by which it obtained the execution result, or the reason for obtaining the execution result. The server can then obtain the reasoning process by which the LLM obtained the execution result and, based on the reasoning process and the aforementioned pre-set knowledge base corresponding to the SOP, verify the correctness of the execution result obtained by the LLM.

[0105] The server can either use another LLM (hereinafter referred to as the verification LLM) to pre-inject the aforementioned pre-set knowledge base corresponding to the SOP into the verification LLM, and use the LLM to verify the correctness of the execution result based on the reasoning process and the aforementioned pre-set knowledge base corresponding to the SOP, or it can still use the LLM that executes the special operator to verify the correctness of the execution result.

[0106] Regardless of the LLM used to verify the correctness of the execution result, the LLM used for correctness verification can identify the entities and relationships between them contained in the above reasoning process and the execution result obtained by executing the special operator. Based on the identified entities and their relationships, each triplet to be verified is determined, where each triplet contains two entities and the relationship between the two entities. For each triplet to be verified, the knowledge graph in the preset knowledge base is used to determine whether the triplet matches at least one triple in the knowledge graph. If it does, the triplet passes the verification; otherwise, it fails. If all triplets pass the verification, the correctness verification of the execution result obtained by the LLM using the special operator is considered successful; if at least one triplet fails, the correctness verification of the execution result obtained by the LLM using the special operator is considered unsuccessful.

[0107] When determining whether a triple to be verified matches at least one triple contained in the knowledge graph, specifically, it can be determined whether the semantics of the triple to be verified matches the semantics of at least one triple contained in the knowledge graph. If so, it is determined that the triple to be verified matches at least one triple contained in the knowledge graph; otherwise, it is determined that the triple to be verified does not match any triple contained in the knowledge graph.

[0108] Let's continue using the risk control task as an example. Suppose the business data to be loaded includes emails, and the calculation logic of a certain special operator is to determine whether the email has a phishing risk based on its content. According to the knowledge graph contained in the preset knowledge base, there are two triples. One triple is the entity "salary" and the entity "wages". The relationship between the two entities is an inclusion relationship, that is, "salary" conceptually includes "wages". The other triple is the entity "salary" and the entity "phishing risk". The relationship between the two entities is a membership relationship. That is, if the email content contains the entity "salary", then the email has a phishing risk.

[0109] If, when the LLM executes the special operator in the aforementioned HOP, the loaded business data (i.e., the email content to be risk-controlled) contains the entity "salary," and the result of executing this special operator is "no phishing risk," the reasoning process is "the email content only contains 'salary,' not 'compensation,' therefore there is no phishing risk." Then, the LLM extracts two triples to be verified from this reasoning process and execution result. One triple is the entity "compensation" and the entity "salary," with a non-inclusion relationship between the two entities, meaning the reasoning process indicates that "compensation" conceptually does not include "salary." The other triple is the entity "compensation" and the entity "phishing risk," with a membership relationship between the two entities. For the first triple, its semantics do not match the semantics of any triple in the aforementioned knowledge graph. Therefore, the first triple fails the verification. Thus, it can be determined that the LLM's execution result of this special operator, "no phishing risk," is incorrect and fails the correctness check.

[0110] Furthermore, in the embodiments of this specification, if pseudocode corresponding to the SOP is generated by LLM, in order for LLM to accurately convert the input natural language description of the SOP into the aforementioned pseudocode, it is still not enough to simply inject the knowledge base corresponding to the SOP into LLM. It is also necessary to fine-tune and train the LLM in advance under the business scenario corresponding to the SOP.

[0111] Specifically, when fine-tuning the LLM, a sample logic process described in natural language can be obtained first. This sample logic process also contains several steps, referred to below as the sample steps. This sample logic process and the aforementioned SOP are SOPs within the same business scenario; that is, the knowledge base corresponding to the sample logic process is exactly the same as the knowledge base corresponding to the aforementioned SOP.

[0112] After obtaining the sample logic process, it can be input into the LLM to be trained. The LLM identifies the sample steps and the logical relationships between them. Based on these relationships, pseudocode is generated to represent them, serving as the framework code to be optimized. For each sample step, at least one special operator is generated to execute it, based on the natural language used to describe it and a pre-defined knowledge base. This operator is also defined in natural language, and its name is a pseudocode function name. Finally, based on the framework code and the operator names generated for each sample step, pseudocode is generated to execute the sample logic process.

[0113] After the LLM to be trained generates the pseudocode to be optimized, the optimized pseudocode can be determined after adjustments. This can be done manually to ensure that the logical relationships between the sample steps expressed in the optimized framework code fully conform to the original sample logic process, and that the natural language corresponding to the operators to be optimized, defined by natural language, is more closely aligned with the knowledge in the knowledge base corresponding to the sample logic process, making the computational logic of the operators to be optimized clearer and more accurate. Of course, the pseudocode to be optimized can also be adjusted using other trained LLMs; this embodiment does not limit this approach.

[0114] After obtaining the optimized pseudocode, supervised fine-tuning (SFT) training can be performed on the LLM to be trained based on the optimized pseudocode. That is, the optimized pseudocode is used as a label corresponding to the sample logic process, which serves as a supervision signal to adjust the model parameters in the LLM to be trained, making the LLM adaptable to generating various SOPs in the business scenario. The device used to train the LLM to be trained is the same as the aforementioned execution... Figure 1 The devices used in the illustrated methods can be the same device or different devices. Furthermore, because the embodiments in this specification aim to preserve the original inference capabilities of the LLM as much as possible, adapting it only to the specific business scenario, when performing SFT on the LLM, not all model parameters of the LLM are adjusted; instead, only a portion of the model parameters are adjusted. The adjusted model parameters may include only the LoRA model parameters within the LLM.

[0115] After obtaining the LLM through the SFT training described above, the server can generate pseudocode corresponding to the SOP for the business scenario using the LLM. When the above business scenario is a risk control business scenario, that is, when the above SOP is the risk control logic process under the risk control business scenario, the loaded business data can be a pre-recorded risk log. This risk log is the risk log of other businesses recorded by the user when performing other businesses, and the risk control business is the business that needs to perform risk control on the other businesses based on the risk log.

[0116] Those skilled in the art should understand that the above description uses SOP as an example of risk control logic process. In fact, the method provided in the embodiments of this specification can execute pseudocode corresponding to business logic processes in any business scenario. Furthermore, the above-described preset type of computer program code is only illustrated using Python code as an example; the preset type of computer program code described in the embodiments of this specification can also be other types of computer program code.

[0117] The above is a pseudocode-based task execution method provided by the embodiments of this specification. Based on the same idea, this specification also provides corresponding devices, storage media and electronic devices.

[0118] Figure 2 This specification provides a schematic diagram of a pseudocode-based task execution device, which includes:

[0119] The acquisition module 201 is used to acquire pre-written pseudocode corresponding to the business logic process. The pseudocode contains the operator name of a special operator, and the calculation logic of the special operator is defined by natural language.

[0120] Compilation module 202 is used to input the pseudocode into a compiler corresponding to a preset type of computer program code, so that the compiler can compile the pseudocode into the preset type of computer program code as object code;

[0121] Loading module 203 is used to run the computer program code of the preset type in the runtime environment and load business data;

[0122] Run module 204 is used to run the target code based on the business data through the runtime environment;

[0123] Module 205 is used to invoke a pre-trained large language model (LLM) when the execution reaches a statement in the target code that contains the operator name.

[0124] Operator execution module 206 is used to input the business data and / or the intermediate results obtained by currently running the target code as input parameters of the special operator corresponding to the operator name into the LLM, so that the LLM executes the calculation logic of the special operator according to the input parameters and the preset knowledge base to determine the execution result of the special operator;

[0125] The running module 204 is further configured to continue executing the statement based on the execution result of the special operator.

[0126] Optionally, the operator name conforms to the function name format of the computer program code of the preset type;

[0127] The calling module 205 is specifically used to call the calculation logic of the special operator corresponding to the operator name when the target code contains a statement with the operator name; and to call the LLM when the calculation logic is defined by natural language through the runtime environment.

[0128] Optionally, the running module 204 is specifically used to perform a correctness check on the execution result; when the correctness check passes, to continue executing the statement based on the execution result of the special operator.

[0129] Optionally, the operator execution module 206 is further configured to, when the correctness check fails, re-execute the calculation logic of the special operator through the LLM to redetermine the execution result of the operator.

[0130] Optionally, the running module 204 is further configured to generate and output an assistance execution request when the number of times the LLM executes the computational logic of the special operator reaches a preset number, and the execution result of the last determined special operator still fails the correctness check; and receive the execution result of the special operator input by the user equipment in response to the assistance execution request as the execution result that has passed the correctness check.

[0131] Optionally, the running module 204 is specifically used to obtain the reasoning process of the LLM to obtain the execution result; and to perform correctness verification on the execution result based on the reasoning process and the preset knowledge base.

[0132] Optionally, the running module 204 is specifically used to: identify the entities contained in the reasoning process and the execution result, as well as the relationships between the entities; determine each triplet to be verified based on the identified entities and the relationships between them, wherein each triplet to be verified contains two entities and the relationship between the two entities; for each triplet to be verified, determine whether the triplet to be verified matches at least one triplet contained in the knowledge graph in the preset knowledge base, if so, determine that the triplet to be verified passes the verification, otherwise, determine that the triplet to be verified fails the verification; if all triplets to be verified pass the verification, then the correctness verification of the execution result is determined to be passed; if at least one triplet to be verified fails the verification, then the correctness verification of the execution result is determined to be failed.

[0133] This specification also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can be used to perform the pseudocode-based task execution method described above.

[0134] based on Figure 1 The pseudocode-based task execution method shown in this specification also provides the following embodiments: Figure 3 The diagram shows the structure of the electronic device. Figure 3At the hardware level, the electronic device includes a processor, internal bus, network interface, memory, and non-volatile storage, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile storage into memory and then runs it to implement the pseudocode-based task execution method described above.

[0135] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A task execution method based on pseudocode, the method comprising: Obtain pre-written pseudocode corresponding to the business logic process. The pseudocode contains operator names of special operators, and the calculation logic of the special operators is defined by natural language. The pseudocode is input into a preset pseudocode compiler, which compiles the pseudocode into computer program code of a preset type, which is then used as target code. The runtime environment for running the preset type of computer program code is configured to load business data. Based on the business data, the target code is run in the aforementioned runtime environment; When the execution reaches a statement in the target code that contains the operator name, the pre-trained Large Language Model (LLM) is invoked; The business data and / or the intermediate results obtained from running the target code are used as input parameters to the LLM for the special operator corresponding to the operator name. The LLM then executes the calculation logic of the special operator based on the input parameters and a preset knowledge base to determine the execution result of the special operator. The statement continues to be executed based on the result of the execution of the special operator.

2. The method as described in claim 1, wherein the operator name conforms to the function name format of the preset type of computer program code; When the execution reaches a statement in the target code that contains the operator name, the pre-trained LLM is invoked, specifically including: When the execution reaches a statement in the target code that contains the operator name, the calculation logic of the special operator corresponding to the operator name is invoked; When the computational logic is defined in natural language by parsing the runtime environment, the LLM is invoked.

3. The method as described in claim 1, wherein the statement is further executed based on the execution result of the special operator, specifically includes: The correctness of the execution result is verified; If the correctness check passes, the statement continues to be executed based on the execution result of the special operator.

4. The method of claim 3, further comprising: If the correctness check fails, the calculation logic of the special operator is re-executed through the LLM to redetermine the execution result of the operator.

5. The method of claim 4, further comprising: When the number of times the LLM executes the computation logic of the special operator reaches a preset number, and the execution result of the last determined special operator still fails the correctness check, an assistance execution request is generated and output. The execution result of the special operator input by the user equipment in response to the assistance execution request is received as the execution result that has passed the correctness check.

6. The method as described in claim 3, wherein the correctness verification of the execution result specifically includes: The reasoning process for obtaining the execution result from the LLM; The correctness of the execution result is verified based on the reasoning process and the preset knowledge base.

7. The method as described in claim 6, wherein the correctness verification of the execution result is performed based on the reasoning process and the preset knowledge base, specifically including: Identify the entities contained in the reasoning process and the execution result, as well as the relationships between the entities; Based on the identified entities and the relationships between them, each triplet to be verified is determined, wherein each triplet to be verified contains two entities and the relationship between the two entities. For each triplet to be verified, based on the knowledge graph in the preset knowledge base, it is determined whether the triplet to be verified matches at least one triplet contained in the knowledge graph. If so, the triplet to be verified is determined to pass the verification; otherwise, the triplet to be verified is determined to fail the verification. If all the triples to be verified pass the verification, then the correctness verification of the execution result is deemed to have passed. If at least one triplet to be verified fails the verification, then the correctness verification of the execution result is determined to have failed.

8. A pseudocode-based task execution device, the device comprising: The acquisition module is used to acquire pre-written pseudocode corresponding to the business logic process. The pseudocode contains the operator name of a special operator, and the calculation logic of the special operator is defined by natural language. A compilation module is used to input the pseudocode into a compiler corresponding to a preset type of computer program code, so that the compiler can compile the pseudocode into the preset type of computer program code as object code. The loading module is used to run the computer program code of the preset type in the runtime environment and load business data; The runtime module is used to run the target code based on the business data through the runtime environment. The calling module is used to invoke the pre-trained Large Language Model (LLM) when the execution reaches a statement in the target code that contains the operator name; The operator execution module is used to input the business data and / or the intermediate results obtained by currently running the target code as input parameters of the special operator corresponding to the operator name into the LLM, so that the LLM executes the calculation logic of the special operator according to the input parameters and the preset knowledge base to determine the execution result of the special operator; The running module is also used to continue executing the statement based on the execution result of the special operator.

9. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in any one of claims 1-7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method described in any one of claims 1-7.

Images