Strategy generation method and apparatus, electronic device, and storage medium

Abstract

The present application discloses a strategy generation method and apparatus, an electronic device, and a storage medium. The method comprises: acquiring a strategy description of a target strategy to be generated, and a strategy generation agent, wherein the strategy generation agent comprises a large language model and a tool module used in conjunction with the large language model, and the tool module comprises a tool used for realizing strategy generation; and inputting the strategy description into the strategy generation agent to generate, by means of the large language model in the strategy generation agent and on the basis of the tool, a target strategy corresponding to the strategy description.

Description

Policy generation method and device, electronic equipment and storage medium

[0001] The present application claims priority to the Chinese patent application No. 202410418124.5, filed on April 8, 2024, the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the field of generative artificial intelligence, for example, to a policy generation method and device, an electronic equipment and a storage medium. BACKGROUND

[0003] Policy generation is applied in many scenarios, for example, in the intelligent marketing scenario, a marketing policy needs to be generated, in the autonomous driving scenario, a driving policy needs to be generated, and in the distributed scenario, a traffic scheduling policy needs to be generated. Therefore, it is crucial to realize automatic generation of the policy.

[0004] However, the adopted policy generation scheme has the problems of high labor cost and low flexibility. SUMMARY

[0005] The present application provides a policy generation method and device, an electronic equipment and a storage medium, to realize flexible generation of the policy with low labor cost.

[0006] According to an aspect of the present application, a policy generation method is provided, which can include:

[0007] Obtaining a policy description of a target policy to be generated and a policy generation agent, wherein the policy generation agent includes a large language model and a tool module applied in cooperation with the large language model, and the tool module includes a tool for realizing policy generation; inputting the policy description into the policy generation agent, to generate the target policy corresponding to the policy description based on the tool through the large language model in the policy generation agent.

[0008] According to another aspect of the present application, a policy generation device is provided, which can include:

[0009] An agent obtaining module is configured to obtain a policy description of a target policy to be generated and a policy generation agent, wherein the policy generation agent includes a large language model and a tool module applied in cooperation with the large language model, and the tool module includes a tool for realizing policy generation; a policy generation module is configured to input the policy description into the policy generation agent, to generate the target policy corresponding to the policy description based on the tool through the large language model in the policy generation agent.

[0010] According to another aspect of the present application, an electronic equipment is provided, which can include:

[0011] At least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the above-mentioned strategy generation method.

[0012] According to another aspect of the present application, a computer readable storage medium is provided, and the computer readable storage medium stores computer instructions for enabling a processor to implement the above-mentioned strategy generation method when the processor executes the computer instructions. BRIEF DESCRIPTION OF DRAWINGS

[0013] Fig. 1 is an example diagram of an architecture of a strategy generation intelligent agent according to an embodiment of the present application;

[0014] Fig. 2 is a flowchart of a strategy generation method according to an embodiment of the present application;

[0015] Fig. 3 is a flowchart of another strategy generation method according to an embodiment of the present application;

[0016] Fig. 4 is a schematic diagram of an example of sequential calling of multiple tools in a strategy generation method according to an embodiment of the present application;

[0017] Fig. 5 is a schematic diagram of an example of planning by a planning module based on ReAct in a strategy generation method according to an embodiment of the present application;

[0018] Fig. 6 is a flowchart of yet another strategy generation method according to an embodiment of the present application;

[0019] Fig. 7 is a flowchart corresponding to a strategy flow generated by using an LLM in a strategy generation method according to an embodiment of the present application;

[0020] Fig. 8 is a schematic diagram of an example of strategy flow generation in a strategy generation method according to an embodiment of the present application;

[0021] Fig. 9 is a flowchart of still another strategy generation method according to an embodiment of the present application;

[0022] Fig. 10 is a structural block diagram of a strategy generation apparatus according to an embodiment of the present application;

[0023] Fig. 11 is a structural schematic diagram of an electronic device for implementing a strategy generation method according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] The terms "first", "second", and the like in the description and in the claims of the present application and above drawings are used for distinguishing between similar objects and not necessarily for describing a specific sequential or chronological order. It is to be understood that the use of these terms herein is merely for distinguishing between the similar objects and the same can be referenced to other terms such as "one", "another", "at least one", "one or more" and / or the like. Similarily, the terms "target", "original" and the like in the description and in the claims of the present application and above drawings are used for distinguishing between similar objects and not necessarily for describing a specific sequential or chronological order. The scope of the present application is not intended to be limited to the exact order or sequence described herein, but to encompass any and all methodologies, which include those in which the various steps based on the embodiments described herein are performed in other sequences. Furthermore, the terms "comprise", "comprising", "include", "including", and the like, used in the description and the claims of the present application (as well as any variations thereof, such as "comprises", "comprising", "includes", "including" and the like) are intended to be used as physical-inclusive, such that when the use of such terms is made in the description and the claims of the present application, for example with respect to a process, method, system, product or apparatus, it is intended that at least the process, method, system, product or apparatus comprising those elements expressly listed is included. Other processes, methods, systems, products, and apparatuses that do not specifically exclude other elements are also "included".

[0025] In the technical solutions of the present application, the collection, collection, updating, analysis, processing, use, transmission, storage and the like of user personal information are in line with relevant laws and regulations, are used for legal purposes, and do not violate public order and good customs. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain user personal information security and network security.

[0026] Before introducing the embodiments of the present application, the causes of the problems existing in the policy generation scheme and the causes of the problems existing in the policy generation scheme are exemplarily described, so that the reasons why the policy generation scheme described in the embodiments of the present application can solve the problems are better understood.

[0027] For example, some policy generation platforms are provided on the market, which provide a visual operation interface, and a plurality of controls are displayed on the operation interface, and the plurality of controls have respective atomic capabilities. For example, in the intelligent marketing scenario, the plurality of controls can be coupon sending controls, push controls, AB experiment controls, and condition judgments. On this basis, the user can generate a policy on the operation interface by dragging and dropping the controls. However, this policy generation scheme requires the user to be familiar with the policy generation platform and understand the functions and use rules of each control, which requires the user to invest a certain learning cost or to apply under professional guidance. This will result in high labor cost of policy generation.

[0028] For example, with the rapid development of large language models (LLMs) in various industries, policy generation can also be based on LLMs. For example, policy templates are formed based on existing policies, the policy templates are respectively annotated with intentions, and then the LLM learns the annotated policy templates. On this basis, the LLM can identify the user's policy generation intention, and then find a matching template from multiple policy templates that matches the policy generation intention to generate a policy. However, this policy generation scheme is based on template matching, which means that it can only generate existing policies and cannot generate other policies other than existing policies, thereby affecting the flexibility of policy generation and failing to meet the individual needs of users.

[0029] To solve the above-mentioned problems of high labor costs and low flexibility, the present application proposes a policy generation scheme based on an agent (Agent). The agent has a policy generation function and can also be called a policy generation agent. On this basis, in order to better understand this policy generation scheme, the policy generation agent applied in the embodiments of the present application is first described.

[0030] For example, referring to FIG. 1, the policy generation agent includes an LLM and four modules, namely a memory module (Memory), a planning module (Plan), an action module (Action), and a tool module (Tools), which cooperate with the LLM. In other words, the policy generation agent can be obtained by externally connecting the four modules to the LLM.

[0031] The memory module includes short-term memory and long-term memory. The short-term memory can store dialogue context information for planning and decision-making in combination with dialogue history. The long-term memory part can store expert policy templates, historical high-frequency policy templates, and control instructions, etc. in a vector library to help the LLM understand policy-related knowledge.

[0032] The planning module can be used to understand the user's input policy description or policy generation task, and on this basis, the overall planning is carried out, the tools in the tool module that need to be called to complete the policy generation task are determined, and the execution order of these tools is formulated, and then subsequent planning is carried out according to the execution results of the tools.

[0033] The action module can be used to call corresponding tools to execute actions based on the planning results of the planning module. In practical applications, this action module can be used in conjunction with other agents, such as object selection agents, function calling agents, and object clarification agents. The object selection agent can extract object information from the policy description and match it with existing objects on the Customer Data Platform (CDP) to perform clarification selection.

[0034] The tool module integrates multiple tools for implementing strategy generation. These tools can be represented through natural language processing (NLP) tasks, code snippets, functions, or application programming interfaces (APIs). Each of these tools has its own atomic capabilities. For example, using the intelligent marketing scenario as an example, these tools may include a basic information identification tool, a strategy flow generation tool, a strategy flow inspection tool, a control information extraction tool, a property setting tool, a strategy template matching tool, and a platform skills tool. The strategy template matching tool can also be called a strategy template recommendation tool.

[0035] The basic information identification tool may be used to identify basic information in the policy description, such as the trigger type, trigger time, and trigger period.

[0036] The policy flow generation tool can be used to generate a usable policy flow based on the policy description.

[0037] The policy flow checking tool can be used to check the generated policy flow to prevent the policy flow from being unusable, for example, checking whether each control used in the policy flow complies with relevant rules.

[0038] The control information extraction tool can be used to extract control information, such as the proportion of each branch in the AB branch.

[0039] The property setting tool can be used to set control properties, such as applying the extracted control information to the control.

[0040] Next, the strategy generation process proposed in the embodiment of the present application will be described in detail.

[0041] FIG. 2 is a flowchart of a strategy generation method according to an embodiment of the present application. The embodiment can be applied to the case of strategy generation by using an agent. The method can be performed by a strategy generation apparatus according to an embodiment of the present application. The apparatus can be implemented in software and / or hardware. The apparatus can be integrated into an electronic device, which can be a user terminal or a server.

[0042] Referring to FIG. 2, the method according to an embodiment of the present application includes the following steps.

[0043] In S110, a strategy description of a target strategy to be generated and a strategy generation agent are obtained.

[0044] The strategy generation agent can include a large language model and a tool module applied in cooperation with the large language model. The tool module can include tools for implementing strategy generation.

[0045] The target strategy can be understood as a strategy to be generated. The strategy description can be used to describe the content of the target strategy. In actual applications, the strategy description can be represented in at least one of the forms of text, audio, picture, and video, without limitation. For example, the strategy description represented in the form of text is taken as an example. In the smart marketing scenario, for example, it can be "send a coupon to a user every morning at 9 o'clock, and if it succeeds, send a push notification, if the push fails, send a short message, if the user is not a member, send a short message to remind the user to open a membership" or "I want to send a coupon to a new user and compare the effects of the text based on the push notification and AB experiment" and the like. In the autonomous driving scenario, for example, it can be "reduce the driving speed on rainy days" or "brake when encountering a red light" and the like. In the distributed scenario, for example, it can be "distribute traffic evenly to multiple servers" and the like. The strategy description is obtained.

[0046] The strategy generation agent can be understood as an agent with a strategy generation function. The strategy generation agent can include an LLM and a tool module applied in cooperation with the LLM. The tool module can include tools for implementing strategy generation. The number of tools can be one, two, or more, which is related to the actual situation and is not limited herein. The strategy generation agent is obtained.

[0047] In S120, the strategy description is input into the strategy generation agent, so as to generate a target strategy corresponding to the strategy description by the large language model in the strategy generation agent based on the tools.

[0048] According to the above description, it can be known that the tools in the tool module can be used to implement strategy generation, which enables the LLM to generate a strategy by means of the capabilities of the tools.

[0049] Therefore, the policy description is input into the policy generation agent, so that the LLM in the policy generation agent can be used to generate a corresponding target policy based on the tool.

[0050] The technical scheme of the embodiment of the present application acquires the policy description of the target policy to be generated and acquires a policy generation agent, the policy generation agent including an LLM and a tool module cooperating with the LLM, the tool module integrating tools for implementing policy generation; the policy description is input into the policy generation agent to generate a corresponding target policy based on the tool by using the LLM in the policy generation agent. The above technical scheme realizes policy generation by using an Agent, which makes the user only need to input the policy description into the Agent without performing other operations, thereby reducing the labor cost (such as the cost of manually dragging a control, the cost of manual learning, and the cost of manual guidance) in the policy generation process. Moreover, the tool module is externally connected to the LLM in the Agent, which makes the LLM generate a personalized policy matching the policy description by means of the tools in the tool module, thereby solving the problem of low flexibility caused by the LLM only being able to generate a policy by means of the relevant knowledge (such as a policy template) learned by itself when the LLM is used alone to generate a policy. The two cooperate with each other to realize flexible policy generation based on a low labor cost.

[0051] FIG. 3 is a flowchart of another policy generation method provided by the embodiment of the present application. The embodiment is explained based on the above technical scheme. In the embodiment, the number of tools is at least one, and the policy generation agent further includes a planning module and an action module cooperating with the large language model; based on the tool, a target policy corresponding to the policy description is generated, including: using the planning module, based on the tool description of at least one tool, decomposing a policy generation task represented by the policy description into at least one subtask; using the action module, by calling the tools in the tool module, executing at least one subtask to generate a target policy corresponding to the policy description. Wherein, the same or corresponding terms as in the above embodiment are not repeated here.

[0052] Referring to FIG. 3, the method of the embodiment can include the following steps:

[0053] S210, acquiring the policy description of the target policy to be generated and the policy generation agent.

[0054] The policy generation agent includes a large language model and a tool module, a planning module, and an action module cooperating with the large language model, and the tool module includes at least one tool for implementing policy generation.

[0055] S220, inputting the policy description into the policy generation agent to execute S230 and S240 by using the large language model in the policy generation agent.

[0056] S230, decompose the policy generation task represented by the policy description into at least one subtask based on the tool description of the at least one tool by using a planning module.

[0057] The tool description can be used to describe the capabilities of the tool, i.e., according to the tool description, it can be known that the plurality of tools integrated in the tool module respectively have the functions. Still taking the agent shown in FIG. 1 as an example, according to the tool description, it can be known that the tool module integrates the basic information recognition tool having the basic information recognition function, the policy flow generation tool having the policy flow generation function, the policy flow checking tool having the policy flow checking function, the control information extraction tool having the control information extraction function, the attribute setting tool having the attribute setting function, and the policy template matching tool having the policy template matching function.

[0058] The policy generation task can be understood as a task of generating a target policy according to the policy description. Generally, the policy generation task is relatively complex, and therefore, in order to ensure the accuracy of the target policy generated based on the policy generation task, the policy generation task can be first decomposed into one or more relatively simple subtasks, and then the target policy is generated by executing the subtasks.

[0059] On this basis, according to the foregoing description, the plurality of tools integrated in the tool module respectively have respective functions, and these functions cooperate with each other to achieve policy generation. In addition, the subtasks are also subtasks decomposed for achieving policy generation. Therefore, the decomposition manner can be determined based on the tool description of the tool, and the task decomposition is performed based on the decomposition manner, and the subtasks obtained thereby can be executed by using the corresponding tools.

[0060] The planning module is used to decompose the policy generation task into at least one subtask based on the tool description. For example, it is assumed that 10 tools are integrated in the tool module, and by analyzing the tool description of the 10 tools and the task content of the policy generation task, it is determined that the policy generation task is decomposed into 6 subtasks, and one of the 6 subtasks corresponds to one of the 10 tools.

[0061] S240, execute the at least one subtask by calling the tools in the tool module by using an action module to generate the target policy corresponding to the policy description.

[0062] According to the foregoing description, the subtasks can be executed by calling the corresponding tools, and therefore, the action module can be used to execute the at least one subtask by calling the tools in the tool module. For each of the at least one subtask, the subtask can be executed by calling the tool in the tool module corresponding to the subtask, so that the target policy can be generated according to the execution result of each subtask.

[0063] The technical solution of the embodiment of the application can decompose a relatively complex strategy generation task into relatively simple subtasks through tool description of the tool, thereby reducing the difficulty of task execution; and the subtasks decomposed in this way can be executed by calling corresponding tools in the tool module, thereby achieving effective execution of the task and further obtaining a target strategy that can reflect the strategy description.

[0064] A technical solution, after decomposing a strategy generation task represented by a strategy description into at least one subtask, the strategy generation method further includes: determining an action sequence corresponding to the at least one subtask based on the tool description, wherein different actions in the action sequence correspond to different tools in the tool module respectively; and executing the at least one subtask by calling the tools in the tool module through the action module to generate a target strategy corresponding to the strategy description, including: calling the tools in the tool module in sequence based on the action sequence through the action module to execute the at least one subtask and generate the target strategy corresponding to the strategy description.

[0065] In actual application, two subtasks in the multiple subtasks can be independent of each other, for example, the basic information extraction subtask and the strategy flow generation subtask are independent of each other; or can be dependent on each other, for example, the strategy flow generation subtask and the strategy flow checking subtask, the latter can be executed only after the former is executed. Therefore, the planning module can also determine the execution order between the multiple subtasks after task decomposition, that is, the multiple subtasks are automatically arranged by combining the tool description and using the logical reasoning ability of the planning module.

[0066] In the agent, one subtask can correspond to one action, and multiple subtasks can correspond to different actions, which makes the action module execute the corresponding subtask by executing the corresponding action. On this basis, since there is an execution order between the multiple subtasks, there is also an execution order between the multiple actions, so the action sequence corresponding to the multiple subtasks as a whole can be determined, wherein the multiple actions correspond to different tools in the tool module respectively, which makes the action module execute the corresponding subtask by calling the corresponding tools in the tool module to execute the corresponding action.

[0067] By using the action module, the multiple subtasks can be executed in sequence by calling the tools in the tool module in sequence based on the multiple actions in the action sequence, thereby achieving the sequential execution of the multiple subtasks.

[0068] Exemplarily, as shown in FIG. 4, it is assumed that the at least one subtask can include a basic information identification subtask, an object information extraction subtask, a control information extraction subtask, an attribute setting subtask, a strategy flow generation subtask and a language conversion subtask, wherein the basic information identification subtask, the object information extraction subtask and the control information extraction subtask are independent of each other, the attribute setting subtask needs to be executed after the execution of the three subtasks; the strategy flow generation subtask and the language conversion subtask depend on each other, the latter needs to be executed after the execution of the former, the language conversion subtask can be used to convert the flow code corresponding to the generated strategy flow into a domain specific language (Domain Specific Language, DSL); and the attribute setting subtask and the language conversion subtask are independent of each other. In addition, in combination with the application scenarios that can be involved in the embodiments of the present application, the above-mentioned flow code can be a mermaid code.

[0069] On this basis, the basic information identification subtask can be executed by calling a basic information identification tool to obtain the basic information, the object information extraction subtask can be executed by calling an object information extraction tool to obtain the object information, the control information extraction subtask can be executed by calling a control information extraction tool to obtain the control information, and the strategy flow generation subtask can be executed by calling a strategy flow generation tool to obtain the flow code corresponding to the strategy flow; then, the language conversion subtask can be executed by calling a language conversion tool to convert the flow code into a DSL, and the attribute setting subtask can be executed by calling an attribute setting tool to perform attribute setting based on the basic information, the object information and the control information; finally, the target strategy is generated according to the DSL and the attribute setting result output by the attribute setting subtask.

[0070] Exemplarily, the planning module can plan based on ReAct, wherein ReAct is a combination of reasoning (Reasoning, Reson) and action (Action, Act). Referring to FIG. 5, the planning module plans based on the strategy description and the tool module by using its own planning thought, and outputs an action sequence to be executed, each action in the action sequence corresponding to a tool in the tool module. Then, the action module performs an action query in the action sequence, and determines the tool input of the tool corresponding to the first action in the action sequence according to the tool description, calls the tool based on the tool input to execute the first action, and then returns the response, i.e., the execution result, to the planning module, so that the planning module determines whether to adjust the previous planning by observing the response. In this way, the multiple actions in the action sequence are executed until the execution of the multiple actions is completed.

[0071] The above example, by using ReAct, constantly adjusts the plan through the execution result of the action, thereby ensuring the accuracy of the final generated target strategy.

[0072] Another technical solution, the action module cooperates with other agents different from the strategy generation agent; based on the tool description of at least one tool, the strategy generation task represented by the strategy description is decomposed into at least one subtask, including: based on the tool description of at least one tool and the agent description of other agents, the strategy generation task represented by the strategy description is decomposed into at least one subtask; by calling the tools in the tool module, at least one subtask is executed, including: in the case that at least one subtask includes a subtask to be executed by other agents, at least one subtask is executed by calling the tools in the tool module and other agents.

[0073] The other agent can be understood as an agent different from the strategy generation agent, and the function of the other agent is different from the function of the tool in the tool module, so that this technical solution is based on the tool, and other agents are also provided to entrust other agents to process the subtasks that they are good at processing and the tools are not good at processing, so as to ensure the effective execution of the subtasks. The number of other agents can be one, two or more, which is related to the actual situation and is not limited here. For example, referring to FIG. 1, the three other agents cooperating with the action module have object selection function, function calling function and object clarification function respectively.

[0074] The agent description can be used to describe at least the function of the other agent, so similar to the tool description, the task decomposition can be performed based on the tool description and the agent description to obtain at least one subtask. On this basis, in the case that at least one subtask includes a subtask X to be executed by other agents, that is, subtask X is decomposed due to other agents, the action module can execute the subtask by calling the tools and other agents, that is, by calling the tools to execute the subtasks except subtask X and by calling other agents to execute subtask X, thereby realizing the effective execution of multiple subtasks.

[0075] FIG. 6 is a flowchart of another strategy generation method according to an embodiment of the present application. The present embodiment is based on the technical solutions described above. In the present embodiment, the at least one subtask includes at least a strategy flow generation subtask, the at least one tool includes at least a strategy flow generation tool corresponding to the strategy flow generation subtask, and the strategy flow generation tool is implemented based on a large language model. The at least one subtask is executed by invoking the tools in the tool module to generate a target strategy corresponding to the strategy description, including: the strategy flow generation subtask is executed by invoking the strategy flow generation tool in the tool module based on the strategy description and the prompt words corresponding to the target strategy injected into the large language model to generate a strategy flow; and the target strategy corresponding to the strategy description is generated based at least on the strategy flow. The explanations of the same or corresponding terms as in the above embodiments are not repeated here.

[0076] Referring to FIG. 6, the method of the present embodiment can include the following steps:

[0077] S310, obtaining a strategy description of a target strategy to be generated and a strategy generation agent.

[0078] The strategy generation agent includes a large language model and a tool module, a planning module and an action module applied in cooperation with the large language model; the tool module includes at least one tool for implementing strategy generation, and the at least one tool includes at least a strategy flow generation tool implemented based on the large language model.

[0079] The strategy flow generation is a core link in the strategy generation task, so the present embodiment predefines a strategy flow generation tool in the tool module to implement strategy flow generation based on the strategy flow generation tool.

[0080] On this basis, in combination with the application scenarios that the embodiments of the present application can involve, considering that DSL is a relatively large language and there are many data in it that do not need to be generated and are not suitable for being directly generated as a generation object, i.e., not suitable for directly generating a strategy flow described by DSL. Therefore, a more concise language (or code language) is considered here to describe the strategy flow, and the LLM itself already has the function of generating a strategy flow based on the code language, so the strategy flow generation tool is implemented based on the LLM in the present embodiment.

[0081] S320, inputting the strategy description into the strategy generation agent to execute S330-S350 by the large language model in the strategy generation agent.

[0082] S330, using the planning module to decompose the strategy generation task represented by the strategy description into at least one subtask based on the tool description of the at least one tool.

[0083] The at least one subtask at least comprises a strategy flow generation subtask.

[0084] Since the strategy flow generation tool is integrated in the tool module, the strategy flow generation subtask can be decomposed based on the corresponding tool description.

[0085] S340, using the action module, by calling the strategy flow generation tool in the tool module, based on the strategy description and the prompt word corresponding to the target strategy injected into the large language model, executing the strategy flow generation subtask, generating the strategy flow.

[0086] Since the strategy flow generation tool is implemented based on LLM, by injecting the prompt word corresponding to the target strategy into LLM, the strategy flow generation tool can execute the strategy flow generation subtask based on the strategy description and the prompt word, so as to obtain the strategy flow according to the execution result of the strategy flow generation subtask.

[0087] For example, assuming that the strategy description is "send coupons to users every morning at 9 o'clock, if successful, send push notifications, if push fails, send SMS, if the user is not a member, send SMS to remind the user to open a membership", using the strategy flow generation tool, based on the strategy description and the corresponding prompt word, the strategy flow described in code language, i.e. the flow code, is as follows:

[0088] A[Every morning at 9 o'clock]-→|Check user|B{Is member}

[0089] B--Yes-→C[Send coupons]

[0090] B--No-→D[Send SMS to remind user to open membership]

[0091] C-→E[push notification success]

[0092] E--Yes-→F[End]

[0093] E--No-→G[Send SMS to user]

[0094] Among them, the above code describes the following strategy flow:

[0095] 1. Start the flow every morning at 9 o'clock

[0096] 2. Check if the user is a member

[0097] 3. If the user is a member, send coupons

[0098] 4. If the push notification is successful after the coupon operation, the flow ends

[0099] 5. If the push notification fails, send SMS to the user

[0100] 6. If the user is not a member, send a short message to remind the user to open a membership

[0101] S350, at least based on the policy flow, generate a target policy corresponding to the policy description.

[0102] The technical scheme of the embodiment of the application sets up a policy flow generation tool based on LLM in the tool module, so that the policy flow generation tool can be used to effectively generate a policy flow based on the policy description and the prompt words injected into the LLM, which is a core link in the generation of a target policy.

[0103] A technical scheme, the target policy is to be executed on a target platform, and the prompt words can be generated based on platform specification of the target platform. The target platform can be understood as a platform that can be used to execute the target policy, that is, the target policy can be executed on the target platform. The platform specification can be understood as a specification related to the target platform. On this basis, in order to make the target policy generated based on the policy flow executable on the target platform, the prompt words can be generated based on the platform specification, so that the LLM understands the target platform better, thereby generating a policy flow adapted to the target platform, and further ensuring the executable of the target policy on the target platform.

[0104] On this basis, the platform specification includes at least one of a control specification of a control defined on the target platform for implementing the policy flow and a flow specification of the policy flow adapted to the target platform.

[0105] The control can be understood as a control defined on the target platform for implementing the policy flow. In combination with the application scenarios that can be involved in the embodiments of the application, for example, it can be a push control, a short message control, an AB experiment control or a conditional control, etc., which are not limited herein. The control specification can be used to specify the functions and usage rules of the control, etc., which helps the LLM to better understand various controls, so as to determine the required controls and their connection relationship when generating the policy flow, and further can be fine-tuned in combination with the policy description to generate an accurate policy flow. On this basis, for example, when generating the policy flow in the above example based on various controls, the corresponding flowchart is shown in FIG. 7.

[0106] The flow specification can be used to specify the policy flow adapted to the target platform, which is the syntax specification and naming specification that the flow code corresponding to the policy flow needs to meet. Although the LLM has learned a lot of knowledge related to the flow code in the pre-training stage, in order to make the policy flow generated by the LLM more controllable, a set of standard flow steps can be given here for the LLM to learn.

[0107] The technical solution described above enables the LLM to learn the control description and / or the process description, so that the subsequently generated strategy process is more suitable for the target platform, thereby ensuring the executability of the target strategy on the target platform.

[0108] For example, referring to FIG. 8, in addition to learning the control description and process description described above, the LLM can also learn the task description and / or related examples (Examples) of the strategy process generation subtask. The task description can be used to describe the execution mode of the strategy process generation subtask. In combination with the application scenarios that can be involved in the embodiments of the present application, for example, it can be directly output or step-by-step output the final strategy process, etc., which is not limited herein. On this basis, considering that the strategy process generation subtask is a relatively complex subtask, if the LLM directly outputs the final strategy process or process code, the effect can be poor. Therefore, the LLM can be guided to think step by step, and the ability of the chain of thought (CoT) is used to enable the LLM to explicitly output the intermediate thinking process, for example, the control to be called and the connection relationship, so that the generated strategy process can be more consistent with the expected result. On this basis, the LLM that has completed learning can achieve accurate generation of the strategy process based on the user input (User Input), i.e., the strategy description.

[0109] On this basis, the strategy generation agent further includes a memory module cooperating with the large language model, and the memory module stores the control description; the strategy generation method further includes: obtaining a task type of a strategy generation task, determining a control type corresponding to the task type, and obtaining the control description from the memory module; recalling a target description corresponding to the control under the control type from the control description, and generating a prompt word based on the target description to inject the prompt word into the large language model.

[0110] In actual application, different task types of strategy generation tasks can be involved, for example, strategy generation tasks for C-end, B-end or algorithm engineers, etc. The target platform can support multiple task types, and the control types covered by the multiple task types are different, for example, the control types can be PUSH control, SMS control, AB experiment control and conditional control, etc. Therefore, in order to ensure the accuracy of the strategy process generation and further ensure the accuracy of the target strategy generation, the control type corresponding to the task type of the strategy generation task can be determined, then the target description for describing the control under the control type is recalled from the control description stored in the memory module, and the prompt word is generated based on the target description, so that the LLM can generate the strategy process suitable for the strategy generation task based on the prompt word.

[0111] In combination with the application scenarios that the embodiments of the present application can involve, the above-mentioned target description recall process can be implemented by a retrieval augmented generation (RAG) mode.

[0112] FIG. 9 is a flowchart of another strategy generation method provided by the embodiments of the present application. The present embodiment is based on the above technical solutions. In the present embodiment, the strategy generation agent can further include a memory module applied in cooperation with the large language model, and the memory module stores strategy templates; based on the tool, the target strategy corresponding to the strategy description can be generated by: obtaining the strategy template from the memory module; in the case where it is determined based on the strategy template and the strategy description that the strategy generation is performed in the tool generation mode, generating the target strategy corresponding to the strategy description based on the tool.

[0113] On this basis, the above-mentioned strategy generation method further includes: in the case where it is determined based on the strategy template and the strategy description that the strategy generation is performed in the template matching mode, generating the target strategy corresponding to the strategy description based on the matching template in the strategy template that matches the strategy description. Wherein, the same or corresponding explanations of the terms in the above-mentioned embodiments are not repeated here.

[0114] Referring to FIG. 9, the method of the present embodiment can include the following steps:

[0115] S410, obtaining the strategy description of the target strategy to be generated and the strategy generation agent.

[0116] The strategy generation agent includes a large language model and a tool module and a memory module applied in cooperation with the large language model, the tool module includes tools for implementing strategy generation, and the memory module stores strategy templates.

[0117] The strategy template can be understood as a template for generating a strategy. In combination with the application scenarios that the embodiments of the present application can involve, it can be a template formed according to an existing strategy, or it can be a template created by an expert (such as a business expert, an operation expert, and a product expert, etc.), etc., which is not limited here. The number of strategy templates can be one, two or more; on this basis, the strategy template can include at least one of an expert strategy template representing the best practice of the strategy in the industry created by experts together and a historical high-frequency strategy template mined by an algorithm, etc., which is not limited here.

[0118] The strategy template is pre-stored in the memory module. In actual application, this pre-storing process can be implemented by the RAG mode, that is, the strategy template is injected into the LLM by the RAG mode.

[0119] S420, input the policy description into the policy generation agent to execute S430-S450 by a large language model in the policy generation agent.

[0120] S430, obtain the policy template from the memory module.

[0121] S440, in the case of determining to generate the policy in the tool generation mode based on the policy template and the policy description, generate the target policy corresponding to the policy description based on the tool.

[0122] The tool generation mode can be understood as a mode of generating the policy based on the tools integrated in the tool module, and in particular, in the case of integrating the policy template matching tool in the tool module, the tool generation mode can be understood as a mode of generating the policy based on the tools other than the policy template matching tool in the plurality of tools.

[0123] After obtaining the policy template and the policy description, it can be determined whether there is a matching template in the plurality of policy templates that matches the policy description. In the embodiments of the present application, this determination process can be implemented based on the policy template matching tool. In the case where there is no matching template in the plurality of policy templates, it is determined to generate the policy in the tool generation mode, and then the target policy can be generated based on the tool.

[0124] S450, in the case of determining to generate the policy in the template matching mode based on the policy template and the policy description, generate the target policy corresponding to the policy description based on the matching template in the policy template that matches the policy description.

[0125] The template matching mode can be understood as a mode of generating the policy based on the policy template. In the case where there is a matching template in the plurality of policy templates, it is determined to generate the policy in the template matching mode (or template recommendation mode), and then the target policy can be generated based on the matching template.

[0126] The technical scheme of the embodiments of the present application embeds the policy template in the memory module, so as to determine the policy generation mode (such as the tool generation mode or the template matching mode) based on the matching between the policy template and the policy description, and then generate the policy based on the policy generation mode, thereby meeting the personalized policy generation needs of the user and recommending more professional policy templates for the user.

[0127] FIG. 10 is a structural block diagram of a policy generation device provided by an embodiment of the present application, which is configured to execute the policy generation method provided by any of the above embodiments. The device and the policy generation method of the above embodiments belong to the same concept, and the details not described in detail in the embodiments of the policy generation device can be referred to the embodiments of the above policy generation method. Referring to FIG. 10, the device can include an agent obtaining module 510 and a policy generation module 520.

[0128] The agent obtaining module 510 is configured to obtain a policy description of a target policy to be generated and a policy generation agent, wherein the policy generation agent includes a large language model and a tool module applied in cooperation with the large language model, and the tool module includes tools for implementing policy generation; and the policy generation module 520 is configured to input the policy description into the policy generation agent, so as to generate a target policy corresponding to the policy description based on the tools by using the large language model in the policy generation agent.

[0129] The number of tools is at least one, and the policy generation agent further includes a planning module and an action module applied in cooperation with the large language model; and the policy generation module 520 can include:

[0130] The task decomposition submodule is configured to decompose a policy generation task represented by the policy description into at least one subtask based on tool descriptions of the at least one tool by using the planning module; and the policy generation first submodule is configured to execute the at least one subtask by calling the tools in the tool module by using the action module, so as to generate the target policy corresponding to the policy description.

[0131] On this basis, the at least one subtask at least includes a policy flow generation subtask, the at least one tool at least includes a policy flow generation tool corresponding to the policy flow generation subtask, and the policy flow generation tool is implemented based on the large language model; and the policy generation first submodule can include:

[0132] The policy flow generation unit is configured to execute the policy flow generation subtask based on the policy description and prompt words corresponding to the target policy injected into the large language model by calling the policy flow generation tool in the tool module, so as to generate a policy flow; and the policy generation unit is configured to generate the target policy corresponding to the policy description based on at least the policy flow.

[0133] On this basis, the target policy is to be executed on a target platform, and the prompt words can be generated based on platform specifications of the target platform.

[0134] On this basis, the platform specifications include at least one of control specifications of controls defined on the target platform for implementing the policy flow and flow specifications of the policy flow adapted to the target platform.

[0135] On this basis, the policy generation agent can further include a memory module applied in cooperation with the large language model, and the memory module stores the control specifications; and the policy generation apparatus further includes:

[0136] The control description obtaining module is configured to obtain a task type of the policy generation task, determine a control type corresponding to the task type, and obtain a control description from the memory module; and the prompt word generation module is configured to recall a target description corresponding to a control under the control type from the control description, generate a prompt word based on the target description, and inject the prompt word into the large language model.

[0137] The action module is applied in cooperation with other agents different from the policy generation agent; and the task decomposition submodule can include:

[0138] The task decomposition unit is configured to decompose the policy generation task represented by the policy description into at least one subtask based on the tool description of the at least one tool and the agent description of the other agent; and the policy generation first submodule can include:

[0139] The subtask execution unit is configured to execute the at least one subtask by calling the tools in the tool module and the other agent in the case where the at least one subtask includes a subtask to be executed by the other agent.

[0140] The above-mentioned policy generation device further includes:

[0141] The action sequence determination module is configured to determine an action sequence corresponding to the at least one subtask based on the tool description after the policy generation task represented by the policy description is decomposed into the at least one subtask, wherein different actions in the action sequence correspond to different tools in the tool module respectively; and the policy generation first submodule is configured to:

[0142] The action module is configured to call the tools in the tool module in sequence based on the action sequence to execute the at least one subtask and generate a target policy corresponding to the policy description.

[0143] The policy generation agent further includes a memory module applied in cooperation with the large language model, and the memory module stores a policy template; and the policy generation module 520 includes:

[0144] The policy template obtaining submodule is configured to obtain the policy template from the memory module; and the policy generation second submodule is configured to generate a target policy corresponding to the policy description based on the tools in the case where it is determined to generate the policy by using the tool generation mode based on the policy template and the policy description.

[0145] On this basis, the above-mentioned policy generation device further includes:

[0146] The template matching module is configured to generate a target policy corresponding to the policy description based on a matching template matched with the policy description in the policy template in the case where it is determined to generate the policy by using the template matching mode based on the policy template and the policy description.

[0147] The policy generation device provided in the embodiments of the present application can obtain the policy description of the target policy to be generated and obtain a policy generation agent through an agent obtaining module. The policy generation agent includes an LLM and a tool module applied in cooperation with the LLM. The tool module is integrated with tools for implementing policy generation. Then, the policy generation module inputs the policy description into the policy generation agent to generate the corresponding target policy based on the tools in the LLM. The device described above implements policy generation by using an Agent, which makes the user only need to input the policy description into the Agent without performing other operations, thereby reducing the labor cost (such as the cost of manually dragging a control, the cost of manual learning, and the cost of manual guidance) in the policy generation process. Moreover, the tool module is externally hung for the LLM in the Agent, which makes the LLM generate a personalized policy matched with the policy description by means of the capabilities of the tools in the tool module, thereby solving the problem of low flexibility caused by the fact that the LLM can only generate a policy by means of the relevant knowledge (such as a policy template) learned by itself when the LLM is used alone for policy generation. The two cooperate with each other to achieve flexible policy generation based on low labor cost.

[0148] The policy generation device provided in the embodiments of the present application can execute the policy generation method provided in any of the embodiments of the present application, and has the corresponding function modules and effects of the execution method.

[0149] In the embodiments of the policy generation device described above, the plurality of units and modules included are only divided according to the functional logic, but are not limited to the division described above, as long as the corresponding functions can be implemented. In addition, the names of the plurality of functional units are only for the convenience of mutual differentiation, and do not serve to limit the protection scope of the present application.

[0150] FIG. 11 shows a structural schematic diagram of an electronic device 10 that can be used to implement the embodiments of the present application. The electronic device is intended to represent a variety of forms of digital computers, such as a laptop computer, a desktop computer, a workstation, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent a variety of forms of mobile devices, such as a personal digital processing, a cellular phone, a smart phone, a wearable device (such as a helmet, glasses, a watch, etc.), and other similar computing devices. The components shown herein, their connections, and relationships, and their functions, are merely examples and are not intended to limit the implementations described and / or claimed in this document.

[0151] As shown in FIG. 11, the electronic device 10 includes at least one processor 11, and a memory, such as a Read-Only Memory (ROM) 12, a Random Access Memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11, wherein the memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the ROM 12 or loaded into the RAM 13 from the storage unit 18. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An Input / Output (I / O) interface 15 is also connected to the bus 14.

[0152] Various components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc., an output unit 17, such as various types of displays, a loudspeaker, etc., a storage unit 18, such as a magnetic disk, an optical disk, etc., and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0153] The processor 11 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, a Digital Signal Processing (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, such as the policy generation method.

[0154] In some embodiments, the policy generation method can be implemented as a computer program tangibly embodied in a computer readable storage medium, e.g., storage unit 18. In some embodiments, portions of the computer program, or all of the computer program, can be loaded onto the electronic device 10 via, e.g., ROM 12 and / or communication unit 19. When the computer program is loaded onto RAM 13 and executed by processor 11, one or more of the steps of the policy generation method described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the policy generation method by any other suitable means, e.g., by way of firmware.

[0155] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), a System on a Chip (SOC), a Complex Programmable Logic Device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0156] Computer programs implementing methods of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program, when executed, implements the functions / acts specified in the flowcharts and / or block diagrams. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as part of a separate software package, and partially on a remote machine or server, or entirely on a remote machine or server.

[0157] In the context of this application, a computer readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer readable storage medium can be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer readable storage medium can be a machine readable signal medium. Examples of a machine readable signal medium will include one or any combination of: a propagating signal, a microwave, a cable, a computer memory, a computer disk, a CD-ROM, an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0158] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a Cathode Ray Tube (CRT) or a Liquid Crystal Display (LCD) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0159] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a Local Area Network (LAN), a Wide Area Network (WAN), a blockchain network, and the Internet.

[0160] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and virtual private server (VPS) services.

[0161] The steps shown above in various forms of flow can be reordered, added to, or deleted from. For example, the steps described in the present application can be executed in parallel, can be executed sequentially, or can be executed in different orders, as long as the desired results of the technical solutions of the present application can be achieved, and the present application does not limit herein.

Claims

1. A strategy generation method, comprising: Obtaining a policy description of a target policy to be generated and a policy generation agent, wherein the policy generation agent includes a large language model and a tool module used in conjunction with the large language model, and the tool module includes a tool for implementing policy generation; The policy description is input into the policy generation agent so as to generate the target policy corresponding to the policy description based on the tool through the large language model in the policy generation agent.

2. The method according to claim 1, wherein The number of the tools is at least one, and the strategy generation agent further includes a planning module and an action module used in conjunction with the large language model; Generating the target policy corresponding to the policy description based on the tool includes: Decomposing, by the planning module, a strategy generation task represented by the strategy description into at least one subtask based on a tool description of the at least one tool; The action module is utilized to execute the at least one subtask by calling a tool in the tool module to generate the target policy corresponding to the policy description.

3. The method according to claim 2, wherein: The at least one subtask includes at least a policy flow generation subtask, the at least one tool includes a policy flow generation tool corresponding to the policy flow generation subtask, and the policy flow generation tool is implemented based on the large language model; The step of executing the at least one subtask by calling a tool in the tool module to generate the target policy corresponding to the policy description includes: By calling the policy flow generation tool in the tool module, based on the policy description and the prompt words corresponding to the target policy injected into the large language model, executing the policy flow generation subtask, a policy flow is generated; Based on the policy process, the target policy corresponding to the policy description is generated.

4. The method according to claim 3, wherein: The target policy is to be executed on a target platform, and the prompt word is generated based on a platform description of the target platform.

5. The method according to claim 4, wherein The platform description includes at least one of a control description of a control defined on the target platform for implementing a policy flow and a flow description of a policy flow adapted to the target platform.

6. The method according to claim 5, wherein: The strategy generation agent further includes a memory module used in conjunction with the large language model, wherein the memory module stores the control description; The method further comprises: Obtaining a task type of the strategy generation task, determining a control type corresponding to the task type, and obtaining a description of the control from the memory module; A target description corresponding to the control under the control type is recalled from the control description, and the prompt word is generated based on the target description to inject the prompt word into the large language model.

7. The method according to claim 2, wherein: The action module is used in conjunction with other agents different from the strategy generation agent; The step of decomposing the strategy generation task represented by the strategy description into at least one subtask based on the tool description of the at least one tool includes: Decomposing the strategy generation task represented by the strategy description into at least one subtask based on the tool description of the at least one tool and the agent description of the other agent; The step of executing the at least one subtask by calling a tool in the tool module includes: In the case that the at least one subtask includes a subtask to be executed by the other intelligent agent, the at least one subtask is executed by calling the tool in the tool module and the other intelligent agent.

8. The method according to claim 2, further comprising, after decomposing the policy generation task represented by the policy description into at least one subtask: Determining an action sequence corresponding to the at least one subtask based on the tool description, wherein different actions in the action sequence correspond to different tools in the tool module; The utilizing the action module to execute the at least one subtask by calling a tool in the tool module to generate the target policy corresponding to the policy description includes: The action module is utilized to sequentially call tools in the tool module based on the action sequence to execute the at least one subtask and generate the target policy corresponding to the policy description.

9. The method according to claim 1, wherein The strategy generation agent also includes a memory module used in conjunction with the large language model, wherein the memory module stores a strategy template; Generating the target policy corresponding to the policy description based on the tool includes: Acquiring the policy template from the memory module; In the case where it is determined to use a tool generation method to generate the policy based on the policy template and the policy description, the target policy corresponding to the policy description is generated based on the tool.

10. The method according to claim 9, further comprising: When it is determined that the policy is generated by template matching based on the policy template and the policy description, the target policy corresponding to the policy description is generated based on a matching template in the policy template that matches the policy description.

11. A strategy generating device, comprising: an agent acquisition module configured to acquire a policy description of a target policy to be generated and a policy generation agent, wherein the policy generation agent includes a large language model and a tool module used in conjunction with the large language model, wherein the tool module includes tools for implementing policy generation; The strategy generation module is configured to input the strategy description into the strategy generation agent so as to generate the target strategy corresponding to the strategy description based on the tool through the large language model in the strategy generation agent.

12. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to perform the policy generation method according to any one of claims 1 to 10.

13. A computer-readable storage medium storing computer instructions, wherein the computer instructions are configured to enable a processor to implement the policy generation method according to any one of claims 1 to 10 when executed.

Images