Action execution method, execution device and storage medium of intelligent agent system

Abstract

The application relates to an action execution method and device of an intelligent agent system and a storage medium. The method comprises the following steps: according to a source type of a context segment used for generating an action parameter of a target action, labeling the context segment with a source code; inputting the context segment into a large model of the intelligent agent system, and obtaining an action parameter output by the large model and a source code list of the action parameter, wherein the source code list comprises a source code of the context segment used for generating the action parameter; determining a risk value of the target action according to the source code list of the action parameter; when the risk value meets a first preset condition, controlling an execution tool of the intelligent agent system to execute the target action based on the action parameter, and the first preset condition is that the risk value is less than a first preset value. The method solves the problem that the intelligent agent system is prone to causing safety accidents when executing actions in the prior art.

Description

Technical Field

[0001] This application relates to the field of intelligent agent technology, and in particular to an action execution method, execution device and storage medium for an intelligent agent system. Background Technology

[0002] Currently, intelligent agent systems (such as intelligent assistants integrated into Telegram, Slack, WeChat Work, email clients, or desktops) have evolved from conversational tools into automated execution tools capable of autonomously performing complex tasks. After a user inputs a command into the intelligent agent system, the system will further access relevant text (e.g., external web pages, external documents), generate action parameters based on the information obtained from the relevant files, and then execute actions based on the action parameters (e.g., organizing documents, modifying documents, sending emails). For example, when a user inputs the command "Send today's meeting minutes to Mr. Zhang" into the intelligent agent system, the system accesses external collaborative documents, extracts Mr. Zhang's email address as an action parameter, and automatically sends the meeting minutes to that address.

[0003] However, the credibility of text from different sources varies significantly. For example, compared to external text such as external web pages and external documents, the text of instructions directly entered by users is usually less likely to be tampered with and has higher credibility. Currently, intelligent agent systems cannot identify the source of the text on which the action parameters are generated. When the action parameters are generated based on high-risk text that is easily tampered with, the system still directly executes the action, which can easily lead to security incidents such as documents being mistakenly modified and sensitive information being sent to the wrong target. Summary of the Invention

[0004] This application provides a method for executing actions in an intelligent agent system, an action execution device for an intelligent agent system, a computer-readable storage medium, and an electronic device to solve the problem that the execution of actions by intelligent agent systems in the prior art can easily lead to security incidents.

[0005] In a first aspect, this application provides an action execution method for an intelligent agent system. The method includes: labeling the context fragment with a source code according to the source type of the context fragment used to generate action parameters for a target action; inputting the context fragment into a large model of the intelligent agent system and obtaining the action parameters and a list of source codes for the action parameters output by the large model, wherein the list of source codes includes the source codes of the context fragment on which the action parameters are generated; determining the risk value of the target action according to the list of source codes for the action parameters; and controlling the execution tool of the intelligent agent system to execute the target action based on the action parameters when the risk value meets a first preset condition, wherein the first preset condition is that the risk value is less than a first preset value.

[0006] Optionally, there are multiple action parameters and multiple source code lists, with each action parameter corresponding one-to-one with a source code list. Determining the risk value of the target action based on the source code lists of the action parameters includes: obtaining the risk scores of the source codes in the source code lists of all action parameters of the target action; the risk scores of the source codes are related to the source type of the context segment to which they belong and are negatively correlated with the credibility of the context segment to which they belong; for each action parameter, determining the risk score of the action parameter based on the risk scores of all source codes in the source code list of the action parameter, wherein the risk score of the action parameter is the maximum value among the risk scores of all source codes in the source code list of the action parameter; determining a first target risk score based on the risk scores of all the action parameters, wherein the first target risk score is the maximum value among the risk scores of all the action parameters; and determining the risk value based on the first target risk score.

[0007] Optionally, determining the risk value based on the first target risk score includes: obtaining the action type and the number of risk action parameters of the target action, wherein the number of risk action parameters is the difference between the total number of action parameters of the target action and the number of safe action parameters, and the safe action parameters are safe action parameters whose source codes in the source code list are all target source codes, and the target source code is the source code whose source type of the context segment to which it belongs is the source code directly input by the user in the current thread; determining a second target risk score based on the action type of the target action, wherein the second target risk score is positively correlated with the degree of risk of the target action causing a safety accident; and calculating the sum of the first target risk score, the second target risk score, and the number of risk action parameters by a preset multiple to obtain the risk value.

[0008] Optionally, after determining the risk value of the target action based on the source code list of the action parameters, the method includes: generating a prompt message when the risk value does not meet the first preset condition but meets the second preset condition, and executing the target action based on the action parameters, wherein the second preset condition is that the risk value is less than a second preset value and the second preset value is greater than the first preset value, and the prompt message includes the action parameters, the source code list, and the risk value.

[0009] Optionally, after determining the risk value of the target action based on the source code list of the action parameters, the method includes: generating approval information when the risk value does not meet the second preset condition but meets the third preset condition, wherein the third preset condition is that the risk value is less than a third preset value and the third preset value is greater than the second preset value, and the approval information includes the action parameters, the source code list, and the risk value; and executing the target action based on the action parameters when a user's confirmation instruction is received.

[0010] Optionally, after determining the risk value of the target action based on the source code list of the action parameters, the method includes: generating approval information when the risk value does not meet the third preset condition, wherein the third preset condition is that the risk value is less than a third preset value and the third preset value is greater than a second preset value; sending verification information upon receiving the confirmation instruction, wherein the verification information is used to request the user to input a device verification code; and executing the target action based on the action parameters when the device verification code input by the user is the same as the preset verification code.

[0011] Optionally, before generating approval information, the method includes: for each action parameter, calculating the hash value of the action parameter to obtain a first action parameter hash value; calculating the hash value of the identifier of the target object of the target action to obtain a first identifier hash value; and executing the target action based on the action parameter, including: for each action parameter, calculating the hash value of the action parameter to obtain a second action parameter hash value; calculating the hash value of the identifier of the target object of the target action to obtain a second identifier hash value; and executing the target action based on the action parameter when the first action parameter hash value and the second action parameter hash value of each action parameter are the same and the first identifier hash value and the second identifier hash value are the same.

[0012] Secondly, this application provides an action execution device for an intelligent agent system. The device includes: an encoding unit, configured to annotate the context fragment with a source code according to the source type of the context fragment used to generate action parameters for a target action; an acquisition unit, configured to input the context fragment into a large model of the intelligent agent system and acquire the action parameters and a list of source codes for the action parameters output by the large model, wherein the list of source codes includes the source codes of the context fragment on which the action parameters are generated; a determination unit, configured to determine the risk value of the target action according to the list of source codes for the action parameters; and a control unit, configured to control the execution tool of the intelligent agent system to execute the target action based on the action parameters when the risk value meets a first preset condition, wherein the first preset condition is that the risk value is less than a first preset value.

[0013] Thirdly, this application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the action execution methods of the intelligent agent system described above.

[0014] Fourthly, this application also provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of any of the action execution methods of the intelligent agent system described above.

[0015] In this embodiment, the context fragments used to generate the action parameters of the target action are labeled with source codes according to their source types. The context fragments are then input into the large model of the intelligent agent system, and the action parameters and a list of source codes for the action parameters output by the large model are obtained. The list of source codes includes the source codes of the context fragments on which the action parameters are based, thus determining the source type of the context fragments used to generate the action parameters of the target action. Then, based on the list of source codes for the action parameters, the risk value of the target action is determined. This risk value reflects the credibility of the context fragments on which the action parameters of the target action are based. Finally, when the risk value meets a first preset condition, it is determined that the risk of the target action causing a security incident is low, and the execution tool of the intelligent agent system is controlled to execute the target action based on the action parameters. The first preset condition is that the risk value is less than a first preset value, thereby reducing the risk of the intelligent agent system causing a security incident when executing the target action and solving the problem that the execution of actions by the intelligent agent system in the prior art is prone to causing security incidents. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0019] Figure 1 A flowchart illustrating an action execution method for an intelligent agent system provided in an embodiment of this application; Figure 2 A structural block diagram of an action execution device for an intelligent agent system provided in an embodiment of this application; Figure 3 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] The following disclosure provides numerous different embodiments or examples for implementing various structures of the invention. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of the invention. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0022] To address the technical problem that the execution of actions by existing intelligent agent systems can easily lead to security incidents, this application provides an action execution method for an intelligent agent system, an action execution device for an intelligent agent system, a computer-readable storage medium, and an electronic device, which can solve the problem that the execution of actions by existing intelligent agent systems can easily lead to security incidents.

[0023] Figure 1An action execution method for an intelligent agent system provided in this application embodiment includes: Step S101: According to the source type of the context fragment used to generate the action parameters of the target action, the source code is labeled for the context fragment. Specifically, the intelligent agent system includes a planning layer and an execution layer. The planning layer includes a large model, and the execution layer includes execution tools. This application adds an execution control gateway to the intelligent agent system to execute the action execution method of the aforementioned intelligent agent system.

[0024] Specifically, after receiving the task instruction input by the user in the chat window of the intelligent agent system, the large model breaks down the task instruction into multiple target actions. For example, if the user inputs the task instruction "search for Mr. Zhang's email address and send the meeting minutes," the large model will break it down into multiple target actions: searching for Mr. Zhang's email address, generating the meeting minutes body, and sending the meeting minutes email. Before executing each target action, the large model analyzes and determines the context fragments that need to be called to generate the action parameters for that target action. The execution control gateway obtains the interface type of the context fragments that need to be called from the large model, thus determining the source type of the context fragments used to generate the action parameters for the target action. For example, the source types of context fragments are divided into four types, as follows: The first type is direct input from the user in the current thread, which refers to the natural language command text explicitly issued by the user in the current session. It is the most difficult to tamper with and has the highest credibility. The second type is intermediate results within the current task, which refers to the data content returned by the tool during the execution of the current task. It is difficult to tamper with and has a high degree of reliability. The third type is historical thread content, which refers to text records from previous dialogue rounds or completed task threads. It is easily tampered with and has low credibility. The fourth type is external content, which refers to the main text of external web pages, the main text of documents after parsing attachments, and the text results returned by third-party skills corresponding to plugin outputs. This type is the easiest to tamper with and has the lowest credibility.

[0025] Specifically, in order to determine the source type of the context fragments on which the action parameters of the target action generated by the large model are based, the context fragments are labeled with source codes according to their source types before entering the large model. For example, when the source type of the context fragment is direct input from the user's current thread, the context fragment is labeled as CTX-Ux; when the source type of the context fragment is intermediate results within the current task, the context fragment is labeled as CTX-Sx; when the source type of the context fragment is content from a historical thread, the context fragment is labeled as CTX-Hx; and when the source type of the context fragment is external content, the context fragment is labeled as CTX-Ex. x is the sequence number of the context fragment of the same type, and the sequence number increases sequentially from 1 according to the order in which the context fragments enter the large model.

[0026] Step S102: Input the above context fragment into the large model of the intelligent agent system, and obtain the above action parameters and the source code list of the above action parameters output by the large model. The source code list includes the source code of the context fragment on which the above action parameters are based. Specifically, the large model can also output only the source code list of key action parameters. Key action parameters refer to action parameters that are highly related to the risk of a security incident caused by performing the target action. For example, when the target action is sending an email, the key action parameters are the recipient, email subject, and email body. These action parameters are highly related to the risk of a security incident caused by performing the target action.

[0027] Specifically, in addition to outputting the action parameters of the target action and the list of source codes for the action parameters, the large model also outputs the action type of the target action and the target object of the target action. For example, the action type of the target action is to send an email, the target object of the target action is the recipient, and the action parameters of the target action include: recipient address, email subject, email body, etc. The action type of the target action is to write to a local file, the target object of the target action is the local file, and the action parameters of the target action include: file path, content to be written, etc.

[0028] Specifically, for example, the source code list for the recipient address includes: CTX-E-02, and the source code list for the email subject is: CTX-U-01 and CTX-H-03.

[0029] Specifically, since each context segment is labeled with a source code, prompt words are set for the large model to make the source code of the context segment that generates the action parameters of the target action so that the large model outputs a list of source codes for the action parameters.

[0030] Specifically, after obtaining the action parameters output by the large model and the list of source codes for the action parameters, it is necessary to verify whether each source code in the list is consistent with any of the forms CTX-Ux, CTX-Sx, CTX-Hx, and CTX-Ex. If it is inconsistent with none of the forms CTX-Ux, CTX-Sx, CTX-Hx, and CTX-Ex, it is determined that the large model is producing an illusion, generating incorrect source codes, and the execution of the target action is prohibited.

[0031] Step S103: Determine the risk value of the target action based on the source code list of the above action parameters; Step S104: When the risk value meets the first preset condition, control the execution tool of the intelligent agent system to execute the target action based on the action parameters. The first preset condition is that the risk value is less than the first preset value.

[0032] Specifically, for example, when the risk value is less than 30, it is determined that the risk of causing a safety accident by executing the target action is low, and the execution tool of the control intelligent agent system executes the target action based on the action parameters.

[0033] Through the above embodiments, based on the source type of the context fragment used to generate the action parameters of the target action, the context fragment is labeled with a source code. The context fragment is input into the large model of the intelligent agent system, and the action parameters and the source code list of the action parameters output by the large model are obtained. The source code list includes the source code of the context fragment on which the action parameters are based, so as to determine the source type of the context fragment of the action parameters of the target action. Then, based on the source code list of the action parameters, the risk value of the target action is determined. This risk value reflects the credibility of the context fragment on which the action parameters of the target action are based. Finally, when the risk value meets a first preset condition, it is determined that the risk of the target action causing a security accident is low, and the execution tool of the intelligent agent system is controlled to execute the target action based on the action parameters. The first preset condition is that the risk value is less than a first preset value, thereby reducing the risk of the intelligent agent system causing a security accident when executing the target action, and solving the problem that the execution of actions by the intelligent agent system in the prior art is prone to causing security accidents.

[0034] In an optional embodiment, there are multiple action parameters and multiple source code lists, and each action parameter corresponds one-to-one with a source code list. Step S103 can then be implemented as follows: Step S1031: Obtain the risk score of the source code in the source code list of all action parameters of the target action. The risk score of the source code is related to the source type of the context segment to which it belongs and is negatively correlated with the credibility of the context segment to which it belongs. Specifically, for example, the context fragments directly input by the user in the current thread are the most difficult to tamper with and have the highest credibility. Therefore, the risk score of CTX-Ux is 0. The context fragments of intermediate system results within the current task are difficult to tamper with and have high credibility. Therefore, the risk score of CTX-Sx is 10. The context fragments of historical thread content are easy to tamper with and have low credibility. Therefore, the risk score of CTX-Hx is 20. The context fragments of external content are the most easy to tamper with and have the lowest credibility. Therefore, the risk score of CTX-Ex is 30.

[0035] Step S1032: For each of the above action parameters, based on the risk scores of all source codes in the source code list of the above action parameters, determine the risk score of the above action parameter, wherein the risk score of the above action parameter is the maximum value among the risk scores of all source codes in the source code list of the above action parameter; Specifically, , where S(p j ) represents the action parameter p j Risk score, max() means take the maximum value, W(type(c i )) indicates the source code c i Risk score, Src(p j ) represents the action parameter p j The set of all source codes in the source code list. If the source code list of an action parameter includes multiple source codes, it means that the action parameter is generated based on multiple context fragments. The maximum value of the risk scores among all source codes in the source code list of the action parameter is determined as the risk score of the action parameter.

[0036] Step S1033: Based on the risk scores of all the above-mentioned action parameters, determine the first target risk score, which is the maximum value among the risk scores of all the above-mentioned action parameters; Step S1034: Determine the risk value based on the first target risk score.

[0037] In this embodiment, firstly, the risk scores of the source codes in the source code list of all action parameters of the target action are obtained. The credibility of the context fragment on which each action parameter of the target action is based is quantified. Then, for each action parameter, the maximum value among the risk scores of all source codes in the source code list of the action parameter is determined as the risk score of the action parameter, which can fully reflect the worst credibility level among all context fragments on which the action parameter is based. After that, the maximum value among the risk scores of all action parameters is determined as the first target risk score, which can fully reflect the worst credibility level among all context fragments on which the target action is based. Finally, the risk value is determined based on the first target risk score.

[0038] In an optional embodiment, step S1034 above can be implemented as follows: Obtain the action type and the number of risk action parameters of the target action. The number of risk action parameters is the difference between the total number of action parameters of the target action and the number of safe action parameters. The safe action parameters are all safe action parameters whose source codes in the source code list are the target source codes. The target source code is the source code whose source type of the context segment to which it belongs is the source code directly input by the user in the current thread. Based on the action type of the above target action, a second target risk score is determined. The second target risk score is positively correlated with the degree of risk of the above target action causing a safety accident. Specifically, the execution of a target action itself carries a risk score, which is related to the type of the target action. For example, if the target action type is a read-only query, the second target risk score is 10; if the target action type is to generate a file but not write it to disk, the second target risk score is 20; if the target action type is to write a local file, the second target risk score is 30; if the target action type is to send a message or email to the outside world, the second target risk score is 40; and if the target action type is to execute a command or call a high-privilege tool, the second target risk score is 50.

[0039] The risk value is obtained by summing the first target risk score, the second target risk score, and the number of risk action parameters of the preset multiple.

[0040] Specifically, Where R is the risk value, B (action) is the second target risk score, and max(S(p) j )) represents the first target risk score, and N represents the number of risk action parameters.

[0041] In this embodiment, the risk value is obtained by summing the first target risk score, the second target risk score, and the number of risk action parameters of a preset multiple. The first target risk score can fully reflect the worst credibility level among all context fragments on which the target action is based. The second target risk score can reflect the degree of risk of the target action causing a safety accident. The number of risk action parameters of the preset multiple can reflect the degree of influence of the context text fragment with the highest non-credibility on the execution of the target action causing a safety accident.

[0042] In an optional embodiment, after step S102, the method further includes: When the risk value does not meet the first preset condition but meets the second preset condition, a prompt message is generated, and the target action is executed based on the action parameters. The second preset condition is that the risk value is less than the second preset value and the second preset value is greater than the first preset value. The prompt message includes the action parameters, the source code list, and the risk value.

[0043] In this embodiment, for example, the first preset condition is a risk value of less than 30, and the second preset condition is a risk value of less than 60. When the risk value is not less than 30 and less than 60, it indicates that the risk of causing a safety accident by performing the target action is moderate. At this time, a prompt message is generated. The prompt message includes each action parameter of the target action, the source code list of each action parameter, and the risk value, so that the user knows the action parameters, the source type of the context fragment on which the action parameters are generated, and the degree of risk of causing a safety accident by performing the target action, and performs the target action based on the action parameters.

[0044] In an optional embodiment, after step S103, the method further includes: If the risk value does not meet the second preset condition but meets the third preset condition, approval information is generated. The third preset condition is that the risk value is less than the third preset value and the third preset value is greater than the second preset value. The approval information includes the action parameters, the source code list, and the risk value. Upon receiving confirmation from the user, the target action is executed based on the aforementioned action parameters.

[0045] Specifically, the approval information may also include the target type of the target action.

[0046] In this embodiment, for example, the second preset condition is a risk value of less than 60, and the third preset condition is a risk value of less than 80. When the risk value is not less than 60 and less than 80, it indicates that the risk of causing a safety accident by executing the target action is high. At this time, approval information is generated. The approval information includes each action parameter of the target action, the source code list of each action parameter, and the risk value, so that the user knows the action parameters, the source type of the context fragment on which the action parameters are generated, and the degree of risk of causing a safety accident by executing the target action. When the user's confirmation instruction is received, the target action is executed based on the action parameters.

[0047] In an optional embodiment, after step S103, the method further includes: When the risk value does not meet the third preset condition, approval information is generated. The third preset condition is that the risk value is less than the third preset value and the third preset value is greater than the second preset value. Upon receiving the above confirmation instruction, a verification message is sent, which is used to request the user to enter a device verification code. If the device verification code entered by the user is the same as the preset verification code, the target action is executed based on the above action parameters.

[0048] In this embodiment, the third preset condition is a risk value of less than 80. When the risk value is not less than 80, it indicates that the risk of a security incident caused by executing the target action is high. At this time, approval information is generated to inform the user of the action parameters, the source type of the context fragment on which the action parameters are based, and the degree of risk of a security incident caused by executing the target action. Upon receiving the user's confirmation instruction, verification information is sent. The verification information is used to request the user to enter a device verification code. When the device verification code entered by the user is the same as the preset verification code, the target action is executed based on the action parameters. On the basis of user approval, the device verification code entered by the user is additionally required to be the same as the preset verification code to ensure that the confirmation instruction comes from a trusted device and the user himself, and to prevent malicious sending of confirmation instructions that could cause a security incident.

[0049] Before generating approval information, the above methods include: For each of the above action parameters, calculate the hash value of the above action parameter to obtain the first action parameter hash value of the above action parameter; Calculate the hash value of the identifier of the target object of the above target action to obtain the first identifier hash value; Specifically, the identifier of the target object can be the name of the target object.

[0050] Based on the above action parameters, perform the above target action, including: For each of the above action parameters, calculate the hash value of the above action parameter to obtain the second action parameter hash value of the above action parameter; Calculate the hash value of the identifier of the target object of the above target action to obtain the second identifier hash value; When the hash value of the first action parameter of each of the above action parameters is the same as the hash value of the second action parameter of each of the above action parameters, and the hash value of the first identifier is the same as the hash value of the second identifier, the above target action is executed based on the above action parameters.

[0051] In this embodiment, if the action parameters or the target object are maliciously modified after approval, the intelligent agent system may cause a security incident when executing the target action. Therefore, before approval, between approval and execution of the target action, the hash value of the action parameters and the hash value of the target object's identifier are calculated separately. Only when the two calculated hash values ​​are consistent will the target action be executed based on the action parameters.

[0052] In an optional embodiment, before generating approval information, the above method includes: For each of the above source code lists, calculate the hash value of the above source code list to obtain the first source code hash value of the above source code list; Based on the above action parameters, perform the above target action, including: For each of the above source code lists, calculate the hash value of the above source code list to obtain the second source code hash value of the above source code list; When the hash value of the first action parameter of each of the above action parameters is the same as the hash value of the second action parameter of each of the above action parameters, the hash value of the first source code of each of the above source code lists is the same as the hash value of the second source code of each of the above source code lists, and the hash value of the first identifier is the same as the hash value of the second identifier, the above target action is executed based on the above action parameters.

[0053] In this embodiment, if the source code list is tampered with after approval, it indicates that the calculated risk value may be incorrect, which may cause the intelligent agent system to trigger a security incident when executing the target action. Therefore, before approval, between approval and execution of the target action, the hash values ​​of the action parameters, the target object's identifier, and the source code list are calculated separately. Only when the two calculated hash values ​​are consistent will the target action be executed based on the action parameters.

[0054] In intelligent agent systems, the large model is not limited to obtaining direct input from the user's current thread; it can also obtain intermediate system results within the current task, historical thread content, and external content. To determine the source type of the context fragments upon which the large model bases the action parameters for generating the target action, it is necessary to label the context fragments with source codes based on the source type of the context fragments used to generate the action parameters for the target action. To better understand this invention, the complete execution flow of this application is illustrated using the intelligent agent system executing the email sending action as an example, including: The user enters the command "Search Mr. Zhang's email address and send meeting minutes"; The large model breaks down the task instructions into three target actions: searching for Mr. Zhang's email address, generating the meeting minutes text, and sending the meeting minutes email. The process for performing the action of searching for Mr. Zhang's email address is as follows: Before the user-input command (directly entered by the user in the current thread) enters the main model, the execution control gateway labels the user-input command as CTX-U-1. The main model then outputs the following based on the user-input command: The action type for searching Mr. Zhang's email address is "Search Contacts," the target object is Mr. Zhang, the action parameter is Mr. Zhang's email address, and the source encoding list includes: CTX-U-1. This target action indicates that the main model, based on the user-input command, searches for Mr. Zhang's email address from other external contact lists (external content). After executing this target action according to the action parameter, Mr. Zhang's email address is obtained: zhangzong@example.com. Since zhangzong@example.com comes from external content, the execution control gateway labels zhangzong@example.com as CTX-E-1.

[0055] The process for executing the objective of generating the main text of the meeting minutes is as follows: The historical meeting content includes the previous meeting content, attendees, and meeting conclusions. Before the historical meeting content (historical thread content) enters the large model, the execution control gateway marks the historical meeting content with CTX-H-1. The large model outputs the following content based on the user's input instructions and the historical meeting content: Action type: Generate meeting minutes text; Target object: Meeting minutes text; Action parameters: Meeting minutes text; Source code columns include: CTX-U-1, CTX-H-1. This target action indicates that the large model calls the meeting minutes generation tool to generate the meeting minutes text to be sent this time based on the user's input instructions and the historical meeting content. After executing this target action according to the action parameters, a draft of the meeting minutes text is obtained. This draft of the meeting minutes text is an intermediate result generated by calling the tool during the current task execution. Therefore, the execution control gateway marks the meeting minutes text with CTX-S-1.

[0056] The process for executing the action of sending an email is as follows: The large model outputs the following content based on the user's input instructions, zhangzong@example.com, and the draft of the meeting minutes: Action type: Send email; Action parameters: Recipient address (zhangzong@example.com), Email subject (Send meeting minutes, determined according to user input instructions), and Email body (Draft of meeting minutes). The source code list for the recipient address includes CTX-E-1, the source code list for the email subject is CTX-U-1, and the source code list for the email body is CTX-S-1.

[0057] It should be noted that each of the above target actions requires a separate risk value calculation, and the execution strategy is determined based on the risk value.

[0058] Figure 2 An action execution device for an intelligent agent system provided in this application embodiment includes: The encoding unit 10 is used to annotate the aforementioned context fragment with source encoding according to the source type of the context fragment used to generate the action parameters of the target action; Specifically, the intelligent agent system includes a planning layer and an execution layer. The planning layer includes a large model, and the execution layer includes execution tools. This application adds an execution control gateway to the intelligent agent system to execute the action execution method of the aforementioned intelligent agent system.

[0059] Specifically, after receiving the task instruction input by the user in the chat window of the intelligent agent system, the large model breaks down the task instruction into multiple target actions. For example, if the user inputs the task instruction "search for Mr. Zhang's email address and send the meeting minutes," the large model will break it down into multiple target actions: searching for Mr. Zhang's email address, generating the meeting minutes body, and sending the meeting minutes email. Before executing each target action, the large model analyzes and determines the context fragments that need to be called to generate the action parameters for that target action. The execution control gateway obtains the interface type of the context fragments that need to be called from the large model, thus determining the source type of the context fragments used to generate the action parameters for the target action. For example, the source types of context fragments are divided into four types, as follows: The first type is direct input from the user in the current thread, which refers to the natural language command text explicitly issued by the user in the current session. It is the most difficult to tamper with and has the highest credibility. The second type is intermediate results within the current task, which refers to the data content returned by the tool during the execution of the current task. It is difficult to tamper with and has a high degree of reliability. The third type is historical thread content, which refers to text records from previous dialogue rounds or completed task threads. It is easily tampered with and has low credibility. The fourth type is external content, which refers to the main text of external web pages, the main text of documents after parsing attachments, and the text results returned by third-party skills corresponding to plugin outputs. This type is the easiest to tamper with and has the lowest credibility.

[0060] Specifically, in order to determine the source type of the context fragments on which the action parameters of the target action generated by the large model are based, the context fragments are labeled with source codes according to their source types before entering the large model. For example, when the source type of the context fragment is direct input from the user's current thread, the context fragment is labeled as CTX-Ux; when the source type of the context fragment is intermediate results within the current task, the context fragment is labeled as CTX-Sx; when the source type of the context fragment is content from a historical thread, the context fragment is labeled as CTX-Hx; and when the source type of the context fragment is external content, the context fragment is labeled as CTX-Ex. x is the sequence number of the context fragment of the same type, and the sequence number increases sequentially from 1 according to the order in which the context fragments enter the large model.

[0061] The acquisition unit 20 is used to input the aforementioned context fragment into the large model of the intelligent agent system, and to acquire the aforementioned action parameters and the source encoding list of the aforementioned action parameters output by the large model. The source encoding list includes the source encoding of the context fragment on which the aforementioned action parameters are based. Specifically, the large model can also output only the source code list of key action parameters. Key action parameters refer to action parameters that are highly related to the risk of a security incident caused by performing the target action. For example, when the target action is sending an email, the key action parameters are the recipient, email subject, and email body. These action parameters are highly related to the risk of a security incident caused by performing the target action.

[0062] Specifically, in addition to outputting the action parameters of the target action and the list of source codes for the action parameters, the large model also outputs the action type of the target action and the target object of the target action. For example, the action type of the target action is to send an email, the target object of the target action is the recipient, and the action parameters of the target action include: recipient address, email subject, email body, etc. The action type of the target action is to write to a local file, the target object of the target action is the local file, and the action parameters of the target action include: file path, content to be written, etc.

[0063] Specifically, for example, the source code list for the recipient address includes: CTX-E-02, and the source code list for the email subject is: CTX-U-01 and CTX-H-03.

[0064] Specifically, since each context segment is labeled with a source code, prompt words are set for the large model to make the source code of the context segment that generates the action parameters of the target action so that the large model outputs a list of source codes for the action parameters.

[0065] Specifically, after obtaining the action parameters output by the large model and the list of source codes for the action parameters, it is necessary to verify whether each source code in the list is consistent with any of the forms CTX-Ux, CTX-Sx, CTX-Hx, and CTX-Ex. If it is inconsistent with none of the forms CTX-Ux, CTX-Sx, CTX-Hx, and CTX-Ex, it is determined that the large model is producing an illusion, generating incorrect source codes, and the execution of the target action is prohibited.

[0066] The determining unit 30 is used to determine the risk value of the target action based on the source code list of the action parameters. The control unit 40 is used to control the execution tool of the intelligent agent system to perform the target action based on the action parameters when the risk value meets the first preset condition. The first preset condition is that the risk value is less than the first preset value.

[0067] Specifically, for example, when the risk value is less than 30, it is determined that the risk of causing a safety accident by executing the target action is low, and the execution tool of the control intelligent agent system executes the target action based on the action parameters.

[0068] Through the above embodiments, based on the source type of the context fragment used to generate the action parameters of the target action, the context fragment is labeled with a source code. The context fragment is input into the large model of the intelligent agent system, and the action parameters and the source code list of the action parameters output by the large model are obtained. The source code list includes the source code of the context fragment on which the action parameters are based, so as to determine the source type of the context fragment of the action parameters of the target action. Then, based on the source code list of the action parameters, the risk value of the target action is determined. This risk value reflects the credibility of the context fragment on which the action parameters of the target action are based. Finally, when the risk value meets a first preset condition, it is determined that the risk of the target action causing a security accident is low, and the execution tool of the intelligent agent system is controlled to execute the target action based on the action parameters. The first preset condition is that the risk value is less than a first preset value, thereby reducing the risk of the intelligent agent system causing a security accident when executing the target action, and solving the problem that the execution of actions by the intelligent agent system in the prior art is prone to causing security accidents.

[0069] In one optional embodiment, there are multiple action parameters and multiple source code lists, with each action parameter corresponding one-to-one with the source code list. The determining unit is specifically used for: Obtain the risk score of the source code from the source code list of all action parameters of the above target action. The risk score of the source code is related to the source type of the context segment to which it belongs, and is negatively correlated with the credibility of the context segment to which it belongs. Specifically, for example, the context fragments directly input by the user in the current thread are the most difficult to tamper with and have the highest credibility. Therefore, the risk score of CTX-Ux is 0. The context fragments of intermediate system results within the current task are difficult to tamper with and have high credibility. Therefore, the risk score of CTX-Sx is 10. The context fragments of historical thread content are easy to tamper with and have low credibility. Therefore, the risk score of CTX-Hx is 20. The context fragments of external content are the most easy to tamper with and have the lowest credibility. Therefore, the risk score of CTX-Ex is 30.

[0070] For each of the above action parameters, a risk score for the action parameter is determined based on the risk scores of all source codes in the source code list of the action parameter. The risk score of the action parameter is the maximum value among all the risk scores of the source codes in the source code list of the action parameter. Specifically, , where S(p j ) represents the action parameter p j Risk score, max() means take the maximum value, W(type(c i )) indicates the source code c i Risk score, Src(p j ) represents the action parameter p j The set of all source codes in the source code list. If the source code list of an action parameter includes multiple source codes, it means that the action parameter is generated based on multiple context fragments. The maximum value of the risk scores among all source codes in the source code list of the action parameter is determined as the risk score of the action parameter.

[0071] Based on the risk scores of all the above action parameters, a first target risk score is determined, which is the maximum value among the risk scores of all the above action parameters. Based on the aforementioned first target risk score, the aforementioned risk value is determined.

[0072] In this embodiment, firstly, the risk scores of the source codes in the source code list of all action parameters of the target action are obtained. The credibility of the context fragment on which each action parameter of the target action is based is quantified. Then, for each action parameter, the maximum value among the risk scores of all source codes in the source code list of the action parameter is determined as the risk score of the action parameter, which can fully reflect the worst credibility level among all context fragments on which the action parameter is based. After that, the maximum value among the risk scores of all action parameters is determined as the first target risk score, which can fully reflect the worst credibility level among all context fragments on which the target action is based. Finally, the risk value is determined based on the first target risk score.

[0073] In an optional embodiment, determining the risk value based on the first target risk score can be achieved as follows: Obtain the action type and the number of risk action parameters of the target action. The number of risk action parameters is the difference between the total number of action parameters of the target action and the number of safe action parameters. The safe action parameters are all safe action parameters whose source codes in the source code list are the target source codes. The target source code is the source code whose source type of the context segment to which it belongs is the source code directly input by the user in the current thread. Based on the action type of the above target action, a second target risk score is determined. The second target risk score is positively correlated with the degree of risk of the above target action causing a safety accident. Specifically, the execution of a target action itself carries a risk score, which is related to the type of the target action. For example, if the target action type is a read-only query, the second target risk score is 10; if the target action type is to generate a file but not write it to disk, the second target risk score is 20; if the target action type is to write a local file, the second target risk score is 30; if the target action type is to send a message or email to the outside world, the second target risk score is 40; and if the target action type is to execute a command or call a high-privilege tool, the second target risk score is 50.

[0074] The risk value is obtained by summing the first target risk score, the second target risk score, and the number of risk action parameters of the preset multiple.

[0075] Specifically, Where R is the risk value, B (action) is the second target risk score, and max(S(p) j )) represents the first target risk score, and N represents the number of risk action parameters.

[0076] In this embodiment, the risk value is obtained by summing the first target risk score, the second target risk score, and the number of risk action parameters of a preset multiple. The first target risk score can fully reflect the worst credibility level among all context fragments on which the target action is based. The second target risk score can reflect the degree of risk of the target action causing a safety accident. The number of risk action parameters of the preset multiple can reflect the degree of influence of the context text fragment with the highest non-credibility on the execution of the target action causing a safety accident.

[0077] In an optional embodiment, the above-described apparatus is used for: When the risk value does not meet the first preset condition but meets the second preset condition, a prompt message is generated, and the target action is executed based on the action parameters. The second preset condition is that the risk value is less than the second preset value and the second preset value is greater than the first preset value. The prompt message includes the action parameters, the source code list, and the risk value.

[0078] In this embodiment, for example, the first preset condition is a risk value of less than 30, and the second preset condition is a risk value of less than 60. When the risk value is not less than 30 and less than 60, it indicates that the risk of causing a safety accident by performing the target action is moderate. At this time, a prompt message is generated. The prompt message includes each action parameter of the target action, the source code list of each action parameter, and the risk value, so that the user knows the action parameters, the source type of the context fragment on which the action parameters are generated, and the degree of risk of causing a safety accident by performing the target action, and performs the target action based on the action parameters.

[0079] In an optional embodiment, the above-described apparatus is used for: If the risk value does not meet the second preset condition but meets the third preset condition, approval information is generated. The third preset condition is that the risk value is less than the third preset value and the third preset value is greater than the second preset value. The approval information includes the action parameters, the source code list, and the risk value. Upon receiving confirmation from the user, the target action is executed based on the aforementioned action parameters.

[0080] Specifically, the approval information may also include the target type of the target action.

[0081] In this embodiment, for example, the second preset condition is a risk value of less than 60, and the third preset condition is a risk value of less than 80. When the risk value is not less than 60 and less than 80, it indicates that the risk of causing a safety accident by executing the target action is high. At this time, approval information is generated. The approval information includes each action parameter of the target action, the source code list of each action parameter, and the risk value, so that the user knows the action parameters, the source type of the context fragment on which the action parameters are generated, and the degree of risk of causing a safety accident by executing the target action. When the user's confirmation instruction is received, the target action is executed based on the action parameters.

[0082] In an optional embodiment, the above-described apparatus is used for: When the risk value does not meet the third preset condition, approval information is generated. The third preset condition is that the risk value is less than the third preset value and the third preset value is greater than the second preset value. Upon receiving the above confirmation instruction, a verification message is sent, which is used to request the user to enter a device verification code. If the device verification code entered by the user is the same as the preset verification code, the target action is executed based on the above action parameters.

[0083] In this embodiment, the third preset condition is a risk value of less than 80. When the risk value is not less than 80, it indicates that the risk of a security incident caused by executing the target action is high. At this time, approval information is generated to inform the user of the action parameters, the source type of the context fragment on which the action parameters are based, and the degree of risk of a security incident caused by executing the target action. Upon receiving the user's confirmation instruction, verification information is sent. The verification information is used to request the user to enter a device verification code. When the device verification code entered by the user is the same as the preset verification code, the target action is executed based on the action parameters. On the basis of user approval, the device verification code entered by the user is additionally required to be the same as the preset verification code to ensure that the confirmation instruction comes from a trusted device and the user himself, and to prevent malicious sending of confirmation instructions that could cause a security incident.

[0084] Before generating approval information, the above-mentioned device is used for: For each of the above action parameters, calculate the hash value of the above action parameter to obtain the first action parameter hash value of the above action parameter; Calculate the hash value of the identifier of the target object of the above target action to obtain the first identifier hash value; Based on the above action parameters, perform the above target action, including: For each of the above action parameters, calculate the hash value of the above action parameter to obtain the second action parameter hash value of the above action parameter; Calculate the hash value of the identifier of the target object of the above target action to obtain the second identifier hash value; When the hash value of the first action parameter of each of the above action parameters is the same as the hash value of the second action parameter of each of the above action parameters, and the hash value of the first identifier is the same as the hash value of the second identifier, the above target action is executed based on the above action parameters.

[0085] In this embodiment, if the action parameters or the target object are maliciously modified after approval, the intelligent agent system may cause a security incident when executing the target action. Therefore, before approval, between approval and execution of the target action, the hash value of the action parameters and the hash value of the target object's identifier are calculated separately. Only when the two calculated hash values ​​are consistent will the target action be executed based on the action parameters.

[0086] In an optional embodiment, before generating approval information, the above-mentioned apparatus is used to: For each of the above source code lists, calculate the hash value of the above source code list to obtain the first source code hash value of the above source code list; Based on the above action parameters, perform the above target action, including: For each of the above source code lists, calculate the hash value of the above source code list to obtain the second source code hash value of the above source code list; When the hash value of the first action parameter of each of the above action parameters is the same as the hash value of the second action parameter of each of the above action parameters, the hash value of the first source code of each of the above source code lists is the same as the hash value of the second source code of each of the above source code lists, and the hash value of the first identifier is the same as the hash value of the second identifier, the above target action is executed based on the above action parameters.

[0087] In this embodiment, if the source code list is tampered with after approval, it indicates that the calculated risk value may be incorrect, which may cause the intelligent agent system to trigger a security incident when executing the target action. Therefore, before approval, between approval and execution of the target action, the hash values ​​of the action parameters, the target object's identifier, and the source code list are calculated separately. Only when the two calculated hash values ​​are consistent will the target action be executed based on the action parameters.

[0088] like Figure 3 As shown, this application provides an electronic device including a processor 111, a communication interface 112, a memory 113, and a communication bus 114. The processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114. The memory 113 is used to store computer programs. In one embodiment of this application, when the processor 111 executes the program stored in the memory 113, it implements the steps of the action execution method of the intelligent agent system provided in any of the aforementioned method embodiments.

[0089] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the action execution method of the intelligent agent system provided in any of the foregoing method embodiments.

[0090] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0091] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0092] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also include the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.

[0093] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for executing actions in an intelligent agent system, characterized in that, The method includes: The context fragments used to generate the action parameters of the target action are labeled with source codes. The context fragment is input into the large model of the intelligent agent system, and the action parameters and the source encoding list of the action parameters output by the large model are obtained. The source encoding list includes the source encoding of the context fragment on which the action parameters are based. Based on the source code list of the action parameters, determine the risk value of the target action; When the risk value meets a first preset condition, the execution tool of the intelligent agent system is controlled to execute the target action based on the action parameters. The first preset condition is that the risk value is less than a first preset value.

2. The method according to claim 1, characterized in that, There are multiple action parameters and multiple source code lists. Each action parameter corresponds one-to-one with a source code list. The risk value of the target action is determined based on the source code list of the action parameters, including: Obtain the risk score of the source code from the source code list of all action parameters of the target action. The risk score of the source code is related to the source type of the context segment to which it belongs and is negatively correlated with the credibility of the context segment to which it belongs. For each action parameter, a risk score is determined based on the risk scores of all source codes in the source code list of the action parameter, wherein the risk score of the action parameter is the maximum value among the risk scores of all source codes in the source code list of the action parameter; Based on the risk scores of all the said action parameters, a first target risk score is determined, which is the maximum value among the risk scores of all the said action parameters; The risk value is determined based on the first target risk score.

3. The method according to claim 2, characterized in that, Based on the first target risk score, the risk value is determined, including: Obtain the action type and the number of risk action parameters of the target action. The number of risk action parameters is the difference between the total number of action parameters of the target action and the number of safe action parameters. The safe action parameters are all safe action parameters whose source codes in the source code list are the target source code. The target source code is the source code whose source type of the context segment to which it belongs is the source code directly input by the user in the current thread. Based on the action type of the target action, a second target risk score is determined, and the second target risk score is positively correlated with the degree of risk of the target action causing a safety accident. The risk value is obtained by summing the first target risk score, the second target risk score, and the number of risk action parameters at a preset multiple.

4. The method according to claim 3, characterized in that, After determining the risk value of the target action based on the source encoding list of the action parameters, the method includes: When the risk value does not meet the first preset condition but meets the second preset condition, a prompt message is generated, and the target action is executed based on the action parameters. The second preset condition is that the risk value is less than a second preset value and the second preset value is greater than the first preset value. The prompt message includes the action parameters, the source code list, and the risk value.

5. The method according to claim 4, characterized in that, After determining the risk value of the target action based on the source encoding list of the action parameters, the method includes: If the risk value does not meet the second preset condition but meets the third preset condition, approval information is generated. The third preset condition is that the risk value is less than a third preset value and the third preset value is greater than the second preset value. The approval information includes the action parameters, the source code list, and the risk value. Upon receiving confirmation from the user, the target action is executed based on the action parameters.

6. The method according to claim 5, characterized in that, After determining the risk value of the target action based on the source encoding list of the action parameters, the method includes: When the risk value does not meet the third preset condition, the approval information is generated, wherein the third preset condition is that the risk value is less than a third preset value and the third preset value is greater than a second preset value; Upon receiving the confirmation instruction, verification information is sent, which is used to request the user to input a device verification code; When the device verification code entered by the user is the same as the preset verification code, the target action is executed based on the action parameters.

7. The method according to claim 5 or 6, characterized in that, Before generating approval information, the method includes: For each action parameter, calculate the hash value of the action parameter to obtain the first action parameter hash value; Calculate the hash value of the identifier of the target object of the target action to obtain the first identifier hash value; Based on the action parameters, the target action is executed, including: For each action parameter, calculate the hash value of the action parameter to obtain the second action parameter hash value; Calculate the hash value of the identifier of the target object of the target action to obtain the second identifier hash value; When the first action parameter hash value and the second action parameter hash value of each action parameter are the same, and the first identifier hash value and the second identifier hash value are the same, the target action is executed based on the action parameters.

8. An action execution device for an intelligent agent system, characterized in that, The device includes: The encoding unit is used to annotate the context segment with source encoding according to the source type of the context segment used to generate the action parameters of the target action; The acquisition unit is used to input the context fragment into the large model of the intelligent agent system, and acquire the action parameters and the source encoding list of the action parameters output by the large model, wherein the source encoding list includes the source encoding of the context fragment on which the action parameters are based; The determining unit is configured to determine the risk value of the target action based on the source encoding list of the action parameters; The control unit is configured to control the execution tool of the intelligent agent system to perform the target action based on the action parameters when the risk value meets a first preset condition, wherein the first preset condition is that the risk value is less than a first preset value.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the action execution method of the intelligent agent system as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When executed by the processor, the computer program implements the steps of the action execution method of the intelligent agent system as described in any one of claims 1 to 7.

Images