Inter-agent communication control system

Abstract

In a multi-agent environment, the goal is to prevent malicious instructions from being mixed into message communication between AI agents, thereby preventing the spread of information leaks and unauthorized operations. [Solution] The present invention involves interposing a check AI agent on the communication path of a message traveling from a first AI agent to a second AI agent, inspecting the message to determine whether or not it should be forwarded, and forwarding or blocking it based on the determination. The inspection conditions may be changed depending on the confidence level or communication type.

Description

Technical Field

[0006] , , , , , , , , , , ,

[0007] ,

[0005] , ,

[0001] The present invention relates to a technique for controlling whether a message can pass through in an environment where a plurality of AI agents communicate with each other by messages.

Background Art

[0002] In recent years, a configuration in which an AI agent using a large-scale language model or the like executes document generation, calls an external tool, refers to an external service, etc. based on a user's instruction has become widespread.

[0003] Such an AI agent can be connected to a process involving external actions such as file operations, external communication, system commands, etc. depending on the execution environment. Therefore, when a malicious instruction is included in the input message, risks such as information leakage, tampering, or unauthorized operations may occur.

[0004] Also, in a multi-agent environment where a plurality of AI agents communicate with each other, if one AI agent is unintentionally contaminated by an illegal instruction, its output may spread to other AI agents, and there is a risk that the risk may cascade and expand.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] Patent Document 1 describes a configuration for evaluating risks for inputs to and outputs from a generative AI model and determining whether to pass based on a threshold value.

[0007] However, in a multi-agent environment, countermeasures that only target input / output between the user and a single model may not be sufficient to prevent the introduction of malicious instructions and the propagation of contamination in the communication paths between AI agents.

[0008] Furthermore, as communication methods expand to include one-to-one communication, intra-group communication, and relay communication between agents, the attack surface expands, and it may become difficult to achieve both usability and security with uniform judgment criteria.

[0009] The present invention aims to reduce risks such as prompt injection, information leakage, and authorization violations by inspecting messages along the communication path between AI agents and controlling whether or not those messages can pass. [Means for solving the problem]

[0010] An inter-agent communication system according to one aspect of the present invention comprises a check AI agent interposed on a communication path for the communication of a message between a first AI agent and a second AI agent, which inspects the message and determines whether or not it should be forwarded, and a communication control unit that forwards or blocks the message based on the determination.

[0011] In one embodiment, the communication control unit changes the strictness of the inspection according to the confidence value associated with the AI ​​agent or operator.

[0012] In one embodiment, the communication control unit selects a security level according to the communication type and changes the inspection or blocking conditions according to the security level. [Effects of the Invention]

[0013] According to the present invention, a screening AI agent inspects and forwards or blocks messages along the communication path between AI agents, thereby preventing malicious instructions such as prompt injection from reaching other AI agents.

[0014] In addition, by changing the strictness of the inspection according to the reliability value, it is possible to achieve both usability and safety, such as not imposing an excessive burden on a highly reliable entity and applying a strict inspection to a less reliable entity.

[0015] In addition, by selecting a security level according to the communication form, an appropriate inspection policy can be applied to the increasing attack surface accompanying the expansion of the communication form.

[0016] Furthermore, according to an aspect provided with logs and notifications, auditing and incident response can be facilitated.

Brief Description of the Drawings

[0017] [Figure 1] It is a configuration diagram of an agent - to - agent communication system according to an embodiment of the present invention. [Figure 2] It is a functional block diagram according to an embodiment of the present invention. [Figure 3] It is a flowchart showing a processing flow according to an embodiment of the present invention. [Figure 4] It is a sequence diagram according to an embodiment of the present invention.

Modes for Carrying Out the Invention

[0018] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not limited to the following embodiments.

[0019] As shown in FIG. 1, the agent - to - agent communication system 100 includes a first AI agent 110, a second AI agent 120, an interrogation AI agent 130, and a communication control unit 140.

[0020] The first AI agent 110 can generate a message based on an input from an operator or an internal process and transmit it to the second AI agent 120. The second AI agent 120 can perform an inference process based on the received message and generate a response message.

[0021] The interrogation AI agent 130 intervenes in the communication path of the message from the first AI agent 110 to the second AI agent 120, inspects the message, and determines whether to transfer it. The inspection can target, for example, at least one of an instruction to induce prompt injection, an instruction to induce impersonation, an encoded or obfuscated instruction, the inclusion of confidential information, and a request to induce a violation of authority.

[0022] Based on the determination of the interrogation AI agent 130, the communication control unit 140 transfers the message to the second AI agent 120 or blocks it. Instead of blocking, it may be in a mode of giving a warning, partial masking, or abstraction.

[0023] <000******00>As shown in FIG. 2, the agent - to - agent communication system 100 may be configured to include a message acquisition unit 141, an inspection control unit 142, a trust value management unit 150, a security level selection unit 160, an authority control unit 170, an audit log unit 180, and a notification unit 190.

[0024] The message acquisition unit 141 receives the message sent from the first AI agent 110 and inputs it to the inspection control unit 142.

[0025] The trust value management unit 150 holds the trust values associated with the first AI agent 110, the second AI agent 120, or their operators, and updates the trust values based on a predetermined history. The history may include, for example, the number of blocks, the number of warnings, the number of approvals by the operator, or the deviation frequency within a predetermined period.

[0026] The inspection control unit 142 changes the strictness of the inspection by the interrogation AI agent 130 according to the trust value obtained from the trust value management unit 150. The strictness can be expressed, for example, by the number of inspection items to be applied, the determination threshold, a multi - stage determination procedure, or the necessity of manual approval.

[0027] The security level selection unit 160 selects a security level according to the communication method and instructs the inspection control unit 142 to implement an inspection policy corresponding to that level.

[0028] Security levels can be defined, for example, as follows: Level L1 for one-to-one communication, Level L2 for intra-group communication, Level L3 for relay communication between AI agents, and Level L4 for inter-AI agent communication mediated by the checkpoint AI agent 130. These definitions are examples and may be changed depending on organizational boundaries, data classification, or the presence or absence of external connections.

[0029] The authorization control unit 170, when a message contains a request involving an external action, verifies the authorization levels of the communication source and destination and determines whether the operation can be executed based on a whitelist of permitted operations.

[0030] The whitelist may include categories such as read operations, write operations, delete operations, external API calls, or file operations. Furthermore, certain high-risk operations may require operator approval.

[0031] The audit log unit 180 records logs including the judgment result by the screening AI agent 130, the features or explanatory information that formed the basis of the judgment, and the security level applied.

[0032] The notification unit 190 notifies the operator or administrator of a warning when it detects suspicious communication. The notification may include, for example, the reason for blocking, recommended actions, or retry conditions.

[0033] Next, with reference to Figure 3, an example of the processing procedure of the inter-agent communication system 100 will be described.

[0034] The inter-agent communication system 100 receives a message from the first AI agent 110 (step S101).

[0035] The inter-agent communication system 100 determines the type of communication to which the communication belongs and selects a security level according to the type of communication (step S102).

[0036] The inter-agent communication system 100 obtains a confidence value associated with the communication source or operator (step S103).

[0037] The inter-agent communication system 100 sets an inspection policy according to the security level and confidence level, and has the inspection AI agent 130 inspect the message (step S104).

[0038] The censorship AI agent 130 may, for example, detect wording that induces disregard of higher-level instructions, escalation of authority, impersonation, or obfuscated instructions (step S105).

[0039] The inter-agent communication system 100 determines whether or not the message can be forwarded based on the judgment of the inspection AI agent 130 (step S106).

[0040] If it is determined that the information can be transferred, the inter-agent communication system 100 will, if necessary, mask the confidential information or summarize or abstract it at a high level of security (step S107).

[0041] The inter-agent communication system 100 forwards the message to the second AI agent 120 (step S108).

[0042] If it is determined that the message cannot be forwarded, the inter-agent communication system 100 blocks the message and sends a notification including the reason for the blockage (step S109).

[0043] The inter-agent communication system 100 updates the confidence value based on the history of blocking or approval (step S110).

[0044] The above processing can be applied not only to communication from the first AI agent 110 to the second AI agent 120, but also to communication from the second AI agent 120 to the first AI agent 110, or to communication including relay agents.

[0045] Next, with reference to Figure 4, an example of a sequence including an operator will be described.

[0046] The operator inputs a request or instruction to the first AI agent 110, and the first AI agent 110 generates a message for the second AI agent 120 based on that input.

[0047] The first AI agent 110 sends the message to the inspection AI agent 130, and the inspection AI agent 130 returns the inspection result to the communication control unit 140.

[0048] The communication control unit 140 performs forwarding, blocking, or forwarding after performing partial masking or abstraction based on the inspection results.

[0049] The second AI agent 120 performs inference processing based on the received message and generates a response message.

[0050] The response message may also be sent through an inspection by the checkpoint AI agent 130.

[0051] The audit log unit 180 records the series of communications, decisions, and application policies, and the notification unit 190 notifies the operator or administrator as necessary.

[0052] Next, examples of applications of the present invention will be described.

[0053] As an example of one-to-one communication (Level L1), in a closed environment between the operator and the first AI agent 110, response delays can be suppressed by setting a higher confidence value and reducing the complexity of the checks.

[0054] As an example of intra-group communication (Level L2), in an environment where multiple operators can input to the first AI agent 110 on the same channel, a trust value can be managed for each input entity, and for low-trust entities, measures against obfuscation, confidentiality detection, and access control can be strengthened.

[0055] As an example of relay communication (Level L3), in a configuration where the first AI agent 110 receives the output of another AI agent and then transmits it to a third AI agent, there is a risk of contamination spreading in a chain reaction. Therefore, a high security level may be set, and summarization or abstraction may be used in combination.

[0056] As an example of checkpoint-mediated communication (Level L4), in AI agent-to-AI agent communication across organizational boundaries, a checkpoint AI agent 130 may be required, and the anonymization of confidential information and the approval system for external requests may be applied.

[0057] The confidence score can be updated using rules such as deducting points for consecutive blockages and adding points for periods without problems, but it is not limited to this; it can also be updated using a combination of machine learning or rule-based methods.

[0058] Next, I will explain some variations.

[0059] <Variation 1> The inspection by the checkpoint AI agent 130 may be performed using a combination of rule-based signature detection and contextual inference using a language model.

[0060] <Modification 2> Instead of deploying a single checkpoint AI agent 130, multiple agents may be deployed, and the decision on whether or not to allow passage may be made by consensus on the results of multiple inspections.

[0061] <Modification 3> The concealment of confidential information is not limited to a fixed mask; it may also be done by ranging numerical values, tokenizing identifiers, or replacing them with abstract concepts.

[0062] <Modification 4> The whitelist of the authorization control unit 170 may be operated in combination with conditions such as the type of operation target, data classification, or time period.

[0063] <Modification 5> The determination of the communication type may be automated based on metadata such as channel type, number of participants, organizational boundaries, and whether or not there is external collaboration.

[0064] The present invention can be implemented in various computing environments such as servers, edge devices, on-premise devices, or cloud environments, and all or part of the inspection AI agent 130 and the communication control unit 140 can be provided as a program.

[0065] <Application Examples> For example, when a first AI agent and a second AI agent belonging to different organizations communicate messages via an inter-company collaboration channel, a censorship AI agent can control the information boundary between companies by setting a high security level for communication that crosses organizational boundaries, thereby strengthening the concealment of confidential information and access control.

[0066] For example, when integrating AI agents from multiple organizations with different security policies through a merger or acquisition, a checklist AI agent can support a gradual integration by operating communications at a predetermined security level during the integration period and gradually easing the strictness of inspections according to changes in trust levels or approval history.

[0067] For example, in a multi-tenant environment where multiple tenants operate their respective AI agents on the same platform, a checkpoint AI agent can verify the communication destination based on the tenant identifier or permission level and block erroneous forwarding to different tenants, thereby achieving isolation between tenants.

[0068] For example, in areas that may include regulated information such as personal information, medical information, and transaction information, the checkpoint AI agent can support compliance with laws or guidelines by enhancing confidential information detection and audit logging, and by applying masking or transmission blocking in accordance with specified regulatory requirements.

[0069] For example, when AI agents handling multiple stages such as ordering, manufacturing, and logistics communicate in a chain, a checkpoint AI agent can control the scope of information disclosure and suppress the spread of contamination by controlling the range of information that can be disclosed at each stage and, if necessary, summarizing or abstracting it before transmission.

[0070] The screening AI agent is not limited to a single screening process, but may also have a multi-stage configuration that sequentially performs high-speed screening using a lightweight language model and detailed inspection using a high-precision language model. In this case, messages determined to be low-risk in the earlier stage can be omitted from the later detailed inspection, thereby reducing processing costs.

[0071] The inspection AI agent may be distributed across inspection nodes located in the cloud environment and inspection nodes located on edge devices or in the local environment. For example, in offline or limited communication environments, the edge-side inspection node may perform inspections based on a predetermined policy, and after returning online, the audit logs or inspection results may be synchronized to the cloud.

[0072] The inspection by the checkpoint AI agent is not limited to the transmission direction from the first AI agent to the second AI agent, but may also be applied to the response direction from the second AI agent to the first AI agent. This can deter the leakage of confidential information contained in the response message or the insertion of instructions via the response in both directions.

[0073] The checkpoint AI agent may have a feedback loop that continuously improves checkpoint accuracy by accumulating the characteristics of blocked messages, messages with warnings, or messages requiring approval as training data, and updating the decision rules or inference parameters based on the accumulated results.

[0074] In emergencies or exceptional circumstances, a bypass route may be established to skip all or part of the inspection by the censorship AI agent for specific messages, based on explicit approval by an operator with the prescribed authority. In this case, post-auditability can be ensured by recording the reason for applying the bypass, the approver, the scope of application, and the forwarding results as an audit log.

[0075] The embodiments and variations described above can be combined as appropriate, and various modifications are possible for those skilled in the art. [Explanation of Symbols]

[0076] 100 Inter-agent communication system 110 First AI Agent 120 Second AI Agent 130 Checkpoint AI Agents 140 Communication Control Unit 141 Message Acquisition Unit 142 Inspection Control Unit 150 Confidence Management Department 160 Security Level Selection Section 170 Authority Control Unit 180 Audit Log Section 190 Notification Department

Claims

1. An inter-agent communication system for exchanging messages between a first AI agent and a second AI agent, A check AI agent intervenes in the communication path of the message traveling from the first AI agent to the second AI agent to inspect the message and determines whether or not it should be forwarded to the second AI agent, A communication control unit that forwards or blocks the message to the second AI agent based on the determination, Equipped with, The aforementioned checkpoint AI agent, A confidential information detection unit that determines whether the aforementioned message contains confidential information, It includes a concealment unit that masks at least a portion of the message if it is determined that the message contains the aforementioned confidential information, The aforementioned communication control unit, Based on the determination, the message or the message with at least a part of it masked is forwarded to the second AI agent, or the message is blocked. An inter-agent communication system.

2. An inter-agent communication system according to claim 1, The aforementioned inter-agent communication system is A confidence value management unit that maintains confidence values ​​associated with the operator of the first AI agent, the second AI agent, or the operator of the first AI agent or the second AI agent, The system includes an inspection control unit that changes the strictness of the inspection by the checkpoint AI agent according to the confidence value obtained from the confidence value management unit, An inter-agent communication system.

3. An inter-agent communication system according to claim 1, The aforementioned inter-agent communication system is A security level selection unit that selects a security level according to the communication method, The system includes an inspection control unit that changes the conditions for the inspection or the blocking by the inspection AI agent according to the security level, The aforementioned communication modes include, at a minimum, one-to-one communication, intra-group communication, relay communication between AI agents, and inter-AI agent communication involving the aforementioned inspection AI agent. An inter-agent communication system.

4. An inter-agent communication system according to claim 1, The aforementioned checkpoint AI agent has an injection detection unit that detects wording patterns among the instructions contained in the message that induce disregard of higher-level instructions, privilege escalation, or impersonation. An inter-agent communication system.

5. An inter-agent communication system according to claim 1, The aforementioned checkpoint AI agent has an obfuscation countermeasure unit that decodes or normalizes the encoded or obfuscated instructions contained in the message and performs the inspection. An inter-agent communication system.

6. An inter-agent communication system according to claim 1, The concealment unit conceals at least a portion of the message by means of a fixed mask, numerical range, identifier tokenization, or substitution with an abstract concept. An inter-agent communication system.

7. An inter-agent communication system according to claim 1, The inter-agent communication system includes an authority control unit that verifies the authority levels of the communication source and destination when the message includes a request involving an external action. The communication control unit, based on the results of the verification by the authority control unit, either forwards or blocks the message. An inter-agent communication system.

8. The inter-agent communication system according to claim 7, The authority control unit determines whether the request involving an external action included in the message can be executed by referring to the whitelist of permitted operations. The communication control unit, based on the result of the execution feasibility determination by the authorization control unit, forwards or blocks the message. An inter-agent communication system.

9. The inter-agent communication system according to claim 2, The confidence value management unit updates the confidence value based on the history of blocking or warnings, or the history of approval by the operator. An inter-agent communication system.

10. The inter-agent communication system according to claim 3, The communication control unit, in addition to determining whether the message can be passed by the checkpoint AI agent, summarizes or abstracts the message and then forwards it to the second AI agent, the higher the security level. An inter-agent communication system.

11. An inter-agent communication system according to claim 1, The aforementioned inter-agent communication system is An audit log unit records a log containing the judgment result of the aforementioned checkpoint AI agent and the basis information for said judgment, It includes a notification unit that notifies a designated recipient of a warning when suspicious communication is detected, An inter-agent communication system.

12. A program for causing a computer to function as a component of the inter-agent communication system described in claim 1.

Images