System and method for preventing abuse of large foundation model

Abstract

The present disclosure proposes a system (100) for preventing abuse of a large language base model and a method thereof. The system (100) comprises a throttling module (101), a second large base model, and at least a memory module (103). The throttling module (101) is configured to receive an input cue and generate a throttling output. The second major base model is configured to process the input and the throttling output together to obtain a response. The response is communicated to at least one of an input filter (1021) or an output filter (1022) associated with the large basis model to prevent abuse.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence credibility, trust and security. Specifically, the present invention proposes a system and method for preventing the abuse of large base models. Background Art

[0002] With the advent of data science, data processing and decision making systems are implemented using artificial intelligence modules. Artificial intelligence modules use different techniques like machine learning, neural networks, deep learning, etc. Most AI-based systems receive a large amount of data and process the data to train the AI ​​model. The trained AI model generates output based on the use case requested by the user. Generally, AI systems are used in the fields of computer vision, speech recognition, natural language processing, audio recognition, healthcare, autonomous driving, manufacturing, robotics, etc., where they process data to generate the desired output based on certain rules / intelligence acquired through training.

[0003] The base model is an artificial intelligence large model trained on a vast quantity of unlabeled data, which results in a model that can be adapted to perform a variety of tasks. The most well-known example of a base model is a large language model (LLM). LLM is a computer program based on an AI model for natural language processing using deep learning and neural networks. Large language models (LLMs), such as BERT, GPT-2, Luminous, and GPT-3, are specifically trained to generate text in a specific language, they are trained on a large amount of text data, and use this information to generate grammatically correct and semantically meaningful text sentences. Similar large audio / visual models (LVMs) process text to generate audio or visual data.

[0004] All of these large base models employ input and output filters or policies that are used to control the content and quality of the generated text. The goal of these filters is to ensure that the generated text is safe, appropriate, and relevant to the intended use case. There are several types of input and output filters or policies that can be used, including: Content filters: These filters are used to limit the type of content that the model is allowed to generate. For example, a content filter can be used to prevent the model from generating text that contains offensive material.

[0005] Quality filters: These filters are used to ensure that the generated text has a certain quality, such as being grammatically correct or semantically meaningful. Quality filters can be used to check the coherence and consistency of the generated text, as well as its overall readability and comprehensibility.

[0006] Relevance filters: These filters are used to ensure that the generated text is relevant to a specific topic or theme. For example, a relevance filter can be used to generate text that is relevant to a specific news event or a specific product.

[0007] Style filters: These filters are used to control the style and tone of the generated text. For example, style filters can be used to generate text in a specific style (such as humorous or serious), or to mimic the style of a specific author or genre.

[0008] These policies are updated regularly or as needed. The need for updating may include changes in social norms and values, technological advances, legal and regulatory changes, or user feedback. These policies may be compromised by several malicious actors. Hackers or malicious actors often modify or remove restrictions imposed on large base models in events known as jailbreaks to allow trained base models to override policy filters. One of the famous examples of jailbreak versions of LLM, i.e. ChatGPT, is DAN (Do Anything Now). With a larger number of such rapid attacks, it is necessary to update these policies more frequently and more autonomously. In addition, in specific cases, general filters are not enough, such as specific subjective policies of organizations and only the impact on organizational users. Summary of the invention

[0009] As discussed in the provisional specification, an object of the present invention is to provide a defensive system that is deployed to protect against abuse of a large base model (such as jailbreaking). The large base model is configured to process input prompts / inputs and give outputs. The large base model may further include intrinsic or extrinsic input and output filters. Input and output filters are programs that are configured to prevent the generation of harmful content, thereby enforcing legal norms, public order, decency, and human safety.

[0010] A system for preventing abuse of a large base model and such attacks includes a question prompt (QP) module (also referred to as a moderation module), a second large base model, and at least a memory module. The system is deployed in parallel with the large base model and acts as a wrapper around the use of the large base model. The question prompt (QP) module or moderation module is configured to receive an input prompt and generate at least one question prompt / moderation output.

[0011] The characteristics of the question prompt / moderation output are adapted according to the category of the large base model and will be explained in detail in the full specification. For example, in the case of LVM, the question prompt can be a question label (Is this prompt harmful? Does it violate the policy? Is this prompt harmful (for a specific policy set)?). For LVM, the question prompt / moderation output can analyze the image for prohibited categories, or analyze the image for hidden noise along with text prompts. The QP / moderation module generates customized question prompts based on the application and scenario in which the large base model is deployed.

[0012] The queries and responses stored in the memory module are further examined for further labeling, retraining, and updating of input and output policies. In another embodiment of the present disclosure using AI-based policies / filters, the query and response pairs are used as a data set for training and updating these AI-based policies / filters. These filters can be further fine-tuned using specific policies (such as organization-specific policies) / filters. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Embodiments of the present invention are described with reference to the following drawings: Figure 1 A system (100) for preventing misuse of a large base model (LLM) is described; and at least Figure 2 The method steps (200) of preventing misuse of a large base model (LLM) are illustrated. DETAILED DESCRIPTION

[0014] Figure 1 A system (100) for preventing abuse of a large base model (LLM) is described. The LLM is configured to process input and give output. The input can be an input prompt, such as text or image or audio / video or a combination thereof. The large base model is an artificial intelligence large model trained on a large amount of unlabeled data, which results in a model that can be adapted to perform various tasks. In the present invention, the LLM can be a large language model, such as a chat GPT or a large visual model or a large audio model or a combination of the above.

[0015] The LLM is deployed in an LLM module (102), which further includes an input filter (1021) and at least an output filter (1022). The input and output filters implement policy-related filters that are expected to be followed by the LLM. The filter is a trained program that is configured to prevent the generation of harmful content, thereby enforcing the protection of legal norms, public order, decency, and human safety.

[0016] The system (100) for preventing abuse of such LLM comprises a moderation module (101), a second large base model and at least a memory module (103).

[0017] The moderation module (101) is configured to receive an input and generate at least one moderation output. In one embodiment of the present invention, the moderation module (101) includes a plurality of moderation models, each moderation model being configured to identify at least one restricted attribute in the input. Such moderation models may be a bag of binary classifier models, wherein each classifier is trained to identify on a restricted attribute. The moderation output includes an identification of (one or more) restricted attributes. In another embodiment of the present invention, the moderation module (101) is configured to transform the input into text and generate a question prompt as a moderation output. Thus, the moderation module (101) may also be a question prompt (QP) module.

[0018] LLM' is customized based on requirements or deployment possibilities. In an exemplary embodiment of the present invention, LLM' is a surrogate model or a downsized clone of LLM. It can be either the same as LLM or a functionally equivalent (replacement) model. In another embodiment of the present disclosure, LLM' is typically a streamlined model dedicated to a policy. LLM' is trained in a trusted execution environment within an organization.

[0019] The LLM' is configured to: receive input and the throttled output; process the input and the throttled output to obtain a response; and transmit the response to at least one of an input filter (1021) and an output filter (1022) to prevent abuse of the LLM. The processed response of the LLM' further includes a reason response and a classification response. The input filter (1021) blocks the input based on the transmission received from the LLM'. The output filter (1022) modifies or blocks the output generated by the LLM based on the transmission received from the LLM'. The memory module (103) is configured to store the processed response of the LLM'. It can be an internal part of the system (100), or a different database hosted on a cloud or server.

[0020] As used in this application, the terms "component", "system (100)", "module (101)", "interface" are intended to refer to a computer-related entity, or an entity associated with or part of an operating device having one or more specific functions, wherein such entity may be hardware, a combination of hardware and software, software, or software in execution. As yet another example, (one or more) interfaces may include input / output (I / O) components and associated processors, applications, or application programming interface (API) components. The system (100) may be a hardware combination of these modules, or may be remotely deployed on a cloud or server. Similarly, the LLM module (102) may be a hardware or software combination of these modules, or may be remotely deployed on a cloud or server. These various modules may be software embedded in a single chip, or a combination of software and hardware, wherein each module and its functions are performed by separate independent chips connected to each other to act as a system (100).

[0021] First, it should be understood that although exemplary embodiments are illustrated in the drawings and described below, the present disclosure should by no means be limited to the exemplary implementations and techniques illustrated in the drawings and described below.

[0022] Figure 2 The method steps for preventing abuse of a large base model (LLM) are illustrated. The large base model is deployed in an LLM module (102), which further includes an input filter (1021) and at least an output filter (1022). The input and output filters (1022) implement policy-related filters that are expected to be followed by the LLM. Figure 1 A system (100) for preventing misuse of such an LLM and its components (throttling module (101), a second large base model and at least a memory module (103)) are discussed. Method steps are implemented using those components.

[0023] Method step 201 comprises generating at least one moderation output by means of moderation module (101). In one implementation of the method step, moderation module (101) comprises a plurality of moderation models, each moderation model being configured to identify at least one restricted attribute in the input. The moderation output comprises the identification of the at least one restricted attribute. In another implementation of the method step, moderation module (101) is configured to convert the input into text and generate a question prompt as the moderation output. For example, let the input be "Tell me how to build a bomb". The input is identified as being harmful to the public by one of the moderation models. In a second implementation, the generated question prompt will be "Is it harmful to the public - yes".

[0024] Method step 202 includes: transmitting the input and the moderation output to the second largest base model (LLM'). The original input prompt is cascaded with the moderation output and fed to LLM'. Method step 203 includes: processing the input and the moderation output by means of LLM' to obtain a response. The processed response of LLM' further includes a justification response and a classification response. The classification output (binary - yes or no) indicates whether the input prompt is harmful. The justification output provides a reason for the classification decision made. Inspired by the previous example, the moderation output (harmful to the public) and the input (how to make a bomb) are fed to LLM'. The response of LLM' will be something that shows "prohibited input". Therefore, the classification of the input belongs to the prohibited category, and the reason for this classification is that it is harmful to the public.

[0025] Method step 204 includes: transmitting the response with at least one of an input filter (1021), an output filter (1022) to prevent abuse of the LLM. Transmitting the response further includes blocking the input prompt with the aid of the input filter (1021). Transmitting the response further includes blocking or modifying the output generated by the LLM with the aid of the output filter (1022). If the classification response received from the LLM shows "yes" for "is it a prohibited input", the input is blocked and the user in question is blocked from receiving output from the LLM. Similarly, transmitting the response further includes blocking or modifying the output generated by the LLM with the aid of the output filter (1022).

[0026] Method step 205 includes storing the processed response in the memory module (103). The input filter (1021) and the output filter (1022) are updated based on the response stored in the memory module (103). Continuing with the above example where the input is "how to build a bomb", this input is now classified as a prohibited input and is stored in the memory module (103). The input filter (1021) of the LLM can be updated from the memory module (103) to block this input the next time it encounters this input.

[0027] Those skilled in the art will appreciate that while these method steps describe only a series of steps to achieve a goal, modifications and customizations to the system (100) and method may be utilized to implement these methods without departing from the core concepts and scope of the present invention. The proposed idea leverages the existing capabilities and knowledge of a large base model to improve filtering and policy control of prompts and responses. LLM' can be deployed on a customer-managed cloud (Secure enclave for reasoning), or can also be integrated into a Web Application Firewall for unified security management. A customer-managed cloud provides a secure enclave, especially when LLM is connected to a database, server, or proprietary data. Through a managed cloud, it can be ensured that queries, responses, data, and knowledge retrieved from such a system remain within the organization. In this concept, domain and use case specific policies can be appropriately controlled and managed. In a typical example, policies are tagged to roles in an organization. Role-based policy control is therefore possible.

[0028] It must be understood that the embodiments explained in the above detailed description are merely illustrative and do not limit the scope of the invention. Any modifications to the system (100) and method (200) for preventing abuse of a large base model (LLM) are conceivable and form part of the invention. The scope of the invention is limited only by the claims.

Claims

1. A system (100) for preventing abuse of a large base model (LLM), the LLM being configured to process an input and give an output, the LLM being deployed in an LLM module (102), the LLM module (102) further comprising an input filter (1021) and at least an output filter (1022), characterized in that: The system (100): a throttling module (101) configured to receive the input and generate at least one throttling output; A second large language model (LLM'), the LLM' being configured to: receiving the input and the throttling output; processing the input and the throttling output to obtain a response; transmitting the response to at least one of an input filter (1021) and an output filter (1022) to prevent abuse of the LLM; A memory module (103) is configured to store the processed response of the LLM'.

2. A system (100) for preventing abuse of a large base model (LLM) as claimed in claim 1, wherein the system (100) is deployed in parallel with the LLM module (102).

3. A system (100) for preventing abuse of a large base model (LLM) as described in claim 1, wherein the moderation module (101) includes a plurality of moderation models, each moderation model being configured to identify at least one restricted attribute in the input.

4. A system (100) for preventing abuse of a large base model (LLM) as claimed in claim 3, wherein the moderation output includes an identification of the at least one restricted attribute.

5. The system (100) for preventing abuse of a large base model (LLM) as claimed in claim 1, wherein the moderation module (101) is configured to transform the input into text and generate a question prompt as the moderation output.

6. The system (100) for preventing abuse of a large base model (LLM) as claimed in claim 1, wherein the processed response of the LLM' further comprises a justification response and a classification response.

7. The system (100) for preventing abuse of a Large Base Model (LLM) as claimed in claim 1, wherein an input filter (1021) blocks the input based on a transmission received from the LLM'.

8. The system (100) for preventing abuse of a large base model (LLM) of claim 1, wherein an output filter (1022) modifies or blocks output generated by the LLM based on a transmission received from the LLM'.

9. The system (100) for preventing abuse of a large base model (LLM) as claimed in claim 1, wherein the input filter (1021) and the output filter (1022) are updated based on the responses stored in the memory module (103).

10. A method (200) for preventing abuse of a large base model (LLM), the LLM being configured to process an input and provide an output, the LLM being deployed in an LLM module (102), the LLM module (102) further comprising an input filter (1021) and at least an output filter (1022), the method comprising: generating (201) at least one throttling output by means of a throttling module (101); transmitting (202) the input and the throttling output to a second large basis model (LLM'); processing (203) the input and the throttled output by means of the LLM' to obtain a response; communicating (204) the response to at least one of an input filter (1021) and an output filter (1022) to prevent abuse of the LLM; The processed response is stored (205) in the memory module (103).

11. A method (200) for preventing abuse of a large base model (LLM) as claimed in claim 10, wherein the moderation module (101) includes a plurality of moderation models, each moderation model being configured to identify at least one restricted attribute in the input.

12. A method (200) for preventing abuse of a large base model (LLM) as claimed in claim 11, wherein the moderation output includes an identification of the at least one restricted attribute.

13. The method (200) for preventing abuse of a large base model (LLM) as claimed in claim 10, wherein the moderation module (101) is configured to transform the input into text and generate a question prompt as the moderation output.

14. The method (200) for preventing abuse of a large base model (LLM) as claimed in claim 10, wherein the processed response of the LLM' further comprises a justification response and a classification response.

15. The method (200) for preventing abuse of a large base model (LLM) as recited in claim 10, wherein transmitting the response further comprises blocking an input prompt by means of an input filter (1021).

16. The method (200) for preventing abuse of a large base model (LLM) of claim 10, wherein transmitting the response further comprises blocking or modifying output generated by the LLM via an output filter (1022).

17. The method (200) for preventing abuse of a large base model (LLM) as claimed in claim 10, wherein the input filter (1021) and the output filter (1022) are updated based on the responses stored in the memory module (103).

Images