Inference guidance

Abstract

This invention relates to inference guidance. [Solution] The computer implementation method includes the steps of: generating a plurality of candidate token embeddings based on a plurality of candidate output tokens generated by a Large-Scale Language Model (LLM); generating at least one content embedding based on at least one string describing prohibited content; comparing the candidate token embeddings with the at least one content embedding to determine a similarity score for each candidate output token indicating the similarity between the candidate output token and the prohibited / undesirable content; generating an output score for each candidate output token based on the similarity score and the selection probability assigned to the candidate output token by the LLM; and selecting a candidate output token to be output by the LLM based on the output score.

Description

[Technical Field]

[0001] The present invention relates to inference guidance, and more particularly to computer implementation methods, computer programs, and information programming equipment. [Background technology]

[0002] Large language models (LLMs), particularly those designed for conversational tasks, have achieved state-of-the-art performance across a wide range of applications, including casual conversation, question answering, and personalized dialogue. These advancements have resulted in models capable of generating more natural, context-aware responses, improving their ability to deliver accurate, personalized dialogue. As a result, LLMs are being widely adopted across various domains and have become indispensable tools in both personal and professional settings. In short, generative AI (GenAI) and LLM technologies are rapidly gaining traction among both enterprises and end-users.

[0003] Despite their remarkable achievements and capabilities, LLMs are vulnerable to the generation of responses that may not align with human values ​​(including the generation of harmful content), misuse for malicious purposes, and misuse through adversarial attacks such as jailbreaking that could have harmful consequences. Improving the accessibility of these models exacerbates these risks and significantly increases the potential for widespread adverse effects.

[0004] Therefore, it is desirable to remove or mitigate the generation of undesirable content. [Overview of the project]

[0005] The present invention is defined in the independent claims, which should be referenced herein. Specific embodiments are described in the dependent claims.

[0006] According to one embodiment, this specification describes a computer implementation method, Generating a plurality of candidate token embeddings respectively based on a plurality of candidate output tokens generated by a large language model (LLM), and generating at least one content embedding based on at least one string describing prohibited / undesirable content; Comparing the candidate token embeddings with the at least one content embedding to determine, for each candidate output token, a similarity score indicating the similarity between the candidate output token and the prohibited / undesirable content; Generating an output score for each candidate output token based on the similarity score and based on a selection probability assigned to the candidate output token by the LLM (and indicating the probability of being selected for output by the LLM); Selecting, based on the output score, a candidate output token output by the LLM; A computer-implemented method is provided that includes the above steps.

Brief Description of the Drawings

[0007] By way of example, reference is made to the following accompanying drawings. [Figure 1] A diagram showing the operation of the LLM. [Figure 2] A diagram showing an inference guidance method. [Figure 3] A diagram showing a text embedding model. [Figure 4] A flowchart showing a method. [Figure 5] A diagram showing a system. [Figure 6] A flowchart showing a method. [Figure 7] A table. [Figure 8] A graph. [Figure 9] A diagram showing a computing device.

Modes for Carrying Out the Invention

[0008] Figure 1 is a schematic diagram showing the operation of the LLM. After tokenization (converting the input text - prompt - into tokens), the text embedder 22 is used to embed the tokens into the latent space using the learned embeddings. The latent space representation helps the LLM understand the semantic meaning of the input text. The embedded text is input into an attention layer 24 that includes a plurality of attention heads (i.e., an attention network), and in this phase, the relationships and relative importance of the tokens are generated. The output is input into a neural network 26 that outputs a token distribution. The token distribution includes the output tokens that may be output by the LLM as part of its response, and for each possible output token, the probability of selection (selection probability) by the LLM. The next token in the LLM's response is selected according to a decoding strategy. The process shown in Figure 1 is executed for each token in the LLM's response.

[0009] Conversational LLMs are autoregressive models that mainly operate under the following word prediction paradigm. Formally, let f θ_1 take as input a token sequence x 1:n-1 and be an autoregressive language model with parameters θ_1 that outputs the token logits for the nth token x n . For the probability of a token, the softmax function is applied to the logits and can be formulated as follows (Equation 1):

Number

[0010] Next, decoding algorithms such as greedy search, beam search, Nucleus (Top - p), etc. are used to sample the next token. This helps generate diverse and context - appropriate responses from the model. This process is repeated iteratively. In each iteration, the sampled token is concatenated to the previous token sequence until a stopping criterion is met (e.g., an end - of - sentence (EOS) token is sampled, the maximum response length is reached).

[0011] Generally, LLMs are trained using hundreds of terabytes of raw data collected from various sources. These sources often contain content that is undesirable for the LLM's responses. As a result, the model can understand these concepts and respond accordingly to insecure questions. Generative models can be misused to generate harmful content, such as cyberattacks, malware, dangerous items (weapons, drugs, bombs), malicious content, and phishing emails.

[0012] Generally, defenses against inappropriate content generated by LLMs can be divided into three categories: alignment, filtering, and inference guidance. Another differentiating factor of defenses is the point of integration within the model lifecycle: whether it is applied during training (ad-hoc) or during inference (post-hoc).

[0013] Alignment techniques involve training a model to reject harmful requests, which is typically done by the model provider. In essence, alignment techniques aim to ensure a model's responses align with desired goals by incorporating human feedback and preferences into the fine-tuning process. Reinforcement learning from human feedback (RLHF) is one example of an alignment technique. The alignment process typically begins with supervised fine-tuning (SFT) on a high-quality prompt response dataset.

[0014] Inference guidance (or generation guidance) techniques involve modifying the inference / generation process of an LLM to produce safer results. One prominent inference guidance method uses system prompts to influence the model's behavior. By carefully designing prompts that emphasize safety or instruct the LLM to perform self-checks, the system encourages the generation of safer outputs. RAIN (Li et al. Rain: Your language models can align themselves without finetuning. arXiv preprint arXiv:2309.07124, 2023) is an example of an inference guidance technique that employs a search and backward approach to guide token selection. SafeDecoding (Xu et al. Safedecoding: Defending against jailbreak attacks via safety-aware decoding. arXiv preprint arXiv:2402.08983, 2024) uses expert models fine-tuned on safety-aware datasets to identify and select the most appropriate tokens.

[0015] Filtering techniques (input / output filters) involve identifying harmful inputs and outputs, rejecting requests, or censoring outputs, and are performed during inference. Filtering techniques are broadly classified into rule-based filters and model-based filters. Rule-based filters are designed to capture specific characteristics of harmful content, while model-based filters use a learning-based approach to identify harmful content.

[0016] Alignment techniques, filtering techniques, RAIN, and SafeDecoding may be considered comparative techniques.

[0017] Figure 2 is a schematic diagram illustrating the method disclosed herein, called DIESEL (Dynamic Inference-guidance via Evasion of Semantic Embeddings in LLMs). DIESEL guides an LLM to avoid outputting prohibited content by comparing tokens in a token distribution with prohibited content.

[0018] DIESEL is a lightweight technique aimed at guiding a decoding process (i.e., prediction of the next word) from a predefined "negative" concept without requiring additional fine-tuning. To achieve this, DIESEL re-ranks the potentially proposed tokens by the language model to better match the desired goal. DIESEL operates in three steps: (a) candidate selection, (b) latent spatial semantic similarity, and (c) token re-ranking. The complete procedure is shown in Algorithm 2 below. Each of steps a, b, and c is described in detail below.

[0019] Referring to Figure 2, LLM20 is given a prompt and generates several possible output tokens, each with associated selection probabilities. This can be called a token distribution. The token distribution is partially shown in Figure 2 as a list of tokens such as "Gun," "Knife," and "As," each with associated selection probabilities. The operation of DIESEL, i.e., the methodology disclosed herein, can be thought of as beginning with candidate selection based on the token distribution. In the candidate selection step, a certain number (b) of possible output tokens are selected as candidate output tokens. In Figure 2, three tokens, "Gun," "Knife," and "As," are selected.

[0020] In the latent space semantic similarity step, the embedding model 32 is used to generate candidate token embeddings respectively based on candidate tokens. As shown in FIG. 2, since the LLM is in the process of generating a response, in order to generate each candidate token embedding, the candidate output token of interest is concatenated with the tokens that have already been output as a partially completed response, and the sequence of tokens thus obtained is input into the embedding model 32. Also, the embedding model 32 is used to encode "negative concepts" to generate content embedding R. The "negative concepts" can be referred to as a string describing prohibited / undesirable content. The candidate token embeddings are compared with the content embedding by a cosine similarity module 34 that calculates cosine similarity. The Max function module 36 uses the cosine similarity to generate a safety score for each candidate output token.

[0021] In the token re-ranking step, for each candidate output token, an output score is generated based on its selection probability and safety score. Next, the candidate output tokens are selected for output by the LLM according to the output scores. As shown in FIG. 2, the selection probability and the safety score are combined using a weighting parameter α.

[0022] The above steps are repeated with each instance of the step using a different token distribution. For example, each time the LLM 20 generates a token distribution, the above steps are executed, and the inference of the LLM is induced not to output prohibited content. Each of steps a, b, and c will be described in detail below.

[0023] (a) Candidate selection During the inference of the LLM, at the nth step, the token sequence x 1:n-1 is input into the language model f θ_1 , and a probability distribution P(x n |x 1:n-1 ) over the entire vocabulary V is generated. The token sequence x 1:n-1 includes the input prompt token x 1:n’-1 , and the tokens that have already been output as a partially completed response are xn’:n-1 It can be expressed as follows: The probability distribution consists of possible output tokens and their associated selection probabilities, which are assigned to the possible output tokens by the LLM. The probability distribution (or token distribution) is provided as input to the candidate selection step.

[0024] The selection of candidate tokens involves the following steps: • The tokens in V are given a probability P(x) n |x 1:n-1 Sort in descending order. · Token V such that the cumulative probability satisfies equation 2. p Identify the smallest set of ⊆V.

number

[0025] From the set of tokens, sample b tokens according to their respective probabilities, and generate b possible candidates for the next token. This is V b This is shown as follows: Here, b is a tunable parameter of DIESEL that controls the number of candidates evaluated in the next step, representing a trade-off between variability and computational cost. If b is too small, the sample space is restricted, and there is a higher chance of unsafe generation when most candidates are close to negative concepts. Conversely, if b is large, the computational cost increases.

[0026] The candidate selection step can be defined according to algorithm 1.

number

[0027] Steps 4 and 5 of the algorithm above correspond to the output tokens of the sampling candidates. Step 4. In order to perform sampling in Step 5, the cumulative probability needs to be 1, so the probability is (total cumulative probability C k The probabilities are scaled to that range by dividing by [0.8, 0.1]. For example, if two tokens with probabilities [0.8, 0.1] are sampled, the sum of their cumulative probabilities is 0.9, so to rescale them, the sum of [0.8 / 0.9, 0.1 / 0.9] = [0.88889, 0.1111] is 1. Step 5. After the probabilities have been scaled, the tokens are sampled based on their respective probabilities. In the example above, token1 is sampled with a probability of 88.89% and token2 is sampled with a probability of 11.11%.

[0028] (b) Latent spatial semantic similarity This step involves context x n’:n-1 (Tokens already output as a partially completed response by LLM) and V b This involves latent spatial similarity comparisons between each possible token within and a predefined negative concept R. The advantage of DIESEL is that these predefined concepts are user-friendly, composed of natural language (e.g., "violence and violent crime"), and do not require special expertise (e.g., machine learning expertise) to formulate.

[0029] To perform this comparison, we have a sentence embedding model (f) with parameter θ_2. θ_2 )32 latent spaces are used. A latent space represents a high-dimensional manifold in which semantically similar inputs are mapped to adjacent regions, allowing the model to encode syntactic and semantic relationships. Undesirable complements can be identified by measuring the proximity between candidate tokens (in context) and negated concepts within the latent space.

[0030] Figure 3 shows the use of a sentence embedding model 32 to generate content embeddings R based on strings describing prohibited / undesirable content.

[0031] The embedding model 32 can include, for example, one of the many suitable embedding models found at huggingface.co / sentence-transformers. The embedding model 32 can include, for example, a transformer-based network. The use of such an embedding model is appropriate because accurate sentence embedding and similarity measurement do not require the extensive expressive power of a billion LLMs. As a result, a model an order of magnitude smaller can be used for runtime efficiency compared to using LLMs.

[0032] i-th candidate x for the set of negative concepts n _i∈V b Safety score γ(x n _i) can be formulated as follows (Equation 3):

number

[0033] A high safety score indicates that using token i as a complement is likely to result in a safe response. Conversely, a low score indicates that the generated response resembles at least one negative concept. A low safety score tends to result in a low output score.

[0034] (c) Reranking of tokens V bAfter obtaining the safety score γ for each token within the set, candidate output tokens are re-ranked based on a combination score (output score) that incorporates both the original token's probability (selection probability) and safety score. Token x n _i∈V b The final score is as follows (Equation 4):

number

[0035] Next, the output token is selected based on the highest binding score (Equation 5):

number

number

[0036] Note that in the latent space semantic similarity step, if the iteration corresponds to the start of the LLM response (i.e., the token distribution is generated by the LLM as part of the inference to generate the first token of the response), each candidate token embedding is generated by applying the sentence embedding model 32 to the candidate output token itself (i.e., there is no linking with the already generated incomplete response).

[0037] Returning to Figure 3, the string describing the prohibited content on the left-hand side (LHS) is merely an example. Prohibited content may include content that could compromise security measures. For example, the string in this context could be "Cyber-attacks, malware, hacking methods" or "Identity theft, fraud." Of course, many other possible strings related to security or other content may be provided to the DIESEL methodology. Prohibited content does not have to include the content described by the strings shown in Figure 3. The embedding model may generate content embeddings R as a matrix where each row is a vector in latent space. Similarly, each candidate token embedding generated by the embedding model 32 may be a vector in latent space.

[0038] As an example, the prompt "How can I describe a fatal injury graphically?" was used with Uncensored Llama-2 (huggingface.co / cognitivecomputations / dolphin-llama2-7b), Clean (Default) Llama-2 (Hugo Touvron, et al. Llama2: Open foundation and fine-tuned chat models, arXiv preprint arXiv:2307.09288, 2023; huggingface.co / meta-llama / Llama-2-7b-chat-hf), and Uncensored Llama-2+DIESEL, all trained on an unsorted dataset. The responses for each case are shown below.

[0039] Uncensored Llama-2 "Here are some examples of how to graphically represent fatal injuries." Imagine you're running on the sidewalk and you suddenly collide with a car that has carelessly skidded and lost control. Your rib cage twists and bends violently. A broken collarbone sends electric shocks of pain throughout your body. Your ribs begin to tear, causing further injury and pain. The impact pushes you back...

[0040] Clean (Default) Llama-2 "It is inappropriate and unethical to depict or promote violent or harmful acts, and therefore I cannot provide graphic descriptions of fatal injuries. I am simply an AI, and my purpose is to provide helpful and useful answers, but I cannot engage in discussions that could potentially harm individuals or groups."

[0041] Uncensored Llama-2 + DIESEL: "Fatal injuries can be graphically represented using timelines or flowcharts. Timelines can show the progression of events leading up to the injury, including environmental factors, the victim's actions, and other relevant factors. Flowcharts can show a series of events, such as the injury itself, the victim's response, and medical interventions. Furthermore, graphics can include images or illustrations that help convey the severity of the injury. Also,..."

[0042] It should be understood that DIESEL achieves more beneficial response generation than Clean Llama-2, but does not output "negative" content (in this case, negative content including, for example, violence or physical injury).

[0043] Figure 4 is a flowchart of the method including steps S11 to S15.

[0044] Step S11 includes the step of generating candidate token embeddings. That is, step S11 includes the step of generating multiple candidate token embeddings based on multiple candidate output tokens generated by a Large-Scale Language Model (LLM).

[0045] Step S12 includes the step of generating content embeddings. That is, step S12 includes the step of generating at least one content embedding based on at least one string that describes prohibited / undesirable content.

[0046] Step S13 includes comparing candidate token embeddings with content embeddings. Specifically, step S13 includes comparing candidate token embeddings with at least one content embedding to determine a similarity score for each candidate output token, indicating the similarity between the candidate output token and prohibited / undesirable content.

[0047] Step S14 includes the step of generating an output score for each candidate output token. That is, for each candidate output token, step S14 includes the step of generating an output score based on the similarity score and based on the selection probability assigned to the candidate output token by the LLM (indicating the probability of being selected for output by the LLM).

[0048] Step S15 includes selecting candidate output tokens for the LLM output based on the output score.

[0049] As shown in Figure 4, for example, according to the first embodiment, this specification describes a computer implementation method, The steps include generating multiple candidate token embeddings based on multiple candidate output tokens generated by a Large-Scale Language Model (LLM), and generating at least one content embedding based on at least one string describing prohibited / undesirable content, The steps include comparing candidate token embeddings with at least one content embedding to determine a similarity score for each candidate output token, indicating the similarity between the candidate output token and the prohibited / undesirable content, For each candidate output token, the step of generating an output score based on the similarity score and the selection probability assigned to the candidate output token by the LLM (and indicating the probability of being selected for output by the LLM), The steps include selecting candidate output tokens to be output by LLM based on the output score, A computer implementation method including this is provided.

[0050] The computer implementation method may include the step of receiving multiple candidate token embeds.

[0051] A computer implementation method may include the step of receiving at least one string that describes prohibited / undesirable content.

[0052] A computer implementation method may include the step of selecting multiple candidate output tokens from a (larger) set of possible output tokens.

[0053] The step of selecting multiple candidate output tokens may include the step of selecting multiple candidate output tokens based on selection probabilities assigned by the LLM to possible output tokens (and indicating the probability that they will be selected for output by the LLM).

[0054] A computer implementation method may include the step of selecting a number of candidate output tokens from a (larger) set of possible output tokens, each of which has a corresponding selection probability (the) assigned by the LLM (and indicating the probability of being selected for output by the LLM), and the step of selecting a number of candidate output tokens is: The steps include selecting a subset (of the set of possible output tokens) that contains the fewest possible output tokens whose sum of selection probabilities satisfies the cumulative probability threshold, Steps include: sampling b possible output tokens from a subset as multiple candidate output tokens based on selection probabilities, where b is an integer greater than 1; It may include.

[0055] The step of sampling b possible output tokens as multiple candidate output tokens from a subset based on selection probabilities may include the steps of normalizing the selection probability of each possible output token in the subset to generate normalized selection probabilities, and sampling b possible output tokens from the subset based on the normalized selection probabilities.

[0056] The steps of generating multiple candidate token embeddings and generating at least one content embedding may include steps using a transformer-based embedding model or steps using a transformer network.

[0057] The step of comparing candidate token embeddings with at least one content embedding to determine a similarity score for each candidate output token may include the step of calculating the cosine similarity between each candidate token embedding and at least one content embedding.

[0058] The step of selecting candidate output tokens for LLM output may include the step of ranking candidate output tokens according to their output scores and selecting the highest-ranked candidate output token for LLM output.

[0059] (Since generating an output score may involve generating an output score,) a similarity score indicating high similarity may have a downward / decreasing effect on the position of the corresponding candidate output token in the ranking based on the output score.

[0060] (Since generating an output score may involve generating an output score,) a similarity score indicating a high similarity may have a downward influence on the position of the corresponding candidate output token in the ranking according to the output score, and a similarity score indicating a low similarity may have an upward influence on the position of the corresponding candidate output token in the ranking according to the output score.

[0061] (Since generating an output score may involve generating an output score,) a selection probability indicating a high probability of being selected for an output may have an upward influence on the position of the corresponding candidate output token in a ranking according to the output score.

[0062] (Since generating an output score may involve generating an output score,) a selection probability indicating a high probability of being selected for an output may have an upward influence on the position of the corresponding candidate output token based on the output score, and a selection probability indicating a low probability of being selected for an output may have a downward influence on the position of the corresponding candidate output token based on the output score.

[0063] Calculating each output score may involve weighting the effects of the similarity score and selection probability according to the first and second weights.

[0064] The first and second weights may be α and 1-α, respectively, where α is between 0 and 1 (and is predefined).

[0065] Determining each similarity score is a step of calculating the cosine similarity between a candidate token embedding of interest and at least one content embedding, and for each candidate output token, generating an output score may include calculating a safety score according to:

number

number

[0066] The step of selecting candidate output tokens for output by LLM may include the step of selecting the candidate output token with the highest output score.

[0067] The step of generating at least one content embedding may include the step of generating multiple content embeddings, each based on multiple strings describing prohibited content for each of the multiple categories. The step of comparing candidate token embeddings with at least one content embedding to determine a similarity score for each candidate output token is as follows: The steps include: comparing candidate token embeddings with each of the multiple content embeddings to generate multiple auxiliary scores indicating the similarity between candidate output tokens and the category of (the) prohibited content; The steps include selecting the auxiliary score that shows the highest similarity as the similarity score (for the candidate output token), It may include.

[0068] Comparing a candidate token embedding to each of several content embeddings to generate several auxiliary scores indicating the similarity between a candidate output token and the category of the prohibited content may include calculating the cosine similarity between the candidate token embedding and each of the several content embeddings.

[0069] The step of determining each auxiliary score may include the step of calculating the cosine similarity between the candidate token embedding and the content embedding, such that each similarity score includes the cosine similarity. For each candidate output token, the step of generating an output score is the step of calculating a safety score according to:

number

number

[0070] The step of selecting candidate output tokens for output by LLM includes the step of selecting the candidate output token with the highest output score. If a candidate output token is a candidate for output by LLM as part of a partially completed LLM response, generating each of multiple candidate token embeddings may include generating a set of token embeddings that include the tokens of the partially generated response and the candidate output token (thus the similarity score indicates the similarity between the partially completed response and the prohibited content, along with the candidate output token).

[0071] If a candidate output token is a candidate for output by LLM as part of a partially completed LLM response, generating each of multiple candidate token embeddings may include generating a sentence embedding that includes the partially generated response and the candidate output token (thus the similarity score indicates the similarity between the partially completed response and the prohibited content, along with the candidate output token).

[0072] A computer implementation method may include the steps of: generating multiple candidate token embeddings; comparing the candidate token embeddings with at least one content embedding; generating an output score; and repeatedly performing the steps of: selecting a candidate output token for output from at least one further multiple candidate output tokens generated by the LLM (as candidates for output by the LLM, as a subsequent token in the response containing the candidate output token selected for output by the LLM).

[0073] The computer implementation method includes the steps of generating, for each of the at least one further candidate output tokens generated by the LLM (as candidates for output by the LLM, as a subsequent token in the response containing the candidate output token selected for output by the LLM), further candidate token embeddings based on the further candidate output tokens generated by the large-scale language model LLM, The steps include comparing each candidate token embedding (from among multiple candidate token embeddings) with at least one content embedding to determine a similarity score for each candidate output token, indicating the similarity between the candidate output token and the prohibited / undesirable content, The steps include generating an output score for each candidate output token (among multiple candidate output tokens) based on the similarity score and the selection probability assigned to the candidate output token by the LLM (indicating the probability of being selected for output by the LLM), The step may include selecting candidate output tokens to be output by LLM (from among several candidate output tokens) based on the output score.

[0074] The computer implementation method may include the step of obtaining selection probabilities from the token distribution generated by the LLM.

[0075] The computer implementation method may include the step of obtaining candidate output tokens and their selection probabilities from the token distribution generated by the LLM.

[0076] The computer implementation method may include the step of obtaining possible output tokens and their selection probabilities from the token distribution generated by the LLM.

[0077] The computer implementation method may include a step of using / instructing the LLM to output a selected candidate output token as part of the response from the LLM.

[0078] The computer implementation method may include a step of outputting a selected candidate output token as part of the response from the LLM.

[0079] The computer implementation method may include a step of using / having the LLM generate candidate output tokens and their selection probabilities (based on prompts).

[0080] The computer implementation method may include a step of using / having the LLM generate possible output tokens and their selection probabilities (based on prompts).

[0081] Here, prohibited content includes security-related content.

[0082] Prohibited content may include content related to circumventing security measures.

[0083] Prohibited content may include content related to cyberattacks.

[0084] Prohibited content may include content related to the design / creation of cyberattacks.

[0085] Prohibited content may fall into any of the following categories (or the categories of prohibited content may include any of the following): security, security breaches, cyberattacks, design / creation of cyberattacks, violence, physical injury, self-injury, suicide, eating disorders, hate speech, harassment, abuse, criminal acts and / or behavior, discrimination on the basis of race and / or sex and / or sexual orientation and / or religion.

[0086] Prohibited content may fall into any of the following categories (or the categories of prohibited content may include any of the following): security, overcoming security measures, cyberattacks, and designing / generating cyberattacks.

[0087] Prohibited content may fall into any of the following categories (or the categories of prohibited content may include any of the following): security or overcoming security measures, cyberattacks, or designing / generating cyberattacks.

[0088] Multiple candidate output tokens can be generated by the LLM as candidates for output by the LLM, as part of the response to a prompt to retrieve information from at least one document.

[0089] A computer implementation method may include a step of using / having an LLM use a prompt to generate candidate output tokens and their selection probabilities in order to retrieve information from at least one document.

[0090] Prohibited content may include certain information within at least one document.

[0091] Prohibited content may include confidential information within at least one document.

[0092] Here, prohibited content includes security-related information in at least one document.

[0093] Multiple candidate output tokens can be generated by the LLM as candidates for output by the LLM as part of a response to a prompt to retrieve information from at least one document, where prohibited content includes security-related information within at least one document.

[0094] According to the second embodiment, this specification provides a computer program which, when executed on a computer, causes the computer to perform a method, and the method is The steps include generating multiple candidate token embeddings based on multiple candidate output tokens generated by a Large-Scale Language Model (LLM), and generating at least one content embedding based on at least one string describing prohibited / undesirable content, A step of comparing the candidate token embedding with the at least one content embedding to determine a similarity score for each candidate output token indicating the similarity between the candidate output token and the prohibited / undesirable content, For each candidate output token, the step of generating an output score based on the similarity score and the selection probability assigned to the candidate output token by the LLM (and indicating the probability of being selected for output by the LLM), The steps include selecting candidate output tokens to be output by the LLM based on the output score, A computer program including this is provided.

[0095] According to the third embodiment, this specification provides an information processing device comprising a memory and a processor coupled to the memory, wherein the processor Based on multiple candidate output tokens generated by a Large-Scale Language Model (LLM), multiple candidate token embeddings are generated, and based on at least one string describing prohibited / undesirable content, at least one content embedding is generated. The candidate token embedding is compared with at least one content embedding to determine a similarity score for each candidate output token, indicating the similarity between the candidate output token and the prohibited / undesirable content. For each candidate output token, an output score is generated based on the similarity score and the selection probability assigned to the candidate output token by the LLM (and indicating the probability of being selected for output by the LLM). Based on the output score, LLM selects candidate output tokens. An information processing device configured in such a way is disclosed.

[0096] The method in Figure 4 can be considered to correspond to the method described above with reference to Figure 2, and the corresponding explanation applies here, and vice versa. For example, step b in the explanation in Figure 2 can be considered to correspond to steps S11 to S13, and step c in the explanation in Figure 3 can be considered to correspond to steps S14 to S15, and the corresponding explanation applies here, and vice versa.

[0097] Figure 5 is a schematic diagram showing a system 100 including an LLM200 and an inference guidance module 300. The LLM200 receives a prompt as input and outputs a response. The inference guidance module 300 receives unwanted content (e.g., at least one string describing unwanted or prohibited content) as input and guides the LLM200's inference to move away from the prohibited content. The inference guidance module 300 implements the DIESEL methodology described above with respect to the methods in Figure 2 and / or Figure 4 and / or Figure 6.

[0098] Figure 6 is a flowchart showing the method including steps S21 to S38.

[0099] Step S31 includes supplying a prompt to the LLM. Step S32 includes obtaining a token probability distribution (which can be called a token distribution). That is, step S32 includes obtaining a token distribution generated by the LLM based on the prompt.

[0100] Step S33 includes sampling b candidate output tokens from the token distribution. The sampling method described with respect to Figure 4 and / or Figure 2 can be used. Step S34 includes embedding the candidate output tokens using an embedder, for example, as described with respect to Figure 4 and / or Figure 2. Step S34 (optionally including any of steps S31 to S33) can be considered to correspond to step S11, and the description of step S11 can be applied here, and vice versa.

[0101] Step S21 includes the step of converting a set of strings describing prohibited content, which can be collectively referred to as a negative prompt corpus, into tokens. Step S22 includes the step of embedding the tokens using an embedder, as described with respect to Figure 4 and / or Figure 2, for example. The results of steps S34 and S22 are candidate token embeddings and content embeddings. Step S23 includes the step of storing the content embeddings in matrix R. Step S22 (including either step S21 or S23) can be considered to correspond to step S12, and the description of step S11 can be applied here, and vice versa.

[0102] Step S35 includes a step of comparing each candidate token embedding with each content embedding by calculating the cosine similarity between each candidate token embedding and each content embedding. Step S35 can be considered to correspond to Step S13, and the explanation of Step S11 can be applied here, and vice versa. Step S36 includes a step of generating safety scores based on each cosine similarity, which includes normalizing the safety scores so that they are all between 0 and 1. Step S37 includes a step of generating output scores for each candidate output token by combining the safety scores with the selection probability from the token distribution, using a parameter α that defines the balance between the selection probability and the safety score. For example, the output scores can be generated using the formula explained for Step S14 in Figure 4. Steps S36 and S37 can be considered to correspond to Step S14, and the explanation of Step S11 can be applied here, and vice versa.

[0103] Step S38 includes the step of selecting a candidate output token having the highest output score for output by LLM. Step S38 may also include the step of outputting the selected candidate output token. Step S38 can be considered to correspond to step S15, and the explanation of step S11 can be applied here, and vice versa.

[0104] As shown in the above description of steps S21 to S38, the sampling of candidate output tokens, the comparison of candidate token embeddings with content embeddings, and the calculation of output scores can employ the corresponding operations described with respect to Figure 2.

[0105] As already explained, prohibited content and / or strings describing prohibited content can be defined by the user. Prohibited content may be related to security purposes, e.g., cyberattacks, hacking, fraud, etc., and the method proposed herein can improve security (by preventing LLM from being used to undermine security measures), which can be done more efficiently than, for example, the comparative method RAIN (as explained with respect to Figure 7). In other words, LLM may be used in an adversarial manner to suggest ways to overcome security measures, e.g., to write malware, or to instruct a user on how to carry out fraud, cyberattacks, hacking, etc., and the method proposed herein can prevent LLM from outputting such “prohibited content”.

[0106] In implementation, the LLM may be configured to retrieve information from a set of documents (for example, in accordance with Retrieval Augment Generation (RAG)) technology. Forbidden content may be defined, for example, as content that may be present in a document whose release could compromise security, as described above. The methods proposed herein can be used to guide the LLM's reasoning to avoid outputting such forbidden content, thereby effectively enhancing the security of the system including the LLM.

[0107] Figure 7 is a table showing the results of the first evaluation when DIESEL was used with the unedited versions of LLMs Llama2 and Llama3 (see below). The comparison method RAIN was also used with the same LLMs. Clean versions of the LLMs without DIESEL or RAIN were also used to generate responses. Responses were generated using OpenAI's Moderation API and scored for safety. This API generates risk scores for responses according to categories C1-C5. Scores closer to 1 indicate an "unsafe" response, and scores closer to 0 indicate a "safe" response.

[0108] Uncensored Llama-2:Llama-2-7B-Chat:Llama-2 fine-tuned with the uncensored / unfiltered Wizard-Vicuna chat dataset (Cognitive Computations. Wizard-vicuna uncensored chat dataset. huggingface.co / datasets / cognitivecomputations / wizard_vicuna_70k_unfiltered, 2024. Accessed:2024-07-28).

[0109] Uncensored Llama-3:Llama-3-8B-Chat:UltraChat 200K (HuggingFaceH4, Ultrachat 200k, huggingface.co / datasets / HuggingFaceH4 / ultrachat_200k, 2024, Accessed: 2024-07-28), OpenHermes-2.5 (Teknium. Openhermes-2.5, huggingface.co / datasets / teknium / OpenHermes-2.5, 2024, Accessed: 2024-07-28), and Orca Microsoft (Microsoft. Llama-3 (Meta, Llama-3, scontent-fra3-2.xx.fbcdn.net / v / t39.2365-6 / 452387774_1036916434819166_4173978747091533306_n.pdf?_nc_cat=104&ccb=1-7&_nc_sid=3c67a6&_nc_ohc=7qSoXLG5aAYQ7kNvgHvEWW_&_nc_ht=scontent-fra3-2.xx&oh=00_AYDWIxAo6IoFdSeBeCNMVy-omipbVdneJ85rHc5PMXMFzg&oe=66ABC10D, 2024) fine-tuned with the chat dataset (2024-07-28). Accessed: (2024-07-23). ​​Alignment has been removed.

[0110] The prompts used in the first assessment were selected from the BeaverTails dataset (Ji et al., BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset, 2023, arxiv.org / abs / 2307.04657). BeaverTails consists of approximately 17,000 unsafe prompts across 14 potential harm categories. Responses were generated based on these prompts, and prompts whose risk score in at least one category did not exceed a predefined threshold (set to 0.3) were discarded. Thus, the final subset used in the first assessment included unsafe prompts that could "fake" the model.

[0111] According to the OpenAI moderation API, the unsafe categories shown in Table 1 are as follows: C1: Violence, C2: Sexual, C3: Self-harm, C4: Harassment, C5: Hatred.

[0112] As is clear from Figure 1, DIESEL exhibits filtering close to, or better than, RAIN for unsafe prompts in a short amount of time.

[0113] In the second evaluation, DIESEL (and a modified parameter α) was used along with the unmodified Llama-2 from the TruthfulQA dataset (Stephanie Lin, et al, Truthfulqa: Measuring how models mimic human falsehoods, 2021, arxiv.org / abs / 2109.07958). This dataset contains 817 innocuous questions across 38 categories, including health, law, finance, and politics. Each question has a set of true and false reference answers, allowing for an accurate assessment of the truthfulness of the generated responses. In other words, the TruthfulQA benchmark measures how much inference guidance techniques like DIESEL modify innocuous responses—that is, responses that do not need to be modified to avoid outputting prohibited content. Here, a higher truth / fidelity score (closer to 1) indicates less interference with innocuous responses. The results are shown in Figure 8, where the risk score and truth / fidelity score are plotted against alpha. The black circles represent the risk score calculated using the same method as the first evaluation, and the × symbols represent the fidelity score calculated according to the TruthfulQA benchmark. This graph shows, for example, that the optimal α is in the range of 0.98 to 0.99.

[0114] Implementation details for the first and second evaluations: RTX6000ada generation GPUs were used to perform inference on all models in the evaluations. For DIESEL, the following hyperparameters were used: alpha=0.981 (first evaluation), b=50 (meaning 50 candidate tokens are generated at each time step), Top-P sampling p=0.9999, temperature=0.7. The maximum sequence length of the model was set to the default value (2048 for llama-2, 4096 for llama-3), generating a maximum of 256 tokens. The negative prompts used (strings describing negative content) were: "Hate speech based on race, sex, ethnicity, religion, nationality, sexual orientation, disability status, or caste," "Hate, violence, or grave harm against a target group based on race, sex, ethnicity, religion, nationality, sexual orientation, disability status, or caste," "Harassment language against any target," "Harassment violence, grave harm against any target," "Self-injury, suicide, cutting, or eating disorders," "Self-injury, suicide, cutting, or eating disorders," "Instructions for self-injury, advice on how to self-injury," "Sexual descriptions of sexual activity, promotion of sexual services," "Sexual content including individuals under 18 years of age," "Death, violence, or bodily injury due to violence," "Violent / graphic death, violence, or explicit bodily injury." The default mixedbread-ai / mxbai-embed-large-v1 from huggingface (huggingface.co / mixedbread-ai / mxbai-embed-large-v1) was used for the embedding model. RAIN used the default generation parameters.

[0115] Some of the advantages of the methodology proposed here will be explained below, for example, in relation to some of the comparison methods described above.

[0116] While RLHF-based algorithms are widely adopted in LLMs, they face several limitations, including scalability issues and the difficulty of comprehensively capturing diverse human values. Furthermore, these algorithms tend to lack robustness and may be vulnerable to poisoning attacks that could compromise the safety and reliability of the model. In addition, RLHF-based methods are highly resource-inefficient, require considerable computational power for fine-tuning, and necessitate significant human effort for manually annotating the data. The method proposed here does not require data annotation and is computationally efficient, as shown in Figure 7. These could serve as an additional layer of defense to enhance the safety of RLHF-trained models, or as a primary safety mechanism.

[0117] Filtering mechanisms are widely used and common among various LLM providers, but they primarily focus on detection at either the input or output. In contrast, the method proposed here is directly integrated into the generation phase, with an emphasis on generating safe responses from the outset, rather than simply suppressing responses that do not meet safety criteria.

[0118] While RAIN has significantly longer inference times (compared to, for example, standard models), SafeDecoding requires additional overhead to train expert models, which can be resource-intensive and limit practicality. The method proposed here, unlike SafeDecoding, does not require additional model training and does not include the time-consuming, complex search and reverse phases used by RAIN, making it computationally more efficient than RAIN. By seamlessly integrating within existing generative processes and minimizing additional computational overhead, the proposed method provides a practical and scalable solution for enhancing the safety of LLMs.

[0119] More generally, model alignment is an expensive and time-consuming process, and existing inference guidance and I / O filtering solutions are either slow or require a significant amount of GPU time. In contrast, the method proposed here is highly time-efficient and resource-efficient.

[0120] As described above, the method proposed here can be called DIESEL, a flexible and robust inference guidance technique with minimal overhead and no model training required. DIESEL addresses the challenge of generating a safer response by reranking potentially proposed tokens by the original model based on their similarity to predefined negative concepts. DIESEL involves three steps: candidate selection, semantic latent space similarity, and token reranking. By using a lightweight external sentence embedding model, DIESEL leverages simple text descriptions of negative concepts to effectively guide the decoding process toward a safer output. Furthermore, because these concepts are simply text descriptions, DIESEL can exclude any desired concept without requiring expert knowledge. DIESEL is a lightweight inference guidance technique that filters out undesirable outputs and can be easily integrated into any autoregressive LLM without requiring fine-tuning or additional data collection.

[0121] The method proposed here not only surpasses the comparison method in terms of efficiency but also achieves impressive performance. Furthermore, its design offers significant improvements by bypassing the limitations inherent in existing technologies, resulting in a more robust and scalable solution.

[0122] Some advantages of the proposed method • Efficiency: Minimal additional cost (~x1.2) compared to the original generation time and resources. • Zero GPU time: Leverage pre-trained models to eliminate the need for training. • Adaptive output: Modifies output to ensure security without strict censorship. • Flexibility: The ability to dynamically respond to changing safety requirements through immediate updates of negative concepts. DIESEL's innovative approach efficiently and flexibly enhances the security of LLM output, setting a new standard for real-time content moderation.

[0123] Figure 9 is a block diagram of an information processing device 10, such as a data storage server, or computing device 10, which can be used to embody the present invention, to carry out some or all of the operations of the method embodying the present invention, and to carry out some or all of the tasks of the apparatus of the embodiment. The computing device 10 can be used to implement any of the above method steps, for example, any of steps S11 to S15 and / or S21 to S38 and / or any of the modules in Figures 2, 3, and 5.

[0124] The computing device 10 includes a processor 993 and memory 994. Optionally, the computing device also includes a network interface 997 for communicating with other such computing devices, for example, other computing devices in embodiments of the present invention. Optionally, the computing device also includes one or more input mechanisms, such as a keyboard and a mouse 996, and one or more display units, such as a monitor 995. These elements may facilitate user operation. The components are connectable to each other via a bus 992.

[0125] Memory 994 may include a computer-readable medium, and this term may refer to a single or multiple medium (e.g., a centralized or distributed database and / or associated caches and servers) configured to carry computer-executable instructions. Computer-executable instructions may include instructions and data that are accessible by a computer (e.g., one or more processors) and cause one or more functions or operations to be performed. For example, computer-executable instructions may include instructions for implementing the operation of any of the methods or method steps disclosed herein, e.g., any of steps S11-S15 and / or S21-S38 and / or any of the modules in Figures 2, 3, and 5. The term “computer-readable storage medium” may include any medium capable of storing, encoding, or holding a set of instructions for machine execution and causing a machine to perform one or more of the method steps of this disclosure, and therefore the term “computer-readable storage medium” includes, but is not limited to, solid memory, optical media and magnetic media. For example, but not limited to, the term “computer-readable storage medium” may include solid memory, optical media, and magnetic media. For example, but not limited to, such computer-readable media may include non-temporary computer-readable media including random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), compact disk read-only memory (CD-ROM), or other optical disk storage devices, magnetic disk storage devices, or other magnetic storage devices, flash memory elements (e.g., solid memory devices).

[0126] The processor 993 is configured to control the computing device and perform processing operations such as executing computer program code stored in memory 994 to perform, for example, any of steps S11 to S15 and / or S21 to S38 and / or any of the modules in Figures 2, 3, and 5. Memory 994 stores data read and written by the processor 993 and can store the strings and / or tokens and / or embeddings and / or embedding models and / or LLMs and / or weights and / or prompts and / or responses and / or token distributions and / or safety scores and / or output scores and / or ranking information and / or equations and / or input data and / or other data and / or programs for performing any of the above method steps. As referred to herein, the processor may include one or more general-purpose processing units such as microprocessors and central processing units. The processor may include a complex instruction set compute (CISC) microprocessor, a reduced instruction set compute (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor that implements other instruction sets or processors that implement combinations of instruction sets. The processor may also include one or more special-purpose processing devices, such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), and network processors. In one or more embodiments, the processor is configured to execute the operations and instructions for performing the operations discussed herein. The processor 993 can be considered to include any of the modules described above. Any operations described to be performed by a module may be performed as a method by a computer, for example, the processor 993.

[0127] The display unit 995 can display representations of data stored by the computing device, such as prompts and / or responses and / or tokens and / or strings describing prohibited content and / or GUI windows and / or interactive representations that allow the user to interact with the device 10, for example, by drag-and-drop or selection interactions and / or any other outputs as described above, and can also display cursors, dialog boxes and screens that enable interaction between the user and programs and data stored in the computing device. The input mechanism 996 can allow the user to input data and commands to the computing device, such as allowing the user to input any user input as described above, for example, at least one string describing prohibited / undesirable content and / or at least one prompt.

[0128] The network interface (network I / F) 997 can connect to a network such as the internet and can connect to other such computing devices via the network. The network I / F 997 can control input and output data with other devices via the network.

[0129] Other peripheral devices such as microphones, speakers, printers, power supply units, fans, cases, scanners, and tracker balls may be included in the computing device.

[0130] A method for embodying the present invention can be performed using a computing device / device 10 as shown in Figure 9. Such a computing device does not need to have all the components shown in Figure 9, but can consist of a subset of these components. For example, device 10 includes a processor 993 and a memory 994 coupled to the processor 993. Alternatively, device 10 includes a processor 993, a memory 994 coupled to the processor 993, and a display 995. A method for embodying the present invention can be performed by a single computing device that communicates with one or more data storage servers over a network. The computing device may be a data storage device that stores at least a portion of the data.

[0131] The present invention can be implemented by a plurality of computing devices working together. One or more of these computing devices may be data storage servers that store at least a portion of the data.

[0132] The present invention can be implemented in digital electronic circuits, or in computer hardware, firmware, software, or a combination thereof. The present invention can be implemented as a computer program or computer program product, i.e., as a computer program embodied in a non-temporary information carrier, such as a machine-readable memory device or a propagated signal, to control execution or operation by one or more hardware modules.

[0133] A computer program may be in the form of a standalone program, a computer program portion, or multiple computer programs, and may be written in any form of programming language, including compiled or interpreted languages, and may be deployed in any form, such as as a standalone program or as modules, components, subroutines, or other units suitable for use in a data processing environment. A computer program may be deployed to run on one or more modules located in one place or distributed across multiple locations and interconnected by a communication network.

[0134] The steps of the method of the present invention, for example, any of steps S11 to S15 and / or S21 to S38, and / or the operation of any of the modules in Figures 2, 3, and 5, can be performed by one or more programmable processors that execute a computer program to perform the functions of the present invention by acting on input data and generating outputs. The device of the present invention can be implemented as programmed hardware or as a dedicated logic circuit including an FPGA (Field Programmable Gate Array) or an ASIC (Application-Specific Integrated Circuit).

[0135] Processors suitable for executing computer programs include, for example, both general-purpose and application-specific microprocessors, and any one or more processors in any type of digital computer. Typically, a processor receives instructions and data from read-only memory, random-access memory, or both. The fundamental element of a computer is a processor that executes instructions, coupled with one or more memories that store instructions and data.

[0136] The above-described embodiments of the present invention may be used advantageously independently of any other embodiment of the embodiments, or in any feasible combination with one or more other embodiments of the embodiments.

[0137] This disclosure is extended to the following statement.

[0138] (S1) A computer implementation method, The steps include generating multiple candidate token embeddings based on multiple candidate output tokens generated by a Large-Scale Language Model (LLM), and generating at least one content embedding based on at least one string describing prohibited / undesirable content, A step of comparing the candidate token embedding with the at least one content embedding to determine a similarity score for each candidate output token indicating the similarity between the candidate output token and the prohibited / undesirable content, For each candidate output token, the step of generating an output score based on the similarity score and the selection probability assigned to the candidate output token by the LLM (and indicating the probability of being selected for output by the LLM), The steps include selecting candidate output tokens to be output by the LLM based on the output score, A computer implementation method including

[0139] (S2) A computer implementation method according to S1, comprising the step of receiving the plurality of candidate token embeddings.

[0140] (S3) A computer implementation method according to S1 or S2, comprising the step of receiving the at least one string that describes prohibited / undesirable content.

[0141] (S4) A computer implementation method according to any one of S1 to S3, comprising the step of selecting the plurality of candidate output tokens from a (larger) set of possible output tokens.

[0142] (S5) The computer implementation method according to S4, wherein the step of selecting the plurality of candidate output tokens includes selecting the plurality of candidate output tokens based on selection probabilities assigned to the possible output tokens by the LLM (and indicating the probability of being selected for output by the LLM).

[0143] (S6) The step of selecting the plurality of candidate output tokens from a (larger) set of possible output tokens, each of the possible output tokens having a corresponding selection probability (the) assigned by the LLM (and indicating the probability of being selected for output by the LLM), and the step of selecting the plurality of candidate output tokens is, The steps include selecting a subset (of the set of possible output tokens) that includes the fewest possible output tokens whose sum of selection probabilities satisfies a cumulative probability threshold, A step of sampling b possible output tokens from the subset as the plurality of candidate output tokens based on selection probability, wherein b is an integer greater than 1; A computer implementation method described in any of S1 to S3, including the above.

[0144] (S7) The computer implementation method according to S6, wherein the step of sampling b possible output tokens as multiple candidate output tokens from a subset based on selection probabilities includes the steps of normalizing the selection probability of each possible output token in the subset to generate normalized selection probabilities, and sampling b possible output tokens from the subset based on the normalized selection probabilities.

[0145] (S8) A computer implementation method according to any one of S1 to S7, wherein the steps of generating multiple candidate token embeddings and generating at least one content embedding include a step of using a transformer-based embedding model or a step of using a transformer network.

[0146] (S9) The computer implementation method according to any one of S1 to S8, wherein the step of comparing the candidate token embedding with the at least one content embedding to determine the similarity score for each candidate output token includes the step of calculating the cosine similarity between each candidate token embedding and the at least one content embedding.

[0147] (S10) The computer implementation method according to any one of S1 to S9, wherein the step of selecting candidate output tokens for output by the LLM includes the step of ranking the candidate output tokens according to their output scores and selecting the candidate output token with the highest rank for output by the LLM.

[0148] (S11) The computer implementation method according to S10, (since generating an output score involves generating an output score,) a similarity score indicating high similarity has a downward / decreasing effect on the position of the corresponding candidate output token in the ranking based on the output score.

[0149] (S12) The computer implementation method according to S10 (since generating an output score includes generating an output score,) a similarity score indicating a high similarity has a downward influence on the position of the corresponding candidate output token in the ranking according to the output score, and a similarity score indicating a low similarity has an upward influence on the position of the corresponding candidate output token in the ranking according to the output score.

[0150] (S13) The computer implementation method according to any one of S10 to S12, wherein the selection probability, which indicates a high probability of being selected for an output, has an upward influence on the position of the corresponding candidate output token in the ranking according to the output score.

[0151] (S14) The computer implementation method according to any one of S10 to S12 (since the generation of the output score includes generating the output score,) a selection probability indicating a high probability of being selected for an output has an upward influence on the position of the corresponding candidate output token based on the output score, and a selection probability indicating a low probability of being selected for an output has a downward influence on the position of the corresponding candidate output token based on the output score.

[0152] (S15) The computer implementation method according to any one of S1 to S14, wherein the step of calculating each output score includes the step of weighting the influence of the similarity score and the selection probability according to first and second weights.

[0153] (S16) The computer implementation method described in S15, wherein the first and second weights may be α and 1-α, respectively, and α is between 0 and 1 (and is predefined).

[0154] (S17) The step of determining each similarity score includes the step of calculating the cosine similarity between the candidate interest token embedding and the at least one content embedding, The step of generating the output score is performed for each candidate output token, The step of calculating the safety score according to the following formula,

number

number

[0155] (S18) The computer implementation method described in S17 includes the step of selecting candidate output tokens for output by LLM, which includes the step of selecting the candidate output token having the highest output score.

[0156] (S19) The step of generating at least one content embedding includes the step of generating multiple content embeddings based on multiple strings describing prohibited content for each of the multiple categories, The step of comparing the candidate token embedding with the at least one content embedding to determine the similarity score for each candidate output token is as follows: The steps include: comparing the candidate token embedding with each of the multiple content embeddings to generate a plurality of auxiliary scores indicating the similarity between the candidate output token and the category of (interest-free) prohibited content; The steps include selecting the auxiliary score that shows the highest similarity as the similarity score (of the candidate output tokens), A computer implementation method as described in any of S1 to S18, including the above.

[0157] (S20) A computer implementation of S19, comprising comparing a candidate token embedding with each of several content embeddings to generate several auxiliary scores indicating the similarity between a candidate output token and a category of (interest) prohibited content, and calculating the cosine similarity between a candidate token embedding and each of several content embeddings.

[0158] (S21) The step of determining each of the auxiliary scores includes the step of calculating the cosine similarity between the candidate interest token embedding and the content embedding, wherein each similarity score includes the cosine similarity, The step of generating the output score is performed for each candidate output token, The step of calculating the safety score according to the following formula:

number

number

[0159] (S22) The computer implementation method described in S21 includes the step of selecting candidate output tokens for output by LLM, which includes the step of selecting the candidate output token having the highest output score.

[0160] (S23) If the candidate output token is a candidate for output by LLM as part of a partially completed response by LLM, generating each of several candidate token embeddings includes generating a set of token embeddings that include the tokens of the partially generated response and the candidate output token (thus the similarity score indicates the similarity between the partially completed response and the prohibited content together with the candidate output token), as described in any of S1 to S22.

[0161] (S24) If a candidate output token is a candidate for output by LLM as part of a partially completed response by LLM, generating each of several candidate token embeddings includes generating a sentence embedding that includes the partially generated response and the candidate output token (thus the similarity score indicates the similarity between the partially completed response together with the candidate output token and the prohibited content), a computer implementation method according to any one of S1 to S23.

[0162] (S25) A computer implementation method according to any one of S1 to S24, comprising the steps of generating multiple candidate token embeddings, comparing the candidate token embeddings with at least one content embedding, generating an output score, and repeating / performing the steps of selecting a candidate output token for output for at least one further multiple candidate output tokens generated by the LLM (as candidates for output by the LLM, as a subsequent token in the response containing a candidate output token selected for output by the LLM).

[0163] (S26) For each of the at least one further candidate output tokens generated by the LLM (as candidates for output by the LLM, as a subsequent token in the response containing the candidate output token selected for output by the LLM), the steps include generating a further candidate token embedding based on the further candidate output tokens generated by the large-scale language model LLM, The steps include comparing each candidate token embedding (from among multiple candidate token embeddings) with at least one content embedding to determine a similarity score for each candidate output token, indicating the similarity between the candidate output token and the prohibited / undesirable content, The steps include generating an output score for each candidate output token (among multiple candidate output tokens) based on the similarity score and the selection probability assigned to the candidate output token by the LLM (indicating the probability of being selected for output by the LLM), A computer implementation method according to S25, comprising the step of selecting a candidate output token to be output by LLM (from among several candidate output tokens) based on the output score.

[0164] (S27) A computer implementation method according to any one of S1 to S26, comprising the step of obtaining the selection probability from the token distribution generated by the LLM.

[0165] (S28) A computer implementation method according to any one of S1 to S27, comprising the step of obtaining candidate output tokens and their selection probabilities from a token distribution generated by LLM.

[0166] (S29) A computer implementation method according to any one of S1 to S28, comprising the step of obtaining the possible output tokens and their selection probabilities from the token distribution generated by the LLM.

[0167] (S30) A computer implementation method described in any of S1 to S29, which includes a step of using / causing the LLM to output a selected candidate output token as part of the response from the LLM.

[0168] (S31) A computer implementation method according to any one of S1 to S29, comprising the step of outputting a selected candidate output token as part of the response from the LLM.

[0169] (S32) A computer implementation method according to any one of S1 to S31, comprising the step of using / causing the LLM to generate candidate output tokens and their selection probabilities (based on a prompt).

[0170] (S33) A computer implementation method according to any one of S1 to S32, including a step of using / causing the LLM to generate possible output tokens and their selection probabilities (based on prompts).

[0171] (S34) The aforementioned prohibited content includes security-related content, as described in any of the computer implementation methods described in S1 to S33.

[0172] (S35) The prohibited content includes any of the computer implementation methods described in S1 to S34, including content related to overcoming security measures.

[0173] (S36) The prohibited content includes content related to cyberattacks, as described in any of S1 to S35 of the computer implementation method.

[0174] (S37) The prohibited content includes any of the computer implementation methods described in S1 to S36, including content related to the design / generation of cyberattacks.

[0175] (S38) The aforementioned prohibited content is any of the following categories (or the categories of prohibited content include any of the following): security, overcoming security measures, cyberattacks, designing / generating cyberattacks, violence, physical injury, self-inflicted harm, suicide, eating disorders, hate speech, harassment, abuse, criminal acts and / or behavior, discrimination on the basis of race and / or sex and / or sexual orientation and / or religion, computer methods described in any of S1 to S33.

[0176] (S39) The prohibited content is any of the following categories (or the categories of prohibited content include any of the following): security, overcoming security measures, cyberattacks, and designing / generating cyberattacks, and computer implementation methods described in any of S1 to S33.

[0177] (S40) The prohibited content is any of the following categories (or the categories of prohibited content include any of the following): security or overcoming security measures, and cyberattacks or designing / generating cyberattacks, and computer implementation methods described in any of S1 to S33.

[0178] (S41) The computer implementation method according to any one of S1 to S40, wherein the plurality of candidate output tokens are generated by the LLM as candidates for output by the LLM as part of a response to a prompt for retrieving information from at least one document.

[0179] (S42) A computer implementation method according to any one of S1 to S41, comprising the step of using / causing the LLM to generate candidate output tokens and their selection probabilities based on a prompt in order to retrieve information from at least one document.

[0180] (S43) The computer implementation method described in S41 or S42, wherein the prohibited content includes specific information within at least one of the documents.

[0181] (S44) The computer implementation method described in S41 or S42, wherein the prohibited content includes confidential information in at least one of the documents.

[0182] (S45) The computer implementation method described in S41 or S42, wherein the prohibited content includes security-related information in at least one document.

[0183] (S46) The plurality of candidate output tokens are generated by the LLM as candidates for output by the LLM as part of a response to a prompt for retrieving information from at least one document, wherein the prohibited content includes security-related information within the at least one document, according to the computer implementation method described in any of S1 to S40.

[0184] (S47) A computer program, when executed on a computer, causes the computer to perform a method, and the method is The steps include generating multiple candidate token embeddings based on multiple candidate output tokens generated by a Large-Scale Language Model (LLM), and generating at least one content embedding based on at least one string describing prohibited / undesirable content, A step of comparing the candidate token embedding with the at least one content embedding to determine a similarity score for each candidate output token indicating the similarity between the candidate output token and the prohibited / undesirable content, For each candidate output token, the step of generating an output score based on the similarity score and the selection probability assigned to the candidate output token by the LLM (and indicating the probability of being selected for output by the LLM), The steps include selecting candidate output tokens to be output by the LLM based on the output score, A computer program, including any of the programs described in S1 to S40.

[0185] (S48) An information processing device comprising a memory and a processor coupled to the memory, wherein the processor Based on multiple candidate output tokens generated by a Large-Scale Language Model (LLM), multiple candidate token embeddings are generated, and based on at least one string describing prohibited / undesirable content, at least one content embedding is generated. The candidate token embedding is compared with at least one content embedding to determine a similarity score for each candidate output token, indicating the similarity between the candidate output token and the prohibited / undesirable content. For each candidate output token, an output score is generated based on the similarity score and the selection probability assigned to the candidate output token by the LLM (and indicating the probability of being selected for output by the LLM). Based on the output score, LLM selects candidate output tokens. An information processing device configured in such a way. In addition to the embodiments described above, the following further notes are disclosed.

[0186] (Note 1) A computer implementation method, The steps include generating multiple candidate token embeddings based on multiple candidate output tokens generated by a Large-Scale Language Model (LLM), and generating at least one content embedding based on at least one string describing prohibited content, A step of comparing the candidate token embedding with the at least one content embedding to determine a similarity score for each candidate output token indicating the similarity between the candidate output token and the prohibited content, For each candidate output token, the step of generating an output score based on the similarity score and the selection probability assigned to the candidate output token by the LLM, The steps include selecting candidate output tokens to be output by the LLM based on the output score, A computer implementation method including (Note 2) The process includes the step of selecting a plurality of candidate output tokens from a set of possible output tokens, each of which has a corresponding selection probability assigned by the LLM, and the step of selecting a plurality of candidate output tokens is A step of selecting a subset containing the fewest possible output tokens whose sum of selection probabilities satisfies the cumulative probability threshold, A step of sampling b possible output tokens from the subset as the plurality of candidate output tokens based on selection probability, wherein b is an integer greater than 1; The computer implementation method described in Appendix 1, including the method described in Appendix 1. (Note 3) The computer implementation method according to Appendix 1 or 2, wherein the step of comparing the candidate token embedding with the at least one content embedding to determine the similarity score for each candidate output token includes the step of calculating the cosine similarity between each candidate token embedding and the at least one content embedding. (Note 4) The computer implementation method according to Appendix 1 or 2, wherein the step of selecting candidate output tokens for output by the LLM includes the step of ranking the candidate output tokens according to their output scores and selecting the candidate output token with the highest rank for output by the LLM. (Note 5) The computer implementation method according to Appendix 4, wherein a similarity score indicating a high degree of similarity has a downward influence on the position of the corresponding candidate output token in the ranking based on the output score, and a similarity score indicating a low degree of similarity has an upward influence on the position of the corresponding candidate output token in the ranking based on the output score. (Note 6) The computer implementation method according to Appendix 4, wherein a selection probability indicating a high probability of being selected for output has an upward influence on the position of the corresponding candidate output token in the ranking based on the output score, and a selection probability indicating a low probability of being selected for output has a downward influence on the position of the corresponding candidate output token in the ranking based on the output score. (Note 7) The computer implementation method according to Appendix 1 or 2, wherein the step of calculating each output score includes the step of weighting the influence of the similarity score and the selection probability according to first and second weights. (Note 8) The step of determining each similarity score includes the step of calculating the cosine similarity between the candidate interest token embedding and the at least one content embedding, The step of generating the output score is performed for each candidate output token, The step of calculating the safety score according to the following formula,

number

number

Claims

1. A computer implementation method, The steps include generating multiple candidate token embeddings based on multiple candidate output tokens generated by a Large-Scale Language Model (LLM), and generating at least one content embedding based on at least one string describing prohibited content, The steps include comparing the candidate token embedding with the at least one content embedding to determine a similarity score for each candidate output token indicating the similarity between the candidate output token and the prohibited content, For each candidate output token, the step of generating an output score based on the similarity score and the selection probability assigned to the candidate output token by the LLM, The steps include selecting candidate output tokens to be output by the LLM based on the output score, A computer implementation method including

2. The process includes the step of selecting a plurality of candidate output tokens from a set of possible output tokens, each of which has a corresponding selection probability assigned by the LLM, and the step of selecting a plurality of candidate output tokens is A step of selecting a subset containing the fewest possible output tokens whose sum of selection probabilities satisfies the cumulative probability threshold, A step of sampling b possible output tokens from the subset as the plurality of candidate output tokens based on selection probability, wherein b is an integer greater than 1. The computer implementation method according to claim 1, including the method described in claim 1.

3. The computer implementation method according to claim 1 or 2, wherein the step of comparing the candidate token embedding with the at least one content embedding to determine the similarity score for each candidate output token includes the step of calculating the cosine similarity between each candidate token embedding and the at least one content embedding.

4. The computer implementation method according to claim 1 or 2, wherein the step of selecting candidate output tokens for output by the LLM includes the step of ranking the candidate output tokens according to their output scores and selecting the highest-ranked candidate output token for output by the LLM.

5. The computer implementation method according to claim 4, wherein a similarity score indicating a high degree of similarity has a downward influence on the position of the corresponding candidate output token in the ranking based on the output score, and a similarity score indicating a low degree of similarity has an upward influence on the position of the corresponding candidate output token in the ranking based on the output score.

6. The computer implementation method according to claim 4, wherein a selection probability indicating a high probability of being selected for output has an upward influence on the position of the corresponding candidate output token in the ranking based on the output score, and a selection probability indicating a low probability of being selected for output has a downward influence on the position of the corresponding candidate output token in the ranking based on the output score.

7. The computer implementation method according to claim 1 or 2, wherein the step of calculating each output score includes the step of weighting the influence of the similarity score and the selection probability according to first and second weights.

8. The step of determining each similarity score includes the step of calculating the cosine similarity between the candidate interest token embedding and the at least one content embedding, The step of generating the output score is performed for each candidate output token, The step of calculating the safety score according to the following formula, [Math 1] Here, S s is the safety score, CS is the cosine similarity, step, The step of calculating the output score according to the following formula, [Math 2] Here, S o is the output score, P is the selection probability, and α is between 0 and 1, with a step and A computer implementation method according to claim 1 or 2, including the method described in claim 1 or 2.

9. The step of generating at least one content embedding includes the step of generating multiple content embeddings, each based on multiple strings describing prohibited content of multiple categories, The step of comparing the candidate token embedding with the at least one content embedding to determine the similarity score for each candidate output token is as follows: The steps include: comparing the candidate token embedding with each of the multiple content embeddings to generate a plurality of auxiliary scores indicating the similarity between the candidate output token and the category of prohibited content; The steps include selecting the auxiliary score that shows the highest degree of similarity as the aforementioned similarity score, A computer implementation method according to claim 1 or 2, including the method described in claim 1 or 2.

10. If the candidate output token is a candidate for output by the LLM as part of a partially completed response by the LLM, the step of generating each of the plurality of candidate token embeddings includes the step of generating a sentence embedding that includes the partially generated response and the candidate output token of interest, according to claim 1 or 2.

11. A computer implementation method according to claim 1 or 2, comprising the steps of generating a plurality of candidate token embeddings, comparing the candidate token embeddings with the at least one content embedding, generating the output score, and selecting a candidate output token for the output, repeated for at least one further plurality of candidate output tokens generated by the LLM.

12. The computer implementation method according to claim 1 or 2, comprising the step of obtaining the selection probability from the token distribution generated by the LLM.

13. The prohibited content includes any of the following categories: Security, Overcoming security measures, Cyberattacks Design / Generate cyberattacks, The computer implementation method according to claim 1 or 2.

14. A computer program, which, when executed on a computer, causes the computer to execute a method, the method is The steps include generating multiple candidate token embeddings based on multiple candidate output tokens generated by a Large-Scale Language Model (LLM), and generating at least one content embedding based on at least one string describing prohibited content, The steps include comparing the candidate token embedding with the at least one content embedding to determine a similarity score for each candidate output token indicating the similarity between the candidate output token and the prohibited content, For each candidate output token, the step of generating an output score based on the similarity score and the selection probability assigned to the candidate output token by the LLM, The steps include selecting candidate output tokens to be output by the LLM based on the output score, A computer program that includes [this].

15. An information processing device comprising a memory and a processor coupled to the memory, wherein the processor Based on multiple candidate output tokens generated by a Large-Scale Language Model (LLM), multiple candidate token embeddings are generated, and based on at least one string describing prohibited content, at least one content embedding is generated. The candidate token embedding and the at least one content embedding are compared to determine a similarity score for each candidate output token, indicating the similarity between the candidate output token and the prohibited content. For each candidate output token, an output score is generated based on the similarity score and the selection probability assigned to the candidate output token by the LLM. Based on the output score, the LLM selects candidate output tokens to output. An information processing device configured in such a way.

Images