Apparatus and computer-implemented method for determining rules for determining answers to questions
The method leverages large-scale language models to generate syntax-compliant rules for diverse content types, improving the accuracy and reliability of automated reasoning in question answering systems.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-08-25
- Publication Date
- 2026-03-10
AI Technical Summary
Existing methods for translating natural language questions into rules for automated reasoning are limited in their ability to handle diverse content types, particularly digital images and sensor data, and often produce incorrect or non-compliant rules.
A computer-implemented method using large-scale language models (LLM) to generate rules by providing a rule set, prompting the LLM with pre-prompts and prompts, and ensuring compliance with syntax to derive accurate answers through automated reasoning, utilizing Answer Set Programming (ASP) for structured content representation.
Enables accurate and syntax-compliant rule generation for diverse content types, enhancing the reliability and effectiveness of automated reasoning in question answering systems.
Smart Images
Figure 2026041696000001_ABST
Abstract
Description
[Technical Field]
[0001] background The present invention relates to an apparatus and computer-implemented method for determining rules for determining answers to questions. [Background technology]
[0002] Adam Ishay, Zhun Yang, and Joohyung Lee, in their paper "Leveraging large language models to generate answer set programs," in Proceedings of the 20th International Conference on Principles of Knowledge Representation and Reasoning, KR2023, Rhodes, Greece, September 2-8, 2023, pages 374-383, 2023, discloses a method for using large language models (LLMs) to translate natural language problem specifications into rules, which can then be passed to an appropriate symbolic inference engine to solve the problem. [Prior art documents] [Non-patent literature]
[0003] [Non-Patent Document 1] Adam Ishay, Zhun Yang, and Joohyung Lee, ““Leveraging large language models to generate answer set programs”, In Proceedings of the 20th International Conference on Principles of Knowledge Representation and Reasoning, KR2023, Rhodes, Greece, September 2-8, 2023, pages 374-383, 2023.” Summary of the Invention [Means for solving the problem]
[0004] Disclosure of the Invention A computer-implemented method for determining rules for determining an answer to a question about content, particularly content representing digital images, e.g., time series of sensor data, audio, synthetic data, or data expressible as objects and relationships between objects, by automated reasoning on the question and the content, the method including: providing a rule set for determining an answer to the question; providing the question, the answer, and the content; determining a pre-prompt for a large-scale language model that describes the rule set; determining a prompt for the large-scale language model that instructs the large-scale language model to add at least one rule to the rule set so that an answer to the question about the content can be derived by automated reasoning on the rule set including the question, the content, and the at least one rule; prompting the large-scale language model with the pre-prompt; prompting the large-scale language model with the prompt; and receiving a response from the large-scale language model to the prompt, the response including the at least one rule. The rule set represents a theory for question answering. The question, answer, and content are an example of a triplet used to extract at least one new rule for the theory.
[0005] The method may include providing a syntax of a language used for at least one rule, prompting a large-scale language model to determine at least one rule according to the syntax, checking whether the at least one rule complies with the syntax of the at least one rule, and adding the at least one rule to a rule set if the at least one rule complies with the syntax of the at least one rule, or not adding the at least one rule to the rule set if the at least one rule does not comply with the syntax of the at least one rule. By prompting the large-scale language model to determine the at least one rule according to the syntax, the large-scale language model is guided to determine at least one new rule in the syntax, and accepting or rejecting the at least one new rule based on the syntax. By adding the at least one rule to the rule set if the at least one rule complies with the syntax, the rule set is prevented from being modified by rules that do not comply with the syntax.
[0006] The method may include performing automated reasoning on a rule set including a question, content, and at least one rule to determine an answer, and adding the at least one rule to the rule set if the answer matches, or not adding the at least one rule to the rule set if the answer does not match. Adding the at least one rule to the rule set if the answer matches prevents the rule set from being modified by a rule that results in an incorrect automated reasoning answer.
[0007] The method may include providing a syntax of a language used in the question, in particular an answer set programming syntax, and prompting a large scale language model to process a question having the syntax of the language. Prompting the large scale language model to process a question having the syntax of the language model guides the large scale language model based on the syntax of the language used in the question.
[0008] The method may include providing a question expression description used to express the question, and prompting the large-scale language model to process the question according to the question expression description. Prompting the large-scale language model to process the question according to the question expression description guides the large-scale language model based on the question expression description.
[0009] The method may include providing a description of a representation of the content used to represent the content, and prompting the large-scale language model to process the content according to the description of the representation of the content. Prompting the large-scale language model to process the content according to the description of the representation of the content guides the large-scale language model based on the description of the representation of the content.
[0010] The representation of the content may include a scene graph, which represents, for example, the content of a scene, particularly the content of a scene within a digital image.
[0011] The method may include providing a syntax of a language used for the content and prompting a large scale language model to process the content having the syntax of the language used for the content. Prompting the large scale language model to process the content having the syntax of the language used for the content guides the large scale language model based on the syntax of the language used for the content.
[0012] The method may include determining a pre-prompt for a rule set that includes at least one rule, prompting a large-scale language model with the pre-prompt for the rule set that includes the at least one rule, determining another prompt that includes a different answer, a different question, and / or different content, and determining at least one other rule for the other prompt using the large-scale language model. Prompting a large-scale language model with the pre-prompt for the rule set that includes the at least one rule provides an update of the rule set to the large-scale language model. Prompting the large-scale language model with the other prompt and adding at least one other rule to the rule set progressively expands the rule set.
[0013] An apparatus for determining rules for determining answers to questions about content by automated reasoning on the questions and content, the apparatus comprising at least one processor and at least one memory, the at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus to perform a method.
[0014] A computer program comprising computer-readable instructions which, when executed by a computer, cause the computer to perform a method.
[0015] A data structure for determining rules for determining an answer to a question about content by automated reasoning on a question and the content, the data structure having at least one data field for a rule set for determining an answer to the question, the data structure having at least one data field for the question, the answer, and the content, the data structure having at least one data field for a pre-prompt for a large-scale language model that describes the rule set, the data structure having at least one data field for a prompt for the large-scale language model that instructs the large-scale language model to add at least one rule to the rule set so that an answer to the question about the content can be derived by automated reasoning on the question, the content, and the rule set including the at least one rule, and the data structure having at least one data field for a response of the large-scale language model to the prompt that is received when the pre-prompt is prompted to the large-scale language model and the prompt is prompted to the large-scale language model, the response including at least one rule.
[0016] Further embodiments can be derived from the following description and drawings. [Brief explanation of the drawings]
[0017] [Figure 1] 1 is a schematic diagram illustrating an apparatus for determining rules for determining answers to questions about content by automated reasoning on the questions and content; [Figure 2] 1 is a flowchart including steps of a method for determining rules. [Figure 3] 1 is a flowchart including steps of an exemplary implementation of the method. DETAILED DESCRIPTION OF THE INVENTION
[0018] FIG. 1 is a schematic diagram illustrating an apparatus 100 for determining rules for determining answers to questions about content by automated reasoning on the questions and the content.
[0019] The apparatus 100 includes at least one processor 102 and at least one memory 104. The at least one memory 104 stores instructions that, when executed by the at least one processor 102, cause the apparatus 100 to perform a method for determining rules.
[0020] FIG. 2 shows a flow chart including the steps of the method.
[0021] The method is based on large-scale language models (LLM).
[0022] The LLM demonstrates a wide range of abilities and is highly versatile across a variety of applications. At its core, the LLM excels in human-like text comprehension and generation, enabling it to perform tasks such as natural language understanding, translation, summarization, and creative content generation.
[0023] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin, "Attention is all you need," In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017, discloses several aspects of LLM.
[0024] The method may use examples, which may be provided as a set of triplets including a question, an answer and a content triplet.
[0025] The content may include a scene graph representation of a visual scene. Scene graph representations are particularly useful for solving visual question answering (VQA) tasks. The method is not limited to VQA or to scene graph representations. A scene graph is an example of content. A visual scene is an example of input data. The method can use other input data that can be translated into content. Content includes, for example, a fact-based symbolic representation of the input data.
[0026] The input data may include digital images of graphs, e.g., scalar time series of sensor data, audio, synthetic data, data that can be represented as objects and relationships between objects. The digital images may include traffic scenes, e.g., scenes including traffic signs, road surfaces, pedestrians, and vehicles.
[0027] The method includes step 202 .
[0028] Step 202 involves providing a rule set for determining the answer to the question.
[0029] The rule set is provided in the syntax of a language used for the rules, for example, the answer set programming syntax of Answer Set Programming (ASP).
[0030] The rule set is, for example, an ASP program.
[0031] ASP is described, for example, in "Answer set programming at a glance," by Gerhard Brewka, Thomas Eiter, and Miroslaw Truszczynski, Commun. ACM, 54(12):92-103, 2011.
[0032] ASP is a declarative, logic-based approach to combinatorial search and optimization that has its roots in knowledge representation and reasoning. It provides a simple modeling language and efficient solvers. In ASP, the search space and properties of a problem's solution are described by a logic program, and the model (called the solution set) encodes the solution to the problem.
[0033] ASP programs are of the following form: a1|…|a m :-b1,…,b n ,not c1,…,not c n is a rule set for all a i ,b j ,c k is a primary literal and not is default negation. The set of atoms to the left of the :- is the head of the rule, and the atoms to the right form the body.
[0034] All b j is true and any c k Whenever there is no evidence about i should be true.
[0035] A rule with an empty body and a single head atom with no variables is a fact and is always true. A rule with an empty head is a constraint and is used to exclude models that would satisfy the body.
[0036] ASP provides further language constructs such as aggregations and weak constraints (also called soft constraints) whose violation is preferably avoided. Further aspects of the ASP language and its semantics are disclosed in "Asp-core-2 input language format", Theory Pract. Log. Program., 20(2):294-309, 2020, by Francesco Calimeri, Wolfgang Faber, Martin Gebser, Giovambattista Ianni, Roland Kaminski, Thomas Krennwallner, Nicola Leone, Marco Maratea, Francesco Ricca, and Torsten Schaub.
[0037] The method includes step 204 .
[0038] Step 204 involves determining a pre-prompt for the LLM that describes the rule set.
[0039] A rule set expresses a theory for question answering.
[0040] Step 204 may include providing the syntax of the language used to query, for example, the ASP answer set programming syntax.
[0041] Step 204 may include providing a syntax for the language used for the content, for example, the ASP answer set programming syntax.
[0042] Step 204 may include providing a description of a representation of the content used to represent the content in the prompt. For example, a scene in a digital image may include the content. For example, the representation may include a scene graph of the scene, particularly a scene graph in ASP's answer set programming syntax.
[0043] Step 204 may include providing a syntax for a language used for the at least one rule, for example, the ASP answer set programming syntax.
[0044] The method includes step 206 .
[0045] Step 206 includes prompting the LLM with a pre-prompt.
[0046] The pre-prompt contains a rule set or theory.
[0047] Step 206 may include prompting the LLM that the question has the syntax of the language used in the question. For example, the LLM is prompted that the question be provided in the answer set programming syntax of the ASP.
[0048] Step 206 may include prompting the LLM that the content has the syntax of the language used for the content. For example, the LLM may be prompted that the content is provided in the answer set programming syntax of the ASP.
[0049] Step 206 may include prompting the LLM to process the content according to the description of the content's representation.
[0050] For example, the LLM is prompted that the representation includes a scene graph of the scene. For example, the LLM is prompted to explain that a scene graph is an answer set programming syntax for ASP.
[0051] Step 206 may include prompting the LLM that at least one rule should have the syntax of the language used for the at least one rule. For example, the LLM is prompted that at least one rule should be provided in the answer set programming syntax of the ASP.
[0052] Each syntax and / or description may be provided to the LLM with the same pre-prompt as the rule set. Different pre-prompts may be used for the rule set and each syntax and / or description.
[0053] The method includes step 208 .
[0054] Step 208 includes providing questions, answers, and content.
[0055] The questions, answers and content can be provided from examples, for example, from one triplet of a set of triplets.
[0056] The method includes step 210 .
[0057] Step 210 includes determining a prompt for the LLM that instructs the LLM to add at least one rule to a rule set such that an answer to a question about the content can be derived by automated reasoning against a rule set that includes a question, content, and at least one rule.
[0058] The method includes step 212 .
[0059] Step 212 includes prompting the LLM with a prompt.
[0060] Step 212 may include prompting the LLM to receive a plurality of prompts and determining at least one rule such that an answer to a question about the content of the plurality of prompts is derivable by automated reasoning against a rule set including the question, the content, and at least one rule.
[0061] The method includes step 214 .
[0062] Step 214 includes receiving the LLM's response to the prompt. If multiple prompts are provided to the LLM, step 214 may include receiving the LLM's response to the multiple prompts. The response includes at least one rule.
[0063] The method may include step 216 .
[0064] Optional step 216 may include checking whether the at least one rule complies with the syntax of the language to be used for the at least one rule.
[0065] Optional step 216 may include performing automated reasoning on a rule set including the question, the content, and at least one rule to determine an automated reasoning answer.
[0066] The method may include, in step 218, adding at least one rule to the rule set if the at least one rule complies with the syntax to be used for the at least one rule, or not adding the at least one rule to the rule set if the at least one rule does not comply.
[0067] A rule set containing at least one rule represents a new theory for question answering.
[0068] Step 218 may include adding at least one rule to the rule set if the answers match, or not adding at least one rule to the rule set if the answers do not match.
[0069] To avoid adding rules to the theory making past examples false, the method can use regression testing in the iterations.
[0070] For example, at each iteration, a rule set is saved. Whenever an additional rule is received, a semantic check is performed on all previously saved rule sets, not just the current rule set. The additional rule is added to the latest rule set if the semantic check is successfully performed on all rule sets, or is not added to the latest rule set if it is not successful. A semantic check may, for example, be to check whether the rule set complies with the semantics of the ASP language.
[0071] Step 204 can then be performed for the rule set that includes at least one rule.
[0072] This means that the pre-prompt includes a rule set including at least one rule, i.e., a new theory. The LLM is prompted with a pre-prompt for the rule set including at least one rule. In step 208, the method includes determining another prompt that includes a different answer, a different question, and / or different content than the previous prompt. This means that the method is used to determine at least one other rule using the LLM for the other prompt based on the new theory.
[0073] The method provides declarative knowledge distillation, where the model the method distills from is an LLM and the knowledge to be distilled is expressed in ASP rules.
[0074] An ASP is an example of a fact-based symbolic representation of questions, answers, and content. Questions used to determine theories may come from neural network components that translate natural language questions into fact-based symbolic representations. Answers used to determine theories may come from neural network components that translate natural language answers into fact-based symbolic representations. Content used to determine theories may come from neural network components that translate natural language content and / or scene graphs into fact-based symbolic representations.
[0075] The content used to determine theories may come from neural network components that translate input data into fact-based symbolic representations.
[0076] Exemplary prompt text after the pre-prompt includes a question Q, a scene S, and an answer A in a tuple p=(Q;S;A) in the language of the ASP.
[0077] The method can begin when the ASP solver's automated reasoning for tuple p and the current theory T, i.e., the current rule set, recognizes that the current theory T cannot provide an answer to tuple p.
[0078] In this method, the LLM is prompted with a tuple p to add rules so that a correct answer can be derived. The LLM outputs, for example, a response R that includes at least one rule.
[0079] The method checks whether the response R is in correct ASP syntax, for example, by passing at least one rule from the response R to an ASP solver for automated inference. If the ASP solver can process the at least one rule without syntax errors, the syntax is correct and the at least one rule from the response R is concatenated with the initial theory, i.e., the current rule set, to generate a resulting theory Tres, i.e., a new theory.
[0080] In the method, checking the semantics may involve running the outcome theory Tres in parallel with an instance pair (Q;S) of question Q and scene S to see if the answer it produces is correct, i.e., consistent with answer A. If this is the case, the method replaces the current theory T with the outcome theory Tres.
[0081] The method may include repeating these steps until all examples have been presented.
[0082] When using the ASP language, the LLM receives instructions to generate an ASP rule. The LLM may not output an ASP rule because the response R may be corrupted by additional natural language comments and other artifacts. To address this, the method includes r retries to prompt the LLM to provide a response R for the same tuple p to improve the chances of answering correctly. This means that the method prompts the LLM r times with the same prompt. After r retries, the method may stop retries and not add a rule to the rule set for tuple p.
[0083] An exemplary algorithmic pseudocode for an exemplary implementation of the method steps is provided as follows for a file initialtheory.lp containing the initial theory:
number
number
[0084] The function pb:generate_preprompt determines the preprompt as above. The function pb:generate_examples samples k examples according to the strategy strat.
[0085] Given an example involving an ASP query formulation, the strategy strat may be based on predicate counting.
[0086] Examples are grouped by the number of predicates that appear in the ASP query expression. For example, a dictionary is created based on the examples, where the dictionary keys are the predicate numbers and the dictionary contents are questions of a given length. The length can range from 4 to 13, for example.
[0087] Given an example containing an ASP query expression, the strategy strat may be based on the relevance of the predicates.
[0088] The examples are grouped based on the predicates that appear in the ASP query expression, e.g., a dictionary is created and the keys of the dictionary are the predicates that appear in the ASP query expression.
[0089] Next, questions in which the key predicate appears in the ASP query formulation are added to the dictionary. For example, there are 22 different predicates that can appear in an ASP query formulation.
[0090] The function pb:generate_examples, for example, samples k examples from a group created with a strategy strat, which can be selected by the user.
[0091] The examples can be processed in a batch process.
[0092] Each prompt for each example can be determined one by one. Each prompt for each example can be sent to the LLM one by one for each example.
[0093] This is useful to find at least one rule that solves that particular example.
[0094] Examples are sampled from the dataset to obtain general rules that can handle a significant portion of the dataset.
[0095] Multiple prompts are sent to the LLM in a batch. Multiple batches can be sent to the LLM.
[0096] A batch may contain b examples resulting in b single prompts. A single prompt may contain multiple questions q = (Q1, ..., Qb) for the same content or for one content per question. In this case, the LLM creates at least one rule that is general enough to pass semantic checks for all examples in the batch.
[0097] FIG. 3 shows a flow chart including steps of an exemplary implementation of the method.
[0098] In step 302, examples, a current theory, and a regression list are provided. The examples can be sampled from a dataset. The current theory can be initialized with a predefined initial theory. The theory can be a rule set. The regression list can be empty or can contain further examples.
[0099] Then, step 304 is executed to check whether the examples have been used, e.g., whether there are no examples, or whether there are any unused examples remaining, e.g., whether there is at least one example remaining.
[0100] If the instance has been used, step 306 is performed. If not, step 308 is performed.
[0101] In step 306, the theory is output.
[0102] In step 308, the current example is sampled from the examples, the number of retries is set to 0, and the maximum number of retries is set to a predetermined value M.
[0103] Step 310 is then executed to check whether the current theory can solve the example. If the current theory can solve the example, step 312 is executed. If not, step 314 is executed.
[0104] In step 312, the sampled example is removed from the examples or marked as a used example.
[0105] In step 314, a check is performed to determine if the number of retries is less than the maximum number of retries. If the number of retries is less than the maximum number of retries, step 316 is performed.
[0106] If not, the method may end.
[0107] Step 316 involves using the current theory to construct prompts and pre-prompts for the current example.
[0108] Step 318 is then executed to obtain a response from the LLM using the prompt and pre-prompt.
[0109] Step 320 is then performed to determine an extended theory by modifying the current theory with the response.
[0110] Thereafter, a syntax check is performed to determine whether the extension theory has correct syntax in step 322. If the syntax check fails, step 324 is performed. If the syntax check determines that the extension theory has correct syntax, step 326 is performed.
[0111] In step 324, the retry count is incremented, for example, by 1. Step 314 is then executed.
[0112] In step 326, a test is performed to determine if the extension theory solves the current example. If the extension theory solves the current example, step 328 is performed. If not, step 324 is performed.
[0113] In step 328, a test is performed to determine if the extension theory solves the examples in the regression list. If the extension theory solves the examples in the regression list, step 330 is performed. If not, step 324 is performed.
[0114] Step 330 involves replacing the current theory with the extended theory.Step 330 involves adding the current example to the regression list.
[0115] Step 312 is then executed.
[0116] A data structure can be provided for determining rules for determining answers to questions about content by automated reasoning on the questions and content.
[0117] The data structure is A set of rules for determining the answer to the question, Questions, answers and content; Pre-prompts for large language models that explain rule sets, a prompt for the large-scale language model that instructs the large-scale language model to add at least one rule to a rule set such that an answer to a question about the content can be derived by automated inference on a rule set that includes the question, the content, and the at least one rule; a response of the large-scale language model to the prompt received upon prompting the large-scale language model with the pre-prompt and prompting the large-scale language model with the prompt, the response comprising at least one rule; It includes at least one data field for
[0118] The data structure is the syntax of the language used for at least one rule, The syntax of the language used for questioning, especially answer set programming syntax, · Explanation of the question wording used to phrase the question; A description of the content representation used to represent the content; The syntax of the language used in the content; Other prompts with different answers, different questions and / or different content; The data field may include at least one data field for:
Claims
1. 1. A computer-implemented method for determining rules for determining answers to questions about content, particularly content representing digital images, time series of e.g., sensor data, audio, synthetic data, or data that can be expressed as objects and relationships between objects, by automated reasoning on the questions and the content, comprising: The method comprises: providing a rule set for determining an answer to the question (202); providing the question, the answer, and the content (208); determining (204) a pre-prompt for a large-scale language model that describes the rule set; determining (210) a prompt for the large-scale language model that instructs the large-scale language model to add the at least one rule to the rule set such that an answer to the question about the content can be derived by automated reasoning on the rule set including the question, the content, and at least one rule; prompting the pre-prompt into the large-scale language model (206); Prompting the prompt into the large-scale language model (212); receiving (214) a response of the large scale language model to the prompt, the response including the at least one rule; A method comprising:
2. The method comprises: providing (204) a syntax for a language used for said at least one rule; prompting the large-scale language model to determine at least one rule according to the syntax (206); Checking whether the at least one rule complies with the syntax of the at least one rule (216); adding the at least one rule to the rule set if the at least one rule complies with the syntax of the at least one rule, or not adding the at least one rule to the rule set if the at least one rule does not comply (218); The method of claim 1 , comprising:
3. The method comprises: performing automated reasoning on the question, the content, and a rule set including the at least one rule to determine an answer (216); adding the at least one rule to the rule set if the answer is a match, or not adding the at least one rule to the rule set if the answer is a match (218); 3. The method of claim 1 or 2, comprising:
4. The method comprises: providing (204) the syntax of the language used in the query, in particular the answer set programming syntax; prompting the large-scale language model to process a question having the syntax of the language (206); 4. The method of claim 1, comprising:
5. The method comprises: providing a description of the question phrase used in the question phrase (204); prompting the large-scale language model to process the question according to a description of the question expression (206); 5. The method of claim 1, comprising:
6. The method comprises: providing a description of a representation of the content used to represent the content (204); prompting the large-scale language model to process the content according to a representational description of the content (206); 6. The method of claim 1, comprising:
7. The method of claim 6 , wherein the representation of the content includes a scene graph.
8. The method comprises: Providing (204) a syntax for a language used in the content; prompting the large-scale language model to process content having a syntax of a language used in the content (206); 8. The method of claim 1, comprising:
9. The method comprises: determining (204) a pre-prompt for a rule set including the at least one rule; prompting the large-scale language model with a pre-prompt for a rule set including the at least one rule (206); determining (208) other prompts including different answers, different questions, and / or different content; determining (210, 212, 214) at least one other rule for the other prompt using the large-scale language model; 9. The method of claim 1, comprising:
10. An apparatus (100) for determining rules for determining an answer to a question about content by automatic reasoning on the question and the content, comprising: The foregoing is at least one processor (102); At least one memory (104); Equipped with The at least one memory (104) stores instructions that, when executed by the at least one processor (102), cause the device (100) to perform the method of any one of claims 1 to 9. An apparatus characterized in that
11. A computer program comprising computer readable instructions which, when executed by a computer, cause the computer to carry out the method of any one of claims 1 to 9.
12. 1. A data structure for determining rules for determining an answer to a question about content by automated reasoning on the question and the content, comprising: the data structure having at least one data field for a rule set for determining an answer to the question; the data structure having at least one data field for the question, the answer, and the content; the data structure having at least one data field for a pre-prompt for a large-scale language model that describes the rule set; the data structure has at least one data field for a prompt for the large-scale language model, the prompt instructing the large-scale language model to add the at least one rule to the rule set, such that an answer to the question about the content can be derived by automated reasoning against a rule set including the question, the content, and at least one rule; the data structure having at least one data field for a response of the large-scale language model to the prompt received upon prompting the large-scale language model with the pre-prompt and prompting the large-scale language model with the prompt, the response including the at least one rule.
1. A data structure comprising: