Prompt engineering computer, prompt engineering method and program

Abstract

A prompt engineering computer, a prompt engineering method, and a program are provided that are capable of providing accurate answers to questions regarding predetermined rules. [Solution] Question data is acquired and one or more first keywords are extracted from the question data. The question data is considered to be a question about whether a specific object matches a rule, and one or more second keywords are extracted from the question data. Rule-related information is acquired from a predetermined database based on the first and second keywords. A prompt for generating an answer is created based on the question data and the rule-related information to be input into a large-scale language model.

Description

[Technical Field]

[0001] The present invention relates to a technology that is effective in utilizing generative AI (Artificial Intelligence). [Background technology]

[0002] Generative AI has become increasingly popular in recent years. In generative AI (herein, this refers specifically to large-scale language models), proper prompt engineering is important, and using appropriate prompts (questions, explanations, instructions, summaries) can improve the accuracy of answers. As an example of the use of generative AI, Patent Document 1 discloses a system that acquires multiple keywords that are recalled from the items searched for by the questioner, and displays information organized by themes from multiple documents based on the multiple keywords and a database that stores multiple document information. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7416508 Summary of the Invention [Problem to be solved by the invention]

[0004] However, with current prompt engineering, when it comes to questions about specific rules, it is possible to accurately answer questions that ask about the content of the specific rules, but it is not possible to accurately answer questions that ask whether a specific object complies with the specific rules. For example, while they were able to accurately answer the question, "Please tell me the rules for setting a password," they were unable to accurately answer questions such as, "Can I use dasdasjdkas as a password?", which asked users to determine the suitability of a password based on an arbitrary combination of characters.

[0005] In view of the above problems, the present invention aims to provide a prompt engineering computer, a prompt engineering method, and a program that are capable of providing accurate answers to questions regarding predetermined rules. [Means for solving the problem]

[0006] A prompt engineering computer according to one embodiment of the present invention comprises an acquisition unit that acquires question data related to a predetermined rule; a first extraction unit that extracts one or more first keywords from the question data; a second generation and extraction unit that regards the question data as asking whether a specific target conforms to the rule and extracts one or more second keywords from the question data; a rule-related information acquisition unit that acquires information related to the rule from a predetermined database based on the first and second keywords; and an answer generation prompt creation unit that creates an answer generation prompt based on the information related to the rule and the question data.

[0007] A prompt engineering computer according to one embodiment of the present invention includes a question acquisition unit that acquires question data related to a predetermined rule; a question classification unit that classifies the question data into whether it inquires about the content of the rule or whether a specific object complies with the rule; a second generation and extraction unit that generates and extracts one or more second keywords from the question data if the question data inquires about whether a specific object complies with the rule; a rule-related information acquisition unit that acquires information related to the rule from a database based on the second keywords; and an answer generation prompt creation unit that creates an answer generation prompt based on the question data, the classification result of the question data, and the rule-related information.

[0008] Although each aspect of the present invention is categorized as a computer, a program that operates the functions of each part of the computer, and a computer-readable recording medium on which the program is recorded, also fall within the scope of the present invention. [Effects of the Invention]

[0009] According to the present invention, it is possible to give an accurate answer to a question regarding a predetermined rule. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a diagram showing various components of a prompt engineering system 1 according to a first embodiment. [Figure 2] FIG. 2 is a diagram illustrating a process executed by a prompt engineering computer 10 in the first embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of a prompt for generating an answer in the first embodiment. [Figure 4] FIG. 10 is a diagram illustrating various components of a prompt engineering system 1 according to a second embodiment. [Figure 5] FIG. 10 is a diagram illustrating a process executed by a prompt engineering computer 10 in the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present invention (hereinafter referred to as "embodiments") will be described in detail with reference to the accompanying drawings. In the following drawings, the same elements are designated by the same numbers or symbols throughout the description of the embodiments.

[0012] First Embodiment [Configuration of Prompt Engineering System 1] The components of the prompt engineering system 1 will be described with reference to Fig. 1. The prompt engineering system 1 is a system comprising at least a prompt engineering computer 10 that has a server function and creates prompts to be input to a large-scale language model. In this embodiment, the prompt engineering system 1 includes a prompt engineering computer 10, a questioner terminal 3 used by a questioner, and an information processing system 20 that provides the functionality of a large-scale language model.

[0013] The questioner terminal 3 is, for example, a terminal device such as a mobile phone, a smartphone, a tablet terminal, a personal computer, a laptop computer, etc. The number of questioner terminals 3 may be any number that corresponds to the number of questioners, is not particularly limited, and can be designed as appropriate.

[0014] The prompt engineering computer 10 has a server function and may be realized, for example, by one computer, or may be realized by multiple computers like a cloud computer.

[0015] In this specification, a cloud computer may refer to either a computer that uses any computer in a scalable manner to perform a specific function, or a computer that includes multiple functional modules to realize a system and uses the functions in any combination.

[0016] The information processing system 20 provides the functionality of a large-scale language model. A large-scale language model is a type of deep learning model with hundreds of millions to hundreds of billions of parameters, and it learns from large amounts of text data to understand natural language and generate responses. Specifically, it analyzes input sentences (prompts) and generates answers that fit the context.

[0017] The information processing system 20 refers to the entire environment that makes the functions of a large-scale language model available, and may be a single device (for example, a system in which a large-scale language model is built into a computer) or a distributed configuration that includes external devices and cloud services.

[0018] The prompt engineering system 1 is a system in which a prompt engineering computer 10 is connected to a questioner terminal 3 and an information processing system 20 so as to be able to communicate data with each other via a network 8 such as the Internet, an in-house LAN (Local Area Network), Wi-Fi, or a VPN (Virtual Private Network).

[0019] In addition, the prompt engineering system 1 may include other terminals and devices in addition to the above-mentioned questioner terminal 3, prompt engineering computer 10, and information processing system 20, and the number, type, and functions thereof are not particularly limited and can be designed as appropriate.

[0020] [Device configuration] The configuration of each device in the prompt engineering system 1 in this embodiment will be described with reference to Fig. 1. The questioner terminal 3 is a terminal device used by the questioner 2, and may be a mobile phone, smartphone, tablet terminal, personal computer, laptop computer, or the like.

[0021] The questioner terminal 3 has a terminal control unit including a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), etc., a communication unit including devices that enable communication with other terminals and devices, etc., and an input / output unit including various devices that receive predetermined inputs, input and output various data, etc.

[0022] The prompt engineering computer 10 has a server function and may be realized, for example, by a single computer or by multiple computers such as a cloud computer. The prompt engineering computer 10 is an information processing device that creates prompts to be input into a large-scale language model.

[0023] The prompt engineering computer 10 includes a CPU, GPU, RAM, ROM, etc. as a control unit, a device for enabling communication with other terminals and devices, etc. as a communication unit, and a data storage unit such as a hard disk, semiconductor memory, recording medium, memory card, etc. as a memory unit. The memory unit stores programs and data used when the processing unit of the prompt engineering computer 10 executes processing, and also stores a rule-related information DB (database) that stores information related to predetermined rules.

[0024] The data and the like described in this embodiment as being stored in the storage unit may be stored in an external device as long as the control unit of the prompt engineering computer 10 can refer to it.

[0025] [Functional configuration of Prompt Engineering Computer 10] In the prompt engineering computer 10, the control unit loads a predetermined program and cooperates with the communication unit as necessary to realize a question acquisition unit, a first extraction unit, a second generation and extraction unit, a rule-related information acquisition unit, a prompt creation unit for answer generation, and an answer receiving and transmitting unit.

[0026] [Processing performed by the prompt engineering computer 10] In this embodiment, the processing executed by the prompt engineering computer 10 will be described with reference to FIG. In this specification, the first keyword and the second keyword are both "search queries for obtaining information about a predetermined rule from a database." However, the first keyword and the second keyword are obtained in different ways (details will be described later).

[0027] The question acquisition unit acquires question data relating to a predetermined rule (step S1). The question data is a prompt that the questioner uses to generate the AI, and the prompt can be a question, an explanation, an instruction, or a summary. The question data must at least contain a question.

[0028] Question data includes, for example, company rules (internal regulations regarding personnel and labor, accounting and finance, information systems, general affairs, legal affairs, sales, work rules, business manuals, etc.), rules at schools and other educational institutions (school rules, academic regulations, course regulations, grading regulations, etc.), rules at medical and nursing care institutions (medical fees, nursing care fees, facility operation standards, ethical regulations, etc.), and rules at stores and facilities (terms of use, service terms, membership terms, reservation and cancellation policies, point program terms, terms of product and service provision, etc.).

[0029] The question acquisition unit acquires question data related to a predetermined rule from the questioner terminal 3. The questioner terminal 3 accepts input of question data in a predetermined format (such as a chatbot format) via a UI (User Interface) for inputting question data. The questioner terminal 3 transmits the accepted input question data to the prompt engineering computer 10. The question acquisition unit receives this question data and acquires it.

[0030] The first extraction unit extracts one or more first keywords from the question data (step S2). The primary keywords are keywords extracted directly from the question data. The first extraction unit performs morphological analysis on the question data and divides the question data into character strings in accordance with the grammar of a predetermined language. The first extraction unit extracts, from the divided character strings, character strings that can be used as search queries for acquiring information about a predetermined rule from a database, and extracts first keywords.

[0031] The processing of the first extraction unit may be performed via a large-scale language model. For example, the first extraction unit inputs a prompt containing instructions and question data such as "Divide the following question sentence into words and extract keywords related to the specified rules. Do not answer the question," to the large-scale language model, and sets one or more keywords output from the large-scale language model as the first keywords.

[0032] The second extraction unit extracts one or more second keywords from the question data (step S3). The second keywords are generated through a large-scale language model after complementing the context of the question data in order to search for information about rules that are premised on the question data. The second generating and extracting unit in this embodiment regards the question data as "a question as to whether a specific subject conforms to a predetermined rule," and generates and extracts second keywords via a large-scale language model.

[0033] Specifically, the second generating and extracting unit: (1) Prerequisite information that assumes that the question data asks whether a specific object conforms to a given rule. (2) the question data, and (3) Instructions to "create keywords that are highly relevant to the rules assumed by the question data." A prompt containing the above is input to the large-scale language model, and the output of the large-scale language model is used as the second keyword. For example, the second generation and extraction unit inputs premise information and instructions such as "The following question is a question to confirm whether the question conforms to a certain rule. Now, in order to answer the following question, we will obtain data related to the premise rule. To do this, please create multiple highly relevant keywords" into the large-scale language model, along with a prompt containing question data such as "May I use cloud service A?" (cloud service A is, for example, OneDrive (registered trademark)).The second generation and extraction unit then sets the keywords "cloud service A," "how to use," and "terms of use" output from the large-scale language model as second keywords. Here, the keywords "usage" and "terms of use" are not included in the question data. In other words, the second generation and extraction unit differs from the first extraction unit in that it does not simply extract character strings from the question data, but is also able to generate keywords that are not included in the question data based on premise information (the context asking whether a specific object complies with a predetermined rule).

[0034] The rule-related information acquisition unit acquires rule-related information from the rule-related information DB based on each of the first keyword and the second keyword (step S4). The rule-related information acquisition unit uses one or more first keywords as a search query, and executes a search in the rule-related information DB using the first keywords.

[0035] In this embodiment, the rule-related information acquisition unit is described as executing a vector search, but other search methods such as a keyword search or a hybrid search may also be used. The rule-related information acquisition unit vectorizes the primary keywords. The vectorization method is not limited to a specific one, and may be, for example, a method using a pre-trained language model (such as Sentence-BERT), a method using statistical information such as TF-IDF (Term Frequency - Inverse Document Frequency), or a combination of these. The rule-related information acquisition unit calculates the similarity (for example, cosine similarity) between the vectorized first keyword and the vector of a document or chunk that has been vectorized in advance in the rule-related information DB and registered in the index, and acquires rule-related information in descending order of similarity. The acquired rule-related information may be a chunk or a document.

[0036] The rule-related information acquisition unit performs the same process for the secondary keywords. The rule-related information acquisition unit sets one or more secondary keywords as a search query. The rule-related information acquisition unit executes a search in the rule-related information DB using the second keyword, and vectorizes the second keyword. The rule-related information acquisition unit calculates the similarity between the vectorized first keyword and the vector of a document or chunk that has been vectorized in advance in the rule-related information DB and registered in the index, and acquires rule-related information in descending order of similarity. The acquired rule-related information may be a chunk or a document.

[0037] The answer generation prompt creating unit creates an answer generation prompt based on the question data and the acquired plurality of rule-related information (step S5). 3 is a diagram showing an example of an answer generation prompt in this embodiment. The answer generation prompt includes at least question data input by the questioner, multiple pieces of acquired rule-related information, and an instruction to generate an answer based on the question and reference materials for a large-scale language model. The answer generation prompt creation unit combines these elements in a predetermined format (string concatenation, embedding in a template, etc.) to create an answer generation prompt.

[0038] The answer receiving / transmitting unit transmits the created answer generation prompt to the large-scale language model and receives the answer generated by the large-scale language model (step S6).

[0039] The answer receiving / transmitting unit transmits the received answer to the questioner terminal (step S7).

[0040] According to the first embodiment, the prompt engineering computer 10 can generate an appropriate answer to a question about a predetermined rule input by a questioner, whether the question is about the content of the rule or whether a specific object conforms to the rule. Furthermore, the questioner does not need to distinguish between a query about the "rule content" and a query about "whether the rule is conformed to," and the prompt engineering computer 10 can accurately answer questions about a predetermined rule input by a questioner in a natural way.

[0041] This effect is achieved by the following means. The first extraction unit extracts keywords (first keywords) directly from the question data. The second generation and extraction unit generates keywords (second keywords) using a large-scale language model, taking into account the context in which the question data is assumed to be a rule-compliant question. These keyword extractions result in keywords that can be used to obtain appropriate rule-related information, even when the question is a rule-compliant question. The rule-related information acquisition unit uses the first and second keywords to acquire related information from the rule-related information DB. At this time, by using a vector search, it is possible to acquire information based on semantic relevance. This makes it possible to acquire more appropriate rule-related information that is less affected by spelling variations and synonym problems. The answer generation prompt generator generates an answer generation prompt based on both the question data and the acquired rule-related information, which enables the large-scale language model to generate an answer taking into account the question context and the associated rule information, thereby improving the accuracy of the answer.

[0042] <Second embodiment> 4 is a diagram showing the configuration of each part of the second embodiment. The components of the system and the configuration of each device of the second embodiment are the same as those of the first embodiment, so a description thereof will be omitted.

[0043] [Functional configuration of Prompt Engineering Computer 10] In the prompt engineering computer 10, the control unit loads a predetermined program and cooperates with the communication unit as necessary to realize a question acquisition unit, a first extraction unit, a second generation and extraction unit, a rule-related information acquisition unit, a prompt creation unit for answer generation, and an answer receiving and transmitting unit, which have the same functions as those of the first embodiment, as well as a question classification unit unique to this embodiment.

[0044] [Processing performed by the prompt engineering computer 10] The processing executed by the prompt engineering computer 10 in this embodiment will be described with reference to FIG. The question acquisition unit acquires question data relating to a predetermined rule (step S10). Question data is a prompt that the questioner uses to use the generation AI, and prompts can be questions, explanations, instructions, or summaries. Question data must at least include a question. Examples of question data include corporate rules (internal regulations regarding human resources and labor, accounting and finance, information systems, general affairs, legal affairs, and sales, work regulations, business manuals, etc.), rules at educational institutions such as schools (school rules, academic regulations, course regulations, grading regulations, etc.), rules at medical and nursing care institutions (medical fees, nursing care fees, facility operation standards, ethical regulations, etc.), and rules at stores and facilities (terms of use, service terms, membership terms, reservation and cancellation policies, point program terms, product and service provision terms, etc.).

[0045] The question acquisition unit acquires question data relating to a predetermined rule from the questioner terminal 3. The questioner terminal 3 accepts input of question data in a predetermined format (such as a chatbot format) via a UI for inputting question data. The questioner terminal 3 transmits the accepted input question data to the prompt engineering computer 10. The question acquisition unit receives the question data and acquires the question data.

[0046] The question classification unit classifies the acquired question data into whether it is a question asking about the content of a specified rule (hereinafter sometimes referred to as a "rule content question") or a question asking whether a specific object conforms to a specified rule (hereinafter sometimes referred to as a "rule conformance question"). The question classification unit in this embodiment classifies questions via a large-scale language model. The question classification unit creates a prompt that includes at least an instruction to classify the question data and the question data to be classified. The prompt may also include a designation of an output format (e.g., output with a label of "rule content question" or "rule compliance question") as needed.

[0047] The question classification unit inputs the generated prompt into a large-scale language model and obtains a question classification result output from the large-scale language model. The question classification result is output in the form of a label such as a "rule content question" or a "rule compliance question," a numerical value, a true / false value, or any other format. The question classification unit analyzes the acquired question classification results and transmits the classification results to subsequent processing. The prompt engineering computer 10 proceeds to the process of step S12 if the question data is a rule content question, and proceeds to the process of step S14 if the question data is a rule conformance question.

[0048] If the question data is a rule content question, the first extraction unit extracts one or more first keywords from the question data (step S12). The primary keywords are keywords extracted directly from the question data. The first extraction unit performs morphological analysis on the question data and divides the question data into character strings according to the grammar of a predetermined language. The first extraction unit extracts, from the divided character strings, character strings that can be used as search queries for obtaining information related to the predetermined rules from a database, and extracts first keywords.

[0049] The processing of the first extraction unit may be performed via a large-scale language model. For example, the first extraction unit inputs a prompt including instructions and question data, such as "Divide the following question sentence into words and extract keywords related to a predetermined rule. Do not answer the question," to the large-scale language model, and sets one or more keywords output from the large-scale language model as the first keywords.

[0050] If the question data is a rule content question, the rule-related information acquisition unit acquires rule-related information from the rule-related information DB based on the first keyword (step S13). The rule-related information acquisition unit sets one or more primary keywords as a search query. The rule-related information acquisition unit executes a search in the rule-related information DB using the first keyword. Note that, although this embodiment is described as the rule-related information acquisition unit executing a vector search, it may also use a search method such as a keyword search or a hybrid search.

[0051] The rule-related information acquisition unit vectorizes the primary keywords. The vectorization method is not limited to a specific one, and may be, for example, a method using a pre-trained language model (such as Sentence-BERT), a method using statistical information such as TF-IDF (Term Frequency - Inverse Document Frequency), or a combination of these.

[0052] The rule-related information acquisition unit calculates the similarity (for example, cosine similarity) between the vectorized first keyword and the vector of a document or chunk that has been vectorized in advance in the rule-related information DB and registered in the index, and acquires rule-related information in descending order of similarity. The acquired rule-related information may be a chunk or a document.

[0053] If the question data is a rule-compliant question, the second generating and extracting unit extracts one or more second keywords from the question data (step S14). The second keywords are generated through a large-scale language model after complementing the context of the question data in order to search for information related to rules assumed by the question data.

[0054] The second generating and extracting unit generates and extracts second keywords via a large-scale language model. Specifically, the second generating and extracting unit: (1) The premise information that "the question data asks whether a specific object conforms to a predetermined rule" (2) the question data, and (3) Instructions to "create keywords that are highly relevant to the rules assumed by the question data." A prompt containing the above is input to the large-scale language model, and the output of the large-scale language model is used as the second keyword.

[0055] For example, the second generation and extraction unit inputs premise information and instructions such as "The following question is a question to confirm whether the question conforms to a certain rule. Now, in order to answer the following question, we will obtain data related to the premise rule. To do this, please create multiple highly relevant keywords" into the large-scale language model, along with a prompt containing the question data "May I use cloud service A?", and sets the keywords "cloud service A," "how to use," and "terms of use" output from the large-scale language model as the second keywords.

[0056] If the question data is a rule-compliant question, the rule-related information acquisition unit acquires rule-related information from the rule-related information DB based on the second keyword, similar to the process based on the first keyword in step S13 (step S15). The rule-related information acquisition unit sets one or more secondary keywords as a search query. The rule-related information acquisition unit executes a search in the rule-related information DB using the second keyword. The rule-related information acquisition unit vectorizes the second keywords. The rule-related information acquisition unit calculates the similarity between the vectorized second keyword and the vector of a document or chunk that has been vectorized in advance in the rule-related information DB and registered in the index, and acquires rule-related information in descending order of similarity. The acquired rule-related information may be a chunk or a document.

[0057] The answer generation prompt creating unit creates an answer generation prompt based on the question data, the one or more pieces of rule-related information that have been acquired, and the question classification results (step S16). The answer generation prompt includes at least question data entered by the questioner, rule-related information, and instructions for generating an answer based on the question and reference materials. Furthermore, in the present embodiment, if the question data is a rule-compliant question, the answer generation prompt includes context information based on the question classification result. The context information is a sentence that emphasizes that the question data is a rule-compliant question, such as "The question data is a question that asks whether the question data conforms to a certain rule," or "Given that a rule called {rule-related information} exists, does the {question data} conform to the rule?" The answer generation prompt creation unit combines the above elements in a predetermined format (for example, embedding in a template, concatenating strings, etc.) according to the question classification result to create an answer generation prompt.

[0058] The answer receiving / transmitting unit transmits the created answer generation prompt to the large-scale language model and receives the answer generated by the large-scale language model (step S17).

[0059] The answer receiving / transmitting unit transmits the received answer to the questioner terminal (step S18).

[0060] According to the second embodiment of the present invention, the prompt engineering computer 10 can generate an appropriate answer to a question about a predetermined rule input by a questioner, whether the question is about the content of the rule or whether a specific object conforms to the rule. Furthermore, the questioner does not need to distinguish between a query about the "rule content" and a query about "whether the rule is conformed to," and the prompt engineering computer 10 can accurately answer questions about a predetermined rule input by a questioner in a natural way.

[0061] In the second embodiment, the prompt engineering computer 10 includes a question classification unit in addition to the configuration of the first embodiment, thereby further enhancing the effects. The question classification unit uses a large-scale language model to classify question data into whether it is a question asking about the content of a rule (rule content question) or a question asking whether a specific object complies with a rule (rule compliance question). For rule content questions, the first extraction unit extracts keywords (first keywords) directly from the question data. For rule conformance questions, the second generation and extraction unit takes into account the context of the question data and generates keywords (second keywords) using a large-scale language model. By optimizing the keyword extraction method according to the type of question in this way, more relevant keywords can be extracted.

[0062] The rule-related information acquisition unit searches the rule-related information DB using the extracted keywords to acquire related information. Because the extracted keywords are more appropriate, search accuracy is improved. The answer generation prompt creation unit adds contextual information (such as whether the question conforms to a rule) to the answer generation prompt based on the question classification results, allowing the large-scale language model to more accurately understand the intent of the question and generate an appropriate answer. [Explanation of symbols]

[0063] 1. Prompt Engineering System 3 Questioner terminal 8 Network 10 Prompt Engineering Computer 20 Information Processing Systems

Claims

1. a prompt engineering computer that generates prompts to input to a large-scale language model, an acquisition unit that acquires question data relating to a predetermined rule; a first extraction unit that directly extracts one or more first keywords from the question data; a second generating and extracting unit that regards the question data as a question as to whether a specific target conforms to the rule, and generates and extracts one or more second keywords from the question data; a rule-related information acquisition unit that acquires information about the rule from a predetermined database based on at least one of the first keyword and the second keyword; an answer generation prompt creation unit that creates an answer generation prompt based on information about the rule and the question data; A prompt engineering computer comprising:

2. a prompt engineering computer that generates prompts to input to a large-scale language model, a question acquisition unit that acquires question data relating to a predetermined rule; a question classification unit that classifies the question data into whether it asks about the content of the rule or whether a specific object conforms to the rule; a second generating and extracting unit that generates and extracts one or more second keywords from the question data based on a classification result of the question data that the question data asks whether a specific target conforms to the rule; a rule-related information acquisition unit that acquires information about the rule from a database based on the second keyword; an answer generation prompt creation unit that creates an answer generation prompt based on the question data, the classification result of the question data, and information about the rule; A prompt engineering computer comprising:

3. a first extraction unit that directly extracts one or more first keywords from the question data based on a classification result of the question data classified by the question classification unit that the question data inquires about the content of the rule; 3. The prompt engineering computer according to claim 2, wherein the rule-related information acquisition unit acquires the rule-related information from a database based on the first keyword.

4. 1. A computer-implemented prompt engineering method for generating prompts to input into a large-scale language model, comprising: obtaining question data relating to a predetermined rule; directly extracting one or more primary keywords from the question data; a step of regarding the question data as a question as to whether a specific object conforms to the rule, and generating and extracting one or more second keywords from the question data; acquiring information about the rule from a predetermined database based on at least one of the first keyword and the second keyword; creating an answer-generating prompt based on information about the rule and the question data.

5. 1. A computer-implemented prompt engineering method for generating prompts to input into a large-scale language model, comprising: obtaining question data relating to a predetermined rule; A step of classifying the question data as to whether it inquires about the content of the rule or whether a specific object complies with the rule; generating and extracting one or more second keywords from the question data based on a classification result of the question data that the question data asks whether a specific object conforms to the rule; acquiring information about the rule from a predetermined database based on the second keyword; creating a prompt for generating an answer based on the question data, the classification results of the question data, and information about the rules.

6. A prompt engineering computer creates prompts to input into a large-scale language model. obtaining question data relating to a predetermined rule; directly extracting one or more first keywords from the question data; a step of regarding the question data as a question as to whether a specific object conforms to the rule, and generating and extracting one or more second keywords from the question data; acquiring information about the rule from a predetermined database based on at least one of the first keyword and the second keyword; and generating a prompt for generating an answer based on information about the rule and the question data.

7. A prompt engineering computer creates prompts to input into a large-scale language model. obtaining question data relating to a predetermined rule; A step of classifying whether the question data inquires about the content of the rule or whether a specific object complies with the rule; generating and extracting one or more second keywords from the question data based on a classification result of the question data that the question data asks whether a specific object conforms to the rule; acquiring information about the rule from a predetermined database based on the second keyword; creating a prompt for generating an answer based on the question data, the classification result of the question data, and information about the rules.

Images