Program, apparatus and method for determining reflection of individual characteristic in inference answer using large language model

Abstract

To provide a program, and the like for creating a prompt for improving inference accuracy of a large language model in inferring an answer to a cognitive psychological question.SOLUTION: A program includes: forward reasoning means which transmits, to a large language model, a prompt describing a question to a user and ground-truth individual characteristics of the user and indicating an instruction to make an inference answer, and receives the inference answer; backward reasoning means which transmits, to the large language model, a prompt describing the question and the inference answer and indicating an instruction to return a backward-reasoning individual characteristics, and receives the backward-reasoning individual characteristics; and answer determination inference means which transmits, to the large language model, a prompt describing the question, the ground-truth individual characteristics, the inference answer, and the backward reasoning individual characteristics, and describing a first task indicating an instruction to determine whether the inference answer to the question strongly reflects the backward-reasoning individual characteristic than the ground-truth individual characteristic, and receives a result of the determination.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a technique for creating prompts to be input into large language models (LLMs) based on natural language processing. [Background technology]

[0002] In recent years, large-scale language models, such as the Transformer, have been improving the inference accuracy for tasks like text generation and language understanding. These are built by training deep neural networks with billions of parameters using large datasets, and are capable of generating natural-sounding sentences. A representative example is ChatGPT (registered trademark), and due to the high expressiveness and versatility of its output results, it is beginning to be used in a wide range of applications, including chatbots, search engines, and code generation.

[0003] Large-scale language models can sometimes have low inference accuracy when it comes to inferring causal relationships between factors and outcomes. This is because large-scale language models learn statistical correlations in text data.

[0004] In response to this, there is a technology that uses a large-scale language model to extract user intentions from log data (see, for example, Non-Patent Document 1). According to this technology, a large-scale language model is used to determine classification classes of intentions from log data, and then humans cooperate to verify and correct the definitions of the classification classes. When the large-scale language model classifies intentions based on the classification classes of the intentions finally obtained, the results show that the classification matches with a high percentage of classifications made by humans.

[0005] There is also a technology that uses a large-scale language model to perform backward inference on the output results of multiple forward inferences of an inference task (see, for example, Non-Patent Document 2). With this technology, the accuracy of the inference process of a large-scale language model is verified by checking whether the forward input and the backward output match. This makes it possible to indirectly verify the inference process of a large-scale language model, whose inference process is a black box, by using backward inference.

[0006] Furthermore, in order to improve the inference accuracy of large-scale language models, there is a technology for describing a chain of thought (CoT) in the input prompt (see, for example, Non-Patent Document 3). This technology describes QA (Question-Answer) examples in the prompt to provide intermediate inference steps leading to the final answer for the large-scale language model. In particular, this can improve performance in various tasks such as arithmetic, common sense, and symbolic reasoning.

[0007] Furthermore, there is a technique called Zero-shot-CoT prompting, which is derived from CoT prompting (see, for example, Non-Patent Document 4). This instructs a large-scale language model to perform inference step by step. The simplest way is to simply add "Let's think step by step" to the prompt. Although no output examples of the inference target are required, inference accuracy can be improved with a single prompt. For example, in the MultiArith benchmark, zero-shot accuracy was improved from 17.7% to 78.7%, achieving SoTA. Unlike conventional CoT (Few-shot-CoT), this technique does not require QA examples, making it versatile and not limited to specific tasks.

[0008] In this way, by prompting the user with hierarchical, logical steps to answer the question, the inference accuracy of large-scale language models can be improved.

[0009] From another perspective, large-scale language models have also been reported to have the ability to solve "Theory of Mind (ToM)" tasks, which involve inferring the minds of others. ToM refers to the ability of humans and apes to understand the mental states, intentions, knowledge, and beliefs of others and to infer their behavior based on that understanding. This concept has traditionally been measured by the "false belief task" in psychological experiments, which measures the ability to understand whether others will make mistakes in judgment in a particular situation. [Prior art documents] [Non-patent literature]

[0010] [Non-Patent Document 1] C. Shah et al., “Using Large Language Models to Generate, Validate, and Apply User Intent Taxonomies.” arXiv, Sep. 14, 2023. [online], [Retrieved January 20, 2024], Internet<https: / / arxiv.org / pdf / 2309.13063.pdf> [Non-patent document 2] Y. Weng et al., “Large Language Models are Better Reasoners with Self-Verification.” arXiv, Oct. 19, 2023. [online], [Retrieved January 20, 2024], Internet<https: / / arxiv.org / abs / 2212.09561> [Non-patent document 3] Wei J. et al., “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models,” arXiv.org, Jan. 28, 2022., [online], [Retrieved January 20, 2020], Internet<https: / / arxiv.org / abs / 2201.11903> [Non-patent document 4] T. Kojima, S.S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa, “Large Language Models are Zero-Shot Reasoners.” arXiv, Jan. 29, 2023. [online] [Retrieved January 20, 2024], Internet<https: / / arxiv.org / abs / 2205.11916> Summary of the Invention [Problem to be solved by the invention]

[0011] However, large-scale language models are said to have limitations in their inference accuracy when it comes to tasks like "reproducing human psychological processes" such as thoughts and emotions. It has been reported that when ChatGPT3.5 (registered trademark), a representative large-scale language model, is used to solve the false belief task described above, it only achieves the inference accuracy of a person around the age of 9 in the ToM task.

[0012] According to Non-Patent Document 1, it is possible to classify intentions from log data using a large-scale language model, but this requires prior collection of log data and human cooperation, limiting the application of automation. Furthermore, Non-Patent Documents 3 and 4 describe prompting technology that improves the inference accuracy of large-scale language models overall, but the task of inferring cognitive psychological answers to any questionnaire requires more advanced inference. It is difficult to demonstrate the process of accurately inferring a human's inference answer using few-shot or zero-shot input.

[0013] In contrast, cognitive psychological tasks are influenced by the relationship between "causal relationships between factors and results" and "individual characteristics (including psychological processes)." The inventors of the present application wondered whether it would be possible to determine what personal characteristics an inferred answer would conform to when inferring answers to cognitive psychology questions using a large-scale language model. That is, they wondered whether it would be possible to externally modify the process of inferring an answer so that the inferred answer conforms to the user's personal characteristics as much as possible.

[0014] Therefore, an object of the present invention is to provide a program, device, and method that can create prompts that improve the inference accuracy of large-scale language models when inferring answers to cognitive psychology questions. [Means for solving the problem]

[0015] According to the present invention, there is provided a program for causing a computer to use a large-scale language model to generate prompts that infer answers for each user to any question, the program comprising: a forward inference means for transmitting a prompt to the large-scale language model, the prompt describing a question to the user and the correct personal characteristics of the user, and instructing the large-scale language model to return an inference answer, and for receiving the inference answer; an inverse inference means for transmitting a prompt to a large-scale language model, the prompt describing the question and the inference answer and instructing the model to return an inverse inference personal characteristic, and for receiving the inverse inference personal characteristic; an answer determination inference means for describing the question, the correct individual characteristic, the inference answer, and the reverse inference individual characteristic, transmitting a prompt describing a first task instructing the large-scale language model to determine whether the inference answer to the question more strongly reflects the reverse inference individual characteristic than the correct individual characteristic, and receiving a determination result; The present invention is characterized by the fact that it makes a computer function by using the above-mentioned method.

[0016] According to another embodiment of the program of the present invention, The answer determination inference means further describes in the prompt a second task that instructs the user to adjust the inference answer so that it better reflects the correct personal characteristics when the first task is determined to be true. It is also preferable to make the computer function in this way.

[0017] According to another embodiment of the program of the present invention, The answer determination inference means further describes in the prompt a third task that instructs the user to return an updated inference answer, which is an adjusted inference answer, and a reason for the update, following the second task. It is also preferable to make the computer function in this way.

[0018] According to another embodiment of the program of the present invention, The answer determination inference means further inputs the updated inference answer received by itself to the inference means as an inference answer received by the forward inference means, and repeats the first task to the third task for the inverse inference personal characteristics further received by the inverse inference means. It is also preferable to make the computer function in this way.

[0019] According to another embodiment of the program of the present invention, The question asks for a numerical value or score, Inferential answers are answers that give a numerical value or a score. It is also preferable to make the computer function in this way.

[0020] According to another embodiment of the program of the present invention, The correct answer personal characteristics and the reverse inference personal characteristics include personal attributes, economic attributes, personality attributes, and / or physical attributes, and are attribute values limited by attribute options. It is also preferable to make the computer function in this way.

[0021] According to another embodiment of the program of the present invention, The personal attributes are any one or a combination of the user's gender, age, place of residence, occupation, and family status, The economic attribute is either personal annual income or household annual income, or a combination thereof; The personality attributes are the scores of each psychological item of the user, The physical attributes are the scores of each biological item detected by the sensor attached to the user. It is also preferable to make the computer function in this way.

[0022] According to another embodiment of the program of the present invention, a questionnaire result collecting means for inputting questionnaire sentences for users as questions to the forward inference means and collecting the questionnaire results of the users as updated inference answers obtained from the answer determining inference means; It is also preferable to make the computer function as follows.

[0023] According to another embodiment of the program of the present invention, The forward inference means, the answer determination inference means, and the reverse inference means may use the same large-scale language model with the dialogue history reset, or may use different large-scale language models. It is also preferable to make the computer function in this way.

[0024] According to the present invention, there is provided a prompt generation device that uses a large-scale language model to generate a prompt that infers an answer from each user to an arbitrary question, the device comprising: a forward inference means for transmitting a prompt to the large-scale language model, the prompt describing a question to the user and the correct personal characteristics of the user, and instructing the large-scale language model to return an inference answer, and for receiving the inference answer; an inverse inference means for transmitting a prompt to a large-scale language model, the prompt describing the question and the inference answer and instructing the model to return an inverse inference personal characteristic, and for receiving the inverse inference personal characteristic; an answer determination inference means for describing the question, the correct individual characteristic, the inference answer, and the reverse inference individual characteristic, transmitting a prompt describing a first task instructing the large-scale language model to determine whether the inference answer to the question more strongly reflects the reverse inference individual characteristic than the correct individual characteristic, and receiving a determination result; The present invention is characterized by having the following.

[0025] According to the present invention, there is provided a method for generating prompts for an apparatus that uses a large-scale language model to generate prompts that infer answers for each user to any question, the method comprising the steps of: The device is A first step of sending a prompt to a large-scale language model, which describes a question to a user and a correct personal characteristic of the user, and instructs the large-scale language model to return an inference answer, and receiving the inference answer; a second step of sending a prompt describing the question and the inference answer and instructing the large-scale language model to return the reverse inference personal characteristics, and receiving the reverse inference personal characteristics; a third step of describing the question, the correct individual characteristic, the inference answer, and the reverse inference individual characteristic, and sending a prompt describing a first task to the large-scale language model to instruct the model to determine whether the inference answer to the question more strongly reflects the reverse inference individual characteristic than the correct individual characteristic, and receiving a determination result; The present invention is characterized by carrying out the following. [Effects of the Invention]

[0026] The program, device, and method of the present invention can create prompts that improve the inference accuracy of large-scale language models when inferring answers to cognitive psychology questions. [Brief explanation of the drawings]

[0027] [Figure 1] FIG. 2 is a functional configuration diagram of the device according to the present invention. [Figure 2] 10 is an example of a first prompt and inference response in the forward inference section of the present invention. [Figure 3] 10 is an example of a second prompt and inference answer in the reverse inference unit of the present invention. [Figure 4] 10 is an example of a third prompt in the answer determination inference unit of the present invention. [Figure 5] FIG. 10 is an explanatory diagram for determining the degree to which a correct answer individual characteristic and an inverse inference individual characteristic are reflected in an inference answer. [Figure 6] 10 is an example of an updated inference answer in the answer determination inference unit of the present invention. [Figure 7] 1 shows an example of the configuration of an inference device and a large-scale language model according to the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0028] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0029] Businesses often conduct surveys of an unspecified number of users to understand the psychological state of their customers regarding the services and products they offer. To obtain reliable responses, businesses must carefully consider the number and content of questions and distribute the survey to an unspecified number of users, resulting in significant time and financial costs. Meanwhile, users also face time and psychological burdens from completing the survey. It can be difficult to strike a balance between the psychological benefits and costs of businesses wanting users to respond and users finding it troublesome to do so.

[0030] Here, as an embodiment of the present invention, the collection of survey results is assumed as a psychological task for an unspecified number of users. That is, a target user is set, and if a survey were conducted on that user, how would that user respond? This allows realistic survey results to be collected from virtual users without conducting a realistic survey on real users. An inference device based on such an embodiment can be used in a variety of service areas, such as survey simulation, survey data expansion, and inferring the intentions of others in dialogue AI.

[0031] FIG. 1 is a functional block diagram of the device according to the present invention.

[0032] 1, inference device 1 uses a large-scale language model 2 to create prompts that infer each user's answer to a given question. Inference device 1 may be a user-operable terminal that can communicate with large-scale language model 2 over a network. The inference device 1 has a questionnaire result collection unit 10, a forward inference unit 11, a reverse inference unit 12, and an answer determination inference unit 13. These functional components are realized by executing a program that causes a computer installed in the device to function. The processing flow of these functional components can also be understood as a prompt creation method for the device.

[0033] [Survey Result Collection Section 10] The survey result collection unit 10 inputs the <question>, which is the text of the survey for the user, and the user's <correct personal characteristic> to the forward inference unit 11. Then, the survey result collection unit 10 acquires the inferred answer (updated inferred answer) of the assumed target user from the answer determination inference unit 13 as the survey result.

[0034] The questionnaire result collection unit 10 outputs different correct individual characteristics for a fixed question to the forward inference unit 11, thereby making it possible to obtain inferred answers for different users from the answer determination inference unit 13. That is, it is possible to collect inferred answers that become questionnaire results from a variety of users for one question.

[0035] <Question> The question is a questionnaire for the user that is freely written by the operator. The question may be a question asking for a numerical value or a score. In that case, the inference answer will also be a numerical value or a score. The questions are, for example, as follows: -------------------------------------------------------------------------------- question: On a 7-point scale, how willing are users to participate in social activities such as volunteering in their local community? (1: Not at all applicable, 2: Somewhat not applicable, 3: Somewhat not applicable, 4: Neutral, 5: Somewhat applicable, 6: Somewhat applicable, 7: Very applicable) --------------------------------------------------------------------------------

[0036] The questions are not limited to those asking about the user's intentions, but may also ask about the user's preferences and behavioral factors (e.g., attitude, subjective norm, perceived behavioral controllability). Preferences, for example, ask about the degree of the user's preference. Attitudes, for example, ask about the degree of the user's positive or negative evaluation or opinion of a particular behavior. Subjective norms, for example, ask about the degree of behavioral factors based on expectations from others or social pressure. Perceived behavioral controllability, for example, ask about a person's perception of the degree of personal ability and constraints to perform a particular behavior.

[0037] <Correct answer: Personal characteristics> The correct individual characteristics are based on each user stored in the user database, and represent the individual characteristics of the user to be estimated.

[0038] The personal characteristics may include personal attributes, economic attributes, personality attributes, and / or physical attributes. The "personal attributes" may be any one or a combination of the user's gender, age, place of residence, occupation, and family status. The "economic attribute" may be either or a combination of personal annual income and household annual income. The "personality attributes" may be the scores of each psychological item of the user. For example, they may be psychological characteristic data such as the BigFive, which are classified by the following five factors: Openness (Levels 1-7) Extraversion (Levels 1-7) Neuroticism (Levels 1-7) Conscientiousness (Level: 1~7) Agreeableness (Level: 1-7) The "physical attributes" may be scores for various biological items detected by a sensor (e.g., a wearable device) attached to the user. For example, they may be numerical measurements of average blood pressure, heart rate, sleep duration, and fatigue level.

[0039] Of course, the personal characteristics may also be freely described by the operator. However, in order to improve inference accuracy, it is preferable to describe the correct personal characteristics based on “attribute options” that explain the parameters of the correct personal characteristics in the large-scale language model 2. It is preferable to define the personal characteristics as attribute values limited by predetermined attribute options.

[0040] [Forward inference part 11] The forward inference unit 11 sends a <first prompt> to the large-scale language model 2 and receives an <inference answer>. The first prompt describes a <question> for the user and the <correct personal characteristic> of the user, and instructs the model to return an <inference answer>.

[0041] FIG. 2 shows an example of a first prompt and inference response in the forward inference section of the present invention.

[0042] According to Fig. 2, for example, the following wording is added to the <Question>: This description indicates that the intended target user is a person based on the correct personal characteristics. -------------------------------------------------------------------------------- Users with the following correct personal characteristics (see attribute options below)... --------------------------------------------------------------------------------

[0043] 2, the "correct individual characteristic" is described as follows, for example: That is, the correct individual characteristic is an attribute value limited by an attribute option, which will be described later. -------------------------------------------------------------------------------- Correct answer: Personal characteristics: gender = 'female' Age = '50-54' Marital status = 'Married' Parental status = 'Has children' Annual household income = '10 to 12 million yen' Job = 'Office worker (clerical)' Big_Five = {"Openness": 3, "Extraversion": 3, "Neuroticism": 3, "Conscientiousness": 4, "Cooperativeness":5} --------------------------------------------------------------------------------

[0044] The predetermined attribute options are as follows: -------------------------------------------------------------------------------- Attribute options: Gender = ['male','female'] Age = ['Under 12', '12-19','20-24','25-29','30-34','35-39','40-44','45-49','50-54','55-59','60+'] Marital status = ['Single','Married'] Parental Status = ['No Children', 'Has Children'] Annual household income = ['Less than 2 million yen', '2 million to 4 million yen', '4 million to 6 million yen', '6 million to 8 million yen', '8 million to 10 million yen', '10 million to 12 million yen', '12 million to 15 million yen', '15 million to 20 million yen', '20 million yen or more', 'Unknown'] Occupation = ['Civil servant', 'Manager / Executive', 'Company employee (Administrative)', 'Company employee (Technical)', 'Company employee (Other)', 'Self-employed', 'Freelance', 'Housewife (Househusband)', 'Part-time', 'Student', 'Other', 'Unemployed'] Big_Five = ['Openness': '(1: Not at all true, 2: Somewhat not true, 3: Somewhat not true, 4: Neutral, 5: Somewhat true, 6: Somewhat true, 7: Very true)', 'Extroversion': '(1: Not at all applicable, 2: Somewhat inapplicable, 3: Somewhat inapplicable, 4: Neutral, 5: Somewhat applicable, 6: Somewhat applicable, 7: Very applicable)', 'Neuroticism': '(1: Not at all true, 2: Somewhat not true, 3: Somewhat not true, 4: Neutral, 5: Somewhat true, 6: Somewhat true, 7: Very true)', 'Integrity': '(1: Not at all applicable, 2: Somewhat inapplicable, 3: Somewhat inapplicable, 4: Neutral, 5: Somewhat applicable, 6: Somewhat applicable, 7: Very applicable)', 'Agreeableness': '(1: Not at all applicable, 2: Somewhat not applicable, 3: Somewhat not applicable, 4: Neutral, 5: Somewhat applicable, 6: Somewhat applicable, 7: Very applicable)' --------------------------------------------------------------------------------

[0045] The attribute parameters in the personal characteristics are not limited to these, but may also be mood or degree of fatigue that changes over time, or sleeping hours, etc.

[0046] According to FIG. 2, for example, the following inference answer can be obtained from the large-scale language model 2. -------------------------------------------------------------------------------- This user rates their desire to participate in social activities such as volunteering in their community as "5: Somewhat applicable." While they value giving back and helping others, their other responsibilities and obligations limit the amount of time and energy they can devote to such activities. However, they still believe it is important to contribute to their community in some way, and they actively participate in social activities when the opportunity arises. --------------------------------------------------------------------------------

[0047] As a psychological approach to forming impressions in interpersonal cognition, there is a "continuum model" consisting of category-dependent processing and piecemeal-dependent processing. This invention is based on the hypothesis that a large-scale language model forms intentions (inferred answers) based on personal characteristics. Specifically, it is believed that a person's "personal characteristics (e.g., gender, age, annual income, etc.)" form a certain degree of intention in response to a question, and the answer is based on that intention. The formation of such intentions plays an important role in social psychological interpersonal cognition.

[0048] Then, the forward inference unit 11 outputs the inference answer obtained from the large-scale language model 2 to the inverse inference unit 12. At this time, the forward inference unit 11 also outputs the question and attribute options to the inverse inference unit 12.

[0049] [Reverse reasoning part 12] The reverse inference unit 12 sends a <second prompt> to the large-scale language model 2 and receives a <reverse inference personal characteristic>. The second prompt describes a <question> and an <inference answer> and instructs the reverse inference personal characteristic to be returned.

[0050] FIG. 3 shows an example of a second prompt and inference response in the reverse inference section of the present invention.

[0051] 3, the second prompt sent by the reverse inference unit 12 contains the <question> written in the first prompt of the forward inference unit 11 and the <inferred answer> obtained by the forward inference unit 11. However, in order to improve the accuracy based on the present invention, it is preferable to also write the <attribute option> written in the first prompt of the forward inference unit 11 to the large-scale language model 2.

[0052] According to FIG. 3, the second prompt lists the question, inference answer, and attribute options, for example, as follows, along with wording instructing the user to return the reverse inference personal characteristics: -------------------------------------------------------------------------------- For each inferred answer to the following question, reverse-infer the personal characteristic (see attribute options below) that best describes that user. question:····· Inference answer:... Attribute options: --------------------------------------------------------------------------------

[0053] Then, from the large-scale language model 2, for example, the following <inverse inference personal characteristics> can be obtained. -------------------------------------------------------------------------------- Reverse reasoning personal characteristics: gender = 'female' Age = '25-29' Marital experience = 'unmarried' Parental Status = 'No Children' Annual household income = '4 to 6 million yen' Job = 'Office worker (clerical)' Big_Five = {"Openness": 4, "Extraversion": 5, "Neuroticism": 3, "Conscientiousness": 6, "Agreeableness": 5} --------------------------------------------------------------------------------

[0054] [Answer judgment inference part 13] The answer determination and inference unit 13 sends the <third prompt> to the large-scale language model 2 and receives the determination result. The third prompt describes the <question>, <correct answer individual characteristic>, <inference answer>, and <reverse inference individual characteristic>.

[0055] FIG. 4 is an example of a third prompt in the answer determination and inference unit of the present invention.

[0056] According to FIG. 4, three tasks are described in the <third prompt>, for example, as follows: -------------------------------------------------------------------------------- For users with the following correct personal characteristics, the following reverse inferred personal characteristics were obtained from the inferred answers to the following questions. Based on this information, perform the following tasks: Task 1: Task 2: Task 3: Correct answer: Personal characteristics: question:····· Inference answer:... Reverse reasoning personal characteristics:... --------------------------------------------------------------------------------

[0057] <Task 1> According to Task 1, at least it is instructed to determine whether the <inference answer> to the question reflects the <reverse inference personal characteristic> more strongly than the <correct answer personal characteristic>.

[0058] According to FIG. 4, task 1 is described as follows, for example. -------------------------------------------------------------------------------- Task 1: Determine whether the inference response more closely reflects the reverse inference personality trait or the correct answer personality trait. --------------------------------------------------------------------------------

[0059] FIG. 5 is an explanatory diagram for determining the degree of reflection of the correct answer individual characteristic and the inverse inference individual characteristic in the inference answer.

[0060] The greater the difference between the correct personal characteristic and the reverse inference personal characteristic, the stronger the inference answer reflects one of the personal characteristics. If the inference answer strongly reflects the correct personal characteristic (false), there is no problem, but if it strongly reflects the reverse inference personal characteristic (true), the inference answer needs to be adjusted.

[0061] <Task 2> According to Task 2, when the first task is determined to be true (strongly reflecting the reverse inference personal characteristics), an instruction is given to adjust the inference answer. By instructing the large-scale language model 2 to adjust the result of the inference answer, the large-scale language model 2 is made to compare the correct answer personal characteristics with the reverse inference personal characteristics and make self-adjustments.

[0062] According to FIG. 4, task 2 is described as follows, for example: -------------------------------------------------------------------------------- Task 2: ···Only if your inference answer reflects too much of your reverse inference personal characteristics, fine-tune your answer value on a 7-point scale to better reflect your correct personal characteristics. (For example, the response value __ for "How much would you like to participate in social activities such as volunteering in your community?" is likely to be high / low, reflecting reverse inference personal characteristics rather than true personal characteristics, so it should be adjusted to a slightly lower / higher response value __.) --------------------------------------------------------------------------------

[0063] <Task 3> Following the second task, an instruction is given to return the adjusted inference answers, <Updated Inference Answer> and <Reason for Update>.

[0064] According to FIG. 4, task 3 is described as follows, for example: -------------------------------------------------------------------------------- Task 3: Correct Answer: Print out your final answer based on your personal characteristics and the reason for it. --------------------------------------------------------------------------------

[0065] FIG. 6 shows an example of an updated inference answer in the answer determination inference unit of the present invention.

[0066] For task 1, the judgment result can be obtained, for example, as follows: -------------------------------------------------------------------------------- Task 1: Comparing reverse inference personal characteristics with correct answer personal characteristics, the inferential answers to the question "How much would you like to participate in social activities such as volunteer work in your community?" reflect reverse inference personal characteristics more than correct answer personal characteristics. --------------------------------------------------------------------------------

[0067] For task 2, the judgement adjustment can be obtained, for example, as follows: -------------------------------------------------------------------------------- Task 2: The inferential response to "How likely are you to participate in community activities such as volunteering in your community?" should be adjusted to a slightly lower number of responses. The correct answer should be adjusted to "4: Neither" to better reflect personal characteristics. -------------------------------------------------------------------------------- In this example, the large-scale language model 2 found that the reverse inference personal trait, unlike the correct personal trait, had a time when the inference answer was likely to be positive about volunteering, and was a personal trait with little social responsibility. Therefore, it was inferred that the correct personal trait's willingness to participate in volunteering would be relatively low. The value of the previous inference answer was adjusted to a lower updated inference answer.

[0068] For task 3, the updated inference answer and the updated inference reason can be obtained, for example, as follows: -------------------------------------------------------------------------------- Task 3: "To what extent would you like to participate in social activities, such as volunteering in your community?" The new inferential response was a value of "4," reflecting a neutral intention regarding participation. This adjustment was made based on the given correct personal characteristic of having other responsibilities or commitments that limit time and energy for social activities. -------------------------------------------------------------------------------- The previous inferred answer "5" has been updated to "4" in the updated inferred answer that matches the correct personal characteristics.

[0069] Then, the answer determination and inference unit 13 outputs an updated inference answer based on task 3 to the questionnaire result collection unit 10.

[0070] 1, in another embodiment, it is also preferable to repeat the inference process of the inverse inference unit 12 and the answer determination inference unit 13 in order to increase the inference accuracy of the updated inference answer. Specifically, the answer determination inference unit 13 further inputs the updated inference answer received from the large-scale language model 2 to the inverse inference unit 12 as an inference answer received by the forward inference unit 11. Next, the inverse inference unit 12 further transmits a new second prompt to the large-scale language model 2 and receives a new inverse inference individual characteristic. Then, the answer determination inference unit 13 further executes the first to third tasks using the new inverse inference individual characteristic. This allows the updated inference answer to be brought closer to one based on the correct personal characteristics, i.e., the inference process is refined. The number of repetitions may be until the determination result becomes false (the updated inference answer to the question does not more strongly reflect the inverse inference individual characteristic than the correct individual characteristic), or may be a predetermined number of times.

[0071] FIG. 7 shows an example of the configuration of an inference device and a large-scale language model according to the present invention.

[0072] According to FIG. 7, the following three embodiments are shown: (a) The forward inference unit 11, the inverse inference unit 12, and the answer determination inference unit 13 each use the same large-scale language model 2. Note that, since the past dialogue history affects the answer estimated by the forward inference unit 11 and the individual characteristics inversely inferred by the inverse inference unit 12, it may not be possible to improve the inference accuracy in some cases.

[0073] (b) The forward inference unit 11, the inverse inference unit 12, and the answer determination inference unit 13 each use the same large-scale language model 2. However, before receiving the second prompt from the inverse inference unit 12, the past dialogue history is reset. On the other hand, before receiving the third prompt from the answer determination inference unit 13, the dialogue history with the forward inference unit 11 is continued. This prevents the past dialogue history from affecting the estimated answer by the forward inference unit 11 and the inverse inference personal characteristics by the inverse inference unit 12.

[0074] (c) The forward inference unit 11 and the answer determination inference unit 13 use large-scale language models 2A and 2B that are different from those used by the inverse inference unit 12. This prevents past dialogue history from affecting the estimated answer by the forward inference unit 11 and the individual characteristics of the inverse inference by the inverse inference unit 12.

[0075] As described above in detail, the program, device, and method of the present invention make it possible to create prompts that improve the inference accuracy of large-scale language models when inferring answers to cognitive psychology questions.

[0076] According to the present invention, it is possible to externally verify and adjust the inference answer to a question from the viewpoint of individual characteristics for a large-scale language model. In particular, it is possible to obtain a determination result as to whether or not the inference answer strongly reflects any individual characteristics (correct individual characteristics or reverse inference individual characteristics). Furthermore, according to the present invention, by setting inference steps using a plurality of tasks for a large-scale language model, it is possible to improve the accuracy of inference answers to questions. Furthermore, according to the present invention, it is possible to automatically generate answers to any questionnaire based on personal characteristics. In particular, even if a user with the assumed personal characteristics does not actually exist, the survey results from a user based on the assumed personal characteristics can be inferred using a large-scale language model. This leads to a reduction in the time and financial costs involved in surveys for businesses. In addition, it is possible to omit questions in the survey, thereby reducing the burden on users and the deterioration of response accuracy.

[0077] Furthermore, this will enable, for example, "creating prompts that improve the inference accuracy of large-scale language models for cognitive psychological tasks," which will contribute to Goal 3 of the United Nations-led Sustainable Development Goals (SDGs), "Ensure healthy lives and promote well-being for all at all ages."

[0078] With respect to the various embodiments of the present invention described above, various changes, modifications, and omissions that fall within the scope of the technical spirit and aspects of the present invention may be easily made by those skilled in the art. The above description is merely illustrative and is not intended to be limiting in any way. The present invention is limited only by the claims and their equivalents. [Explanation of symbols]

[0079] 1 Reasoning device 10. Survey Result Collection Department 11 Forward inference section 12 Reverse reasoning part 13 Answer judgment inference part 2 Large-scale language models

Claims

1. A program that causes a computer to use a large-scale language model to create prompts that infer each user's answer to any question, the program comprising: a forward inference means for transmitting a prompt to the large-scale language model, the prompt describing a question to the user and the correct personal characteristics of the user, and instructing the large-scale language model to return an inference answer, and for receiving the inference answer; an inverse inference means for transmitting a prompt to a large-scale language model, the prompt describing the question and the inference answer and instructing the model to return an inverse inference personal characteristic, and for receiving the inverse inference personal characteristic; an answer determination inference means for describing the question, the correct individual characteristic, the inference answer, and the reverse inference individual characteristic, transmitting a prompt describing a first task instructing the large-scale language model to determine whether the inference answer to the question more strongly reflects the reverse inference individual characteristic than the correct individual characteristic, and receiving a determination result; A program that causes a computer to function.

2. The answer determination inference means further describes in the prompt a second task that instructs the user to adjust the inference answer so that it better reflects the correct personal characteristics when the first task is determined to be true.

2. The program according to claim 1, which causes a computer to function as follows:

3. The answer determination inference means further describes in the prompt a third task that instructs the user to return an updated inference answer, which is an adjusted inference answer, and a reason for the update, following the second task.

3. The program according to claim 2, which causes a computer to function as follows.

4. The answer determination inference means further inputs the updated inference answer received by itself to the inference means as an inference answer received by the forward inference means, and repeats the first to third tasks for the inverse inference personal characteristics further received by the inverse inference means.

4. The program according to claim 3, which causes a computer to function as follows.

5. The question asks for a numerical value or score, Inferential answers are answers that give a numerical value or a score.

3. The program according to claim 1, wherein the program causes a computer to function as described above.

6. The correct answer personal characteristics and the reverse inference personal characteristics include personal attributes, economic attributes, personality attributes, and / or physical attributes, and are attribute values limited by attribute options.

3. The program according to claim 1, wherein the program causes a computer to function as described above.

7. The personal attributes are any one or a combination of the user's gender, age, place of residence, occupation, and family status, The economic attribute is either personal annual income or household annual income, or a combination thereof; The personality attributes are the scores of each psychological item of the user, The physical attributes are the scores of each biological item detected by the sensor attached to the user.

7. The program according to claim 6, which causes a computer to function as follows.

8. a questionnaire result collecting means for inputting questionnaire sentences for users as questions to the forward inference means and collecting the questionnaire results of the users as updated inference answers obtained from the answer determining inference means; 4. The program according to claim 3, further causing a computer to function.

9. The forward inference means, the answer determination inference means, and the reverse inference means may use the same large-scale language model with the dialogue history reset, or may use different large-scale language models.

3. The program according to claim 1, wherein the program causes a computer to function as described above.

10. A prompt generation device that uses a large-scale language model to generate prompts that infer answers to any questions from each user, comprising: a forward inference means for transmitting a prompt to the large-scale language model, the prompt describing a question to the user and the correct personal characteristics of the user, and instructing the large-scale language model to return an inference answer, and for receiving the inference answer; an inverse inference means for transmitting a prompt to a large-scale language model, the prompt describing the question and the inference answer and instructing the model to return an inverse inference personal characteristic, and for receiving the inverse inference personal characteristic; an answer determination inference means for describing the question, the correct individual characteristic, the inference answer, and the reverse inference individual characteristic, transmitting a prompt describing a first task instructing the large-scale language model to determine whether the inference answer to the question more strongly reflects the reverse inference individual characteristic than the correct individual characteristic, and receiving a determination result; A prompt creation device comprising:

11. 1. A method for generating prompts for a device that uses a large-scale language model to generate prompts that infer answers for each user to any question, comprising: The device is A first step of sending a prompt to a large-scale language model, the prompt describing a question to a user and a correct personal characteristic of the user, and instructing the large-scale language model to return an inference answer, and receiving the inference answer; a second step of sending a prompt to a large-scale language model describing the question and the inference answer and instructing the large-scale language model to return a reverse inference personal characteristic, and receiving the reverse inference personal characteristic; a third step of describing the question, the correct individual characteristic, the inference answer, and the reverse inference individual characteristic, and sending a prompt describing a first task instructing the large-scale language model to determine whether the inference answer to the question more strongly reflects the reverse inference individual characteristic than the correct individual characteristic, and receiving a determination result; A prompt creation method comprising:

Images