Program, method, information processing device, and system

Abstract

To solve a problem that a user cannot obtain an appropriate reply from a large-scale language model (LLM) system.SOLUTION: A program is to be executed by a computer having a processor and a storage unit. The program causes the processor to execute: a user reception step of receiving first user identification information for identifying a first user; a query reception step of receiving a query related to a predetermined question; an extraction step of extracting user information associated with the first user based on the first user identification information received in the user reception step and the query received in the query reception step; and a prompt generation step of generating a prompt that serves as an input sentence for a large-scale language model based on the query received in the query reception step and the user information extracted in the extraction step.SELECTED DRAWING: Figure 10

Description

[Technical field]

[0001] The present disclosure relates to a program, a method, an information processing device, and a system. [Background technology]

[0002] Large-scale language models (LLMs) are becoming increasingly recognized as useful in various information processing tasks. Patent Document 1 discloses a technique for providing a context-aware conversational agent based on a journaling model. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-121360 A Summary of the Invention [Problem to be solved by the invention]

[0004] There is a problem that large-scale language model (LLM) systems are not able to provide appropriate answers for users. Therefore, the present disclosure has been made to solve the above problem, and its purpose is to provide a technology for obtaining a response appropriate to a user from an LLM system. [Means for solving the problem]

[0005] A program to be executed by a computer having a processor and a memory unit, the program executing the following steps: a user reception step in which the processor receives first user identification information for identifying a first user; a query reception step in which the processor receives a query related to a specific question; an extraction step in which the processor extracts user information associated with the first user based on the first user identification information received in the user reception step and the query received in the query reception step; and a prompt generation step in which the processor generates a prompt that serves as an input sentence for a large-scale language model based on the query received in the query reception step and the user information extracted in the extraction step. Effect of the Invention

[0006] According to the present disclosure, an LLM system can provide an appropriate answer for the user. [Brief description of the drawings]

[0007] [Figure 1] FIG. 2 is a block diagram showing the functional configuration of the system 1. [Diagram 2] FIG. 2 is a block diagram showing the functional configuration of the server 10. [Diagram 3] 2 is a block diagram showing a functional configuration of a first user terminal 20. FIG. [Figure 4] FIG. 13 is a diagram showing the data structure of a user table 1012. [Diagram 5] FIG. 13 is a diagram showing the data structure of a query table 1013. [Figure 6] FIG. 10 illustrates the data structure of a prompt table 1014. [Figure 7] FIG. 13 is a diagram showing the data structure of a store table 1015. [Figure 8] 13 is a flowchart showing an operation of a question process. [Figure 9] 13 is a flowchart showing the operation of a proposal process. [Figure 10] 11 is a screen example showing a question processing operation. [Figure 11]FIG. 2 is a block diagram showing the basic hardware configuration of a computer 90. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. In all the drawings explaining the embodiment, the same reference numerals are given to common components, and repeated explanations are omitted. Note that the following embodiment does not unduly limit the contents of the present disclosure described in the claims. In addition, not all of the components shown in the embodiment are essential components of the present disclosure. In addition, each figure is a schematic diagram and is not necessarily illustrated strictly.

[0009] <System 1 Configuration> The system 1 in this disclosure is an information processing system that provides an information service that generates prompts for querying a large-scale language model (LLM) system. The system 1 includes a server 10, a first user terminal 20, and an information processing device of an LLM system 50, which are connected via a network N. FIG. 1 is a block diagram showing the functional configuration of the system 1. FIG. 2 is a block diagram showing the functional configuration of the server 10. As shown in FIG. FIG. 3 is a block diagram showing the functional configuration of the first user terminal 20. As shown in FIG.

[0010] Each information processing device is configured by a computer having an arithmetic unit and a storage device. The basic hardware configuration of the computer and the basic functional configuration of the computer realized by the hardware configuration will be described later. For each of the server 10, the first user terminal 20, and the LLM system 50, descriptions that overlap with the basic hardware configuration and basic functional configuration of the computer described later will be omitted.

[0011] <Server 10 Configuration> The server 10 is an information processing device that provides an information service that generates prompts for querying a large-scale language model (LLM) system. The server 10 includes a storage unit 101 and a control unit 104 .

[0012] <Configuration of the storage unit 101 of the server 10> The storage unit 101 of the server 10 includes an application program 1011 , a user table 1012 , a query table 1013 , a prompt table 1014 , and a store table 1015 .

[0013] The application program 1011 is a program for causing the control unit 104 of the server 10 to function as each functional unit. Application programs 1011 include applications such as a web browser application.

[0014] User table 1012 is a table that stores and manages information on member users (hereinafter, users) who use the service. When a user registers to use the service, the user's information is stored in a new record in user table 1012. This allows the user to use the service according to the present disclosure. A user according to the present disclosure is an end user who is the final user (customer) of a product, service, system, application, etc. The user table 1012 is a table having columns of user ID, user name, and user data, with the user ID as the primary key. FIG. 4 is a diagram showing the data structure of the user table 1012. As shown in FIG.

[0015] The user ID is an item for storing user identification information for identifying a user. The user identification information is an item for which a unique value is set for each user. The user name is an item for storing the name of the user. The user name may be set to any character string such as a nickname instead of a name. User data is an item that stores information related to a user that is associated with the user. The user data includes information regarding the physical condition of the user. The information on the user's physical condition specifically includes the user's biometric indicators, vital data (e.g., heart rate, blood pressure, body temperature, etc.), physiological state (e.g., body fat percentage, height, weight, etc.), health state (e.g., presence or absence of disease, allergy information, medical history of illness, etc.), athletic performance (e.g., walking speed, jumping power, etc.), dietary and sleep state (e.g., calorie intake, sleep time, etc.), other information on the user's physical condition and lifestyle habits, such as gender, age (including information indicating whether the user is an adult), etc. User data may also include information on the user's friendships with other people, dining history, and contact situation. The user data includes information about the devices owned by the user. Information about the devices owned by the user specifically includes the type of device (e.g. PC, smartphone, tablet, wearable device, etc.), the brand and model of the device, the type of operating system and its version, hardware specifications (e.g. CPU type and speed, memory capacity, storage capacity, etc.), the physical condition of the device (e.g. new or used, repair history, etc.), a list of installed software and applications and their versions, and other settings and usage information (e.g. battery life, usage time, communication status, etc.). The devices are not limited to information devices, but also include means of transportation (e.g., automobiles, bicycles, motorcycles, etc.), home appliances (e.g., refrigerators, washing machines, televisions, etc.), tools and machines (e.g., power tools, gardening tools, etc.), etc. In this case, the user data may include information such as the type of product, brand and model, year of manufacture and year of purchase, product condition (new or used, current functional state, etc.), product specifications and performance, maintenance history and repair history, frequency and method of use, and the presence or absence of accessories.

[0016] The control unit 104 of the server 10 may be configured to obtain user data from an information device owned by the user via a communication line such as the Internet, and store the user data in the user table 1012 in association with the user ID. The control unit 104 of the server 10 may be configured to store, in the user data item of the user table 1012, a message document previously received from a user via information exchange means such as e-mail, a chat service, etc. Specifically, the user data does not need to be structured data in which the name and type of data are associated with the value of the data, and may be, for example, text such as the subject and body of a message document in an e-mail, a chat service, etc. Specifically, in the case where the operator of the information processing system according to the present disclosure has traditionally provided user support through e-mail, chat services, etc., the message text exchanged with the user during the user support can be included in the user data. This allows the user data to include information on the user's physical condition and the devices owned by the user, which is included in the message text. The user data includes any database, such as CSV, Excel, SQL, etc., generated by providing a predetermined process to the message text. The items of the user data may also include a file name, path name, FQDN (Full Qualified Domain Name), IP address, etc., for referencing any database, such as CSV, Excel, SQL, etc. (not shown). In this case, the user data includes information stored in the referenced database.

[0017] The query table 1013 is a table for storing and managing information regarding queries related to questions (query information). The query table 1013 is a table having a query ID as a primary key, and columns of a query ID, a user ID, a query, and a date and time. FIG. 5 is a diagram showing the data structure of the query table 1013. As shown in FIG.

[0018] The query ID is an item for storing query identification information for identifying a query. The query identification information is an item in which a unique value is set for each piece of query information. The user ID is an item for storing user identification information for identifying a user. The query is an item for storing character string (text) information related to a question. Specifically, a query related to the user's physical condition, a query related to a device owned by the user, etc. are stored. For example, queries regarding a user's physical condition include queries such as, "What kind of diet should I follow to lower my blood pressure?", "What kind of exercise is suitable for lowering blood pressure?", and "What is the best method for losing weight in a healthy way?" For example, queries regarding devices owned by users include "My PC won't turn on, what should I do?" and "My PC application won't start, what should I do?". The date and time is an item for storing the date and time when a record is newly stored or updated in the query table 1013.

[0019] The prompt table 1014 is a table for storing and managing information (prompt information) relating to a prompt that serves as an input sentence to the LLM system 50 . The prompt table 1014 is a table having a prompt ID as a primary key, and columns of a prompt ID, a user ID, a prompt, an answer result, and a date and time. FIG. 6 is a diagram showing the data structure of the prompt table 1014. As shown in FIG.

[0020] The prompt ID is an item for storing prompt identification information for identifying a prompt. The prompt identification information is an item for which a unique value is set for each piece of prompt information. The user ID is an item for storing user identification information for identifying a user. The prompt is an item for storing a prompt that is an input sentence to the LLM system 50. Specifically, character string (text) information related to the prompt is stored. The answer result is an item for storing the answer result to the prompt received from the LLM system 50. Specifically, character string (text) information related to the answer sentence to the prompt is stored. The date and time is an item for storing the date and time when a record is newly stored or updated in the prompt table 1014.

[0021] The store table 1015 is a table for storing and managing information about stores (store information). The store table 1015 is a table having a store ID as a primary key, and columns of store ID, user ID, store name, store data, and word-of-mouth data. FIG. 7 is a diagram showing the data structure of the store table 1015.

[0022] The store ID is an item for storing store identification information for identifying a store. The store identification information is an item for which a unique value is set for each store information. The user ID is an item for storing user identification information for identifying a user associated with a store. Specifically, the user identification information of a user related to the operation of the store, such as a manager, owner, or employee of the store, is stored. The store name is an item for storing the name of the store. Any character string can be set as the store name. The store data is an item for storing information about a store. Store data includes information such as the store name, store location, industry, products sold, store opening and closing times, store size (e.g., sales floor area, number of floors, etc.), types of products handled and their stock status, store layout, services provided in the store, staff information (e.g., number of employees, skill level, etc.), sales status and customer flow at the store, and other marketing-related information (e.g., sale days, sale information, etc.). The word-of-mouth data is an item that stores information about word-of-mouth reviews by posters. Word-of-mouth data includes the poster's ID, email address, posting date and time, rating value (e.g. number of stars, score, etc.), posted text (specific review or impression), name of the target product or service, the poster's purchasing or usage experience, information about the store or company that is the subject of the post, ratings and reactions from other users (e.g. number of "likes", comments, etc.), etc.

[0023] <Configuration of the control unit 104 of the server 10> The control unit 104 of the server 10 includes a user registration control unit 1041, a question unit 1042, and a suggestion unit 1043. The control unit 104 executes an application program 1011 stored in the storage unit 101, thereby realizing each functional unit.

[0024] The user registration control unit 1041 performs processing to store, in the user table 1012, information on users who wish to use the service according to the present disclosure. The information stored in the user table 1012 is generated when a user opens a web page operated by a service provider from any information processing terminal, enters information into a specific input form, and transmits the information to the server 10. The user registration control unit 1041 stores the received information in a new record in the user table 1012, completing the user registration. This allows the user stored in the user table 1012 to use the service. Before the user registration control unit 1041 registers user information in the user table 1012, the service provider may carry out a predetermined examination to restrict whether or not the user is permitted to use the service. The user ID may be any character string or number that can identify the user, any character string or number desired by the user, or an arbitrary character string or number may be automatically set by the user registration control unit 1041.

[0025] The interrogator 1042 executes interrogation processing, the details of which will be described later.

[0026] The proposing unit 1043 executes a proposing process, the details of which will be described later.

[0027] <Configuration of First User Terminal 20> The first user terminal 20 is an information processing device operated by a user who uses a service. The first user terminal 20 may be, for example, a mobile terminal such as a smartphone or a tablet, a stationary PC (Personal Computer), or a laptop PC. It may also be a wearable terminal such as an HMD (Head Mount Display) or a wristwatch-type terminal. The first user terminal 20 includes a storage unit 201 , a control unit 204 , an input device 206 , and an output device 208 .

[0028] <Configuration of the storage unit 201 of the first user terminal 20> The storage unit 201 of the first user terminal 20 includes a first user ID 2011 and an application program 2012 .

[0029] The first user ID 2011 is a user's account ID. The first user transmits the first user ID 2011 from the first user terminal 20 to the server 10. The server 10 identifies the user based on the first user ID 2011 and provides the user with the service according to the present disclosure. The first user ID 2011 includes information such as a session ID temporarily assigned by the server 10 to identify the user using the first user terminal 20.

[0030] The application program 2012 may be stored in advance in the storage unit 201, or may be configured to be downloaded from a web server operated by a service provider via a communication IF. The application programs 2012 include applications such as a web browser application. The application program 2012 includes an interpreted programming language such as JavaScript™ that runs on a web browser application stored on the first user terminal 20.

[0031] <Configuration of the control unit 204 of the first user terminal 20> The control unit 204 of the first user terminal 20 includes an input control unit 2041 and an output control unit 2042. The control unit 204 realizes each functional unit by executing the application program 2012 stored in the storage unit 201.

[0032] <Configuration of the input device 206 of the first user terminal 20> The input device 206 of the first user terminal 20 includes a camera 2061, a microphone 2062, a position information sensor 2063, a motion sensor 2064, and a touch device 2065.

[0033] <Configuration of the output device 208 of the first user terminal 20> The output device 208 of the first user terminal 20 includes a display 2081 and a speaker 2082.

[0034] <Configuration of the LLM system 50> The LLM (Large Language Model) system 50 refers to a large-scale artificial intelligence model used in the field of natural language processing (NLP). These models can understand the patterns of the language used by humans and effectively perform natural language generation (NLG) tasks by learning a large amount of text data (web pages, books, articles, etc.). The LLM system 50 is used in many NLP tasks such as generating responses to specific questions, automatically generating articles, summarizing text, translation, sentiment analysis, etc. It can also be applied to various uses such as education, entertainment, customer service, product development, etc. The following exist in the LLM system 50. ·OpenAI ChatGPT ·Google Bard ·Stable Diffusion ·midjourney

[0035] <Operation of system 1> The following describes each process of system 1. Figure 8 is a flowchart showing the operation of the question processing. FIG. 9 is a flowchart showing the operation of the proposal process. FIG. 10 is a screen example showing the operation of question processing.

[0036] <Question Processing> The question processing is a process for accepting a question sentence (query) from a user, generating a prompt that serves as an input sentence to a large-scale language model (LLM) system based on the query, and obtaining an answer to the prompt.

[0037] <Question processing overview> Question processing is a series of processes that include accepting a question (query) from a user, extracting user data based on user identification information and the query, generating a prompt based on the query and user data, sending the prompt to a large-scale language model (LLM) system, receiving an answer result from the LLM system, storing the answer result, and presenting the answer result to the user. The question processing may be configured to be executed by an operator who accepts an inquiry from a customer. For example, the operator may accept a question (query) from a customer (user) and transmit information for identifying the customer as user identification information together with the query to the information processing system according to the present disclosure. In this case, the answer result from the large-scale language model (LLM) is presented to the operator. The operator may present the presented answer result by replying to the customer. Furthermore, the question processing may be configured to be executed by an end user. For example, the question processing may be executed by the end user without the intervention of an operator, and the presented answer result may be presented directly to the end user.

[0038] <Question processing details> The question processing will be described in detail below.

[0039] In step S101, the interrogator 1042 of the server 10 executes a query receiving step of receiving a query regarding a predetermined question from a first user. The interrogator 1042 of the server 10 executes a user receiving step of receiving first user identification information for identifying the first user. Specifically, the first user operates the input device 206 of the first user terminal 20 to execute a browser application or the like, and opens the question page D1 by inputting the URL of a web page (question page) for executing a question process, etc. The control unit 204 of the first user terminal 20 transmits a request including the first user ID 2011 for opening the question page to the server 10.

[0040] When the server 10 receives the request, it generates a question page and transmits it to the first user terminal 20. The control unit 204 of the first user terminal 20 displays the question page on the display 2081 of the first user terminal 20 for presentation. 10 is an example of a question screen in question processing. A question page D1 is displayed on the display 2081 of the first user terminal 20. The question page D1 includes a query input field D101 for inputting character string (text) information related to the question, an answer area D102 for displaying character string (text) information related to an answer result to the question, and a send button D103. On the question screen, the first user can input various questions in the query input field D101 and obtain answers to the questions from the answer results displayed in the answer area D102.

[0041] The question page D1 includes a create button D111, a DB construction button D112, and a system setting button D113. The user can switch the screen displayed on the question page D1 by selecting one of the create button D111, the DB construction button D112, and the system setting button D113 by operating the input device 206 of the first user terminal 20. Specifically, the question page D1 shown in FIG. 10 is a prompt creation screen displayed by selecting the create button D111. The user can display a DB construction screen for registering and editing user data in the user table 1012 by operating the input device 206 of the user terminal 20 and selecting the DB construction button D112. The user can change various setting values ​​of the information processing system according to the present disclosure by operating the input device 206 of the user terminal 20 and selecting the system setting button D113.

[0042] The question page D1 includes a product selection input field D121 for selecting a target product related to the question, and reference data D122 and D123. The user inputs product specification information for specifying a product, such as a product name of a product or service related to the question, a product ID, etc., into the product selection input field D121. The product specification information input into the product selection input field D121 is stored in the user data item of the user table 1012 in association with the user ID 2011.

[0043] The first user operates the input device 206 of the first user terminal 20 to input a character string related to a question into the query input field D101. The first user operates the input device 206 of the first user terminal 20 to select the send button D103. The control unit 204 of the first user terminal 20 transmits the query and the first user ID 2011 input into the query input field D101 to the server 10. The question unit 1042 of the server 10 receives and accepts the query and the first user ID 2011. In the present disclosure, a configuration in which a query and first user ID 2011 are received from a first user has been disclosed as an example, but the first user ID 2011 may be input, received, and accepted by an input operation by an operator, not from the first user terminal 20. For example, the operator operates his / her own information processing terminal to input the first user ID 2011 in a predetermined user identification information input field and transmits it. The questioning unit 1042 of the server 10 receives and accepts the first user ID 2011.

[0044] The query receiving step includes a step of receiving a query regarding a physical condition of the first user. Specifically, the first user can input a query regarding the physical condition of the first user in the query input field D101 by operating the input device 206 of the first user terminal 20. The query regarding the physical condition is described in the query item of the query table 1013. In this case, the interrogator 1042 of the server 10 receives and accepts a query regarding the physical condition of the first user.

[0045] The query receiving step includes a step of receiving a query regarding a device owned by the first user. Specifically, the first user can input a query related to a device owned by the first user into the query input field D101 by operating the input device 206 of the first user terminal 20. The query related to the device owned is described in the query item of the query table 1013. In this case, the interrogator 1042 of the server 10 receives and accepts a query regarding the device owned by the first user.

[0046] In step S102, the questioning unit 1042 of the server 10 executes an extraction step of extracting user information associated with the first user based on the first user identification information received in the user receiving step and the query received in the query receiving step. The extracting step includes a step of extracting a portion of user information related to the query from information associated with the first user, the portion of user information not related to the query being excluded. Specifically, the interrogation unit 1042 of the server 10 searches the user ID item of the user table 1012 based on the first user ID 2011, and acquires user data of the first user (first user data). Based on the query received in step S101, the interrogation unit 1042 of the server 10 extracts information related to the query from various information related to the first user included in the first user data, and does not extract information unrelated to the query. Specifically, the questioning unit 1042 of the server 10 calculates the similarity between the query and the multiple first user data based on an algorithm such as cosine similarity, TF-IDF score, or LexRank between the received query and information included in the first user data (multiple first user data). This calculates the similarity for each of the multiple first user data. The questioning unit 1042 of the server 10 extracts a predetermined number of first user data in order of similarity according to the similarity or first user data with a similarity equal to or greater than a predetermined value from the multiple first user data, and specifies the first user data by excluding the other first user data. Alternatively, the first user data related to the query may be extracted from the multiple first user data with reference to past questions, answer information, and the like. The question unit 1042 of the server 10 may receive the contents of the input value of the product selection input field D121 from the first user terminal 20, and search for user data in the user table 1012 based on the input value. Specifically, based on the product name input in the product selection input field D121, user data including the product name may be searched for from the user table 1012. That is, the query received in the query receiving step includes information indicating a product name such as the input value of the product selection input field D121. For example, when a query such as "What kind of diet should I follow to lower my blood pressure?" is received from a first user, first user data that is less relevant to the query, such as information about devices owned by the first user, is excluded, and first user data that is more relevant to the query, such as the first user's gender, age, blood pressure, diet, calorie intake, medical history, and illness history, is extracted.

[0047] In step S102, the extracting step performs a step of extracting user information including information related to a physical condition of the first user associated with the first user. For example, when a query regarding the physical condition of a first user is received from a first user, the question unit 1042 of the server 10 extracts first user data related to the physical condition of the first user, and does not extract first user data unrelated to the physical condition of the first user.

[0048] In step S102, the extracting step executes a step of extracting user information including information on a device owned by the first user. For example, when a query is received from a first user regarding a device owned by the first user, the question unit 1042 of the server 10 extracts first user data related to the device owned by the first user, and does not extract first user data not related to the device owned by the first user.

[0049] In step S102, the questioning unit 1042 of the server 10 transmits the extracted first user data to the first user terminal 20. The control unit 204 of the first user terminal 20 displays and presents the received first user data in the reference data D122, D123. This allows the first user to explicitly confirm what first user data related to the first user is referenced to generate the prompt. The first user may operate the input device 206 of the first user terminal 20 to select one of the reference data D122, D123, etc., and to exclude it from the first user data to be included in the prompt. In addition, the first user may operate the input device 206 of the first user terminal 20 to manually add any user data among the user data stored in the user table 1012 in association with the first user to the reference data D122, D123, thereby allowing the first user data to be included in the first user data. Alternatively, the first user may operate the input device 206 of the first user terminal 20 to include any information in the first user data. This allows the first user to manually generate prompts that serve as input sentences for an appropriate large-scale language model (LLM). The reference data D122 and D123 may be the file name or path name of a CSV file related to the user data, the FQDN or IP address of a database that is a reference source of the user data, or the like.

[0050] In step S103, the question unit 1042 of the server 10 executes a prompt generation step of generating a prompt that serves as an input sentence for the large scale language model (LLM) based on the query received in the query reception step and the user information extracted in the extraction step. Specifically, the question unit 1042 of the server 10 generates a prompt based on the query received in step S101 and the first user data extracted in step S102. For example, the questioner 1042 of the server 10 generates a prompt such as the following: 〔prompt〕 # Order You are a chatbot that responds to questions from users by outputting answers to assist the user, taking into account the following user information: # Question from the user Query # User information First user data # Answer text

[0051] For example, when the question unit 1042 of the server 10 receives a query from the first user regarding the physical condition of the first user, the question unit 1042 of the server 10 generates a prompt such as the following: 〔prompt〕 # Order You are a chatbot that responds to questions from users by outputting answers to assist the user, taking into account the following user information: # Question What kind of diet should I follow to lower my blood pressure? # User information ·blood pressure: ●●(YMD),●●(YMD),●●(YMD),●●(YMD) ·body weight ●●(YMD),●●(YMD),●●(YMD),●●(YMD) Meal details ●●(YMD),●●(YMD),●●(YMD),●●(YMD) - Medical history ●●(YMD),●●(YMD),●●(YMD),●●(YMD) # Answer text

[0052] The questioning unit 1042 of the server 10 may generate a prompt including a character string that imposes an age restriction on the question sentence based on information indicating the age and whether the user is an adult included in the first user data. For example, the questioning unit 1042 of the server 10 may include a character string such as "Please give an answer appropriate for a person aged XX" or "Please exclude answers that are inappropriate for minors" in the prompt. In addition, the question unit 1042 of the server 10 may perform a process of including in the prompt a character string for restricting the output content based on the first user data, or may perform a process of including in the prompt a character string for restricting the output content without being based on the first user data. This allows the LLM system 50 to provide an appropriate answer based on the first user's age and information on whether the first user is an adult or a minor.

[0053] In step S104, the interrogator 1042 of the server 10 executes a prompt sending step of sending the prompt generated in the prompt generating step to an external large scale language model (LLM) system operated by a business operator different from that of the computer. Specifically, the interrogator 1042 of the server 10 sends a request including the prompt generated in step S103 to an API endpoint provided by the LLM system 50.

[0054] In step S105, the question unit 1042 of the server 10 executes an answer receiving step of receiving an answer result to the prompt transmitted in the prompt transmitting step. The answer receiving step executes a step of receiving an answer result to the prompt based on user information associated with the first user, without training a large scale language model (LLM) system. Specifically, the LLM system 50 is configured to be able to output a character string (text) in response to a prompt received at an API endpoint. The LLM system 50 transmits a response including an answer result output based on the received prompt to the server 10. The questioning unit 1042 of the server 10 receives and accepts the answer result to the prompt included in the response from the LLM system 50.

[0055] In step S106, the questioning section 1042 of the server 10 stores the prompt generated in the prompt generating step and the answer result received in the answer receiving step in association with each other. Specifically, the questioning unit 1042 of the server 10 stores the first user ID 2011, the prompt generated in step S103, and the answer result received in step S105 in the user ID, prompt, and answer result items of the prompt table 1014, respectively.

[0056] In step S107, the question unit 1042 of the server 10 executes a proposal step of proposing to the first user advice information regarding the first user's response measures to the query received in the query receiving step, based on the answer result received in the answer receiving step. Specifically, the question unit 1042 of the server 10 transmits the character string (text) related to the answer result received from the LLM system 50 in step S105 to the first user terminal 20. The control unit 204 of the first user terminal 20 outputs the character string (text) related to the answer result to the answer area D102 of the question page D1 displayed on the display 2081 of the first user terminal 20. As a result, the question unit 1042 of the server 10 proposes to the first user a sentence regarding advice for the first user's query (advice information) in response to the query received in step S101. When a question process is executed by an operator, the answer result is presented to the operator. The operator transmits the presented answer result by replying to the customer (first user) who inquired the question. This allows the information inquiry service to provide the customer with a suitable answer result in response to the customer's inquiry.

[0057] <Proposal processing> The suggestion process is a process of giving advice (suggestion) regarding a store's word-of-mouth received from a third party to a user associated with the store.

[0058] <Proposal process overview> The proposal process is a series of processes that accept reviews about stores etc. from third parties, extract store data for the store etc. that accepted the reviews, generate prompts based on the content of the reviews and the store data, send the prompts to a large-scale language model (LLM) system, receive response results from the LLM system, store the response results, and present to the user advice information regarding how the user should respond to the reviews based on the response results.

[0059] <Details of the proposal process> The proposal process will be described in detail below.

[0060] In S301, a query receiving step is executed to receive a query regarding word-of-mouth about a store associated with a first user. Specifically, a user related to a poster (hereinafter, a poster) operates the poster's information processing terminal to open a website for posting reviews about a store in a web browser, etc. The site for posting reviews does not need to be a website operated by the business that provides the information service according to the present disclosure, and may be an external website operated by another business. A poster operates his / her information processing terminal to input and send information about a store, such as a review about the store, into an input form or the like provided on the website. This posts a review to the website for posting reviews about stores. The suggestion unit 1043 of the server 10 accesses the website to obtain the review posted by the poster and store identification information for identifying the posted store. The suggestion unit 1043 of the server 10 accepts the obtained review as a query. Based on the acquired store identification information, the suggestion unit 1043 of the server 10 stores the acquired word-of-mouth in the word-of-mouth data field of the record identified by referring to the store ID in the store table 1015. Note that the record may be identified based on the store name instead of the store identification information.

[0061] In step S302, the suggestion unit 1043 of the server 10 executes an extraction step of extracting store information associated with a store based on the acquired word-of-mouth. The suggestion unit 1043 of the server 10 searches the store table 1015 for a store ID based on the store identification information acquired in step S301, and acquires store data of the store identified by the store identification information and a user ID (user identification information of a user associated with the store, hereinafter referred to as a first user). Note that the record may be identified based on the store name instead of the store identification information. The suggestion unit 1043 of the server 10 extracts store data related to the word-of-mouth from various information related to the word-of-mouth included in the store data based on the word-of-mouth acquired in step S301, and does not extract store data unrelated to the word-of-mouth. Specifically, the suggestion unit 1043 of the server 10 calculates the similarity between the word-of-mouth and the multiple store data based on an algorithm such as cosine similarity, TF-IDF score, or LexRank between the received word-of-mouth and information included in the store data (multiple store data). This calculates the similarity for each of the multiple store data. The suggestion unit 1043 of the server 10 extracts a predetermined number of store data in order of similarity according to the similarity or store data with a similarity equal to or greater than a predetermined value from the multiple store data, and specifies the store data by excluding other store data. Alternatively, the configuration may be such that store data related to the word-of-mouth is extracted from the multiple store data by referring to past questions, answer information, etc. For example, if a review is received from a poster stating that "the menu variety was limited," store data is extracted that includes store data that is highly relevant to the review, such as the products sold by the store, the types of products handled and their stock status, services offered in the store, and other marketing information.

[0062] In step S303, the suggestion unit 1043 of the server 10 executes a prompt generation step of generating a prompt that serves as an input sentence for the large-scale language model (LLM) based on the word-of-mouth received in the query reception step and the store information extracted in the extraction step. Specifically, the suggestion unit 1043 of the server 10 generates a prompt based on the word-of-mouth received in step S301 and the store data extracted in step S302. For example, the suggestion unit 1043 of the server 10 generates a prompt as follows: 〔prompt〕 # Order You are a chatbot that responds to reviews about a store by taking into account the following store information and outputting responses to help the store respond to the reviews. # Store reviews Reviews # Store information Store Data # Answer text

[0063] For example, when the suggestion unit 1043 of the server 10 receives a word-of-mouth from a contributor, the suggestion unit 1043 of the server 10 generates a prompt such as the following. 〔prompt〕 # Order You are a chatbot that responds to reviews about a store by taking into account the following store information and outputting responses to support store initiatives in response to reviews. # Store reviews The menu variety was poor # Store information ·Products for sale: ●●●●,●●●●,●●●●,●●●●,●●●●,●●●● Products ●●●●,●●●●,●●●●,●●●●,●●●●,●●●● Availability ●●●●,●●●●,●●●●,●●●●,●●●●,●●●● In-store services # Answer text

[0064] Steps S304 to S306 are the same as steps S104 to S106 in the question processing, and therefore a description thereof will be omitted. The suggestion unit 1043 of the server 10 stores the user identification information of the user associated with the store, the prompt generated in step S303, and the answer result received in step S305 in the items of user ID, prompt, and answer result in the prompt table 1014.

[0065] In step S307, the suggestion unit 1043 of the server 10 executes a suggestion step of proposing to the first user advice information regarding the first user's response measures for the query received in the query receiving step, based on the answer result received in the answer receiving step. Specifically, the suggestion unit 1043 of the server 10 transmits a character string (text) related to the answer result received from the LLM system 50 in step S305 to the first user terminal 20. For example, the suggestion unit 1043 of the server 10 searches the user ID item of the user table 1012 based on the user identification information of the first user associated with the store identified in step S101, and acquires contact information for contacting the first user, such as the first user's email address and the first user's chat ID. The suggestion unit 1043 of the server 10 transmits a character string (text) related to the answer result received from the LLM system 50 in response to the contact information. The control unit 204 of the first user terminal 20 outputs a character string (text) related to the received answer result to the display 2081 of the first user terminal 20. As a result, the suggestion unit 1043 of the server 10 suggests to the first user associated with the store, in accordance with the word-of-mouth received in step S301, a sentence (advice information) related to advice on the first user's response measures.

[0066] <Basic computer hardware configuration> 11 is a block diagram showing the basic hardware configuration of a computer 90. The computer 90 includes at least a processor 901, a main storage device 902, an auxiliary storage device 903, and a communication IF 991 (interface). These are electrically connected to each other by a communication bus 921.

[0067] The processor 901 is hardware for executing an instruction set described in a program, and is composed of an arithmetic unit, a register, a peripheral circuit, and the like.

[0068] The main memory device 902 is for temporarily storing programs, data to be processed by the programs, etc. For example, it is a volatile memory such as a DRAM (Dynamic Random Access Memory).

[0069] The auxiliary storage device 903 is a storage device for saving data and programs, such as a flash memory, a hard disk drive (HDD), a magneto-optical disk, a CD-ROM, a DVD-ROM, or a semiconductor memory.

[0070] The communication IF 991 is an interface for inputting and outputting signals for communicating with other computers via a network using a wired or wireless communication standard. The network is composed of the Internet, a LAN, various mobile communication systems constructed by wireless base stations, etc. For example, the network includes 3G, 4G, 5G mobile communication systems, LTE (Long Term Evolution), wireless networks that can connect to the Internet via a specified access point (e.g., Wi-Fi (registered trademark)), etc. In the case of wireless connection, communication protocols include, for example, Z-Wave (registered trademark), ZigBee (registered trademark), Bluetooth (registered trademark), etc. In the case of wired connection, the network also includes a network that is directly connected by a USB (Universal Serial Bus) cable or the like.

[0071] It should be noted that the computer 90 can be virtually realized by distributing all or part of each hardware configuration among multiple computers 90 and connecting them together via a network. In this way, the computer 90 is a concept that includes not only a computer 90 housed in a single housing or case, but also a virtualized computer system.

[0072] <Basic functional configuration of computer 90> A description will now be given of the functional configuration of a computer realized by the basic hardware configuration (FIG. 11) of a computer 90. The computer comprises at least the functional units of a control unit, a storage unit, and a communication unit.

[0073] The functional units of the computer 90 can also be realized by distributing all or part of the functional units among multiple computers 90 connected to each other via a network. The computer 90 is a concept that includes not only a single computer 90 but also a virtualized computer system.

[0074] The control unit is realized by the processor 901 reading out various programs stored in the auxiliary storage device 903, expanding the programs in the main storage device 902, and executing processes according to the programs. The control unit can realize functional units that perform various information processing depending on the type of program. In this way, the computer is realized as an information processing device that performs information processing.

[0075] The storage unit is realized by a main storage device 902 and an auxiliary storage device 903. The storage unit stores data, various programs, and various databases. Furthermore, the processor 901 can secure a storage area corresponding to the storage unit in the main storage device 902 or the auxiliary storage device 903 in accordance with a program. Furthermore, the control unit can cause the processor 901 to execute processes of adding, updating, and deleting data stored in the storage unit in accordance with the various programs.

[0076] The term database refers to a relational database, which is used to manage sets of data called masters and tables in a tabular format structurally defined by rows and columns, by associating them with each other. In a database, a table is called a table or master, a column in a table is called a column, and a row in a table is called a record. In a relational database, relationships between tables and masters can be set and associated. Usually, a column that serves as a primary key for uniquely identifying a record is set in each table and each master, but setting a primary key in a column is not essential. The control unit can cause the processor 901 to add, delete, or update records in a specific table or master stored in the storage unit according to various programs. Furthermore, by storing data, various programs, and various databases in the storage unit, it can be considered that the information processing device and information processing system according to the present disclosure have been manufactured.

[0077] In addition, the database and master in this disclosure may include any data structure (such as a list, a dictionary, an associative array, or an object) in which information is structurally defined. The data structure also includes data that can be considered as a data structure by combining data with a function, class, method, or the like written in any programming language.

[0078] The communication unit is realized by the communication IF 991. The communication unit realizes a function of communicating with other computers 90 via a network. The communication unit can receive information transmitted from other computers 90 and input the information to the control unit. The control unit can cause the processor 901 to execute information processing on the received information in accordance with various programs. In addition, the communication unit can transmit information output from the control unit to other computers 90.

[0079] <Additional Notes> The matters described in the above embodiments will be supplemented below.

[0080] (Appendix 1) A program to be executed by a computer having a processor and a memory unit, the program executing a user reception step (S101) in which the processor receives first user identification information for identifying a first user, a query reception step (S101) in which a query related to a specific question is received, an extraction step (S102) in which user information associated with the first user is extracted based on the first user identification information received in the user reception step and the query received in the query reception step, and a prompt generation step (S103) in which a prompt is generated as an input sentence for a large scale language model (LLM) based on the query received in the query reception step and the user information extracted in the extraction step. This allows the user information to be included in the prompt for querying the LLM system. By using the prompt, the LLM system can provide a response that is more suited to the user. For example, the user's problem can be resolved in a shorter time.

[0081] (Appendix 2) The program according to claim 1, wherein the extracting step (S102) is a step of excluding a portion of user information not related to the query from the information associated with the first user and extracting a portion of user information related to the query. This allows the prompt for querying the LLM system to include information related to the query from among the user information. By using this prompt, even if there is a large amount of user information, it is possible to extract only the user information that is appropriate for the query and query the LLM system.

[0082] (Appendix 3) The program according to claim 1 or 2, wherein the extracting step (S102) is a step of extracting user information including information related to a physical condition of the first user associated with the first user. This allows users to receive prompts from the LLM system to answer questions about their health status, for example to get answers more tailored to the user's problem.

[0083] (Appendix 4) The program according to claim 3, wherein the query receiving step (S101) is a step of receiving a query regarding a physical condition of the first user. This allows users to receive prompts from the LLM system to answer questions about their health status, for example to get answers more tailored to the user's problem.

[0084] (Appendix 5) 3. The program according to claim 1, wherein the extracting step (S102) is a step of extracting user information including information on a device owned by the first user. This allows users to receive prompts that provide answers more tailored to the problems they are facing when asked questions about their own information processing devices, such as PCs and smartphones.

[0085] (Appendix 6) The program according to claim 5, wherein the query receiving step (S101) is a step of receiving a query regarding a device owned by the first user. This allows users to receive prompts that provide answers more tailored to the problems they are facing when asked questions about their own information processing devices, such as PCs and smartphones.

[0086] (Appendix 7) The program according to claim 1 or 2, wherein the query receiving step (S301) is a step of receiving a query regarding word-of-mouth reviews of a store associated with the first user. This allows the user to receive a prompt to ask the LLM system a question regarding the system's policy for responding to reviews posted by third parties about the store or other business that the user manages or is involved with.

[0087] (Appendix 8) A program described in any one of Appendices 1 to 7, in which a processor executes a prompt sending step (S104) of sending a prompt generated in the prompt generation step to an external large scale language model (LLM) system operated by an operator different from the computer, and an answer receiving step (S105) of receiving an answer result to the prompt sent in the prompt sending step. This allows the LLM system to provide answers that are more tailored to the user, for example solving the user's problem in a shorter time.

[0088] (Appendix 9) The program according to claim 8, wherein the answer receiving step (S105) is a step of receiving an answer result to the prompt based on user information associated with the first user, without training a large-scale language model (LLM) system. This allows the LLM system to obtain answers that are appropriate for each user, without having to train the LLM system using user information for each user. For example, this can save the effort of training a model for each user, as well as the cost of storing the model.

[0089] (Appendix 10) A program as described in Appendix 8 or 9, in which a processor executes a proposal step (S107) of proposing advice information to the first user regarding the first user's response measures for the query received in the query receiving step based on the answer result received in the answer receiving step. This allows users to obtain advice from the LLM system regarding reviews posted by third parties regarding stores that they manage or are involved in. For example, users can resolve issues regarding reviews posted about their stores in a shorter time.

[0090] (Appendix 11) A method implemented on a computer having a processor and a memory, the processor performing all of the steps performed in the invention according to any one of claims 1 to 10. This allows the user information to be included in the prompt for querying the LLM system. By using the prompt, the LLM system can provide a response that is more suited to the user. For example, the user's problem can be resolved in a shorter time.

[0091] (Appendix 12) An information processing device comprising a control unit and a memory unit, wherein the control unit executes all of the steps executed in the invention according to any one of Supplementary Note 1 to Supplementary Note 10. This allows the user information to be included in the prompt for querying the LLM system. By using the prompt, the LLM system can provide a response that is more suited to the user. For example, the user's problem can be resolved in a shorter time.

[0092] (Appendix 13) A system comprising means for performing all the steps performed in any one of claims 1 to 10. This allows the user information to be included in the prompt for querying the LLM system. By using the prompt, the LLM system can provide a response that is more suited to the user. For example, the user's problem can be resolved in a shorter time. [Explanation of symbols]

[0093] 1 System, 10 Server, 101 Memory unit, 104 Control unit, 106 Input device, 108 Output device, 20 First user terminal, 201 Memory unit, 204 Control unit, 206 Input device, 208 Output device, 50 LLM system, 501 Memory unit, 504 Control unit, 506 Input device, 508 Output device

Claims

1. A program to be executed by a computer having a processor and a storage unit, The processor, a user receiving step of receiving first user identification information for identifying a first user; a query receiving step of receiving a query regarding a predetermined question; an extraction step of extracting user information associated with the first user based on the first user identification information received in the user receiving step and the query received in the query receiving step; a prompt generating step of generating a prompt to be an input sentence to a large scale language model (LLM) based on the query received in the query receiving step and the user information extracted in the extracting step; A program that executes.

2. The extracting step is a step of excluding a portion of the user information not related to the query from the information associated with the first user and extracting a portion of the user information related to the query. The program according to claim 1.

3. The extracting step is a step of extracting user information including information regarding a physical condition of the first user associated with the first user.

3. The program according to claim 1 or 2.

4. The query receiving step is a step of receiving the query regarding a physical condition of the first user. The program according to claim 3.

5. The extraction step is a step of extracting user information including information about a device owned by the first user.

3. The program according to claim 1 or 2.

6. The query receiving step is a step of receiving the query regarding a device owned by the first user. The program according to claim 5.

7. The query receiving step is a step of receiving a query regarding word-of-mouth reviews of a store associated with the first user.

3. The program according to claim 1 or 2.

8. The processor, a prompt sending step of sending the prompt generated in the prompt generating step to an external large scale language model (LLM) system operated by a different operator than the computer; a response receiving step of receiving a response result to the prompt transmitted in the prompt transmitting step; Execute The program according to any one of claims 1 to 7.

9. receiving the answer based on the user information associated with the first user, without training the large-scale language model (LLM) system; The program according to claim 8.

10. The processor, a suggestion step of suggesting to the first user advice information regarding a countermeasure to be taken by the first user with respect to the query received in the query receiving step, based on the answer result received in the answer receiving step; Execute 10. The program according to claim 8 or 9.

11. A method implemented on a computer having a processor and a memory, the processor performing all of the steps performed in any of the inventions according to claims 1 to 10.

12. 11. An information processing apparatus comprising a control unit and a storage unit, the control unit executing all steps executed in the invention according to any one of claims 1 to 10.

13. A system comprising means for executing all the steps performed in any one of claims 1 to 10.

Images