Programs, methods, information processing devices, systems

Abstract

This program provides a way to obtain user-appropriate answers from a large-scale language model (LLM) system. [Solution] A program to be executed by a computer comprising a processor and a memory unit, wherein the processor executes steps including: a user reception step of receiving first user identification information for identifying a first user; a query reception step S101 of receiving a query relating to a predetermined question; an extraction step S102 of extracting user information associated with a first user based on the first user identification information received in the user reception step and the query received in the query reception step; and a prompt generation step S103 of generating a prompt that will be an input sentence for a large-scale language model based on the query received in the query reception step and the user information extracted in the extraction step.

Description

Technical Field

[0001] This disclosure relates to programs, methods, information processing apparatuses, and systems.

Background Art

[0002] In various information processing tasks, the usefulness of large language models (LLMs) is being recognized. Patent Document 1 discloses a technique for providing a context recognition conversational agent based on a journaling model.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] There is a problem that an answer suitable for a user cannot be obtained from a large language model (LLM) system. Therefore, this disclosure has been made to solve the above problems, and its object is to provide a technique for obtaining an answer suitable for a user from an LLM system.

Means for Solving the Problems

[0005] A program to be executed by a computer having a processor and a memory unit, wherein the processor executes a user reception step of receiving first user identification information for identifying a first user; a query reception step of receiving a query relating to a predetermined question; an extraction step of extracting user information associated with the first user based on the first user identification information received in the user reception step and the query received in the query reception step; and a prompt generation step of generating a prompt that will be an input sentence for a large-scale language model based on the query received in the query reception step and the user information extracted in the extraction step. [Effects of the Invention]

[0006] According to this disclosure, the LLM system can provide answers tailored to the user. [Brief explanation of the drawing]

[0007] [Figure 1] This is a block diagram showing the functional configuration of System 1. [Figure 2] This block diagram shows the functional configuration of Server 10. [Figure 3] This is a block diagram showing the functional configuration of the first user terminal 20. [Figure 4] This diagram shows the data structure of user table 1012. [Figure 5] This diagram shows the data structure of query table 1013. [Figure 6] This diagram shows the data structure of prompt table 1014. [Figure 7] This diagram shows the data structure of the store table 1015. [Figure 8] This is a flowchart showing the operation of the question processing. [Figure 9] This is a flowchart showing the operation of the proposal process. [Figure 10] This is an example screen showing how the question processing works. [Figure 11]A block diagram showing the basic hardware configuration of Computer 90. [Modes for carrying out the invention]

[0008] The embodiments of this disclosure will be described below with reference to the drawings. In all the drawings illustrating the embodiments, common components are denoted by the same reference numerals, and repeated explanations are omitted. The following embodiments are not intended to unduly limit the content of this disclosure as described in the claims. Not all components shown in the embodiments are necessarily essential components of this disclosure. Also, each drawing is a schematic diagram and is not necessarily a strict illustration.

[0009] <Configuration of System 1> System 1 in this disclosure is an information processing system that provides information services for generating prompts to query a Large-Scale Language Model (LLM) system. System 1 comprises a server 10, a first user terminal 20, and an LLM system 50 information processing device, all connected via network N. Figure 1 is a block diagram showing the functional configuration of System 1. Figure 2 is a block diagram showing the functional configuration of server 10. Figure 3 is a block diagram showing the functional configuration of the first user terminal 20.

[0010] Each information processing device consists of a computer equipped with an arithmetic unit and a memory device. The basic hardware configuration of the computer and the basic functional configuration of the computer realized by said hardware configuration will be described later. For each of the server 10, the first user terminal 20, and the LLM system 50, explanations that overlap with the basic hardware configuration and basic functional configuration of the computer described later will be omitted.

[0011] <Server 10 Configuration> Server 10 is an information processing device that provides information services for generating prompts to query a Large-Scale Language Model (LLM) system. Server 10 includes a storage unit 101 and a control unit 104.

[0012] <Configuration of the storage unit 101 of server 10> The storage unit 101 of server 10 includes an application program 1011, a user table 1012, a query table 1013, a prompt table 1014, and a store table 1015.

[0013] The application program 1011 is a program for causing the control unit 104 of server 10 to function as each functional unit. The application program 1011 includes applications such as a web browser application. [[ID=...]]

[0014] The user table 1012 is a table for storing and managing information of member users (hereinafter referred to as users) who use the service. By registering for use of the service, the information of the user is stored in a new record of the user table 1012. Thereby, the user can use the service according to the present disclosure. The user according to the present disclosure is an end user who is the ultimate user (customer) of products, services, systems, applications, etc. The user table 1012 is a table having columns of user ID, user name, and user data with the user ID as the primary key. FIG. 4 is a diagram showing the data structure of the user table 1012.

[0015] The user ID is an item for storing user identification information for identifying a user. The user identification information is an item for which a unique value is set for each user. The user name is an item for storing the name of the user. The user name may be set to any string such as a nickname instead of the actual name. The user data is an item for storing information related to the user associated with the user. The user data includes information related to the physical condition of the user. Information regarding a user's physical condition specifically includes the user's biometric indicators, vital data (e.g., heart rate, blood pressure, body temperature), physiological status (e.g., body fat percentage, height, weight), health status (e.g., presence or absence of disease, allergy information, medical history, etc.), exercise performance (e.g., walking speed, jumping ability, etc.), diet and sleep patterns (e.g., calorie intake, sleep duration, etc.), and other information regarding the user's physical condition and lifestyle, such as gender and age (including information indicating whether or not they are an adult). User data may also include information regarding the user's relationships with other people, dining history, and contact status. User data includes information about the devices owned by the user. Information about a user's devices specifically includes the type of device (e.g., personal computer, smartphone, tablet, wearable device, etc.), the brand and model of the device, the type and version of the operating system, hardware specifications (e.g., CPU type and speed, memory capacity, storage capacity, etc.), the physical condition of the device (e.g., new or used, repair history, etc.), a list of installed software and applications and their versions, and other information regarding settings and usage (e.g., battery life, usage time, communication status, etc.). The term "devices" is not limited to information devices; it also includes means of transportation (e.g., automobiles, bicycles, motorcycles, etc.), home appliances (e.g., refrigerators, washing machines, televisions, etc.), tools and machinery (e.g., power tools, gardening equipment, etc.). In this case, user data may include information such as the type of product, brand and model, year of manufacture and purchase, condition of the product (new or used, current functional status, etc.), product specifications and performance, maintenance and repair history, frequency and method of use, and presence or absence of accessories.

[0016] The control unit 104 of the server 10 may be configured to acquire user data from information devices owned by the user via a communication line such as the Internet, and store it in the user data items of the user table 1012 in association with the user ID. The control unit 104 of the server 10 may be configured to store message documents previously received from users via email, chat services, or other information exchange means in the user data items of the user table 1012. Specifically, the user data does not need to be structured data in which the data name, type, and data value are associated; for example, it may be text such as the subject and body of a message document from an email or chat service. Specifically, if the operator of the information processing system related to this disclosure has traditionally provided user support via email, chat services, etc., the message documents exchanged with the user during such user support can be included in the user data. This allows information such as the user's physical condition and the equipment the user owns, contained in the message documents, to be included in the user data. The user data includes any database such as CSV, Excel, SQL, etc., generated by providing predetermined processing to the message documents. Furthermore, the items in the user data may also include file names, path names, FQDNs (Fully Qualified Domain Names), IP addresses, etc., for referencing any database such as CSV, Excel, SQL, etc. (not shown). In this case, the user data also includes information stored in the referenced database.

[0017] Query table 1013 is a table used to store and manage information about queries related to a question (query information). Query table 1013 is a table with Query ID as its primary key, and contains columns for Query ID, User ID, Query, and Date / Time. Figure 5 shows the data structure of query table 1013.

[0018] The query ID is an item that stores query identification information to identify a query. Query identification information is an item where a unique value is assigned to each query. The User ID is an item that stores user identification information used to identify a user. A query is an item that stores string (text) information related to a question. Specifically, queries about the user's physical condition, queries about the devices the user owns, etc., are stored. For example, queries related to a user's physical condition may include questions such as, "What kind of diet should I follow to lower my blood pressure?", "What kind of exercise is suitable for lowering blood pressure?", and "What methods are preferable for healthy weight loss?". For example, queries related to a user's equipment may include questions such as, "My computer won't turn on, what should I do?" or "My computer applications won't start, what should I do?" The date and time field stores the date and time when a new record was added or updated in query table 1013.

[0019] The prompt table 1014 is a table for storing and managing information about prompts (prompt information) that will be input statements to the LLM system 50. The prompt table 1014 is a table with prompt ID as its primary key, and has columns for prompt ID, user ID, prompt, response result, and date and time. Figure 6 shows the data structure of prompt table 1014.

[0020] The prompt ID is an item that stores prompt identification information to identify a prompt. Each prompt has a unique value assigned to it. The User ID is an item that stores user identification information used to identify a user. The prompt is an item that stores the prompt, which is the input statement for the LLM system 50. Specifically, it stores the string (text) information related to the prompt. The "Response Result" field stores the response to a prompt received from the LLM system 50. Specifically, it stores the string (text) information of the response to the prompt. The date and time field stores the date and time when a new record was added or updated in the prompt table 1014.

[0021] Store table 1015 is a table for storing and managing information about stores (store information). The store table 1015 is a table with store ID as the primary key and contains columns for store ID, user ID, store name, store data, and review data. Figure 7 shows the data structure of the store table 1015.

[0022] The Store ID is an item that stores store identification information used to identify a store. Store identification information is an item with a unique value assigned to each store. The User ID is an item that stores user identification information to identify users associated with a store. Specifically, it stores user identification information for users related to the operation of the store, such as store managers, owners, and employees. The store name field is used to store the store's name. The store name can be any string of characters. Store data is an item that stores information about a store. Store data includes store name, location, industry, products sold, opening and closing times, store size (e.g., sales area, number of floors, etc.), types of products handled and their inventory status, store layout, services offered in the store, staff information (e.g., number of employees, skill level, etc.), sales performance and customer flow at the store, and other marketing information (e.g., special sale days, sale information, etc.). The review data is an item that stores information about reviews submitted by users. Review data includes the poster's ID, email address, posting date and time, rating (e.g., number of stars, score, etc.), post text (specific review or impression), name of the product or service, poster's purchase or usage experience, information about the store or company being reviewed, and ratings and reactions from other users (e.g., number of "likes", comments, etc.).

[0023] <Configuration of the control unit 104 of server 10> The control unit 104 of the server 10 includes a user registration control unit 1041, a question unit 1042, and a suggestion unit 1043. The control unit 104 realizes each functional unit by executing the application program 1011 stored in the storage unit 101.

[0024] The user registration control unit 1041 processes information of users who wish to use the services related to this disclosure and stores it in the user table 1012. Information stored in the user table 1012 is obtained when a user opens a web page operated by the service provider from any information processing terminal, enters information into a designated input form, and sends it to the server 10. The user registration control unit 1041 stores the received information in a new record in the user table 1012, and user registration is completed. As a result, users stored in the user table 1012 can use the service. Prior to the registration of user information in the user table 1012 by the user registration control unit 1041, the service provider may perform a prescribed review and restrict whether or not the user can use the service. The user ID can be any string or number that can identify the user, and may be any string or number desired by the user, or the user registration control unit 1041 may automatically set any string or number.

[0025] The questioning unit 1042 performs the question processing. Details will be described later.

[0026] The proposal unit 1043 executes the proposal processing. Details will be described later.

[0027] <Configuration of the first user terminal 20> The first user terminal 20 is an information processing device operated by a user of the service. The first user terminal 20 may be, for example, a mobile device such as a smartphone or tablet, or a stationary PC (Personal Computer) or laptop PC. It may also be a wearable device such as an HMD (Head Mount Display) or a smartwatch. The first user terminal 20 includes a storage unit 201, a control unit 204, an input device 206, and an output device 208.

[0028] <Configuration of the storage unit 201 of the first user terminal 20> The storage unit 201 of the first user terminal 20 includes a first user ID 2011 and an application program 2012.

[0029] The first user ID 2011 is the user's account ID. The first user transmits the first user ID 2011 from the first user terminal 20 to the server 10. The server 10 identifies the user based on the first user ID 2011 and provides the services related to this disclosure to the user. The first user ID 2011 includes information such as a session ID that is temporarily assigned by the server 10 to identify the user using the first user terminal 20.

[0030] The application program 2012 may be pre-stored in the memory unit 201, or it may be configured to be downloaded from a web server operated by the service provider via a communication interface. Application Program 2012 includes applications such as web browser applications. Application program 2012 includes an interpreted programming language such as JavaScript (registered trademark) that runs on a web browser application stored on the first user terminal 20.

[0031] <Configuration of the control unit 204 of the first user terminal 20> The control unit 204 of the first user terminal 20 includes an input control unit 2041 and an output control unit 2042. The control unit 204 realizes each functional unit by executing the application program 2012 stored in the storage unit 201.

[0032] <Configuration of the input device 206 of the first user terminal 20> The input device 206 of the first user terminal 20 includes a camera 2061, a microphone 2062, a position information sensor 2063, a motion sensor 2064, and a touch device 2065.

[0033] <Configuration of the output device 208 of the first user terminal 20> The output device 208 of the first user terminal 20 includes a display 2081 and a speaker 2082.

[0034] <Configuration of the LLM system 50> The LLM (Large Language Model) system 50 refers to a large artificial intelligence model used in the field of natural language processing (NLP). These models learn from a large amount of text data (web pages, books, articles, etc.) to understand the patterns of the language used by humans and can effectively perform natural language generation (NLG) tasks. The LLM system 50 is used in many NLP tasks such as generating responses to specific questions, automatically generating articles, summarizing text, translation, sentiment analysis, etc. It can also be utilized in various applications such as education, entertainment, customer service, product development, etc. The following exist in the LLM system 50. ·OpenAI ChatGPT ·Google Bard ·Stable Diffusion ·midjourney

[0035] <Operation of system 1> The following describes each process of system 1. Figure 8 is a flowchart showing the operation of the question processing. Figure 9 is a flowchart showing the operation of the proposal process. Figure 10 shows an example screen illustrating the operation of the question processing process.

[0036] <Question Processing> Query processing involves receiving a question (query) from the user, generating a prompt that serves as input to the Large-Scale Language Model (LLM) system based on the query, and obtaining a response to that prompt.

[0037] <Overview of Question Processing> Question processing is a series of processes that involves receiving a question (query) from the user, extracting user data based on user identification information and the query, generating a prompt based on the query and user data, sending the prompt to a large-scale language model (LLM) system, receiving the answer result from the LLM system, storing the answer result, and presenting the answer result to the user. Furthermore, the question processing may be configured to be performed by an operator who receives an inquiry from a customer. For example, the operator may receive a question (query) from a customer (user) and send customer-identifying information as user identification information along with the query to the information processing system related to this disclosure. In this case, the answer result from the Large-Scale Language Model (LLM) is presented to the operator. The operator may also present the presented answer result to the customer by replying to them. Furthermore, the question processing may be configured to be performed by the end user. For example, the question processing may be performed by the end user without the intervention of an operator, and the provided answer results may be presented directly to the end user.

[0038] <Details of Question Processing> The details of the question processing are described below.

[0039] In step S101, the query unit 1042 of the server 10 executes a query reception step to receive a query from the first user regarding a predetermined question. The query unit 1042 of the server 10 executes a user reception step to receive first user identification information to identify the first user. Specifically, the first user operates the input device 206 of the first user terminal 20, runs a browser application or the like, and opens the question page D1 by entering the URL of the web page (question page) for executing the question processing. The control unit 204 of the first user terminal 20 sends a request to the server 10 that includes the first user ID 2011 to open the question page.

[0040] When server 10 receives a request, it generates a question page and sends it to the first user terminal 20. The control unit 204 of the first user terminal 20 displays and presents the question page on the display 2081 of the first user terminal 20. Figure 10 shows an example of a question screen in question processing. The display 2081 of the first user terminal 20 shows the question page D1. The question page D1 includes a query input field D101 for entering text information related to the question, an answer area D102 for displaying text information related to the answer to the question, and a submit button D103. On the question screen, the first user can enter various questions in the query input field D101 and obtain the answer to the question from the answer result displayed in the answer area D102.

[0041] Question page D1 includes a generate button D111, a database construction button D112, and a system settings button D113. By operating the input device 206 of the first user terminal 20, the user can switch the screen displayed on question page D1 by selecting one of the generate button D111, the database construction button D112, or the system settings button D113. Specifically, question page D1 shown in Figure 10 is the prompt generation screen displayed by selecting the generate button D111. By operating the input device 206 of the user terminal 20 and selecting the database construction button D112, the user can display the database construction screen for registering and editing user data in user table 1012. By operating the input device 206 of the user terminal 20 and selecting the system settings button D113, the user can change various settings of the information processing system related to this disclosure.

[0042] Question page D1 includes a product selection input field D121 for selecting the product related to the question, and reference data D122 and D123. The user enters product identification information, such as the product name and product ID of the product or service related to the question, into the product selection input field D121. The product identification information entered into the product selection input field D121 is stored in a user data item in user table 1012, associated with user ID 2011.

[0043] The first user enters a string related to the question into the query input field D101 by operating the input device 206 of the first user terminal 20. The first user selects the send button D103 by operating the input device 206 of the first user terminal 20. The control unit 204 of the first user terminal 20 sends the query entered in the query input field D101 and the first user ID 2011 to the server 10. The question unit 1042 of the server 10 receives and accepts the query and the first user ID 2011. In this disclosure, a configuration in which a query and the first user ID 2011 are received from the first user is disclosed as an example. However, the first user ID 2011 may be entered and received by an operator through an input operation, rather than from the first user terminal 20. For example, the operator enters the first user ID 2011 into a predetermined user identification information input field by operating their own information processing terminal and sends it. The query unit 1042 of the server 10 receives and accepts the first user ID 2011.

[0044] The query reception step performs a step that receives queries regarding the physical condition of the first user. Specifically, the first user can input queries regarding their physical condition into the query input field D101 by operating the input device 206 of the first user terminal 20. Queries regarding physical condition are explained in the query items of the query table 1013. In this case, the query unit 1042 of server 10 receives and accepts a query regarding the physical condition of the first user.

[0045] The query reception step executes a step that receives queries regarding equipment owned by the first user. Specifically, the first user can input queries related to the equipment they own into the query input field D101 by operating the input device 206 of the first user terminal 20. Queries related to owned equipment are explained in the query column of the query table 1013. In this case, the query unit 1042 of server 10 receives and accepts queries regarding equipment owned by the first user.

[0046] In step S102, the query unit 1042 of the server 10 performs an extraction step to extract user information associated with the first user based on the first user identification information received in the user reception step and the query received in the query reception step. The extraction step involves excluding some user information associated with the first user that is not relevant to the query, and extracting some of the user information that is relevant to the query. Specifically, the query unit 1042 of server 10 searches the user ID column in user table 1012 based on the first user ID 2011 and retrieves the user data (first user data) of the first user. Based on the query received in step S101, the query unit 1042 of server 10 extracts information related to the query from the various information about the first user contained in the first user data, and does not extract information unrelated to the query. Specifically, the query unit 1042 of server 10 calculates the similarity between the received query and the information contained in the first user data (which consists of multiple first user data sets) based on algorithms such as cosine similarity, TF-IDF score, and LexRank. This calculates the similarity for each of the multiple first user data sets. The query unit 1042 of server 10 extracts a predetermined number of first user data sets from the multiple first user data sets in order of similarity, or first user data sets with a similarity of a predetermined value or higher, and excludes the other first user data to identify the first user data. Alternatively, the system may be configured to extract first user data related to the query from the multiple first user data sets by referring to past questions, answers, etc. Furthermore, the query unit 1042 of server 10 may receive the input value from the first user terminal 20 in the product selection input field D121 and search for user data in user table 1012 based on that input value. Specifically, it may search for user data containing the product name entered in product selection input field D121 in user table 1012. In other words, the query received in the query reception step includes information indicating a product name, such as the input value in product selection input field D121. For example, if the system receives a query from the first user asking, "What kind of diet should I follow to lower my blood pressure?", it will exclude first-user data that is not highly relevant to the query, such as information about the devices the first user owns, and extract first-user data that is highly relevant to the query, such as the first user's gender, age, blood pressure, diet, calorie intake, medical history, and illness history.

[0047] In step S102, the extraction step performs the step of extracting user information, which includes information about the physical condition of the first user associated with the first user. For example, if the server 10 receives a query from the first user regarding the first user's physical condition, the query unit 1042 of the server 10 extracts first user data related to the first user's physical condition, and does not extract first user data that is not related to the first user's physical condition.

[0048] In step S102, the extraction step performs the step of extracting user information, which includes information about the equipment owned by the first user. For example, if the server 10 receives a query from the first user regarding equipment owned by the first user, the query unit 1042 of the server 10 extracts first user data related to the equipment owned by the first user, and does not extract first user data unrelated to the equipment owned by the first user.

[0049] In step S102, the questioning unit 1042 of the server 10 sends the extracted first user data to the first user terminal 20. The control unit 204 of the first user terminal 20 displays and presents the received first user data in reference data D122 and D123. This allows the first user to explicitly confirm which first user data relating to the first user is referenced to generate the prompt. The first user may also operate the input device 206 of the first user terminal 20 to select any of the reference data D122, D123, etc. and exclude it from the first user data to be included in the prompt. Alternatively, the first user may also operate the input device 206 of the first user terminal 20 to manually add any user data from the user data stored in the user table 1012 in association with the first user to reference data D122 and D123, thereby including it in the first user data. Alternatively, the first user may include arbitrary information in the first user data by operating the input device 206 of the first user terminal 20. This allows the first user to manually generate prompts that serve as input sentences for an appropriate Large-Scale Language Model (LLM). Reference data D122 and D123 may include the filename and pathname of the CSV file related to the user data, the FQDN and IP address of the database from which the user data is referenced, etc.

[0050] In step S103, the query unit 1042 of the server 10 executes a prompt generation step that generates prompts to be input statements for the Large-Scale Language Model (LLM) based on the queries received in the query reception step and the user information extracted in the extraction step. Specifically, the query unit 1042 of server 10 generates a prompt based on the query received in step S101 and the first user data extracted in step S102. For example, the question unit 1042 of server 10 generates a prompt like the following: 〔prompt〕 # Order You are a chatbot that, in response to a user's question, outputs an answer that assists the user, taking into account the following user information. # User Question query # User Information First User Data # Answer text

[0051] For example, if the query unit 1042 of server 10 receives a query from the first user regarding the first user's physical condition, the query unit 1042 of server 10 generates a prompt like the following. 〔prompt〕 # Order You are a chatbot that, in response to a user's question, outputs an answer that assists the user, taking into account the following user information. # Question What kind of diet should I follow to lower my blood pressure? # User Information ·blood pressure: ●●(YMD),●●(YMD),●●(YMD),●●(YMD) ·body weight ●●(YMD),●●(YMD),●●(YMD),●●(YMD) • Meal contents ●●(YMD),●●(YMD),●●(YMD),●●(YMD) Medical history, past medical history ●●(YMD),●●(YMD),●●(YMD),●●(YMD) # Answer text

[0052] The question unit 1042 of server 10 may generate a prompt that includes a string to impose an age restriction on the question, based on the information in the first user data indicating age and whether or not the user is an adult. For example, the question unit 1042 of server 10 may include strings such as "Please provide an answer appropriate for a ●●-year-old." or "Please exclude answers inappropriate for minors." in the prompt. The questioning unit 1042 of server 10 may either include a string in the prompt to restrict the output content based on the first user data, or it may include a string in the prompt to restrict the output content without being based on the first user data. This allows the LLM system 50 to provide appropriate responses based on the first user's age and whether they are an adult or a minor.

[0053] In step S104, the questioning unit 1042 of the server 10 executes a prompt transmission step, which sends the prompt generated in the prompt generation step to an external large-scale language model (LLM) system operated by a different operator than the computer. Specifically, the questioning unit 1042 of the server 10 sends a request including the prompt generated in step S103 to the API endpoint provided by the LLM system 50.

[0054] In step S105, the question unit 1042 of the server 10 performs a response receiving step to receive the response result to the prompt sent in the prompt sending step. The response receiving step performs a step of receiving the response result to the prompt based on user information associated with the first user, without training the large-scale language model (LLM) system. Specifically, the LLM system 50 is configured to output a string (text) in response to a prompt received at the API endpoint. The LLM system 50 sends a response to the server 10 that includes the answer result output based on the received prompt. The question unit 1042 of the server 10 receives and accepts the answer result to the prompt included in the response from the LLM system 50.

[0055] In step S106, the question unit 1042 of the server 10 stores the prompt generated in the prompt generation step and the answer result received in the answer reception step in association with each other. Specifically, the questioning unit 1042 of server 10 stores the first user ID 2011, the prompt generated in step S103, and the answer result received in step S105 in the user ID, prompt, and answer result fields of the prompt table 1014, respectively.

[0056] In step S107, the question unit 1042 of the server 10 executes a proposal step in which it proposes to the first user advice information regarding the first user's response measures to the query received in the query reception step, based on the answer results received in the answer reception step. Specifically, the question unit 1042 of the server 10 sends the string (text) related to the answer result received from the LLM system 50 in step S105 to the first user terminal 20. The control unit 204 of the first user terminal 20 outputs the string (text) related to the answer result to the answer area D102 of the question page D1 displayed on the display 2081 of the first user terminal 20. As a result, the question unit 1042 of the server 10 proposes a text (advice information) regarding advice for the first user's query to the first user, in accordance with the query received in step S101. Furthermore, if the question is processed by an operator, the answer will be presented to the operator. The operator will then send the presented answer to the customer (first user) who submitted the inquiry. This allows the information inquiry service to provide customers with the most appropriate answer to their inquiries.

[0057] <Proposal Processing> The suggestion processing is the process of providing advice (suggestions) regarding store reviews received from third parties to users associated with that store.

[0058] <Overview of Proposal Processing> The suggestion processing is a series of processes that involves receiving reviews about stores, etc., from third parties, extracting store data for the stores, etc. that received the reviews, generating prompts based on the content of the reviews and the store data, sending the prompts to a large-scale language model (LLM) system, receiving the response results from the LLM system, storing the response results, and presenting the user with advice information regarding user response measures based on the response results.

[0059] <Details of proposal processing> The details of the proposal processing are described below.

[0060] In S301, the query reception step executes a query reception step that receives queries regarding store reviews associated with the first user. Specifically, a user who posts a review (hereinafter referred to as "poster") opens a website to post a review about a store using a web browser or similar device by operating the poster's information processing terminal. The website to which the review is posted does not have to be a website operated by the business providing the information service related to this disclosure; it may be an external website operated by another business. The poster uses their information processing terminal to input information about a store, such as reviews, into an input form on the website and submits it. This posts the review to the website where reviews about stores are posted. The suggestion unit 1043 of the server 10 accesses the website and obtains the reviews posted by the poster and store identification information to identify the store to which the reviews were posted. The suggestion unit 1043 of the server 10 accepts the obtained reviews as queries. The suggestion unit 1043 of server 10 stores the acquired reviews in the review data field of the record identified by referring to the store ID in the store table 1015 based on the acquired store identification information. Note that the record may also be identified based on the store name instead of the store identification information.

[0061] In step S302, the suggestion unit 1043 of the server 10 performs an extraction step to extract store information associated with the store based on the acquired reviews. Based on the store identification information obtained in step S301, the suggestion unit 1043 of server 10 searches for the store ID in the store table 1015 to obtain the store data and user ID (user identification information of the user associated with the store, hereinafter referred to as the first user) of the store identified by the store identification information. Note that the record may also be identified based on the store name instead of the store identification information. Based on the reviews obtained in step S301, the suggestion unit 1043 of server 10 extracts store data related to the reviews from various information about the reviews included in the store data, and does not extract store data unrelated to the reviews. Specifically, the suggestion unit 1043 of server 10 calculates the similarity between the received reviews and the information contained in the store data (multiple store data) based on algorithms such as cosine similarity, TF-IDF score, and LexRank. This calculates the similarity for each of the multiple store data. The suggestion unit 1043 of server 10 extracts a predetermined number of store data or store data with a similarity of a predetermined value or higher from the multiple store data in order of similarity, and excludes the other store data to identify the store data. Alternatively, the system may be configured to extract store data related to the reviews from the multiple store data by referring to past questions, answers, etc. For example, if a customer leaves a review stating that "the menu selection was limited," the system will extract store data that is highly relevant to the review, including information such as the products sold at the store, the types of products it carries and their availability, the services offered in the store, and other marketing-related information.

[0062] In step S303, the suggestion unit 1043 of the server 10 executes a prompt generation step that generates a prompt to be an input sentence for the Large-Scale Language Model (LLM) based on the reviews received in the query reception step and the store information extracted in the extraction step. Specifically, the suggestion unit 1043 of server 10 generates a prompt based on the customer reviews received in step S301 and the store data extracted in step S302. For example, the suggestion unit 1043 of server 10 generates a prompt like the following: 〔prompt〕 # Order You are a chatbot that, in response to customer reviews about a store, outputs a reply that takes the following store information into consideration to help the store implement its response strategies to those reviews. # Store reviews Customer reviews # Store Information Store data # Answer text

[0063] For example, when the suggestion unit 1043 of server 10 receives a review from a poster, the suggestion unit 1043 of server 10 generates a prompt like the following: 〔prompt〕 # Order You are a chatbot that, in response to customer reviews about a store, outputs a reply that supports the store's response strategy, taking into account the following store information. # Store reviews The menu selection was limited. # Store Information • Products for sale: ●●●●,●●●●,●●●●,●●●●,●●●●,●●●● • Products handled ●●●●,●●●●,●●●●,●●●●,●●●●,●●●● • Stock status ●●●●,●●●●,●●●●,●●●●,●●●●,●●●● • Services offered within the store # Answer text

[0064] Steps S304 to S306 are the same as steps S104 to S106 in the question processing, so their explanation is omitted. The suggestion unit 1043 of the server 10 stores the user identification information of the user associated with the store, the prompt generated in step S303, and the answer result received in step S305 in the user ID, prompt, and answer result fields of the prompt table 1014.

[0065] In step S307, the suggestion unit 1043 of the server 10 executes a suggestion step in which, based on the response results received in the response reception step, it suggests to the first user advice information regarding the first user's response measures for the query received in the query reception step. Specifically, the suggestion unit 1043 of the server 10 sends the string (text) related to the response result received from the LLM system 50 in step S305 to the first user terminal 20. For example, based on the user identification information of the first user associated with the store identified in step S101, the suggestion unit 1043 of the server 10 searches the user ID item in the user table 1012 and obtains contact information for contacting the first user, such as the first user's email address and chat ID. The suggestion unit 1043 of the server 10 sends the string (text) related to the response result received from the LLM system 50 to the contact information. The control unit 204 of the first user terminal 20 outputs the string (text) related to the received response result to the display 2081 of the first user terminal 20. As a result, the suggestion unit 1043 of the server 10 proposes to the first user associated with the store, in accordance with the word-of-mouth received in step S301, a text (advice information) regarding advice on the first user's response measures.

[0066] <Basic Computer Hardware Configuration> Figure 11 is a block diagram showing the basic hardware configuration of computer 90. Computer 90 comprises at least a processor 901, main memory 902, auxiliary memory 903, and a communication interface IF991. These are electrically connected to each other by a communication bus 921.

[0067] The processor 901 is hardware for executing the instruction set written in a program. The processor 901 consists of an arithmetic unit, registers, peripheral circuits, etc.

[0068] Main memory 902 is used to temporarily store programs and data processed by programs, etc. For example, it is a volatile memory such as DRAM (Dynamic Random Access Memory).

[0069] Auxiliary storage device 903 refers to a storage device for saving data and programs. Examples include flash memory, HDD (Hard Disc Drive), magneto-optical disk, CD-ROM, DVD-ROM, and semiconductor memory.

[0070] The IF991 communication interface is an interface for inputting and outputting signals for communication with other computers via a network using wired or wireless communication standards. A network consists of various mobile communication systems, such as the internet, LANs, and wireless base stations. For example, a network includes 3G, 4G, and 5G mobile communication systems, LTE (Long Term Evolution), and wireless networks that can connect to the internet via designated access points (e.g., Wi-Fi®). When connecting wirelessly, communication protocols include, for example, Z-Wave®, ZigBee®, and Bluetooth®. When connecting via a wired connection, the network also includes connections made directly via USB (Universal Serial Bus) cables, etc.

[0071] Furthermore, by distributing all or part of each hardware configuration across multiple computers 90 and connecting them to each other via a network, a computer 90 can be virtually realized. Thus, the concept of computer 90 includes not only a computer 90 housed in a single enclosure or case, but also a virtualized computer system.

[0072] <Basic Functional Configuration of Computer 90> The functional configuration of the computer realized by the basic hardware configuration of computer 90 (Figure 11) is described below. The computer comprises at least one functional unit: a control unit, a memory unit, and a communication unit.

[0073] Furthermore, the functional units of computer 90 can also be realized by distributing all or part of each functional unit across multiple computers 90 interconnected via a network. The concept of computer 90 includes not only a single computer 90 but also a virtualized computer system.

[0074] The control unit is realized when the processor 901 reads various programs stored in the auxiliary storage device 903, loads them into the main memory device 902, and executes processing according to those programs. The control unit can realize various functional units that perform information processing depending on the type of program. In this way, the computer is realized as an information processing device that performs information processing.

[0075] The memory unit is implemented by the main memory 902 and the auxiliary memory 903. The memory unit stores data, various programs, and various databases. The processor 901 can also reserve memory areas corresponding to the memory unit in the main memory 902 or the auxiliary memory 903 according to the program. The control unit can also cause the processor 901 to perform operations such as adding, updating, and deleting data stored in the memory unit according to the various programs.

[0076] A database, specifically a relational database, is used to manage and link together tabular data sets called masters, which are structurally defined by rows and columns. In a database, tables are called tables, masters are called masters, the columns of tables are called columns, and the rows of tables are called records. In a relational database, relationships can be established and linked between tables and masters. Typically, each table and master has a primary key column to uniquely identify records, but setting a primary key column is not mandatory. The control unit can instruct the processor 901 to add, delete, or update records in specific tables and masters stored in the memory unit, according to various programs. Furthermore, by storing data, various programs, and various databases in the memory unit, the information processing device and information processing system related to this disclosure can be considered to have been manufactured.

[0077] Furthermore, the databases and masters in this disclosure may include any data structures (lists, dictionaries, associative arrays, objects, etc.) in which information is structurally defined. Data structures also include data that can be considered as data structures by combining data with functions, classes, methods, etc., written in any programming language.

[0078] The communication unit is implemented by the communication IF991. The communication unit provides the functionality to communicate with other computers 90 via the network. The communication unit can receive information transmitted from other computers 90 and input it to the control unit. The control unit can cause the processor 901 to perform information processing on the received information according to various programs. The communication unit can also transmit information output from the control unit to other computers 90.

[0079] <Note> The details described in each of the above embodiments are noted below.

[0080] (Note 1) A program to be executed by a computer having a processor and a memory unit, wherein the processor executes a user reception step (S101) in which it receives first user identification information for identifying a first user; a query reception step (S101) in which it receives a query relating to a predetermined question; an extraction step (S102) in which it extracts user information associated with the first user based on the first user identification information received in the user reception step and the query received in the query reception step; and a prompt generation step (S103) in which it generates a prompt that will be an input statement for a large-scale language model (LLM) based on the query received in the query reception step and the user information extracted in the extraction step. This allows user information to be included in prompts for querying the LLM system. By using these prompts, the LLM system can provide answers tailored to the user. For example, it can resolve the user's problem in a shorter amount of time.

[0081] (Note 2) The extraction step (S102) is a step in the program described in Appendix 1, which extracts a portion of the user information related to the query from the information associated with the first user, excluding a portion of the user information that is not related to the query. This allows the prompt for querying the LLM system to include user information relevant to the query. By using this prompt, even when there is a large amount of user information, only the user information suitable for the query can be extracted and queried to the LLM system.

[0082] (Note 3) The program as described in Appendix 1 or 2, wherein the extraction step (S102) is a step of extracting user information, which includes information about the physical condition of the first user associated with the first user. This allows users to receive prompts from the LLM system that provide appropriate answers to questions about their health status. For example, they can receive answers that are more relevant to their specific problems.

[0083] (Note 4) The query reception step (S101) is a step in the program described in Appendix 3 that receives queries regarding the physical condition of the first user. This allows users to receive prompts from the LLM system that provide appropriate answers to questions about their health status. For example, they can receive answers that are more relevant to their specific problems.

[0084] (Note 5) The program as described in Appendix 1 or 2, wherein the extraction step (S102) is a step of extracting user information, including information about equipment owned by the first user. This allows users to receive prompts that provide answers more relevant to their specific problems when they ask questions about their own PCs, smartphones, and other information processing devices.

[0085] (Note 6) The query reception step (S101) is a step in the program described in Appendix 5 that receives queries regarding equipment owned by the first user. This allows users to receive prompts that provide answers more relevant to their specific problems when they ask questions about their own PCs, smartphones, and other information processing devices.

[0086] (Note 7) The query reception step (S301) is a step in the program described in Appendix 1 or 2 that receives queries regarding store reviews associated with a first user. This allows users to receive prompts to ask questions to the LLM system regarding policies on how to respond to reviews made by third parties about stores or other businesses they manage or are involved with.

[0087] (Note 8) A program as described in any of Appendix 1 to 7, wherein the processor performs a prompt transmission step (S104) in which it transmits a prompt generated in a prompt generation step to an external large-scale language model (LLM) system operated by a different entity than the computer, and a response reception step (S105) in which it receives a response result to the prompt transmitted in the prompt transmission step. This allows users to obtain answers tailored to their needs from the LLM system. For example, it can help resolve user problems in a shorter amount of time.

[0088] (Note 9) The program as described in Appendix 8, wherein the response receiving step (S105) is a step of receiving a response result to a prompt based on user information associated with the first user, without training a large-scale language model (LLM) system. This allows users to obtain user-appropriate answers from the LLM system without having to train the LLM system with user information for each individual user. For example, it saves the effort of training a model for each user and reduces the cost of storing the model.

[0089] (Note 10) The program described in Appendix 8 or 9, wherein the processor performs a suggestion step (S107) in which it suggests to the first user advice information regarding the first user's response measures to the query received in the query reception step, based on the response result received in the response reception step. This allows users to receive advice from the LLM system regarding reviews made by third parties about stores or businesses they manage or are involved with. For example, users can resolve issues related to reviews made about their stores in a shorter amount of time.

[0090] (Note 11) A method to be performed on a computer comprising a processor and memory, wherein the processor performs all steps performed in any of the inventions described in Appendix 1 to Appendix 10. This allows user information to be included in prompts for querying the LLM system. By using these prompts, the LLM system can provide answers tailored to the user. For example, it can resolve the user's problem in a shorter amount of time.

[0091] (Note 12) An information processing apparatus comprising a control unit and a storage unit, wherein the control unit performs all steps performed in the invention according to any of the appendices 1 to 10. This allows user information to be included in prompts for querying the LLM system. By using these prompts, the LLM system can provide answers tailored to the user. For example, it can resolve the user's problem in a shorter amount of time.

[0092] (Note 13) A system comprising means for performing all steps performed in any of the inventions described in Appendix 1 to Appendix 10. This allows user information to be included in prompts for querying the LLM system. By using these prompts, the LLM system can provide answers tailored to the user. For example, it can resolve the user's problem in a shorter amount of time. [Explanation of symbols]

[0093] 1 System, 10 Servers, 101 Storage Unit, 104 Control Unit, 106 Input Device, 108 Output Device, 20 First User Terminal, 201 Storage Unit, 204 Control Unit, 206 Input Device, 208 Output Device, 50 LLM System, 501 Storage Unit, 504 Control Unit, 506 Input Device, 508 Output Device

Claims

1. A program to be executed by a computer having a processor and a memory unit, The aforementioned processor, A user reception step that receives first user identification information to identify the first user, A query reception step that accepts queries regarding a specified question, An extraction step for extracting user information associated with the first user based on the first user identification information received in the user reception step and the query received in the query reception step, A prompt generation step generates a prompt that will be an input statement for a Large-Scale Language Model (LLM) based on the query received in the query reception step and the user information extracted in the extraction step, A program that executes this task.

2. The extraction step is to extract a portion of the user information related to the query from the information associated with the first user, excluding a portion of the user information that is not related to the query. The program according to claim 1.

3. The extraction step is a step of extracting user information, which includes information relating to the physical condition of the first user associated with the first user. The program according to claim 1 or 2.

4. The query reception step is a step of receiving the query regarding the physical condition of the first user. The program according to claim 3.

5. The extraction step is a step of extracting user information, which includes information about the equipment owned by the first user. The program according to claim 1 or 2.

6. The query reception step is a step of receiving the query relating to the equipment owned by the first user. The program according to claim 5.

7. The query reception step is a step of receiving queries regarding reviews of stores associated with the first user. The program according to claim 1 or 2.

8. The aforementioned processor, A prompt transmission step that transmits the prompt generated in the prompt generation step to an external Large-Scale Language Model (LLM) system operated by a different operator than the aforementioned computer, A response receiving step, which receives the response result to the prompt sent in the prompt sending step, Execute A program according to any one of claims 1 to 7.

9. The response receiving step is a step of receiving the response result to the prompt based on the user information associated with the first user, without training the large-scale language model (LLM) system. The program according to claim 8.

10. The aforementioned processor, A proposal step in which, based on the response results received in the response receiving step, advice information regarding the first user's response measures for the query received in the query receiving step is proposed to the first user, Execute The program according to claim 8 or 9.

11. A method to be performed on a computer comprising a processor and memory, wherein the processor performs all steps performed in the invention according to any one of claims 1 to 10.

12. An information processing apparatus comprising a control unit and a storage unit, wherein the control unit performs all steps performed in the invention according to any one of claims 1 to 10.

13. A system comprising means for performing all steps performed in the invention according to any one of claims 1 to 10.

Images