Program, method, information processing device, and system

Abstract

To solve a problem that a user cannot obtain an appropriate reply from a large-scale language model (LLM) system.SOLUTION: A program is to be executed by a computer having a processor and a storage unit. The program causes the processor to execute: a user reception step of receiving first user identification information for identifying a first user; a query reception step of receiving a query related to a predetermined question; an extraction step of extracting user information associated with the first user based on the first user identification information received in the user reception step and the query received in the query reception step; and a prompt generation step of generating a prompt that serves as an input sentence for a large-scale language model based on the query received in the query reception step and the user information extracted in the extraction step.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] The present disclosure relates to a program, a method, an information processing device, and a system. [Background technology]

[0002] The usefulness of large-scale language models (LLMs) is becoming increasingly recognized in various information processing tasks. Patent Document 1 discloses a technique for providing a context-aware conversational agent based on a journaling model. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-121360 Summary of the Invention [Problem to be solved by the invention]

[0004] There is a problem that large-scale language model (LLM) systems are unable to provide answers that are appropriate for the user. Therefore, the present disclosure has been made to solve the above problem, and its purpose is to provide a technology for obtaining a response appropriate for a user from an LLM system. [Means for solving the problem]

[0005] A program to be executed by a computer having a processor and a memory unit, the program executing the following steps: a user reception step in which the processor receives first user identification information for identifying a first user; a query reception step in which the processor receives a query related to a predetermined question; an extraction step in which the processor extracts user information associated with the first user based on the first user identification information received in the user reception step and the query received in the query reception step; and a prompt generation step in which the processor generates a prompt that serves as an input sentence for a large-scale language model based on the query received in the query reception step and the user information extracted in the extraction step. [Effects of the Invention]

[0006] According to the present disclosure, an LLM system can provide an answer that is appropriate for the user. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 2 is a block diagram showing the functional configuration of the system 1. [Figure 2] FIG. 2 is a block diagram showing the functional configuration of the server 10. [Figure 3] 2 is a block diagram showing the functional configuration of a first user terminal 20. FIG. [Figure 4] FIG. 10 is a diagram showing the data structure of a user table 1012. [Figure 5] FIG. 10 is a diagram showing the data structure of a query table 1013. [Figure 6] FIG. 10 is a diagram showing the data structure of a prompt table 1014. [Figure 7] FIG. 10 is a diagram showing the data structure of a store table 1015. [Figure 8] 10 is a flowchart showing the operation of question processing. [Figure 9] 10 is a flowchart showing the operation of a proposal process. [Figure 10] 10 is a screen example showing the operation of question processing. [Figure 11]FIG. 2 is a block diagram showing the basic hardware configuration of a computer 90. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In all drawings describing the embodiments, common components are designated by the same reference numerals, and repeated description will be omitted. Note that the following embodiments do not unduly limit the content of the present disclosure described in the claims. Furthermore, not all components shown in the embodiments are necessarily essential components of the present disclosure. Furthermore, each drawing is a schematic diagram and is not necessarily a precise illustration.

[0009] <System 1 Configuration> The system 1 in this disclosure is an information processing system that provides an information service that generates prompts for querying a large-scale language model (LLM) system. The system 1 includes a server 10, a first user terminal 20, and an information processing device of an LLM system 50, which are connected via a network N. FIG. 1 is a block diagram showing the functional configuration of the system 1. As shown in FIG. FIG. 2 is a block diagram showing the functional configuration of the server 10. As shown in FIG. FIG. 3 is a block diagram showing the functional configuration of the first user terminal 20. As shown in FIG.

[0010] Each information processing device is configured by a computer equipped with an arithmetic unit and a storage device. The basic hardware configuration of the computer and the basic functional configuration of the computer realized by the hardware configuration will be described later. For each of the server 10, the first user terminal 20, and the LLM system 50, descriptions that overlap with the basic hardware configuration and basic functional configuration of the computer will be omitted.

[0011] <Server 10 configuration> The server 10 is an information processing device that provides an information service that generates prompts for querying a large-scale language model (LLM) system. The server 10 includes a storage unit 101 and a control unit 104 .

[0012] <Configuration of the storage unit 101 of the server 10> The storage unit 101 of the server 10 includes an application program 1011 , a user table 1012 , a query table 1013 , a prompt table 1014 , and a store table 1015 .

[0013] The application program 1011 is a program for causing the control unit 104 of the server 10 to function as each functional unit. Application programs 1011 include applications such as a web browser application.

[0014] User table 1012 is a table that stores and manages information about member users (hereinafter referred to as users) who use the service. When a user registers to use the service, the user's information is stored in a new record in user table 1012. This allows the user to use the service according to the present disclosure. A user according to the present disclosure is an end user who is the final user (customer) of a product, service, system, application, etc. The user table 1012 is a table having columns of user ID, user name, and user data, with the user ID as the primary key. FIG. 4 is a diagram showing the data structure of the user table 1012. As shown in FIG.

[0015] The user ID is an item that stores user identification information for identifying a user. The user identification information is an item that is set with a unique value for each user. The user name is an item for storing the name of the user. The user name may be set to any character string such as a nickname instead of a name. User data is an item that stores information about a user that is associated with the user. The user data includes information about the user's physical condition. Specifically, information related to the user's physical condition includes the user's biometric indicators, vital data (e.g., heart rate, blood pressure, body temperature, etc.), physiological state (e.g., body fat percentage, height, weight, etc.), health condition (e.g., presence or absence of illness, allergy information, medical history of illness, etc.), exercise performance (e.g., walking speed, jumping power, etc.), dietary and sleep status (e.g., calorie intake, sleep duration, etc.), and other information related to the user's physical condition and lifestyle, such as gender, age (including information indicating whether the user is an adult), etc. User data may also include information related to the user's friendships with other people, dining history, and contact status. The user data includes information about the devices owned by the user. Information about devices owned by users specifically includes the type of device (e.g., PC, smartphone, tablet, wearable device, etc.), device brand and model, operating system type and version, hardware specifications (e.g., CPU type and speed, memory capacity, storage capacity, etc.), the physical condition of the device (e.g., new or used, repair history, etc.), a list of installed software and applications and their versions, and other information about settings and usage (e.g., battery life, usage time, communication status, etc.). The devices are not limited to information devices, but also include means of transportation (e.g., automobiles, bicycles, motorcycles, etc.), home appliances (e.g., refrigerators, washing machines, televisions, etc.), tools and machines (e.g., power tools, gardening tools, etc.), etc. In this case, the user data may include information such as the product type, brand and model, year of manufacture and year of purchase, product condition (new or used, current functional state, etc.), product specifications and performance, maintenance history and repair history, frequency and method of use, and whether or not accessories are included.

[0016] The control unit 104 of the server 10 may be configured to acquire user data from an information device owned by the user via a communication line such as the Internet, and store the data in the user data item of the user table 1012 in association with the user ID. The control unit 104 of the server 10 may be configured to store, in the user data item of the user table 1012, message documents previously received from users via information exchange means such as e-mail or chat service. Specifically, the user data does not need to be structured data in which the name, type and value of the data are associated with each other, and may be, for example, text such as the subject and body of a message document in e-mail, chat service or the like. Specifically, if the operator of the information processing system according to the present disclosure has traditionally provided user support via email, chat services, or the like, the user data can include message documents exchanged with the user during such user support. This allows information about the user's physical condition and the devices the user owns, which are contained in the message documents, to be included in the user data. The user data can also include any database, such as CSV, Excel, or SQL, generated by performing a predetermined process on the message documents. The user data items can also include file names, path names, FQDNs (Full Qualified Domain Names), IP addresses, and the like, for referencing any database, such as CSV, Excel, or SQL (not shown). In this case, the user data also includes information stored in the referenced database.

[0017] The query table 1013 is a table for storing and managing information about queries related to questions (query information). The query table 1013 is a table having a query ID as a primary key and columns of a query ID, a user ID, a query, and a date and time. FIG. 5 is a diagram showing the data structure of the query table 1013.

[0018] The query ID is an item that stores query identification information for identifying a query. The query identification information is an item that has a unique value set for each piece of query information. The user ID is an item for storing user identification information for identifying a user. The query is an item that stores character string (text) information related to a question. Specifically, a query related to the user's physical condition, a query related to a device owned by the user, etc. are stored. For example, queries regarding a user's physical condition include, "What kind of diet should I follow to lower my blood pressure?", "What kind of exercise is suitable to lower my blood pressure?", and "What is the best method to lose weight in a healthy way?" For example, queries regarding devices owned by users include "My PC won't turn on, what should I do?" and "My PC application won't start, what should I do?". The date and time is an item for storing the date and time when a record is newly stored or updated in the query table 1013.

[0019] The prompt table 1014 is a table for storing and managing information (prompt information) relating to prompts that are input statements to the LLM system 50. The prompt table 1014 is a table having a prompt ID as a primary key and columns of prompt ID, user ID, prompt, answer result, and date and time. FIG. 6 is a diagram showing the data structure of the prompt table 1014.

[0020] The prompt ID is an item that stores prompt identification information for identifying a prompt. The prompt identification information is an item that has a unique value set for each piece of prompt information. The user ID is an item for storing user identification information for identifying a user. The prompt is an item that stores a prompt that serves as an input sentence to the LLM system 50. Specifically, character string (text) information related to the prompt is stored. The answer result is an item for storing the answer result to the prompt received from the LLM system 50. Specifically, character string (text) information related to the answer sentence to the prompt is stored. The date and time is an item for storing the date and time when a new record is stored or updated in the prompt table 1014.

[0021] The store table 1015 is a table for storing and managing information about stores (store information). The store table 1015 is a table having a store ID as a primary key and columns of store ID, user ID, store name, store data, and word-of-mouth data. FIG. 7 is a diagram showing the data structure of the store table 1015.

[0022] The store ID is an item for storing store identification information for identifying a store. The store identification information is an item for which a unique value is set for each store information. The user ID is an item for storing user identification information for identifying a user associated with a store. Specifically, the user identification information of a user related to the operation of the store, such as the store manager, owner, or employee, is stored. The store name is an item for storing the name of the store. Any character string can be set as the store name. The store data is an item for storing store information. Store data includes the store name, store location, industry, products sold, store opening and closing times, store size (e.g., sales floor area, number of floors, etc.), types of products sold and their inventory status, store layout, services provided in the store, staff information (e.g., number of employees, skill level, etc.), store sales status and customer flow status, and other marketing information (e.g., sale days, sale information, etc.). The word-of-mouth data is an item for storing information about word-of-mouth reviews by posters. Word-of-mouth data includes the poster's ID, email address, posting date and time, rating value (e.g., number of stars, score, etc.), posted text (specific review or impression), name of the product or service in question, the poster's purchasing or usage experience, information about the store or company that is the subject of the post, and ratings and reactions from other users (e.g., number of "likes," comments, etc.).

[0023] <Configuration of the control unit 104 of the server 10> The control unit 104 of the server 10 includes a user registration control unit 1041, a question unit 1042, and a suggestion unit 1043. The control unit 104 executes an application program 1011 stored in the storage unit 101, thereby realizing each functional unit.

[0024] The user registration control unit 1041 performs processing to store information about users who wish to use the service according to the present disclosure in the user table 1012. The information stored in the user table 1012 is generated when a user opens a web page operated by a service provider from any information processing terminal, enters information into a predetermined input form, and transmits the information to the server 10. The user registration control unit 1041 stores the received information in a new record in the user table 1012, completing the user registration. This allows the user stored in the user table 1012 to use the service. Before the user registration control unit 1041 registers the user information in the user table 1012, the service provider may conduct a predetermined examination to restrict whether or not the user is permitted to use the service. The user ID may be any character string or number that can identify the user, any character string or number desired by the user, or may be automatically set by the user registration control unit 1041.

[0025] The interrogator 1042 executes interrogation processing, the details of which will be described later.

[0026] The proposing unit 1043 executes a proposing process, the details of which will be described later.

[0027] <Configuration of First User Terminal 20> The first user terminal 20 is an information processing device operated by a user who uses a service. The first user terminal 20 may be, for example, a mobile terminal such as a smartphone or tablet, a stationary personal computer (PC) or a laptop PC, or a wearable terminal such as a head mounted display (HMD) or a wristwatch terminal. The first user terminal 20 includes a storage unit 201 , a control unit 204 , an input device 206 , and an output device 208 .

[0028] <Configuration of the storage unit 201 of the first user terminal 20> The storage unit 201 of the first user terminal 20 includes a first user ID 2011 and an application program 2012 .

[0029] The first user ID 2011 is a user's account ID. The first user transmits the first user ID 2011 from the first user terminal 20 to the server 10. The server 10 identifies the user based on the first user ID 2011 and provides the user with the service according to the present disclosure. The first user ID 2011 includes information such as a session ID temporarily assigned by the server 10 to identify the user using the first user terminal 20.

[0030] The application program 2012 may be stored in advance in the storage unit 201, or may be configured to be downloaded from a web server or the like operated by a service provider via a communication IF. The application programs 2012 include applications such as a web browser application. The application program 2012 includes an interpreted programming language such as JavaScript (registered trademark) that runs on a web browser application stored on the first user terminal 20.

[0031] <Configuration of the control unit 204 of the first user terminal 20> The control unit 204 of the first user terminal 20 includes an input control unit 2041 and an output control unit 2042. The control unit 204 realizes each functional unit by executing the application program 2012 stored in the storage unit 201.

[0032] <Configuration of the input device 206 of the first user terminal 20> The input device 206 of the first user terminal 20 includes a camera 2061, a microphone 2062, a position information sensor 2063, a motion sensor 2064, and a touch device 2065.

[0033] <Configuration of the output device 208 of the first user terminal 20> The output device 208 of the first user terminal 20 includes a display 2081 and a speaker 2082.

[0034] <Configuration of the LLM system 50> The LLM (Large Language Model) system 50 refers to a large-scale artificial intelligence model used in the field of natural language processing (NLP). These models learn a large amount of text data (web pages, books, articles, etc.) to understand the patterns of the language used by humans and can effectively perform natural language generation (NLG) tasks. The LLM system 50 is used in many NLP tasks such as generating responses to specific questions, automatically generating articles, summarizing text, translation, sentiment analysis, etc. It can also be utilized in various applications such as education, entertainment, customer service, product development, etc. The following exist in the LLM system 50. ·OpenAI ChatGPT ·Google Bard ·Stable Diffusion ·midjourney

[0035] <Operation of system 1> The following describes each process of system 1. Figure 8 is a flowchart showing the operation of the question processing. FIG. 9 is a flowchart showing the operation of the proposal process. FIG. 10 is an example of a screen showing the operation of question processing.

[0036] <Question Processing> Question processing is a process for accepting a question (query) from a user, generating a prompt based on the query as an input sentence to a large-scale language model (LLM) system, and obtaining an answer to the prompt.

[0037] <Question processing overview> Question processing is a series of processes that accept a question (query) from a user, extract user data based on the user identification information and the query, generate a prompt based on the query and user data, send the prompt to a large-scale language model (LLM) system, receive an answer result from the LLM system, store the answer result, and present the answer result to the user. The question processing may be performed by an operator who receives an inquiry from a customer. For example, the operator may receive a question (query) from a customer (user) and transmit information identifying the customer as user identification information along with the query to the information processing system according to the present disclosure. In this case, the answer result from the large-scale language model (LLM) is presented to the operator. The operator may present the presented answer result by replying to the customer. Furthermore, the question processing may be configured to be executed by the end user. For example, the question processing may be executed by the end user without the intervention of an operator, and the presented answer results may be presented directly to the end user.

[0038] <Question processing details> The question processing will be described in detail below.

[0039] In step S101, the interrogator 1042 of the server 10 executes a query receiving step of receiving a query regarding a predetermined question from a first user. The interrogator 1042 of the server 10 executes a user receiving step of receiving first user identification information for identifying the first user. Specifically, the first user operates the input device 206 of the first user terminal 20 to execute a browser application or the like, and opens the question page D1 by inputting the URL of a web page (question page) for executing the question processing. The control unit 204 of the first user terminal 20 transmits a request including the first user ID 2011 to the server 10 to open the question page.

[0040] When the server 10 receives the request, it generates a question page and transmits it to the first user terminal 20. The control unit 204 of the first user terminal 20 displays the question page on the display 2081 of the first user terminal 20 for presentation. 10 is an example of a question screen in question processing. A question page D1 is displayed on the display 2081 of the first user terminal 20. The question page D1 includes a query input field D101 for inputting character string (text) information related to the question, an answer area D102 for displaying character string (text) information related to the answer result to the question, and a send button D103. On the question screen, the first user can input various questions into the query input field D101 and obtain answers to the questions from the answer results displayed in the answer area D102.

[0041] The question page D1 includes a generate button D111, a DB construction button D112, and a system settings button D113. The user can switch the screen displayed on the question page D1 by operating the input device 206 of the first user terminal 20 to select one of the generate button D111, the DB construction button D112, and the system settings button D113. Specifically, the question page D1 shown in FIG. 10 is a prompt generation screen displayed by selecting the generate button D111. The user can display a DB construction screen for registering and editing user data in the user table 1012 by operating the input device 206 of the user terminal 20 and selecting the system settings button D113. The user can change various setting values ​​of the information processing system according to the present disclosure by operating the input device 206 of the user terminal 20 and selecting the system settings button D113.

[0042] The question page D1 includes a product selection input field D121 for selecting a target product related to the question, and reference data D122 and D123. The user inputs product identification information for identifying the product, such as the product name and product ID of the product, service, etc. related to the question, into the product selection input field D121. The product identification information input into the product selection input field D121 is associated with the user ID 2011 and stored in the user data item of the user table 1012.

[0043] The first user operates the input device 206 of the first user terminal 20 to input a character string related to a question into the query input field D101. The first user operates the input device 206 of the first user terminal 20 to select the send button D103. The control unit 204 of the first user terminal 20 transmits the query and first user ID 2011 input into the query input field D101 to the server 10. The question unit 1042 of the server 10 receives and accepts the query and first user ID 2011. Although the present disclosure has disclosed an example of a configuration in which a query and first user ID 2011 are received from the first user, the first user ID 2011 may be input, received, and accepted by an input operation by an operator, rather than from the first user terminal 20. For example, the operator operates his or her own information processing terminal to input the first user ID 2011 in a predetermined user identification information input field and transmit it. The interrogation unit 1042 of the server 10 receives and accepts the first user ID 2011.

[0044] The query receiving step includes receiving a query regarding the physical condition of the first user. Specifically, the first user can input a query about the first user's physical condition into the query input field D101 by operating the input device 206 of the first user terminal 20. Queries about physical conditions are described in the query section of the query table 1013. In this case, the interrogator 1042 of the server 10 receives and accepts the query regarding the physical condition of the first user.

[0045] The query receiving step includes a step of receiving a query regarding a device owned by the first user. Specifically, the first user can input a query related to a device owned by the first user into the query input field D101 by operating the input device 206 of the first user terminal 20. Queries related to owned devices are described in the query section of the query table 1013. In this case, the interrogator 1042 of the server 10 receives and accepts a query regarding the device owned by the first user.

[0046] In step S102, the questioning unit 1042 of the server 10 executes an extraction step of extracting user information associated with the first user based on the first user identification information received in the user reception step and the query received in the query reception step. The extracting step includes extracting a portion of user information related to the query from the information associated with the first user, excluding a portion of user information that is not related to the query. Specifically, the interrogation unit 1042 of the server 10 searches the user ID item in the user table 1012 based on the first user ID 2011, and acquires the user data of the first user (first user data). Based on the query received in step S101, the interrogation unit 1042 of the server 10 extracts information related to the query from various information related to the first user included in the first user data, and does not extract information unrelated to the query. Specifically, the interrogation unit 1042 of the server 10 calculates the similarity between the query and the plurality of first user data based on an algorithm such as cosine similarity, TF-IDF score, or LexRank between the received query and information included in the first user data (a plurality of first user data). This calculates the similarity for each of the plurality of first user data. The interrogation unit 1042 of the server 10 extracts a predetermined number of first user data or first user data whose similarity is equal to or greater than a predetermined value from the plurality of first user data in order of similarity according to the similarity, and excludes the other first user data to identify the first user data. Alternatively, the interrogation unit 1042 may be configured to extract first user data related to the query from the plurality of first user data with reference to past questions, answer information, etc. The query unit 1042 of the server 10 may receive the contents of the input value in the product selection input field D121 from the first user terminal 20, and may search for user data in the user table 1012 based on the input value. Specifically, based on the product name input in the product selection input field D121, user data including the product name may be searched for from the user table 1012. In other words, the query received in the query receiving step includes information indicating the product name, such as the input value in the product selection input field D121. For example, when a query such as "What kind of diet should I follow to lower my blood pressure?" is received from a first user, first user data that is less relevant to the query, such as information about devices owned by the first user, is excluded, and first user data that is more relevant to the query, such as the first user's gender, age, blood pressure, diet, calorie intake, medical history, and so on, is extracted.

[0047] In step S102, the extracting step executes a step of extracting user information including information related to the physical condition of the first user associated with the first user. For example, when a query regarding the physical condition of the first user is received from the first user, the question unit 1042 of the server 10 extracts first user data related to the physical condition of the first user, and does not extract first user data not related to the physical condition of the first user.

[0048] In step S102, the extraction step executes a step of extracting user information including information about a device owned by the first user. For example, when a query is received from a first user regarding a device owned by the first user, the query unit 1042 of the server 10 extracts first user data related to the device owned by the first user, and does not extract first user data that is not related to the device owned by the first user.

[0049] In step S102, the interrogator 1042 of the server 10 transmits the extracted first user data to the first user terminal 20. The controller 204 of the first user terminal 20 displays and presents the received first user data in the reference data D122 and D123. This allows the first user to explicitly confirm what first user data related to the first user will be referenced when generating the prompt. The first user may operate the input device 206 of the first user terminal 20 to select one of the reference data D122, D123, etc., and exclude it from the first user data to be included in the prompt. Furthermore, the first user may operate the input device 206 of the first user terminal 20 to manually add any user data from the user data stored in the user table 1012 in association with the first user to the reference data D122 and D123, thereby including it in the first user data. Alternatively, the first user may operate the input device 206 of the first user terminal 20 to include any information in the first user data. This allows the first user to manually generate prompts that serve as input sentences for an appropriate large-scale language model (LLM). The reference data D122 and D123 may be the file name and path name of the CSV file related to the user data, the FQDN and IP address of the database that is the reference source of the user data, etc.

[0050] In step S103, the question unit 1042 of the server 10 executes a prompt generation step to generate a prompt that will be an input sentence for the large-scale language model (LLM) based on the query received in the query reception step and the user information extracted in the extraction step. Specifically, the questioning unit 1042 of the server 10 generates a prompt based on the query received in step S101 and the first user data extracted in step S102. For example, the question section 1042 of the server 10 generates a prompt such as the following: 〔prompt〕 # Order You are a chatbot that responds to questions from users by outputting answers to assist the user, taking into account the following user information: # Question from user Query # User information First user data # Answer text

[0051] For example, when the interrogator 1042 of the server 10 receives a query from the first user regarding the physical condition of the first user, the interrogator 1042 of the server 10 generates a prompt such as the following: 〔prompt〕 # Order You are a chatbot that responds to questions from users by outputting answers to assist the user, taking into account the following user information: # Question What kind of diet should I follow to lower my blood pressure? # User information ·blood pressure: ●●(YMD),●●(YMD),●●(YMD),●●(YMD) ·body weight ●●(YMD),●●(YMD),●●(YMD),●●(YMD) Meal contents ●●(YMD),●●(YMD),●●(YMD),●●(YMD) - Medical history ●●(YMD),●●(YMD),●●(YMD),●●(YMD) # Answer text

[0052] The questioning unit 1042 of the server 10 may generate a prompt including a character string that imposes an age restriction on the question sentence based on information indicating age and whether the user is an adult included in the first user data. For example, the questioning unit 1042 of the server 10 may include a character string such as "Please provide an answer appropriate for age XX" or "Please exclude answers that are inappropriate for minors" in the prompt. In addition, the question unit 1042 of the server 10 may perform a process of including in the prompt a character string for restricting the output content based on the first user data, or may perform a process of including in the prompt a character string for restricting the output content without being based on the first user data. This allows the LLM system 50 to provide an appropriate response based on the first user's age and information on whether the first user is an adult or a minor.

[0053] In step S104, the interrogator 1042 of the server 10 executes a prompt sending step of sending the prompt generated in the prompt generating step to an external large scale language model (LLM) system operated by a different operator from the computer. Specifically, the interrogator 1042 of the server 10 sends a request including the prompt generated in step S103 to the API endpoint provided by the LLM system 50.

[0054] In step S105, the question unit 1042 of the server 10 executes an answer receiving step of receiving an answer result to the prompt sent in the prompt sending step. The answer receiving step executes a step of receiving an answer result to the prompt based on user information associated with the first user without training a large-scale language model (LLM) system. Specifically, the LLM system 50 is configured to be able to output a character string (text) in response to a prompt received at an API endpoint. The LLM system 50 transmits a response including an answer result output based on the received prompt to the server 10. The questioning unit 1042 of the server 10 receives and accepts the answer result to the prompt included in the response from the LLM system 50.

[0055] In step S106, the questioning unit 1042 of the server 10 associates the prompt generated in the prompt generating step with the answer result received in the answer receiving step and stores them. Specifically, the questioning unit 1042 of the server 10 stores the first user ID 2011, the prompt generated in step S103, and the answer received in step S105 in the user ID, prompt, and answer items of the prompt table 1014, respectively.

[0056] In step S107, the question unit 1042 of the server 10 executes a proposal step of proposing to the first user advice information regarding the first user's response measures to the query received in the query reception step, based on the answer result received in the answer reception step. Specifically, the question unit 1042 of the server 10 transmits the character string (text) related to the answer result received from the LLM system 50 in step S105 to the first user terminal 20. The control unit 204 of the first user terminal 20 outputs the character string (text) related to the answer result to the answer area D102 of the question page D1 displayed on the display 2081 of the first user terminal 20. As a result, the question unit 1042 of the server 10 proposes to the first user a sentence related to advice for the first user's query (advice information) in accordance with the query received in step S101. When a question is processed by an operator, the answer result is presented to the operator. The operator sends the presented answer result by replying to the customer (first user) who inquired about the question. This allows the information inquiry service to provide the customer with an appropriate answer result in response to the customer's inquiry.

[0057] <Proposal processing> The suggestion process is a process of giving advice (suggestion) regarding a store's word-of-mouth received from a third party to a user associated with the store.

[0058] <Proposal process overview> The proposal process is a series of steps that accepts reviews about stores, etc. from third parties, extracts store data for the stores, etc. that accepted the reviews, generates prompts based on the content of the reviews and the store data, sends the prompts to a large-scale language model (LLM) system, receives response results from the LLM system, stores the response results, and presents the user with advice information regarding how the user should respond to the reviews based on the response results.

[0059] <Details of proposal process> The proposal process will be described in detail below.

[0060] In S301, a query receiving step is executed to receive a query regarding word-of-mouth about a store associated with a first user. Specifically, a user related to a poster (hereinafter referred to as a poster) operates the poster's information processing terminal to open a website for posting reviews about a store in a web browser, etc. The site for posting reviews does not need to be a website operated by the business that provides the information service according to the present disclosure, but may be an external website operated by another business. A poster operates the poster's information processing terminal to input and send information about the store, such as a review about the store, into an input form or the like provided on the website. This allows the review to be posted to the website for posting store reviews. The suggestion unit 1043 of the server 10 accesses the website to acquire the review posted by the poster and store identification information for identifying the posted store. The suggestion unit 1043 of the server 10 accepts the acquired review as a query. Based on the acquired store identification information, the suggestion unit 1043 of the server 10 stores the acquired word-of-mouth information in the word-of-mouth data field of the record identified by referencing the store ID in the store table 1015. Note that the record may be identified based on the store name instead of the store identification information.

[0061] In step S302, the suggestion unit 1043 of the server 10 executes an extraction step of extracting store information associated with the store based on the acquired word-of-mouth. The suggestion unit 1043 of the server 10 searches the store ID in the store table 1015 based on the store identification information acquired in step S301, and acquires store data of the store identified by the store identification information and a user ID (user identification information of a user associated with the store, hereinafter referred to as the first user). Note that the record may be identified based on the store name instead of the store identification information. Based on the word-of-mouth information acquired in step S301, the suggestion unit 1043 of the server 10 extracts store data related to the word-of-mouth information from various information related to the word-of-mouth information included in the store data, and does not extract store data not related to the word-of-mouth information. Specifically, the suggestion unit 1043 of the server 10 calculates the similarity between the word-of-mouth review and the multiple store data based on algorithms such as cosine similarity, TF-IDF score, and LexRank between the received word-of-mouth review and the information included in the store data (multiple store data). This calculates the similarity for each of the multiple store data. The suggestion unit 1043 of the server 10 extracts a predetermined number of store data items in descending order of similarity according to the similarity, or store data items with a similarity equal to or greater than a predetermined value, and excludes other store data items to identify the store data. Alternatively, the suggestion unit 1043 may be configured to extract store data related to the word-of-mouth review from the multiple store data items by referring to past questions, answer information, etc. For example, if a user posts a review saying that "the menu selection was limited," store data is extracted, including store data that is highly relevant to the review, such as the products sold by the store, the types of products available and their stock status, the services offered in the store, and other marketing information.

[0062] In step S303, the suggestion unit 1043 of the server 10 executes a prompt generation step to generate a prompt that serves as an input sentence for the large-scale language model (LLM) based on the word-of-mouth received in the query reception step and the store information extracted in the extraction step. Specifically, the suggestion unit 1043 of the server 10 generates a prompt based on the word-of-mouth received in step S301 and the store data extracted in step S302. For example, the suggestion unit 1043 of the server 10 generates the following prompt: 〔prompt〕 # Order You are a chatbot that responds to reviews about a store by taking into account the following store information and outputting responses to support the store's response measures to the reviews. # Store reviews Reviews # Store information Store Data # Answer text

[0063] For example, when the suggestion unit 1043 of the server 10 receives a word-of-mouth message from a contributor, the suggestion unit 1043 of the server 10 generates the following prompt. 〔prompt〕 # Order You are a chatbot that responds to reviews about a store by taking into account the following store information and outputting responses to support store initiatives in response to reviews. # Store reviews The menu variety was poor # Store information ·Products for sale: ●●●●,●●●●,●●●●,●●●●,●●●●,●●●● ·Products ●●●●,●●●●,●●●●,●●●●,●●●●,●●●● Availability ●●●●,●●●●,●●●●,●●●●,●●●●,●●●● In-store services # Answer text

[0064] Steps S304 to S306 are the same as steps S104 to S106 in the question processing, and therefore will not be described here. The suggestion unit 1043 of the server 10 stores the user identification information of the user associated with the store, the prompt generated in step S303, and the answer result received in step S305 in the user ID, prompt, and answer result fields of the prompt table 1014.

[0065] In step S307, the suggestion unit 1043 of the server 10 executes a suggestion step of suggesting to the first user advice information regarding the first user's response measures to the query received in the query reception step, based on the answer result received in the answer reception step. Specifically, the suggestion unit 1043 of the server 10 transmits a character string (text) related to the answer result received from the LLM system 50 in step S305 to the first user terminal 20. For example, the suggestion unit 1043 of the server 10 searches the user ID field in the user table 1012 based on the user identification information of the first user associated with the store identified in step S101, and acquires contact information for contacting the first user, such as the first user's email address and chat ID. The suggestion unit 1043 of the server 10 transmits the character string (text) related to the answer result received from the LLM system 50 in response to the contact information. The control unit 204 of the first user terminal 20 outputs a character string (text) related to the received answer result to the display 2081 of the first user terminal 20. As a result, the suggestion unit 1043 of the server 10 suggests to the first user associated with the store, in accordance with the word-of-mouth received in step S301, a sentence (advice information) related to advice on the first user's response measures.

[0066] <Basic computer hardware configuration> 11 is a block diagram showing the basic hardware configuration of a computer 90. The computer 90 includes at least a processor 901, a main memory device 902, an auxiliary memory device 903, and a communication IF 991 (interface), which are electrically connected to one another by a communication bus 921.

[0067] The processor 901 is hardware for executing an instruction set written in a program, and is composed of an arithmetic unit, registers, peripheral circuits, and the like.

[0068] The main memory device 902 is used to temporarily store programs, data to be processed by the programs, etc. For example, it is a volatile memory such as a DRAM (Dynamic Random Access Memory).

[0069] The auxiliary storage device 903 is a storage device for saving data and programs, such as a flash memory, a hard disk drive (HDD), a magneto-optical disk, a CD-ROM, a DVD-ROM, or a semiconductor memory.

[0070] The communication IF 991 is an interface for inputting and outputting signals for communicating with other computers via a network using wired or wireless communication standards. The network is composed of the Internet, a LAN, various mobile communication systems constructed by wireless base stations, etc. For example, the network includes 3G, 4G, and 5G mobile communication systems, LTE (Long Term Evolution), and wireless networks (e.g., Wi-Fi (registered trademark)) that can connect to the Internet via a predetermined access point. In the case of a wireless connection, communication protocols include, for example, Z-Wave (registered trademark), ZigBee (registered trademark), and Bluetooth (registered trademark). In the case of a wired connection, the network also includes a direct connection using a USB (Universal Serial Bus) cable, etc.

[0071] It should be noted that the computer 90 can be virtually realized by distributing all or part of each hardware configuration across multiple computers 90 and interconnecting them via a network. In this way, the computer 90 is a concept that includes not only a computer 90 housed in a single housing or case, but also a virtualized computer system.

[0072] <Basic functional configuration of computer 90> The following describes the functional configuration of a computer realized by the basic hardware configuration (FIG. 11) of the computer 90. The computer includes at least the functional units of a control unit, a storage unit, and a communication unit.

[0073] The functional units of the computer 90 can also be realized by distributing all or part of the functional units among multiple computers 90 interconnected via a network. The computer 90 is a concept that includes not only a single computer 90 but also a virtualized computer system.

[0074] The control unit is realized by the processor 901 reading out various programs stored in the auxiliary storage device 903, expanding them in the main storage device 902, and executing processing in accordance with the programs. The control unit can realize functional units that perform various types of information processing depending on the type of program. In this way, the computer is realized as an information processing device that performs information processing.

[0075] The storage unit is realized by a main storage device 902 and an auxiliary storage device 903. The storage unit stores data, various programs, and various databases. Furthermore, the processor 901 can allocate a storage area corresponding to the storage unit in the main storage device 902 or the auxiliary storage device 903 in accordance with the programs. Furthermore, the control unit can cause the processor 901 to execute processes for adding, updating, and deleting data stored in the storage unit in accordance with the various programs.

[0076] A database refers to a relational database, which manages data sets called masters and tables in a tabular format structurally defined by rows and columns, by relating them to each other. In a database, a table is called a table, a master, a column in a table is called a column, and a row in a table is called a record. In a relational database, relationships between tables and masters can be set and associated. Typically, each table and each master has a column set as a primary key to uniquely identify a record, but setting a primary key to a column is not essential. The control unit can cause the processor 901 to add, delete, or update records in specific tables and masters stored in the storage unit according to various programs. Furthermore, by storing data, various programs, and various databases in the storage unit, it can be considered that the information processing device and information processing system according to the present disclosure have been manufactured.

[0077] Note that the databases and masters in this disclosure may include any data structure in which information is structurally defined (such as a list, dictionary, associative array, or object). The data structure also includes data that can be considered as a data structure by combining data with functions, classes, methods, etc. written in any programming language.

[0078] The communication unit is realized by the communication IF 991. The communication unit realizes a function of communicating with other computers 90 via a network. The communication unit can receive information transmitted from other computers 90 and input the information to the control unit. The control unit can cause the processor 901 to execute information processing on the received information in accordance with various programs. In addition, the communication unit can transmit information output from the control unit to other computers 90.

[0079] <Additional Notes> The matters described in the above embodiments will be supplemented below.

[0080] (Appendix 1) A program to be executed by a computer having a processor and a memory unit, wherein the processor executes a user reception step (S101) of receiving first user identification information for identifying a first user, a query reception step (S101) of receiving a query regarding a predetermined question, an extraction step (S102) of extracting user information associated with the first user based on the first user identification information received in the user reception step and the query received in the query reception step, and a prompt generation step (S103) of generating a prompt that serves as an input sentence for a large-scale language model (LLM) based on the query received in the query reception step and the user information extracted in the extraction step. This allows user information to be included in prompts sent to the LLM system. By using these prompts, the LLM system can provide answers that are tailored to the user. For example, it can resolve the user's problem in a shorter time.

[0081] (Appendix 2) The program according to appendix 1, wherein the extraction step (S102) is a step of excluding a portion of user information not related to the query from the information associated with the first user and extracting a portion of user information related to the query. This allows the prompt for querying the LLM system to include the query-related information from the user information. By using this prompt, even if there is a large amount of user information, it is possible to extract only the user information that is appropriate for the query and query the LLM system.

[0082] (Appendix 3) 3. The program according to claim 1, wherein the extracting step (S102) is a step of extracting user information including information related to the physical state of the first user associated with the first user. This allows users to receive prompts from the LLM system that can provide better answers to questions about their health, for example, answers that are more tailored to the user's problem.

[0083] (Appendix 4) The program according to Supplementary Note 3, wherein the query receiving step (S101) is a step of receiving a query regarding the physical state of the first user. This allows users to receive prompts from the LLM system that can provide better answers to questions about their health, for example, answers that are more tailored to the user's problem.

[0084] (Appendix 5) 3. The program according to claim 1, wherein the extracting step (S102) is a step of extracting user information including information about a device owned by the first user. This allows users to receive prompts that provide answers that are more tailored to the user's problem when asked questions about their own information processing devices, such as PCs and smartphones.

[0085] (Appendix 6) The program according to claim 5, wherein the query receiving step (S101) is a step of receiving a query regarding a device owned by the first user. This allows users to receive prompts that provide answers that are more tailored to the user's problem when asked questions about their own information processing devices, such as PCs and smartphones.

[0086] (Appendix 7) The program according to appendix 1 or 2, wherein the query receiving step (S301) is a step of receiving a query regarding word-of-mouth reviews of a store associated with the first user. This allows the user to receive a prompt to ask the LLM system a question regarding the company's policy for responding to reviews posted by third parties about stores or other businesses that the user manages or is involved in.

[0087] (Appendix 8) A program described in any one of Appendices 1 to 7, in which a processor executes a prompt sending step (S104) of sending a prompt generated in the prompt generation step to an external large-scale language model (LLM) system operated by a different operator than the computer, and an answer receiving step (S105) of receiving an answer result to the prompt sent in the prompt sending step. This allows the LLM system to provide the user with the most appropriate answer, for example, to solve the user's problem in a shorter time.

[0088] (Appendix 9) The program according to claim 8, wherein the answer receiving step (S105) is a step of receiving an answer result to the prompt based on user information associated with the first user without training a large-scale language model (LLM) system. This allows the LLM system to obtain answers that are appropriate for each user, without having to train the LLM system using user information for each user. For example, this saves the effort of training a model for each user, as well as the cost of storing the model.

[0089] (Appendix 10) A program described in Appendix 8 or 9, in which a processor executes a proposal step (S107) in which, based on the answer result received in the answer receiving step, the processor proposes to the first user advice information regarding the first user's response measures for the query received in the query receiving step. This allows users to obtain advice from the LLM system regarding reviews posted by third parties about stores they manage or are involved in. For example, users can resolve issues regarding reviews posted about their stores in a shorter time.

[0090] (Appendix 11) A computer-implemented method comprising a processor and a memory, wherein the processor performs all of the steps performed in the invention according to any one of appendices 1 to 10. This allows user information to be included in prompts sent to the LLM system. By using these prompts, the LLM system can provide answers that are tailored to the user. For example, it can resolve the user's problem in a shorter time.

[0091] (Appendix 12) An information processing device comprising a control unit and a memory unit, wherein the control unit executes all of the steps executed in the invention according to any one of Supplementary Note 1 to Supplementary Note 10. This allows user information to be included in prompts sent to the LLM system. By using these prompts, the LLM system can provide answers that are tailored to the user. For example, it can resolve the user's problem in a shorter time.

[0092] (Appendix 13) A system comprising means for performing all steps performed in any of the inventions according to any one of appendixes 1 to 10. This allows user information to be included in prompts sent to the LLM system. By using these prompts, the LLM system can provide answers that are tailored to the user. For example, it can resolve the user's problem in a shorter time. [Explanation of symbols]

[0093] 1 System, 10 Server, 101 Memory Unit, 104 Control Unit, 106 Input Device, 108 Output Device, 20 First User Terminal, 201 Memory Unit, 204 Control Unit, 206 Input Device, 208 Output Device, 50 LLM System, 501 Memory Unit, 504 Control Unit, 506 Input Device, 508 Output Device

Claims

1. A program to be executed by a computer having a processor and a storage unit, the processor: a user receiving step of receiving first user identification information for identifying the first user; a product receiving step of receiving input of information indicating a predetermined product; a query receiving step of receiving a query regarding a predetermined question; an extraction step of extracting user information associated with the first user based on the first user identification information received in the user receiving step, the query received in the query receiving step, and information indicating the predetermined product received in the product receiving step; a prompt generating step of generating a prompt to be an input sentence to a large-scale language model (LLM) based on the query received in the query receiving step and the user information extracted in the extracting step; Run the query receiving step is a step of receiving the query regarding a physical condition of the first user or the query regarding a device owned by the first user; the extracting step is a step of extracting user information associated with the first user based on the query from information on the physical condition of the first user and information on a device owned by the first user that is stored in advance in association with the first user; program.

2. The program, a prompt sending step of sending the prompt generated in the prompt generating step to a large language model (LLM) system; receiving an answer from the large-scale language model (LLM) system to answer the prompt; a presentation step of presenting the user information extracted in the extraction step together with the answer result; To execute The program according to claim 1.

3. the extracting step is a step of excluding a portion of the user information not related to the query from the information associated with the first user and extracting a portion of the user information related to the query; The program according to claim 1.

4. The extracting step is a step of excluding a portion of the user information not related to the query from the information associated with the first user and extracting a portion of the user information related to the query. The program according to claim 2.

5. receiving the answer based on the user information associated with the first user, without training the large-scale language model (LLM) system; The program according to claim 2.

6. the processor: a suggestion step of suggesting to the first user advice information regarding a response measure to be taken by the first user in response to the query received in the query receiving step, based on the answer result received in the answer receiving step; To execute The program according to claim 2.

7. A method implemented on a computer having a processor and a memory, wherein the processor performs all of the steps performed in the invention according to any one of claims 1 to 6.

8. 10. An information processing device comprising a control unit and a storage unit, wherein the control unit executes all of the steps executed in the invention according to any one of claims 1 to 6.

9. A system having means for executing all steps performed in an invention according to any one of claims 1 to 6.

Images