Interview system, interview method, and interview program

Abstract

We provide interview systems, interview methods, and programs. [Solution] An interview method for interviewing a user via a user terminal 30 is a method in which the processor 21 performs multiple question-and-answer sessions, each question-and-answer session including sending one question to the user terminal 30 and obtaining the user's answer to the question from the user terminal 30, generating a summary sentence that summarizes the multiple answers obtained through the multiple question-and-answer sessions using a language model 14, sending a confirmation sentence asking whether the summary sentence is acceptable to the user terminal 30 along with the summary sentence, and obtaining a confirmation answer to the confirmation sentence from the user terminal 30.

Description

Technical Field

[0001] The present disclosure relates to an interview system, an interview method, and an interview program.

Background Art

[0002] In recent years, chatbots have sometimes been used as tools for conducting questionnaires. Generally speaking, a chatbot refers to an application configured to output a response according to the input content for the characters or voices input by a user. When conducting a questionnaire using a chatbot, for the questions output by the chatbot to the user terminal, the user who is the respondent inputs answers, and thus questionnaire results can be obtained.

[0003] For a chatbot that uses a large language model to respond, it is possible to conduct an interview-style questionnaire in which question-and-answer sessions are repeated multiple times. For example, assume that for the question "Have you ever used this product?" from the chatbot, the respondent answers "Yes, I have." In this case, the chatbot can develop questions based on the respondent's answer, such as "Please tell me the good points of using that product."

[0004] For example, Patent Document 1 discloses a system for conducting a questionnaire using a trained model that has learned the dialogue patterns of specific user attributes as a large language model. This system includes evaluation inquiry means to improve the accuracy of the trained model.

[0005] The evaluation inquiry means sends a notification asking for an evaluation of a chat executed on a certain user terminal to another user terminal, and obtains an evaluation result such as "Agree" or "Disagree" from the other user terminal. The trained model weighted based on such evaluation results can respond according to the attributes.

Prior Art Documents

Patent Documents

[0006] [Patent Document 1] Patent No. 7542904 [Overview of the project] [Problems that the invention aims to solve]

[0007] Even when a human interviewer conducts an in-person interview, they may summarize the survey results and evaluate them afterward. However, evaluating the results after the survey is completed presents the challenge that the quality of poorly evaluated survey results cannot be guaranteed. Furthermore, when using a pre-trained model, the accuracy of the survey results cannot be guaranteed until appropriate weighting is completed.

[0008] This disclosure aims to provide an interview system, interview method, and interview program that can obtain more stable and high-quality survey results. [Means for solving the problem]

[0009] An interview system according to one aspect of the present disclosure is a system for conducting an interview with a user via a user terminal, comprising: a memory for storing a plurality of instructions; and at least one processor, wherein the at least one processor executes the plurality of instructions to conduct a plurality of question-and-answer sessions, each question-and-answer session including: sending one question to the user terminal; and obtaining the user's answer to the one question from the user terminal; generating a summary sentence that summarizes the plurality of answers obtained through the plurality of question-and-answer sessions using a large-scale language model; sending a confirmation sentence asking whether the summary sentence is acceptable to the user terminal along with the summary sentence; and obtaining a confirmation answer to the confirmation sentence from the user terminal.

[0010] An interview method according to one aspect of the present disclosure is an interview method for conducting an interview with a user via a user terminal, wherein at least one processor conducts multiple question-and-answer sessions, each question-and-answer session including transmitting one question to the user terminal and obtaining the user's answer to the one question from the user terminal; generating a summary sentence that summarizes the multiple answers obtained through the multiple question-and-answer sessions using a large-scale language model; transmitting a confirmation sentence asking whether the summary sentence is acceptable to the user terminal along with the summary sentence; and obtaining a confirmation answer to the confirmation sentence from the user terminal.

[0011] An interview program according to one aspect of the present disclosure is an interview program for conducting an interview with a user via a user terminal, wherein at least one processor is configured to perform multiple question-and-answer sessions, each of which includes sending one question to the user terminal and obtaining the user's answer to the one question from the user terminal; generating a summary sentence that summarizes the multiple answers obtained through the multiple question-and-answer sessions using a large-scale language model; sending a confirmation sentence asking whether the summary sentence is acceptable to the user terminal along with the summary sentence; and obtaining a confirmation answer to the confirmation sentence from the user terminal. [Brief explanation of the drawing]

[0012] [Figure 1] Figure 1 is a schematic diagram of the interview system according to the embodiment. [Figure 2] Figure 2 is a flowchart illustrating the questionnaire method according to the embodiment. [Figure 3] Figure 3 is an example of a task selection screen displayed on a user's terminal. [Figure 4] Figure 4 is an example of an explanatory screen displayed on the user's terminal. [Figure 5] Figure 5 shows an example of the chat screen when an interview begins. [Figure 6] Figure 6 shows an example of the chat screen when an interview is concluded. [Figure 7] Figure 7 is a flowchart illustrating an interview method according to the embodiment. [Modes for carrying out the invention]

[0013] Examples of the interview system 11, interview method, and interview program are described with reference to Figures 1 to 7. The present invention is not limited to these examples and is intended to be expressed by the claims, with all modifications being made in the sense and scope equivalent to the claims.

[0014] [Interview System] The interview system 11 shown in Figure 1 (hereinafter simply referred to as "System 11") includes an interview server 20. The interview server 20 is configured to conduct interviews with users via a user terminal 30. The interviews are conducted for a questionnaire that includes multiple questions on a certain topic. In this disclosure, an interview means that a chatbot using a language model 14 acts as an interviewer and conducts a questionnaire by obtaining answers from the user in a question-and-answer format (or Q&A format).

[0015] The language model 14 may be a language model capable of editing and creating text. The language model may be a natural language processing model trained on a large amount of text data. The language model 14 may be a general-purpose large language model (LLM) that can be adapted to any natural language processing task, such as information extraction, text summarization, text generation, or question and answer. The language model may be a small language model or a medium-sized language model.

[0016] The language model 14 may be an interactive generative AI (Artificial Intelligence) that repeats input and output in a chat format, such as ChatGPT (Generative Pre-trained Transformer) for example. The language model 14 may also be a vision language model (VLM) capable of inputting images and text.

[0017] The language model 14 is configured to output, as a completion, data generated according to an instruction when a prompt including an instruction for generating some data is input to the language model. In the present disclosure, the processor 21 of the interview server 20 is configured to input (transmit) a prompt for conducting a questionnaire to the language model 14.

[0018] In the present disclosure, an example in which the interview server 20 uses the language model 14, which is a general large-scale language model not included in the system 11, will be described. Instead of this, the interview server 20 may include the language model 14, or the system 11 may include a server that holds the language model 14.

[0019] The interview server 20 may be implemented as a computer including, for example, at least one processor 21, at least one memory 22, and a communication interface (IF) 23. Hereinafter, for the sake of convenience, a case where the interview server 20 includes one processor 21 and one memory 22 will be described. The communication IF 23 enables communication with other devices via a network.

[0020] The memory 22 stores a program 24 and a database 25 that are executed by the processor 21. The program 24 includes an application and an operating system. The processor 21 realizes various functions by executing a plurality of instructions (program codes) included in the program 24.

[0021] Database 25 may include user data relating to multiple users, client data relating to multiple clients, and survey data for conducting surveys. The survey data may include prompts for input into language model 14 and interview results.

[0022] The interview server 20 is configured to conduct a survey in an interview format to obtain answers to multiple questions about a given topic. The interview server 20 has chatbot functionality using the language model 14 as a web application for conducting interviews.

[0023] The interview server 20 is operated by the provider of the interview service (hereinafter referred to as the "service provider"). The interview server 20 is configured to conduct questionnaires requested by the client via the client terminal 40 to multiple users. The number of users to be surveyed (hereinafter also referred to as the "number of respondents") may be determined by the client or the service provider.

[0024] System 11 may have one or more user terminals 30. System 11 does not have to have user terminals 30, but below we will describe an example in which System 11 has one user terminal 30.

[0025] The user terminal 30 may be implemented as a computer comprising at least one processor 31, at least one memory 32, and a communication interface 33. For convenience, the following description will explain an example in which the user terminal 30 comprises one processor 31 and one memory 32. The communication interface 33 enables communication with other devices via a network.

[0026] The user terminal 30 may be a mobile device such as a smartphone or tablet. The user terminal 30 may be equipped with a display 34, which is an output device. The display 34 may be equipped with a touch panel, which is an input device 35.

[0027] System 11 may have one or more client terminals 40. System 11 does not have to have client terminals 40, but below we will describe an example in which System 11 has one client terminal 40.

[0028] The client terminal 40 may be implemented as a computer comprising at least one processor 41, at least one memory 42, and a communication interface 43. For convenience, the following description will explain an example in which the client terminal 40 comprises one processor 41 and one memory 42. The client terminal 40 may also be a mobile device such as a smartphone or tablet.

[0029] The communication IF 43 enables communication with other devices via the network. The client terminal 40 may include an input device 44 and an output device 45, or these may be connected externally. The input device 44 may include, for example, a keyboard and a mouse, and the output device 45 may be, for example, a display. The client terminal 40 may also be equipped with a touch panel, which is an input / output device.

[0030] The requester is, for example, a seller who sells goods or services in a store. The store may be a physical store that conducts face-to-face sales, or an online shop that operates on an e-commerce (EC) site. The requester operates the requester terminal 40 to send a survey request to the interview server 20.

[0031] The interview server 20 may generate multiple questions to be used in the questionnaire based on the questionnaire request received from the client terminal 40. For example, the interview server 20 may generate one or more of the multiple questions to be used in the questionnaire using the language model 14.

[0032] The service provider may award users points that can be used in a points service as a reward or token of appreciation for completing the survey. In this case, system 11 may include a points management server 50. The points management server 50 is configured to award points to users who have completed the survey upon receiving a point awarding instruction from the interview server 20.

[0033] The point management server 50 may be implemented as a computer comprising at least one processor 51, at least one memory 52, and a communication interface 53. For convenience, the following description will explain an example in which the point management server 50 comprises one processor 51 and one memory 52. ​​The communication interface 53 enables communication with other devices via the network.

[0034] Memory 52 stores a program 54 and a database 55 that are executed by the processor 51. The program 54 includes applications and an operating system. The processor 51 implements various functions by executing multiple instructions (program code) contained in the program 54.

[0035] Database 55 may include merchant data for participating merchants in the points service. The merchant data may include multiple merchant records for each of multiple participating merchants. Each merchant record may include, for example, a merchant ID as an identifier, the merchant name, merchant ID, merchant address, email address, authentication information, and point processing history.

[0036] Database 55 may include user data relating to users registered for the points service. User data may include, for example, multiple user records for each of multiple users, with the user ID as the identifier. Each user record may include, but is not limited to, name, age, gender, address, account, email address, point acquisition history, and point usage history. Point acquisition history and point usage history may include, but is not limited to, the name of the store where points were acquired or used, date and time, transaction item, genre of transaction item, number of points acquired, or number of points used.

[0037] System 11 may include a web server 60 that provides an e-commerce site. The e-commerce site may use a points service managed by a points management server 50. In this case, when using the e-commerce site, users can use points to make payments on the e-commerce site or earn points based on the amount paid.

[0038] The web server 60 may be implemented as a computer comprising at least one processor 61, at least one memory 62, and a communication interface 63. For convenience, the following description will explain an example in which the web server 60 comprises one processor 61 and one memory 62. The communication interface 63 enables communication with other devices via the network.

[0039] Memory 62 stores a program 64 and a database 65 that are executed by the processor 61. Program 64 includes applications and the operating system. The processor 61 implements various functions by executing multiple instructions (program code) contained in program 64.

[0040] The e-commerce site may be a marketplace type with multiple businesses or stores participating, or it may be an online shop operated for one business or store. The e-commerce site may enable electronic payments through processing by the web server 60 or other servers, and points from a points program may be awarded according to the payment amount.

[0041] Database 65 may include seller data. The seller data may include multiple seller records for each of multiple sellers. Each seller record may include, for example, a seller ID as an identifier, seller name, store name, store ID, store address, e-commerce site account, email address, authentication information, payment receiving account, electronic payment history, e-commerce site sales history, and point accrual history.

[0042] Database 65 may include customer data relating to customers using the e-commerce site. The customer data may include, for example, multiple customer records for each of multiple customers, with the customer ID as the identifier. Each customer record may include, but is not limited to, name, address, age, gender, account, email address, and electronic payment history, purchase history on the e-commerce site, point acquisition history, and point usage history. The payment history and purchase history may include, but is not limited to, the name of the store where the payment was made, the date and time, purchased items, genre of purchased items, and purchase amount.

[0043] The web server 60 and the point management server 50 send point processing requests to each other each time points are awarded or used. Therefore, the seller data in database 65 may contain at least partially the same data as the merchant data in database 55. Similarly, the customer data in database 65 may contain at least partially the same data as the user data in database 55. The web server 60 and the point management server 50 may be managed by the same business operator, in which case databases 55 and 65 may contain at least partially the same data.

[0044] [Summary of the survey] A survey request from a client may include, but is not limited to, the attributes of the survey participants, the time allotted for the survey (chat time), the number of respondents, the survey period, the purpose of the survey, and one or more survey questions. The attributes of the survey participants may include, but is not limited to, one or more of the following: age or age group, gender, and place of residence. The number of respondents can be set to any number, for example, 50 or 100 people. The survey period is the period from the start of the survey to the delivery of the survey results, and can be set to any period, for example, 5 or 10 days, depending on the number of respondents.

[0045] The purpose of the survey may be, for example, a survey on a specific product, or it may be to gather user feedback. For example, the purpose of the survey may be to gather information on users' interest in, experience with, or feelings about products sold by the client, or it may be to gather information on users' concerns or problems that could lead to the discovery of latent needs. The one or more questions specified by the client may include only the first question, include multiple questions in a specified order, or include multiple questions in an unspecified order.

[0046] The first question could be something that asks about the user's awareness of the product that is the theme of the survey, such as "Have you ever used it?", "Are you familiar with it?", or "Are you interested in it?", or it could be something that asks about the user's situation, such as "Do you own a car?" or "How many credit cards do you have?", but it is not limited to these. By setting at least the first question in advance, even when the language model 14 generates other questions, it is possible to maintain consistency in the questions asked to multiple users.

[0047] The service provider may provide a request form for survey requests. The request form may allow users to select from multiple survey size patterns (e.g., 50 or 100 people) and the survey method. The selectable survey methods may include, but are not limited to, Category Entry Point (CEP), Customer Journey, Concept Evaluation, or Laddering.

[0048] The survey participants may be selected from a user group that includes multiple users. Multiple users may be registered in the user group as collaborators who perform several tasks in exchange for rewards or compensation, such as points. The user group data may be included in the user data of database 25.

[0049] When awarding points to users who participate in a survey, the user data in database 25 may contain at least partially the same data as the user data stored in database 55 of the point management server 50. Furthermore, the interview server 20 may use the user data from database 55 or the customer data from database 65 to select users for the survey, in accordance with the data usage agreement.

[0050] A user may log in to the task execution site by entering login information via the user terminal 30 on the login screen (not shown) provided by the interview server 20. In this case, the interview server 20 performs user authentication based on the login information. The login information may be, for example, a user ID and password, but is not limited to these.

[0051] For unregistered users, the interview server 20 may display a registration screen (not shown) on the user terminal 30. In this case, the interview server 20 performs user registration by adding the user data entered on the registration screen to the database 25.

[0052] [Survey Method] The questionnaire method described in this disclosure will be explained with reference to Figure 2. In step S11, the interview server 20 receives a questionnaire request from the client terminal 40. In the following step S12, the interview server 20 generates a questionnaire task to execute the questionnaire based on the questionnaire request.

[0053] Generating a survey task involves, for example, creating a prompt and determining several configuration items. These configuration items may include, but are not limited to, the question items, the survey format, the number of respondents (let's call it "M" in this example), the user attributes of the target audience, the expected response time, the time threshold, the number of question-and-answer sessions threshold, the completion conditions, the number of points awarded for completing the survey, and the number of times each user can participate.

[0054] The termination conditions are the conditions under which an interview is terminated during a single interview, even if the questionnaire is not yet completed. The interview server 20 may be configured to terminate the interview based on the termination conditions, regardless of the progress of the chat by the language model 14. The termination conditions may include, but are not limited to, at least one of the following: the time elapsed since the start of the interview exceeds a time threshold; the number of questions and answers exceeds a threshold; or communication with the user terminal 30 is interrupted for a specified period of time or longer during the interview.

[0055] The interview server 20 terminates the interview by turning off the completion flag when the termination conditions are met. This allows the interview server 20 to forcibly terminate the interview if the user repeatedly gives unclear answers to questions or makes statements unrelated to the questions. Conversely, when the questionnaire is completed successfully, the interview server 20 turns on the completion flag and terminates the interview.

[0056] In step S13, the interview server 20 extracts multiple target users from the user group that match the user attributes determined in step S12. In step S14, the interview server 20 configures task display settings for the target users. With the task display settings configured, when a target user logs into the task execution site, the survey task will be displayed.

[0057] As shown in Figure 3, the task execution site may be designed to display a task selection screen 15 that allows the selection of multiple tasks. For example, the task selection screen 15 may include multiple task buttons 66 (66A, 66B, 66C, 66D). Each of the multiple tasks selectable on the task selection screen 15 has a user group that it can run assigned to it. Therefore, the task buttons 66 displayed on the task selection screen 15 will differ depending on the user.

[0058] In this example, the task button 66A labeled "Survey via Chat" in Figure 3 corresponds to the survey task. When the user operates the survey task button 66A, the survey explanation screen 67 (see Figure 4) may be displayed on the user terminal 30.

[0059] As shown in Figure 4, the explanation screen 67 may include an explanation field 68 that describes the overview of the questionnaire. The explanation field 68 may display conditions such as the expected response time, the number of points awarded, and the number of times participation is possible. Furthermore, the explanation screen 67 may include an interview start button 69.

[0060] In step S15, the interview server 20 enters the initial value "0" into the number of completed questionnaires N in order to count the number of completed questionnaires N. When the user operates the interview start button 69 on the explanation screen 67, step S16 is determined to be positive (YES), and in step S17, the interview server 20 conducts an interview with that user. The interview execution process in step S17 will be described later. While step S16 is determined to be negative (NO), the interview server 20 waits, repeating S16 at predetermined intervals.

[0061] In step S18, the interview server 20 determines whether the completion flag for the performed interview is ON. If the result in step S18 is YES, the interview server 20 adds 1 to the number of completed questionnaires N in step S19 and proceeds to step S20. If the result in step S18 is NO, the interview server 20 proceeds to step S20 without adding to the number of completed questionnaires N.

[0062] In step S20, the interview server 20 determines whether the number of completed questionnaires N exceeds the number of target users (number of respondents) M. If step S20 is negative (NO), the interview server 20 returns to step S16. If step S20 is positive (YES), the interview server 20 proceeds to step S21 and outputs the results for M questionnaires. Then, in step S22, the interview server 20 sends the questionnaire results to the requester terminal 40 and terminates the questionnaire.

[0063] [prompt] The interview server 20 obtains survey results by mediating repeated question-and-answer sessions between the user terminal 30 and the language model 14. In other words, the interview server 20 conducts multiple question-and-answer sessions in a single interview. A series of interactions between the interview server 20 and the language model 14 for conducting multiple question-and-answer sessions with the user terminal 30 in succession is called a "session."

[0064] The prompt may include an initiation prompt to start a session, and a series of response prompts that are entered into the language model 14 to mediate the chat after the session has started. Each response prompt may simply input the user's response directly into the language model 14.

[0065] Each question-and-answer session includes inputting a response prompt to the language model 14, obtaining one question from the language model 14, sending one question to the user terminal 30, obtaining the user's answer to one question from the user terminal 30, and inputting the answer to the language model 14.

[0066] The introductory prompt may include role-setting and task descriptions for the session. Role-setting may be, for example, "You are an interviewer for a survey." Task descriptions may include the purpose and content of the survey. The introductory prompt may include response instructions indicating what information should be presented to the respondent first. Response instructions may include, for example, instructions for outputting an introductory message and the content to include in the introductory message. The introductory message may include, for example, an interview opening greeting, an explanation of the survey content, and the first question.

[0067] The opening prompt may include instructions to guide the interview. These instructions may include, for example, "Please ask the next question after confirming the answer to the previous one," indicating that the next question should be asked only after an answer has been obtained to the previous question.

[0068] The start prompt may include a question generation instruction to generate multiple questions. In this case, the question generation instruction may include an instruction to generate the next question based on the answer to the previous question. The question generation instruction could be written as, for example, "Please repeat the question based on the answer to the previous question in order to delve deeper into the user's needs," but is not limited to this.

[0069] The first question in an interview may be determined by the interviewer. In this case, the introductory prompt may include the first question and question generation instructions for generating one or more questions following the first question. The question generation instructions may include instructions for generating the next question based on the answer to the first question.

[0070] The question generation instruction may include generation conditions to define the length of the interview. The generation conditions may include, for example, at least one of the following: the estimated length of the interview or the estimated number of questions. The question generation instruction may also include an estimated response time of "10 minutes," for example, "Please ask questions so that the interview can be completed in about 10 minutes." As a result, the language model 14 generates one or more questions so that the interview automatically ends with a length of time or an estimated number of questions corresponding to the estimated response time.

[0071] The initial prompt may include instructions for taking appropriate action if the answer to question 1 is inappropriate. These instructions may include at least one of the following: requesting a re-answer, rephrasing question 1, or rephrasing question 1. The instructions may be written, for example, "If the respondent gives an unclear answer or makes a statement unrelated to the question, take the following actions." The instructions may also be followed by the following actions:

[0072] If the respondent's answer is unclear, please ask a follow-up question such as, "Could you please elaborate a little more on your answer?" If a respondent makes a comment unrelated to the question, please guide them back to the original question by saying something like, "Thank you. Now, let's go back to the original question."

[0073] If the answer is not directly related to the question, prompt the respondent to return to the original question by saying something like, "That's a very interesting point. With that in mind, I'd like to ask you about the original question."

[0074] The start prompt may include a summarization instruction to generate a summary sentence that summarizes all the answers once all answers have been received for all questions. The summarization instruction can be written, for example, "Once you have finished the questions, please create a summary sentence that summarizes all the answers received up to that point," but is not limited to this. For example, if the survey request includes multiple questions, the start prompt may include a summarization instruction such as "Once you have finished the questions, please summarize the answers for each question." The summary message 77 illustrated in Figure 6 includes a summary sentence that summarizes the answers for each question.

[0075] An initiation prompt containing a summary instruction may further include a confirmation instruction for sending a confirmation message asking whether the summary is acceptable. The confirmation instruction could, but is not limited to, "After presenting the summary, please confirm with [Is this content acceptable?]".

[0076] The initial prompt may include a correction instruction to generate a revised summary if the user's response to the summary is a negative response indicating that the summary is unsuitable. The initial prompt may also include a reconfirmation instruction to send a reconfirmation message along with the revised summary asking whether the revised summary is acceptable. The correction instruction could be written, for example, "If the user responds that the content of the summary is incorrect, please ask what needs to be corrected and revise the summary." The reconfirmation instruction could be written, for example, "Please reconfirm whether the content of the revised summary is acceptable," but is not limited to this.

[0077] Furthermore, the start prompt may include an end instruction to conclude the interview if the user gives an affirmative response acknowledging the summary. The end instruction could, but is not limited to, "If you agree that the summary is good, please end the interview by saying, [Thank you. Thank you so much for taking the time to participate in this interview today. You may now leave this page.]"

[0078] The start prompt may include a control instruction to output a control command to notify the interview server 20 that the interview is complete when all questions have been answered and the questionnaire has been successfully completed. After the interview server 20 inputs the response prompt to the language model 14 and receives the control command output by the language model 14, it determines that the interview is complete and performs the interview termination process.

[0079] [Interview Examples] Referring to Figures 5 and 6, we will explain an example of an interview focused on a concept survey for shampoo, which is an example of a new product.

[0080] Figure 5 shows the chat screen 70 that the interview server 20 displays on the user terminal 30 after the user operates the interview start button 69 on the explanation screen 67 displayed on the user terminal 30. In the chat screen 70 shown in Figures 4 and 5, the interviewer messages 71, 73, 75, 77, and 79 output by the interview server 20 are displayed on the left side of the chat screen 70, while the user messages 72, 74, 76, and 78 entered by the user via the user terminal 30 are displayed on the right side of the chat screen 70.

[0081] The interview server 20 first displays an opening message 71 at the top of the chat screen 70. The opening message 71 includes, for example, a greeting to start the interview, an explanation of the questionnaire content, and the first question. When the user enters a first answer message 72 containing the first answer to the first question into the chat screen 70, the interview server 20 displays a question message 73 containing the second question into the chat screen 70. The second question is generated by the language model 14 based on the prompt and the first answer.

[0082] Question message 73 may include a summary of the user's first response or a response to the first response before the second question. The response summary can be generated by the language model 14 by including a summary instruction in the start prompt input to the language model 14 to summarize the user's response each time a response is made. The response to the response can be generated by the language model 14 by including an interview guideline in the start prompt, such as "Please respond to the response in a friendly tone, nodding and showing empathy."

[0083] Figure 6 shows a chat screen 70 illustrating the scene where an interview concludes. The user message 76 on the chat screen 70 is the final answer to the last question. The language model 14 determines whether or not it is the final answer according to the generation conditions included in the prompt (for example, the estimated length of the interview or the estimated number of questions). Having received the final answer, the language model 14 generates a summary. The chat screen 70 then displays a summary message 77, which includes the interview summary and a confirmation message asking, "Is this content correct?"

[0084] In the example in Figure 6, the response message 78 to the confirmation statement contains an affirmative response acknowledging the summary, "Yes, that's fine." Therefore, the chat screen 70 displays a completion message 79, which is generated in accordance with the termination instruction. This completes the interview appropriately.

[0085] [Interview Method] The interview method of this application will be explained with reference to Figure 7. The process shown in Figure 7 corresponds to the interview execution process shown in step S17 of Figure 2. When the user operates the interview start button 69 on the explanation screen 67, the interview execution process shown in Figure 7 is started.

[0086] First, in step S30, the interview server 20 inputs a start prompt to the language model 14. Upon reading the start prompt, the language model 14 starts a session with the interview server 20. Then, the language model 14 outputs a start message 71 according to the response instructions. In step S31, the interview server 20 displays the start message 71 on the chat screen 70.

[0087] In step S32, when the interview server 20 receives the first response message 72 from the user terminal 30, the interview server 20 starts a question-and-answer session as loop L10. In loop L10, the interview server 20 conducts multiple question-and-answer sessions with the user terminal 30 until the language model 14 finishes asking the question.

[0088] Each question-and-answer session includes inputting a response prompt containing the previous answer (initially the answer obtained in step S32) into the language model 14 (step S51), obtaining an interviewer message containing the next question from the language model 14 (step S52), sending an interviewer message containing the next question to the user terminal 30 (step S53), and obtaining the user's answer to the question from the user terminal 30 (step S54).

[0089] Loop L10 may further include step S55, in which the interview server 20 determines whether a threshold (for example, at least one or both of a time threshold or a count threshold) has been exceeded since the start of the interview. If step S55 is negative (NO), the interview server 20 proceeds to step S56 and inputs the response obtained in step S54 into the language model 14.

[0090] If step S55 is a positive result (YES), the interview server 20 proceeds to step S57 and sends a termination message to the user terminal 30. The termination message may be, for example, "The interview has ended because the response time has been exceeded. The task is not complete, so you can restart the questionnaire from the task list screen," but is not limited to this.

[0091] In the following step S58, the interview server 20 turns off the completion flag. Then, in step S59, the interview server 20 terminates the session with the language model 14, ending the interview with the questionnaire still incomplete.

[0092] In loop L10, once multiple question-and-answer sessions have been successfully completed, the interview server 20 proceeds to step S33, where it uses the language model 14 to generate a summary sentence that summarizes the multiple answers obtained through the multiple question-and-answer sessions. More specifically, the interview server 20 retrieves the summary sentence generated by the language model 14 according to the summarization instructions. In the subsequent step S34, the interview server 20 sends a summary message 77 to the user terminal 30, which includes the summary sentence and a confirmation sentence asking whether the summary sentence is acceptable.

[0093] In step S35, the interview server 20 obtains a response message 78 from the user terminal 30. The response message 78 includes the user's confirmation response to the confirmation statement. In step S36, the interview server 20 determines whether the user's confirmation response obtained in step S35 is an affirmative response approving the summary statement or a negative response rejecting the summary statement.

[0094] In step S36, if the confirmation response is negative (step S36=NO), the interview server 20 generates a revised summary using the language model 14 to modify the summary. More specifically, when the interview server 20 inputs the response message 78 into the language model 14, the language model 14 generates a revised summary and a reconfirmation statement asking whether the revised summary is acceptable, depending on the response (step S33). The interview server 20 retrieves the revised summary and the reconfirmation statement from the language model 14 and sends them to the user terminal 30 (step S34). Subsequently, the interview server 20 obtains a reconfirmation response to the reconfirmation statement from the user terminal 30 (step S35). This process of revising the summary is repeated until the user approves the summary.

[0095] In step S36, if the user's confirmation response is an affirmative response approving the summary (step S36 = YES), the interview server 20 inputs the affirmative response into the language model 14. The language model 14 then outputs a completion message 79 in accordance with the termination instruction. In step S37, the interview server 20 sends the completion message 79 to the user terminal 30. This completes the survey for that user.

[0096] Subsequently, the interview server 20 performs processing to terminate the interview. For example, in step S38, the interview server 20 turns on the completion flag. Also, in step S39, the interview server 20 performs task completion processing. More specifically, it adds data indicating that the survey task has been completed to the registration information of the user who was surveyed. If a user is allowed to participate in the survey task only once, after the task completion processing, the survey task button 66A will no longer be displayed on the user's task selection screen 15, or the survey task button 66A will become inoperable.

[0097] In step S40, the interview server 20 performs a point awarding process to award points to the user who has completed the interview. More specifically, the interview server 20 sends a user identifier to identify the user, the number of points to be awarded, and a point processing request to the point management server 50. This completes the interview. The point management server 50 updates the user data stored in the database 55 based on the user identifier and performs the point awarding process for that user.

[0098] Once the interview is successfully completed, the survey results obtained from the completed interview are output in step S21 of Figure 2. These survey results may include all messages displayed on the chat screen 70. In other words, the survey results may include, for each of the number of users surveyed, multiple questions asked in the survey, multiple answers to each of the multiple questions, and a summary. The survey results may further include information such as the attributes of the users who answered the survey and the date the survey was conducted.

[0099] [Effects of this disclosure] The interview server 20 can automate user interviews by utilizing the language model 14. In this interview, a summary instruction is included in the prompt, and at the end of the interview, a summary of the user's answers is presented to the user. If the user gives a negative answer, a revised summary is automatically generated, and the user is asked to reconfirm it.

[0100] The interview is completed when the user gives an affirmative response, approving the summary of the interview results. Survey results approved by the user can be said to accurately reflect the user's opinion. Therefore, it is possible to obtain survey results of more consistent quality.

[0101] In face-to-face interviews, it's necessary to summarize and evaluate the results of a user questionnaire afterward. In contrast, this disclosure allows for summarizing the questionnaire results within the interview itself. Furthermore, user approval of the questionnaire results can be obtained at the end of the interview.

[0102] As a result, the interview server 20 can quickly compile the survey results once all interviews for the required number of participants have been completed. This significantly shortens the survey implementation period. Furthermore, the requester who receives the survey results can easily grasp the overview of the interviews by reading the summary.

[0103] Furthermore, once the survey task is completed, the interview server 20 processes the task completion and points awarding. Therefore, handling multiple users who answer the survey can be completed quickly and without human intervention.

[0104] For example, if the communication status deteriorates during an interview and the time threshold is exceeded, the interview will end unfinished, and in that case, the task completion process will not be performed. Therefore, the user can restart the survey task by operating the survey task button 66A again from the task selection screen 15.

[0105] The interview server 20 can terminate the interview forcibly according to termination conditions such as a time threshold, or perform processing to terminate the interview according to control commands output by the language model 14. Therefore, the system 11 can use the language model 14 to manage the progress of the interview, while the interview server 20 can automate all or most of the questionnaire.

[0106] [Effects of this disclosure] According to this disclosure, the following effects can be achieved. (1) The interview server 20 sends a summary of the responses generated using the language model 14 to the user terminal 30, allowing the user who provided the responses to confirm whether the summary is appropriate. If the summary is appropriate, it can be said that appropriate questions and answers were given, thus guaranteeing the quality of the survey results for that user. Therefore, the interview server 20 can obtain survey results of more stable quality.

[0107] (2) If the user gives a negative response indicating that the summary is inappropriate, the interview server 20 uses the language model 14 to generate a revised summary. By sending the revised summary to the user terminal 30, the user who provided the response can review the revised summary. In this way, if the user determines that the summary is inappropriate, the interview server 20 automatically generates a revised summary, thus preventing a decline in the quality of the survey results.

[0108] (3) The interview server 20 terminates the interview when the user approves the summary. Thus, the interview server 20 can complete the interview after confirming that the questions and answers were appropriate.

[0109] (4) For example, if a user repeatedly gives vague answers to questions such as "I don't know" or "What do you mean?", the interview cannot be conducted properly. In such cases, the interview server 20 may terminate the interview based on predetermined termination conditions, such as when the time elapsed since the start of the interview exceeds a time threshold, or when the number of questions and answers exceeds a count threshold.

[0110] (5) The interview server 20 can perform user authentication or user registration, and after the interview is completed, it can award points as a reward to the user who answered the questionnaire.

[0111] (6) The interview server 20 can conduct the interview under the guidance of the language model 14 by inputting a start prompt containing instructions for conducting the interview into the language model 14. The language model 14 automatically generates a summary after obtaining answers to all questions included in the questionnaire, in accordance with the summary instructions. As a result, the interview server 20 can terminate the interview after obtaining confirmation from the user that the questionnaire results are appropriate.

[0112] (7) If the prompt includes a question generation instruction, the language model 14 generates the next question based on the user's answer to the previous question. This allows the interview server 20 to send questions to the user terminal 30 in a timely manner to delve deeper into the user's intentions in a more natural flow.

[0113] (8) If the question generation instruction includes generation conditions for defining the length of the interview, the interview time or the number of questions can be made uniform even if the language model 14 generates multiple questions. Therefore, when conducting a survey with multiple users, it is possible to generate questions that correspond to each user's answers while making the volume of survey results uniform.

[0114] (9) Even when conducting a survey with multiple users, the interview server 20 can start the interview with a common first question included in the prompt. Therefore, even if the language model 14 generates one or more questions following the first question, the interview content can be made consistent. Thus, it is possible to generate questions tailored to each of the multiple users while obtaining consistent survey results.

[0115] (10) If the prompt includes instructions for action, even if the user's response is inappropriate, the interview server 20 can obtain an appropriate response from the user by taking measures such as requesting a response again, rephrasing the question, or rewording the question.

[0116] (11) By including a summary in the survey results in addition to the answers to each of the multiple questions, the requester can easily grasp an overview of the survey results from the summary. Furthermore, since this summary is approved by the respondent user, the requester can be provided with survey results of guaranteed quality.

[0117] This embodiment can be implemented with the following modifications. This embodiment and the following modifications can be combined with each other to the extent that they do not contradict each other technically. [Example of changing the interview server] The interview server 20 may also have the functionality of a point management server 50. In this case, the system 11 does not need to have the point management server 50. Alternatively, or in addition to this, the interview server 20 may also have the functionality of a web server 60. In this case, the system 11 does not need to have the web server 60.

[0118] [Example of changes to interview results] System 11 may be configured to use the language model 14 to summarize or analyze the survey results. For example, after step S21, the interview server 20 may input an analysis prompt to the language model 14, which includes multiple summary sentences for each of the multiple users, the attributes of the multiple users, and analysis instructions. User attributes may be, but are not limited to, age or age group, gender, and place of residence (e.g., prefecture).

[0119] The analysis instructions may include, for example, instructions to summarize response trends for each attribute (e.g., by age group or gender). In addition, the analysis instructions may include instructions to propose strategies on how to promote the product for each attribute. As a result, the language model 14 outputs the survey analysis results for each attribute in accordance with the analysis prompts. Similarly, the system 11 may use the language model 14 to generate an overall summary that summarizes the summaries of all interviews. In this case, the survey results sent in step S22 may include the survey analysis results and the overall summary.

[0120] [Examples of changes to interview methods] The interview may be conducted not only via text chat, but also partially or entirely by voice.

[0121] [Example of changes to the question] The initial question does not need to be included in the opening message 71. In this case, after step S31, step S32 may be omitted, and the question-and-answer loop L10 may be entered. Alternatively, the language model 14 may generate all questions, including the initial question.

[0122] [Example of changes to summary instructions] The summarization instructions may include, in addition to, instructions to generate partial summaries divided into several parts, such as for a specified number of questions or for each question item, instead of instructing to summarize the answers to all questions. In this case, the summaries may be checked each time a partial summary is generated, or a final check of the overall summary, which combines all the partial summaries, may be performed.

[0123] [Examples of changes to flowcharts and configuration diagrams] The flowcharts and configuration diagrams of this disclosure illustrate the architecture, functionality, and operation of the apparatus, system, method, and program according to embodiments of this disclosure. Each step included in these flowcharts and each component included in the configuration diagrams may correspond to a part of a program containing one or more instructions for realizing a logical functional unit. In other embodiments, some of the illustrated steps may be omitted, other steps may be included, the order of the steps may be different, or some steps may be executed simultaneously. Furthermore, a flowchart described as a series of actions may be divided into several parts and executed, or multiple flowcharts may be executed sequentially or in relation to each other. Also in other embodiments, some of the illustrated components may be omitted, other components may be included, or the arrangement of components may be changed. Furthermore, the functions realized by these steps and components may be realized by hardware, software, or a combination of hardware and software.

[0124] [Example of changes to the interview system] System 11 may be implemented as a single interview server 20 (e.g., a computer), or it may be distributed across multiple devices (e.g., computers) or subsystems that collaborate to execute the program.

[0125] [Example of memory modification] The memory included in the apparatus of this disclosure is a computer-readable storage medium, including non-transitory computer-readable medium. The memory may include, but is not limited to, ROM, hard disks, storage, removable media, flash memory, memory sticks, optical media, magneto-optical media, and CD-ROMs.

[0126] [Example of processor change] One or more processors included in the apparatus of this disclosure may include, but are not limited to, a CPU (central processing unit), a GPU (graphics processing unit), an APU (accelerated processing unit), an NPU (Neural network Processing Unit), a microprocessor, a microcontroller, a DSP (digital signal processor), an FPGA (field programmable gate array), a CPLD (Complex Programmable Logic Device), an application-specific integrated circuit (ASIC), a general-purpose processor, or any combination thereof designed to perform the functions described herein.

[0127] [Example of network changes] Communication between multiple devices or systems may be carried out via one or more communication networks in accordance with well-known communication protocols. The communication network may be, but is not limited to, an intranet, the internet, a local area network, a wide area network, a wireless network, a wired network, a virtual network, a software-defined network, or any other type of network, or a combination thereof.

[0128] [Example of changing the communication interface] A communication interface (IF) enables one device to communicate with other devices via a communication network. The communication interface may be, but is not limited to, a LAN (Local Area Network), Wi-Fi (registered trademark), Bluetooth (registered trademark), NFC (Near Field Communication), or other wireless communication interfaces.

[0129] The embodiments and modifications described above are listed below. [1] A system for conducting interviews with users via their terminals. Memory that stores multiple instructions, At least one processor, The system comprises, and the at least one processor executes the plurality of instructions, The process involves conducting multiple question-and-answer sessions, each of which includes sending one question to the user terminal and obtaining the user's answer to the one question from the user terminal. The process involves generating a summary sentence by summarizing the multiple answers obtained through the aforementioned multiple question-and-answer sessions, using a large-scale language model, A confirmation message asking whether the summary is acceptable is sent to the user terminal along with the summary, Obtaining a confirmation response to the confirmation message from the user terminal, An interview system configured to perform an interview.

[0130] [2] The aforementioned at least one processor, If the response to the confirmation is a negative response that the summary is unsuitable, the large-scale language model is used to generate a revised summary, A confirmation message asking whether the aforementioned revised summary is acceptable is sent to the user terminal along with the aforementioned revised summary, Obtaining a confirmation response to the confirmation statement from the user terminal, Configured to perform, The interview system described in [1] above.

[0131] [3] The aforementioned at least one processor, The system is configured to terminate the interview if the response to the confirmation is an affirmative response approving the summary. The interview system described in [1] or [2] above.

[0132] [4] The at least one processor is configured to terminate the interview based on a defined termination condition, The termination condition includes at least one of the following: the time elapsed since the start of the interview exceeds a time threshold, or the number of question-and-answer sessions exceeds a count threshold. An interview system as described in any of the above [1] to [3].

[0133] [5] The aforementioned at least one processor, The authentication process for the user or the registration process for the user, A point awarding process for awarding points to the user after the interview has been completed, Configured to perform, An interview system as described in any of the above [1] to [4].

[0134] [6] The aforementioned interview was conducted for a questionnaire that included multiple questions on a certain topic. The at least one processor is configured to input prompts for conducting the survey to the large-scale language model, The aforementioned prompt, Instructions for conducting the aforementioned interview, including asking the next question after obtaining an answer to one of the aforementioned multiple questions, A summarization instruction to generate the aforementioned summary text that summarizes all the answers when answers to all questions have been obtained, If an affirmative response is received in response to the aforementioned confirmation, acknowledging the summary, the following is included: a termination instruction to end the interview; An interview system as described in any of the above [1] to [5].

[0135] [7] The prompt includes a question generation instruction for generating the plurality of questions, The aforementioned question generation instruction includes an instruction to generate the next question based on the answer to the previous question, The interview system described in [6] above.

[0136] [8] The question generation instruction includes generation conditions for defining the length of the interview, The generation conditions include at least one of the following: the target length of the interview or the target number of questions. The interview system described above [7].

[0137] [9] The aforementioned prompt, The first question and, A question generation instruction for generating one or more questions following the aforementioned first question, including, The question generation instruction includes an instruction to generate the next question based on the answer to the first question, An interview system as described in any of the above [6] to [8].

[0138]

[10] The prompt includes instructions for taking corrective action to guide an appropriate answer if the answer to the one question is inappropriate. The aforementioned instructions for countermeasures include at least one of the following: a request for a response, a rephrasing of the question in item 1, or a rewording of the question in item 1. An interview system as described in any of the above [6] to [9].

[0139]

[11] The aforementioned interview was conducted for a questionnaire that included multiple questions on a certain topic. The aforementioned at least one processor, The request for the aforementioned survey is received from the requester's terminal, The results of the questionnaire obtained through the interview are transmitted to the client's terminal, wherein the results of the questionnaire include the answers to each of the multiple questions and the summary text. Configured to perform, An interview system as described in any of the above [1] to

[10] .

[0140]

[12] An interview method for conducting interviews with users via a user terminal, At least one processor, The process involves conducting multiple question-and-answer sessions, each of which includes sending one question to the user terminal and obtaining the user's answer to the one question from the user terminal. The process involves generating a summary sentence by summarizing the multiple answers obtained through the aforementioned multiple question-and-answer sessions, using a large-scale language model, A confirmation message asking whether the summary is acceptable is sent to the user terminal along with the summary, Obtaining a confirmation response to the confirmation message from the user terminal, Interview methods, including...

[0141]

[13] An interview program for conducting interviews with users via a user terminal, At least one processor, The process involves conducting multiple question-and-answer sessions, each of which includes sending one question to the user terminal and obtaining the user's answer to the one question from the user terminal. The process involves generating a summary sentence by summarizing the multiple answers obtained through the aforementioned multiple question-and-answer sessions, using a large-scale language model, A confirmation message asking whether the summary is acceptable is sent to the user terminal along with the summary, Obtaining a confirmation response to the confirmation message from the user terminal, An interview program designed to execute the process. [Explanation of Symbols]

[0142] 11...Interview system, 14...Language model, 15...Task selection screen, 20...Interview server, 21,31,41,51,61...Processor, 22,32,42,52,62...Memory, 24,54,64...Program, 25,55,65...Database, 30...User terminal, 34...Display, 35...Input device, 40...Requester terminal, 44...Input device, 45...Output device, 50...Point management server, 60...Web server, 66...Task button, 66A...Questionnaire task button, 67...Explanation screen, 68...Explanation field, 69...Start interview button, 70...Chat screen, 71...Start message, 72...First response message, 73...Question message, 77...Summary message, 78...Response message, 79...Completion message.

Claims

1. A system for conducting interviews with users via their terminals. Memory that stores multiple instructions, At least one processor, The system comprises, and the at least one processor executes the plurality of instructions, The process involves conducting multiple question-and-answer sessions, each of which includes sending one question to the user terminal and obtaining the user's answer to the one question from the user terminal. The process involves generating a summary sentence by summarizing the multiple answers obtained through the aforementioned multiple question-and-answer sessions, using a large-scale language model, A confirmation message asking whether the summary is acceptable is sent to the user terminal along with the summary, Obtaining a confirmation response to the confirmation message from the user terminal, An interview system configured to perform an interview.

2. The aforementioned at least one processor, If the response to the confirmation is a negative response that the summary is unsuitable, the large-scale language model is used to generate a revised summary, A confirmation message asking whether the aforementioned revised summary is acceptable is sent to the user terminal along with the aforementioned revised summary, Obtaining a confirmation response to the confirmation statement from the user terminal, Configured to perform, The interview system according to claim 1.

3. The aforementioned at least one processor, The system is configured to terminate the interview if the response to the confirmation is an affirmative response approving the summary. The interview system according to claim 1.

4. The at least one processor is configured to terminate the interview based on a defined termination condition, The termination condition includes at least one of the following: the time elapsed since the start of the interview exceeds a time threshold, or the number of question-and-answer sessions exceeds a count threshold. The interview system according to claim 1.

5. The aforementioned at least one processor, The authentication process for the user or the registration process for the user, A point awarding process for awarding points to the user after the interview has been completed, Configured to perform, The interview system according to claim 1.

6. The aforementioned interview was conducted for a questionnaire that included multiple questions on a certain topic. The at least one processor is configured to input prompts for conducting the survey to the large-scale language model, The aforementioned prompt, Instructions for conducting the aforementioned interview, including asking the next question after obtaining an answer to one of the aforementioned multiple questions, A summarization instruction to generate the aforementioned summary text that summarizes all the answers when answers to all questions have been obtained, If an affirmative response is received in response to the aforementioned confirmation, acknowledging the summary, the following is included: a termination instruction to end the interview; An interview system according to any one of claims 1 to 5.

7. The prompt includes a question generation instruction for generating the plurality of questions, The aforementioned question generation instruction includes an instruction to generate the next question based on the answer to the previous question, The interview system according to claim 6.

8. The question generation instruction includes generation conditions for defining the length of the interview, The generation conditions include at least one of the following: the target length of the interview or the target number of questions. The interview system according to claim 7.

9. The aforementioned prompt, The first question and, A question generation instruction for generating one or more questions following the aforementioned first question, including, The question generation instruction includes an instruction to generate the next question based on the answer to the first question, The interview system according to claim 6.

10. The prompt includes instructions for taking corrective action to guide an appropriate answer if the answer to the one question is inappropriate. The aforementioned instructions for countermeasures include at least one of the following: a request for a further response, a rephrasing of the question in item 1, or a rewording of the question in item 1. The interview system according to claim 6.

11. The aforementioned interview was conducted for a questionnaire that included multiple questions on a certain topic. The aforementioned at least one processor, The request for the aforementioned survey is received from the requester's terminal, The results of the questionnaire obtained through the interview are transmitted to the client's terminal, wherein the results of the questionnaire include the answers to each of the multiple questions and the summary text. Configured to perform, An interview system according to any one of claims 1 to 5.

12. An interview method for conducting interviews with users via a user terminal, At least one processor, The process involves conducting multiple question-and-answer sessions, each of which includes sending one question to the user terminal and obtaining the user's answer to the one question from the user terminal. The process involves generating a summary sentence by summarizing the multiple answers obtained through the aforementioned multiple question-and-answer sessions, using a large-scale language model, A confirmation message asking whether the summary is acceptable is sent to the user terminal along with the summary, Obtaining a confirmation response to the confirmation message from the user terminal, Interview methods, including...

13. An interview program for conducting interviews with users via a user terminal, At least one processor, The process involves conducting multiple question-and-answer sessions, each of which includes sending one question to the user terminal and obtaining the user's answer to the one question from the user terminal. The process involves generating a summary sentence by summarizing the multiple answers obtained through the aforementioned multiple question-and-answer sessions, using a large-scale language model, A confirmation message asking whether the summary is acceptable is sent to the user terminal along with the summary, Obtaining a confirmation response to the confirmation message from the user terminal, An interview program designed to execute the process.

Images