Information processing equipment, methods, programs and systems

Abstract

The purpose of this disclosure is to improve the convenience of AI-based conversation services. [Solution] A program for operating a computer comprising a processor and memory, the program causing the program to perform the following steps: storing a video relating to a dialogue between a person and a generating AI as first video data; restarting the dialogue from a position related to the question topic at the time of departure, if the person leaves the dialogue midway through and then returns to the dialogue; and storing the video relating to the dialogue after the restart as second video data.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, method, program, and system.

Background Art

[0002] In Patent Document 1, a device is described that acquires information about an interview subject and specifies an interviewer who conducts an interview with the interview subject from a plurality of interviewers and the content of questions to be asked to the interview subject based on the acquired information about the interview subject. The device also describes a virtual interviewer. The virtual interviewer stores a plurality of questions, answer examples for each question, the next question corresponding to each answer example, etc. in a server. When the interview subject accesses the server via a network using his / her own terminal, an interview is conducted by transmitting and receiving questions and answers between the server and the terminal.

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In Patent Document 1, by performing processing for dividing the conversation part of the interviewer and the conversation part of the interview subject based on differences in voice feature amounts between the interviewer and the interview subject, processing for detecting the continuation or interruption of the conversation, etc., it is described that the duration of a series of conversations (conversation time), the time between conversations (time between conversations), the time required for questions and answers to the questions (question·answer time) can be obtained. However, in Patent Document 1, interruption of the conversation during the dialogue is not assumed. In a dialogue with a virtual dialogue partner, the fact that interruption of the dialogue is not assumed can hinder the convenience of the dialogue service with AI.

[0005] An object of the present disclosure is to improve the convenience of the dialogue service with AI.

Means for Solving the Problems

[0006] A program for operating a computer comprising a processor and memory, the program causing the processor to perform the following steps: storing a video relating to a dialogue between a person and a generating AI as first video data; restarting the dialogue from a position related to the question topic at the time of departure, if the person leaves the dialogue midway through and then returns to the dialogue; and storing the video relating to the dialogue after the restart as second video data. [Effects of the Invention]

[0007] According to this disclosure, the convenience of AI-based conversation services can be improved. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram showing an example of the overall configuration of System 1. [Figure 2] This is a block diagram showing an example of the functional configuration of terminal device 10 (the functional configuration of terminal device 40 is the same as that of terminal device 10, so its explanation is omitted). [Figure 3] This figure shows an example of the functional configuration of server 20. [Figure 4] This diagram shows the data structure of the Interactor Table 2021. [Figure 5] This diagram shows the data structure of Question Table 2022. [Figure 6] This diagram shows the data structure of the dialogue session table 2023. [Figure 7] This flowchart illustrates an example of what happens in System 1 when a dialogue session between a person and an AI avatar is interrupted after it has started, and the first video data is stored. [Figure 8] This flowchart illustrates an example of how a dialogue session interrupted in Figure 7 resumes and the second video data is stored in System 1. [Figure 9]This flowchart illustrates an example of how an administrator plays a video of a conversation in System 1. [Figure 10] This is an example of a screen display when a user interacts with an avatar using terminal device 10. [Figure 11] This is an example of a screen displayed when the interlocutor selects an option related to resuming the interaction on terminal device 10. [Figure 12] This is an example of the screen displayed when the administrator plays a video on terminal device 40. [Figure 13] A block diagram showing the basic hardware configuration of Computer 90. [Modes for carrying out the invention]

[0009] The embodiments of this disclosure will be described below with reference to the drawings. In all the drawings illustrating the embodiments, common components are denoted by the same reference numerals, and repeated explanations are omitted. The following embodiments are not intended to unduly limit the content of this disclosure as described in the claims. Not all components shown in the embodiments are necessarily essential components of this disclosure. Also, each drawing is a schematic diagram and is not necessarily a strict illustration.

[0010] Furthermore, in the following description, "processor" refers to one or more processors. At least one processor is typically a microprocessor such as a CPU (Central Processing Unit), but may be another type of processor such as a GPU (Graphics Processing Unit). At least one processor may be single-core or multi-core.

[0011] Furthermore, at least one processor may be a broad-sense processor, such as a hardware circuit that performs some or all of the processing (e.g., an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit)).

[0012] In the following description, the expression "xxx table" may be used to describe the information from which the output is obtained for the input. However, this information may be data of any structure or a learning model such as a neural network that generates the output for the input. Therefore, "xxx table" can be referred to as "xxx information".

[0013] In the following description, the configuration of each table is an example. One table may be divided into two or more tables, or all or part of two or more tables may be combined into one table.

[0014] In the following description, the "program" may be used as the subject to describe the process. However, since the program is executed by the processor to perform the defined process while appropriately using the storage unit and / or the interface unit, etc., the subject of the process may be the processor (or a device such as a controller having the processor).

[0015] The program may be installed in a device such as a computer or may be in, for example, a program distribution server or a computer-readable (e.g., non-temporary) recording medium. In the following description, two or more programs may be realized as one program, or one program may be realized as two or more programs.

[0016] In the following description, an identification number is used as the identification information for various objects, but other types of identification information (e.g., an identifier including letters or symbols) may be adopted.

[0017] In the following description, when describing elements of the same type without distinction, reference signs (or common signs among the reference signs) are used, and when describing elements of the same type by distinction, the identification numbers (or reference signs) of the elements may be used.

[0018] Furthermore, in the following explanation, only control lines and information lines deemed necessary for the explanation are shown, and not all control lines and information lines in the product are necessarily shown. All components may be interconnected.

[0019] Each information processing device consists of a computer equipped with an arithmetic unit and a memory device. The basic hardware configuration of the computer and the basic functional configuration of the computer realized by said hardware configuration will be described later for each of the terminal devices 10, 20, and 40, but explanations that overlap with the basic hardware configuration and basic functional configuration of the computer described later will be omitted.

[0020] <Overview> The system according to this embodiment provides an AI avatar dialogue service in which a dialogue participant (user) and an AI avatar interact. In a dialogue (for example, a job interview), if a dialogue participant leaves the dialogue midway and then returns, the dialogue resumes. The dialogue may resume from the point in time when the participant left, or from the beginning of the question topic from which the participant left off. The video from the start of the dialogue until the participant leaves is stored as first video data (hereinafter referred to as first video data), and the video from the resumption of the dialogue until the end of the dialogue is stored as second video data (hereinafter referred to as second video data).

[0021] In this embodiment, a continuous exchange of user statements and AI responses is collectively referred to as a single session of dialogue. While this embodiment primarily uses the example of a single-session job interview, any type of dialogue can constitute a single session. Examples of single-session dialogues include a 10-minute casual conversation, a 30-minute business negotiation, a 60-minute media interview, and other types of interactions such as face-to-face meetings, interviews, hearings, sales calls, telephone reception, initial consultations for specialized tasks, and role-playing (e.g., customer service practice).

[0022] In this embodiment, we will consider an example of a dialogue format in which a user interacts with a generative AI model through an avatar. The avatar is, for example, an interface that has the appearance and voice of a human and can interact with the user based on output data from the generative AI model. The avatar can take the form of an icon of any human appearance, for example.

[0023] <1. System Configuration Diagram> Figure 1 is a block diagram showing an example of the overall configuration of System 1. As shown in Figure 1, System 1 includes, for example, a participant's terminal device 10, a server 20, a generating AI system 30, and an administrator's terminal device 40. The terminal devices 10, server 20, generating AI system 30, and administrator's terminal device 40 communicate with each other via, for example, a network 80.

[0024] Figure 1 shows an example where System 1 includes two terminal devices 10, but the number of terminal devices 10 included in System 1 is not limited to two. System 1 may include one terminal device 10, or it may include three or more terminal devices 10.

[0025] Figure 1 shows an example where System 1 includes one Generative AI System 30, but the number of Generative AI Systems 30 included in System 1 is not limited to one. System 1 may include two or more Generative AI Systems 30. Also, Figure 1 shows an example where the Generative AI System 30 is independent of the Server 20, but the Server 20 may include the functions of the Generative AI System 30. In other words, the Server 20 may store the Generative AI.

[0026] In this embodiment, a collection of multiple devices may be treated as a single server. The method of allocating the multiple functions required to implement the server 20 according to this embodiment to one or more hardware can be appropriately determined in view of the processing capacity of each hardware and / or the specifications required for the server 20.

[0027] Terminal device 10 is an information processing device used by the conversational user in the conversational service provided by server 20. The conversational user may also be referred to as the end user of the conversational service. The conversational user is, for example, an applicant for a job interview using an AI avatar. Terminal device 10 provides the applicant with a user interface for interacting with the generated AI provided by the generated AI system 30. Terminal device 10 can be implemented as, for example, a stationary PC (Personal Computer), a laptop PC, a head-mounted display, etc. Alternatively, terminal device 10 may be a portable computer such as a smartphone or a tablet device.

[0028] The terminal device 10 comprises a communication interface (IF) 12, an input device 13, an output device 14, a memory 15, storage 16, and a processor 19. The communication interface 12 is an interface for inputting and outputting signals so that the terminal device 10 can communicate with devices in system 1, such as a server 20. The input device 13 is a device for receiving input operations from the user (e.g., a touch panel, touchpad, pointing device such as a mouse, keyboard, etc.). The output device 14 is a device for presenting information to the user (display, speaker, etc.). The memory 15 is for temporarily storing programs and data processed by programs, etc., and is a volatile memory such as DRAM (Dynamic Random Access Memory). The storage 16 is for saving data, and is a flash memory, HDD (Hard Disk Drive), etc. The processor 19 is hardware for executing the instruction set written in the program, and is composed of an arithmetic unit, registers, peripheral circuits, etc.

[0029] Server 20 is an information processing device used by a service provider that provides interactive services (hereinafter referred to as the provider). Server 20 is an information processing device implemented by a computer connected to network 80, for example.

[0030] The server 20 comprises a communication interface 22, an input / output interface 23, a memory 25, storage 26, and a processor 29. The communication interface 22 is an interface for inputting and outputting signals so that the server 20 can communicate with devices in system 1, such as a terminal device 10. The input / output interface 23 functions as an interface to an input device for receiving input operations from a user and an output device for outputting information to the user. The memory 25 is for temporarily storing programs and data processed by programs, etc., and is a volatile memory such as DRAM. The storage 26 is for saving data, and is a flash memory or HDD, for example. The processor 29 is hardware for executing the instruction set written in the program, and is composed of an arithmetic unit, registers, peripheral circuits, etc.

[0031] Each information processing device consists of a computer equipped with an arithmetic unit and a memory device. The basic hardware configuration of the computer and the basic functional configuration of the computer realized by said hardware configuration will be described later. For each of the terminal device 10 and the server 20, explanations that overlap with the basic hardware configuration and basic functional configuration of the computer described later will be omitted.

[0032] The generative AI system 30 is, for example, a system on which generative AI (generative artificial intelligence) is constructed. The generative AI system 30 functions as a system for realizing dialogue between a user and the generative AI. That is, the AI ​​system 30 also plays a role in generating the speech content of the AI ​​avatar, such as by posing statements to the user or responding to statements from the user.

[0033] The generative AI built into the generative AI system 30 is, for example, a Large Language Model (LLM). A Large Language Model is a natural language model designed to perform multiple tasks of natural language processing. A Large Language Model is an example of a trained model, trained using a large number of parameters (e.g., billions to hundreds of billions) and high-level computing resources. A natural language model refers to a computer program or algorithm designed to perform tasks of natural language processing. For example, in natural language processing, processes such as morphological analysis, syntactic analysis, information extraction, and text generation are performed, enabling a computer to analyze human language (i.e., natural language) and perform predetermined processing. A Large Language Model generates output based on the text, image, etc., of a prompt (instruction) that is input. The prompt can be defined in natural language.

[0034] Examples of large-scale language models include the GPT series (Generative Pre-trained Transformer) developed by OPEN AI, BERT (Bidirectional Encoder Representations from Transformers) developed by Google, StableLM developed by StabilityAI, and Llama2, Palm2®, and LamDA2® developed by Meta. Large-scale language models tend to have high training costs because they are trained using a very large number of parameters and computational resources. In this embodiment, the training cost is reduced by using a publicly available pre-trained model by the generative AI system 30. In this embodiment, for example, a definition is passed to GPT, allowing it to understand the context and then generate information.

[0035] A prompt is primarily a query (including text, strings, images, videos, audio, etc.) input to a generating AI. Users of a generating AI can instruct it to process information by inputting prompts. Users can input prompts to the generating AI to produce the desired output. Prompts do not necessarily have to be strings; they can also be images, videos, audio, etc. For example, gestures and voice commands from the user can also serve as prompts.

[0036] Prompts can be entered through input operations on the user's terminal. Document files, images, videos, audio, etc., can also be uploaded and used as prompts. Users can create prompts by combining instructions entered on their terminal with other instructions, data, etc. In this case, a prompt may contain one or more instructions, data, etc. Alternatively, a prompt may contain only one or more instructions, or only one or more data. A prompt can be created by including other instructions, data, etc., as part of a predetermined set of instructions, data, etc. A prompt can be created by inserting other instructions, data, etc., as part of a predetermined set of instructions, data, etc. A prompt can be created by combining predetermined instructions, data, etc., with other instructions, data, etc. A prompt can be created by combining predetermined instructions, data, etc., with other instructions, data, etc. A prompt can be created by adding other instructions, data, etc., to predetermined instructions, data, etc.

[0037] In this disclosure, expressions such as "include," "insert," "combine," "combine," and "add" used when creating prompts from predetermined instructions, data, etc., are used as terms that refer to the same information processing. In other words, they can be treated as the same information processing in that the same prompt is created based on one or more instructions, data, etc. For example, the term "include" includes information processing such as "insert," "combine," "combine," and "add." Similarly, "insert," "combine," "combine," and "add" also include information processing such as "include," "insert," "combine," "combine," and "add," respectively.

[0038] Terminal device 40 is an information processing device used by a service provider (hereinafter referred to as the administrator) that manages the dialogue service and allows the dialoguer to interact with the AI ​​avatar. The administrator is, for example, a recruiter in a job interview using an AI avatar. The administrator checks information related to the dialogue and plays the video related to the dialogue via terminal device 40. Terminal device 40 can be implemented as, for example, a mobile device such as a smartphone or tablet, a stationary PC (Personal Computer), or a laptop PC. Terminal device 40, like terminal device 10, is implemented as a computer equipped with hardware such as a communication interface, input device, output device, memory, storage, and processor.

[0039] Each information processing device consists of a computer equipped with an arithmetic unit and a memory device. The basic hardware configuration of the computer and the basic functional configuration of the computer realized by said hardware configuration will be described later. For each of terminal devices 10, server 20, generation AI system 30, and terminal device 40, explanations that overlap with the basic hardware configuration and basic functional configuration of the computer described later will be omitted.

[0040] <2. Configuration of terminal equipment> Figure 2 is a block diagram showing an example of the functional configuration of terminal device 10 (the functional configuration example of terminal device 40 is the same as that of terminal device 10, so its explanation is omitted). As shown in Figure 2, terminal device 10 includes a communication unit 120, an input device 13, an output device 14, an audio processing unit 17, a microphone 171, a speaker 172, a camera 160, a location information sensor 150, a storage unit 180, and a control unit 190. Each block included in terminal device 10 is electrically connected, for example, by a bus.

[0041] The communication unit 120 performs processing such as modulation and demodulation processing for the terminal device 10 to communicate with other devices. The communication unit 120 performs transmission processing on the signal generated by the control unit 190 and transmits it to an external source (for example, the server 20). The communication unit 120 performs reception processing on the signal received from an external source and outputs it to the control unit 190.

[0042] The input device 13 is a device for a user operating the terminal device 10 to input instructions or information. The input device 13 can be implemented, for example, by a touch-sensitive device 131 on which instructions are input by touching the operating surface. If the terminal device 10 is a PC, the input device 13 may be implemented by a reader, keyboard, mouse, etc. The input device 13 converts the instructions input by the user into electrical signals and outputs the electrical signals to the control unit 190. The input device 13 may also include, for example, a receiving port that accepts electrical signals input from an external input device.

[0043] The output device 14 is a device for presenting information to the user operating the terminal device 10. The output device 14 is implemented, for example, by a display 141. The display 141 displays data according to the control of the control unit 190. The display 141 is implemented, for example, by an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display.

[0044] The audio processing unit 17 performs, for example, digital-to-analog conversion processing of the audio signal. The audio processing unit 17 converts the signal received from the microphone 171 into a digital signal and provides the converted signal to the control unit 190. The audio processing unit 17 also provides the audio signal to the speaker 172. The audio processing unit 17 is implemented, for example, by an audio processing processor. The microphone 171 receives an audio input and provides the audio signal corresponding to that audio input to the audio processing unit 17. The speaker 172 converts the audio signal received from the audio processing unit 17 into audio and outputs the audio to the outside of the terminal device 10.

[0045] Camera 160 is a device that receives light using a photodetector and outputs it as a shooting signal.

[0046] The location information sensor 150 is a sensor that detects the position of the terminal device 10, and is, for example, a GPS (Global Positioning System) module. A GPS module is a receiving device used in a satellite positioning system. In a satellite positioning system, signals are received from at least three or four satellites, and the current position of the terminal device 10, which is equipped with a GPS module, is detected based on the received signals. The location information sensor 150 may also detect the current position of the terminal device 10 from the position of the wireless base station to which the terminal device 10 is connected.

[0047] The storage unit 180 is implemented by, for example, memory 15 and storage 16, and stores data and programs used by the terminal device 10. The storage unit 180 stores, for example, user information 181.

[0048] User information 181 includes, for example, information about the user who uses the terminal device 10. User information includes, for example, the user's name, age, address, date of birth, contact information, etc.

[0049] The control unit 190 is realized when the processor 19 reads a program stored in the memory unit 180 and executes instructions contained in the program. The control unit 190 controls the operation of the terminal device 10. By operating according to the program, the control unit 190 performs the functions of an operation reception unit 191, a transmission / reception unit 192, and a presentation control unit 193.

[0050] The operation reception unit 191 processes instructions or information input from the input device 13. Specifically, for example, the operation reception unit 191 receives instructions or information input from a touch-sensitive device 131 or the like.

[0051] Furthermore, the operation reception unit 191 receives voice instructions input from the microphone 171. Specifically, for example, the operation reception unit 191 receives voice signals input from the microphone 171 and converted into digital signals by the voice processing unit 17. The operation reception unit 191 obtains instructions from the user by, for example, analyzing the received voice signals and extracting predetermined nouns.

[0052] The transmitting / receiving unit 192 performs processing to enable the terminal device 10 to send and receive data with an external device such as the server 20 in accordance with a communication protocol. Specifically, for example, the transmitting / receiving unit 192 sends information input by the user or instructions from the user to the server 20. The transmitting / receiving unit 192 also receives information provided by the server 20.

[0053] The presentation control unit 193 controls the output device 14 to present information provided by the server 20 to the user. Specifically, for example, the presentation control unit 193 displays the information transmitted from the server 20 on the display 141. As an example, the presentation control unit 193 displays an AI avatar in the browser of the terminal device 10. The presentation control unit 193 may also display the AI ​​avatar in a dedicated dialogue application of the terminal device 10. The presentation control unit 193 also outputs the information transmitted from the server 20 through the speaker 172.

[0054] <3. Functional Configuration of the Server> Figure 3 shows an example of the functional configuration of server 20. As shown in Figure 3, server 20 functions as a communication unit 201, a storage unit 202, and a control unit 203.

[0055] The communications unit 201 performs processing to enable the server 20 to communicate with external devices.

[0056] The memory unit 202 includes, for example, a dialogue table 2021, a question table 2022, a dialogue session table 2023, and so on. The tables stored in the memory unit 202 are not limited to these.

[0057] The Interlocutor Table 2021 is a table that stores information related to the interlocutor.

[0058] Question Table 2022 is a table that stores information related to question themes and question items.

[0059] Dialogue session table 2023 is a table that stores information related to a dialogue session.

[0060] The control unit 203 operates according to a program, thereby performing the function of controlling all processing related to the dialogue service.

[0061] Specifically, for example, the control unit 203 controls the transmission and reception of information with an external device. The control unit 203 controls the transmission and reception of information related to the interaction with an external device via the communication unit 201.

[0062] Specifically, for example, the control unit 203 controls the presentation of information related to the dialogue. The control unit 203 controls the speech and actions of the avatar displayed on the display 141, for example, via the presentation control unit 193.

[0063] Specifically, for example, the control unit 203 controls the start, interruption, resumption, and termination of the dialogue. The control unit 203 starts, interrupts, resumes, and terminates the dialogue in response to, for example, the status of the dialogue, the communication status with the terminal device 10, and requests from the terminal device 10.

[0064] Specifically, for example, the control unit 203 manages the video related to the dialogue. The control unit 203 stores the video related to the dialogue. The control unit 203 presents information about the video related to the dialogue in response to an operation of the terminal device 40 by the administrator, and the control unit 203 plays the video related to the dialogue in response to an operation of the terminal device 40 by the administrator.

[0065] The control unit 203 is implemented by the processor 29 reading a program stored in the memory unit 202 and executing instructions contained in the program. The program includes applications such as web browser applications. The program includes programming languages ​​such as JavaScript® that are executed on the web browser application stored in the terminal device 10. The control unit 203 may implement the functions according to this embodiment by cooperating with external systems or services using an API (Application Programming Interface) as needed during the process of executing the program. That is, the control unit 203 may, for example, call a program stored in an external system using an API.

[0066] <4. Data Structure> This section describes the data structure of the tables stored by server 20. Note that the described data structure is an example and does not exclude data not listed. Furthermore, even data listed in the same table may be stored in separate memory areas within the storage unit 202. Each table may have columns other than those described in that table. Each table may not have any of the columns described in that table.

[0067] Figure 4 shows the data structure of the Interactor Table 2021. As shown in Figure 4, the Interactor Table 2021 has columns such as name, date of birth, email address, etc., with the Interactor ID as the key.

[0068] The Interlocutor ID is a column that stores an identifier to uniquely identify the interlocutor.

[0069] The "Name" column is used to remember the name of the person you are speaking with.

[0070] The date of birth column stores the date of birth of the person being spoken to.

[0071] The email address column stores the email address of the person you are speaking with.

[0072] The interlocutor table 2021 may have columns to store other information about the interlocutor (such as attributes, background, and abilities).

[0073] Figure 5 shows the data structure of the question table 2022. As shown in Figure 5, the question table 2022 has columns such as question theme, question time, supplementary explanation, question item ID, and question item, with the question theme ID as the key. The contents of the question table 2022 are set, for example, by the administrator via the terminal device 40.

[0074] The Question Theme ID is a column that stores an identifier to uniquely identify the question theme.

[0075] The question theme is a column that stores text indicating the question theme. The question theme represents the concept, purpose, objective, or perspective of the question to be asked to the interlocutor during the dialogue.

[0076] The "Question Time" column stores the time limit allocated to the question topic.

[0077] The supplementary explanation column is a column for storing text that provides supplementary explanations to a question topic, in order to guide the determination of multiple items for further exploration of the question topic. A supplementary explanation for a single question topic can consist of one or more sentences.

[0078] The Question Item ID is a column that stores an identifier to uniquely identify a question item. A question item is a sub-item that makes up a question theme. An item can also be called a detailed theme that subdivides a question theme. One or more question items are associated with one question theme.

[0079] The question items are columns that store the text representing the question items.

[0080] Figure 6 shows the data structure of the dialogue session table 2023. As shown in Figure 6, the dialogue session table 2023 has columns such as dialogue session ID, video ID, video data, dialogue log, reason for leaving, leaving position, question theme ID, and playback position, with dialogue session ID as the key. The dialogue session ID and question theme ID for each dialogue session in the dialogue session table 2023 are set, for example, via the administrator's terminal device 40 before the dialogue session.

[0081] The Dialogue Session ID is a column that stores an identifier to uniquely identify a dialogue session. Each dialogue session is assigned a unique Dialogue Session ID.

[0082] The Interlocutor ID is a column that stores an identifier (same as in Figure 4) to uniquely identify the interlocutor who is the subject of the dialogue session.

[0083] The video ID is a column that stores an identifier to uniquely identify the video recording of the conversation. If there is no resumption of a conversation session, the first video data of session 1 is associated with that conversation session. If there is both departure and resumption of a conversation session, multiple video data will be associated with that conversation session. Specifically, for example, if there is one departure and one resumption, the first video data of session 1 and the second video data of session 1 will be associated.

[0084] The video data column stores the video recording of the conversation. The video data may also store the link address to the video recording of the conversation.

[0085] The dialogue log is a column that stores the statements made by the dialoguer and the AI ​​avatar during the conversation.

[0086] The "Reason for Exiting" column stores the reason why the interlocutor left the conversation. If the interlocutor voluntarily ends the conversation on the conversation screen, the exit location is stored as, for example, "Voluntary Exit." If the conversation is forcibly terminated due to poor communication, the exit location is stored as, for example, "Communication Failure." For videos where the interlocutor did not exit the conversation, the reason for exit is indicated by a dash (-).

[0087] The "exit position" column is a column that records the position in the video where the interlocutor exited the conversation.

[0088] The Question Theme ID is a column that stores the Question Theme ID (same as Figure 5) of the question themes heard during the dialogue. Video 1 contains one or more question themes.

[0089] The playback position column stores the playback position in the video where the question topic began during the dialogue. The playback position of the first question topic in each video data is set to 00:00 for that data.

[0090] <5. Operation> An example of the operation flow in System 1 is described below.

[0091] <5.1. Operation 1>

[0092] Figure 7 is a flowchart illustrating an example of what happens in System 1 when a dialogue session between the interlocutor and the AI ​​avatar is interrupted after it has started, and the first video data is stored.

[0093] In step S1001, the server 20 accepts a request to start a dialogue. Specifically, the dialogue participant accesses the dialogue platform (dialogue participant screen) via a web browser or a dedicated dialogue application on the terminal device 10 after performing a predetermined operation. The predetermined operation is, for example, logging in. On the dialogue platform on the terminal device 10, the dialogue participant selects one dialogue session that they wish to start from among the dialogue sessions assigned to them. The transmitting / receiving unit 192 sends a request to start the selected dialogue session to the server 20 via the communication unit 120. The control unit 203 receives the request to start a dialogue session via, for example, the communication unit 201. The control unit 203 inputs a prompt containing an instruction to start the dialogue to the generating AI system 30 via the communication unit 201, for example. The instruction is, for example, "Conduct a 30-minute job interview with the dialogue participant. Ask the dialogue participant questions according to the question theme "...". Respond to the dialogue participant's statements in a way that allows you to obtain answers to the questions on each question theme."

[0094] In step S1002, the server 20 inputs the utterance made by the interlocutor to the generation AI system 30. Specifically, for example, the operation reception unit 191 receives the utterance made by the interlocutor as voice input via the input device 13. The transmission / reception unit 192 transmits the voice data of the utterance to the server 20 via, for example, the communication unit 120. The control unit 203 receives the voice data of the utterance via, for example, the communication unit 201. The control unit 203 converts the voice data of the utterance into text data. The control unit 203 transmits the text data of the utterance to the generation AI system 30 via, for example, the communication unit 201.

[0095] In step S1003, the server 20 instructs the generating AI system 30 to output questions for the interlocutor and responses to statements made by the interlocutor. Specifically, the generating AI system 30 generates questions according to the question themes instructed in the instruction text. For example, the generating AI system 30 outputs question texts as text data for each question item in accordance with the supplementary explanation text under one question theme. Specifically, the generating AI system 30 generates responses in accordance with statements made by the interlocutor. For example, the generating AI system 30 outputs responses such as interest in the statement, praise, summary, rephrasing, nod, fuller, etc., as text data. The generating AI system 30 sends, for example, the text data of the questions or the text data of the responses to the server 20.

[0096] In step S1004, the server 20 presents the information transmitted from the generating AI system 30 to the interlocutor. Specifically, the control unit 203 converts, for example, the text data of the question or the text data of the response transmitted from the generating AI system 30 into audio data. The conversion process from text data to audio data may be performed, for example, by the audio processing unit 17. The control unit 203 transmits the text data and audio data to the terminal device 10, for example, via the communication unit 201.

[0097] The control unit 203 controls, for example, the speech and actions of the avatar displayed on the display 141 via the presentation control unit 193. The control unit 203 causes the avatar to speak based on voice data via the presentation control unit 193. The control unit 203 may, for example, control the presentation control unit 193 to display response text data on the display 141. In this way, the server 20 presents the question and answer to the interlocutor.

[0098] During one session of interaction, the processes from steps S1002 to S1004 are repeated.

[0099] During the repetition of the process from step S1002 to step S1004, the control unit 203 records the dialogue session. For example, the control unit 203 records a video of the avatar during the dialogue. For example, the control unit 203 receives and records a video of the dialogue participants captured by the camera 160 via the communication unit 201.

[0100] In the repetition of the process from step S1002 to step S1004, the control unit 203 associates a question theme with the playback position in the video where the question theme begins. For example, the control unit 203 determines that a question theme has begun under predetermined conditions. These predetermined conditions include, for example, when the control unit 203 performs voice analysis on the ongoing dialogue and detects that the avatar has spoken a line that suggests the start of a question theme (for example, "First, here is the first question theme," or "We will now move on to the next question theme"). Each time the control unit 203 determines that a question theme has begun, it associates the playback position where the question theme was determined to have started with that question theme.

[0101] In step S1005, the server 20 detects that the interlocutor has left the conversation. Specifically, when the interlocutor voluntarily leaves the conversation by operating the conversation screen (for example, the interlocutor taps a button to end the conversation on the conversation screen via the terminal device 10), the control unit 203 receives notification of the departure via the communication unit 201. More specifically, if communication from the terminal device 10 to the server 20 is interrupted for a certain period of time due to external factors (for example, a failure in the network environment surrounding the terminal device 10), the control unit 203 determines that the interlocutor has left the conversation. The control unit 203 then finishes recording the conversation video.

[0102] Furthermore, if the control unit 203 detects a departure, it may invalidate the association between the question topic at the time of departure and the playback position where that topic started.

[0103] Furthermore, the control unit 203 marks the question topic at the time the interlocutor left the conversation, for reference when resuming the conversation.

[0104] In step S1006, the server 20 stores the recorded dialogue video (from the start of the dialogue until the dialogue participant leaves) and information related to the video (e.g., dialogue session ID, dialogue participant ID, video ID, dialogue log, reason for leaving, departure location, question theme ID, playback location) as first video data. Specifically, the control unit 203 stores the first video data by storing information corresponding to each column of the dialogue session table 2023 (dialogue session ID, dialogue participant ID, video ID, video data, dialogue log, reason for leaving, departure location, question theme ID, playback location) based on the information stored as the dialogue video. The first video data includes, for example, at least a video of the dialogue participant speaking. The first video data may also include, for example, a video of an avatar and a video of the dialogue participant recorded in parallel during the same dialogue.

[0105] <5.2. Operation 2>

[0106] Figure 8 is a flowchart illustrating an example of how the dialogue session interrupted in Figure 7 resumes and the second video data is stored in System 1.

[0107] In step S1101, the server 20 detects the participant's return to the dialogue. Specifically, the participant accesses the dialogue platform (participant screen) via a web browser or a dedicated dialogue application on the terminal device 10, following a predetermined procedure. The predetermined procedure is, for example, logging in. On the dialogue platform on the terminal device 10, the participant selects one dialogue session from those assigned to them that they wish to resume. The transmitting / receiving unit 192 transmits a message to the server 20 via the communication unit 120 indicating the return to the selected dialogue session. The control unit 203 receives the message indicating the return to the dialogue session via the communication unit 201.

[0108] In step S1102, the server 20 presents the interlocutor with a message regarding resuming the session. Specifically, the control unit 203 displays, for example, on the display 141 of the terminal device 10 a message asking whether to resume the dialogue session from the beginning of the question topic that was in progress when the interlocutor left the dialogue platform, and a button to select whether to resume or not.

[0109] In step S1103, the server 20 receives a request from the interlocutor to resume the conversation. Specifically, the interlocutor taps a button on the terminal device 10 to resume the conversation session from the beginning of the question topic displayed. The transmitting / receiving unit 192 sends a resume request to the server 20 via the communication unit 120. The control unit 203 receives the selected resume request via the communication unit 201.

[0110] In step S1104, the server 20 restarts the dialogue session. Specifically, the control unit 203 restarts the dialogue session from the beginning of the question topic at the time of departure, which has been marked for reference during restart. During the dialogue of the restarted session, the processes from steps S1002 to S1004 are repeated.

[0111] During the repetition of the process from step S1002 to step S1004, the control unit 203 records the conversation session after the restart. The control unit 203 records, for example, a video of the avatar during the conversation. The control unit 203 receives, for example, a video of the conversational participants captured by the camera 160 via the communication unit 201 and records it.

[0112] In the repetition of the process from step S1002 to step S1004, the control unit 203 associates a question theme with the playback position in the video where the question theme begins. For example, the control unit 203 determines that a question theme has begun under predetermined conditions. These predetermined conditions include, for example, when the control unit 203 performs voice analysis on the ongoing dialogue and detects that the avatar has spoken a line that suggests the start of a question theme (for example, "First, here is the first question theme," or "We will now move on to the next question theme"). Each time the control unit 203 determines that a question theme has begun, it associates the playback position where the question theme was determined to have started with that question theme.

[0113] In step S1105, the server 20 terminates the dialogue session under predetermined conditions. These predetermined conditions include, for example, the dialogue session exceeding the set time, or all matters to be heard for all question themes set in the dialogue session being heard. Once the dialogue session ends, the control unit 203 finishes recording the dialogue video.

[0114] In step S1106, the server 20 stores the recorded dialogue video (from the time the dialogue resumes until it ends) and information related to the video (e.g., dialogue session ID, dialogue participant ID, video ID, dialogue log, question theme ID, playback position) as second video data. Specifically, the control unit 203 stores the second video data by storing information corresponding to each column of the dialogue session table 2023 (dialogue session ID, dialogue participant ID, video ID, video data, dialogue log, question theme ID, playback position) based on the information stored as the dialogue video. The second video data includes, for example, at least a video of the dialogue participants speaking. The second video data may also include, for example, a video of an avatar and a video of the dialogue participants recorded in parallel during the same dialogue.

[0115] Furthermore, the server 20 stores, for video related to a dialogue session in which the interlocutor did not leave, one recorded dialogue video (from the start of the dialogue to the end of the dialogue) and information related to the video (for example, dialogue session ID, interlocutor ID, video ID, dialogue log, question theme ID, playback position) through steps S1001, S1002, S1003, S1004, S1104, S1105, and S1106. Specifically, the control unit 203 stores one video data by storing information corresponding to each column of the dialogue session table 2023 (dialogue session ID, interlocutor ID, video ID, video data, dialogue log, question theme ID, playback position) based on the information stored as the dialogue video. The one video data includes, for example, at least a video of the interlocutor speaking. The one video data may also include, for example, a video of an avatar and a video of the interlocutor that were recorded in parallel during the same dialogue.

[0116] <5.3. Operation 3>

[0117] Figure 9 is a flowchart illustrating an example of how an administrator plays a video of a conversation in System 1.

[0118] In step S1201, the server 20 receives a request from the administrator to present video information. Specifically, the administrator accesses the conversation platform (administrator screen) via a predetermined operation using a web browser or a dedicated conversation application on the terminal device 40. The predetermined operation is, for example, logging in. On the conversation platform on the terminal device 40, the administrator selects one conversation session from among the conversation sessions they manage in which they wish to play a video. The transmitting / receiving unit 192 sends a request to the server 20 via the communication unit 120 to present video information for the selected conversation session. The control unit 203 receives the request to present video information for the conversation session via the communication unit 201.

[0119] In step S1202, the server 20 presents video information to the administrator. Specifically, the control unit 203 displays a complete set of basic information related to the dialogue (for example, video data, dialogue session ID, names of the dialogue participants, question themes included in the dialogue, and dialogue logs) and video operation buttons on the display 141 of the terminal device 40 as a viewing screen for the dialogue platform. For dialogue sections where there were withdrawals and resumptions, information related to the first video data and information related to the second video data are displayed. For dialogue sections where there were no withdrawals, information related to the first video data is displayed.

[0120] In step S1203, the server 20 receives a question topic from the administrator. Specifically, the administrator specifies the question topic by tapping the question topic they want to play on the viewing screen of the dialogue platform on the terminal device 40. The control unit 203 receives a request to present video information of the dialogue session via the communication unit 201.

[0121] In step S1204, the server 20 plays the video from the position associated with the specified question theme. Specifically, the control unit 203 plays the first video data or the second video data on the viewing screen of the dialogue platform from the playback position associated with the question theme.

[0122] <6. Screen Example> Examples of screens on the display 141 of terminal device 10 or terminal device 40 in this disclosure are described below.

[0123] Figure 10 shows an example screen when a person interacts with an avatar on the terminal device 10. This screen is a user interface for interaction between the person interacting with the generating AI, for example, to receive statements from the person interacting with the AI ​​and to present the user with responses from the generating AI. The presentation control unit 193 controls, for example, the speech and actions of the avatar displayed on the display 141, and causes the avatar to speak voice data. Note that this screen example is just one example, and various screen configurations and screen contents can be adopted. For example, only the avatar 3011, which will be described later, may be displayed on the display 141.

[0124] The avatar display area 3001 is the area where the image or animation of the avatar 3011 is displayed. The avatar 3011 is, for example, an interface that has the appearance and voice of a human and speaks and acts under the control of the control unit 203. When the avatar 3011 speaks, the movements of the avatar 3011's mouth and other features may be expressed in sync. The avatar 3011's speech is realized, for example, by sound output from the speaker 172, but it may also be realized by displaying the content of the speech as text near the display area of ​​the avatar 3011. In this case, the avatar 3011 may function only as an icon without speaking. The text 3012 is text indicating the ongoing question topic. The button 3013 is a button for the interlocutor to voluntarily withdraw from the conversation.

[0125] The speech display area 3002 is an area where speech entered by the user via voice or text is displayed in text format.

[0126] The response display area 3003 is an auxiliary display area for the user to recognize the response from the avatar 3011. Since the avatar 3011 responds by voice, the text of the response may not be displayed in this area, or if text data is sent from the server 20 along with the voice data, that text may be displayed.

[0127] The input means display area 3004 is an area that displays means for the user to input a statement to the avatar 3011. In this example screen, for example, a microphone icon 3021 for starting voice input and a field 3022 for supplementary text input are displayed in this area. For example, it is assumed that when the operation reception unit 191 receives a tap of the microphone icon 3021 by the user, voice input from the microphone 171 will be started to be received.

[0128] The send button 3005 is a button used to send a statement (mainly voice input or text input) entered by the user via the input means display area 3004 to the server 20.

[0129] The dialogue log display area 3006 is an area where the history of a series of interactions between the user and the avatar 3011 (specifically, the generated AI) is displayed in chronological order. The user can, for example, scroll through and review past conversations.

[0130] Figure 11 shows an example screen when the interlocutor selects an option related to resuming the interaction on the terminal device 10. Note that this is merely one example, and various screen configurations and content can be adopted.

[0131] The message display area 4001 is an area that displays a message asking whether or not to resume the dialogue session from the beginning of the question topic that was in progress when the interlocutor left.

[0132] Button 4002 is a button that allows the interlocutor to select whether or not to resume the dialogue session from the beginning of the question topic that was in progress at the time the interlocutor left. If the interlocutor selects the resume button, the control unit 203 resumes the dialogue session from the beginning of the question topic at the time of the interlocutor's departure.

[0133] Figure 12 shows an example screen when the administrator plays a video on terminal device 40. Note that this is just one example, and various screen configurations and content can be adopted.

[0134] The dialogue basic information display area 5001 is an area where basic information of the dialogue session is displayed. This basic information includes, for example, the dialogue session ID and the names of the participants.

[0135] The video display area 5002 is the area where the video of the dialogue session is displayed. In the example in Figure 12, the video of the dialogue participant is displayed in the video display area 5002, but the video of the dialogue participant and the video of the avatar may be displayed in a divided video display area 5002.

[0136] Bar 5003 is a bar that displays the playback position in the video. Administrators can adjust the playback position by moving the dots on bar 5003 left or right. Markers indicating the point where the question topic changes may also be displayed on bar 5003.

[0137] Button 5004 is a button used to control the video. Button 5004 consists of, for example, a play button, a rewind button, and a fast forward button.

[0138] The video basic information display area 5005 is the area where basic information about the video is displayed. A video basic information display area 5005 is placed for each video data. When the video basic information display area 5005 is clicked, it displays the basic information of the video data (video name and a set of question themes. The set of question themes includes the question content, summary, and a collapsible full text display) and icons for operation.

[0139] Furthermore, the video basic information display area 5005 may be arranged not for each individual video data, but rather by consolidating all video data into a single video basic information display area. In this case, multiple sets of question themes can be arranged within the video basic information display area 5005 without partitions separating each video data.

[0140] Icon 5006 is an icon that, when tapped, will start playing the video data from the beginning.

[0141] Icon 5007 is an icon for downloading video data to the terminal device 40 in response to a tap. Separately from icon 5007, an icon for comprehensively downloading the first and second video data of the conversation may also be displayed on the screen shown in Figure 12. When the administrator taps the comprehensive download icon, for example, the control unit 203 combines the first and second video data and sends it to the terminal device 40.

[0142] Icon 5008 is an icon that, when tapped, copies the entire dialogue log of the video data.

[0143] Icon 5009 is an icon that, when tapped, plays video data from the beginning of the question topic.

[0144] Icon 5010 is an icon that, when tapped, copies the entire dialogue log of the question topic.

[0145] Icon 5011 is an icon that, when tapped, displays the full text of the hidden dialogue log for the question topic.

[0146] <7.Summary> As described above, in the above embodiment, the server 20 stores a video of the conversation between the interlocutor and the generated AI as first video data. If the interlocutor leaves the conversation midway and then returns to it, the server 20 resumes the conversation from a point related to the question topic at the time of departure. The server 20 resumes the conversation from the beginning of the question topic that was in progress when the interlocutor left. The server 20 presents the interlocutor with a message regarding the resumption of the conversation, receives a request from the interlocutor to resume the conversation, and then resumes the conversation. This makes it easier to resume the conversation from an appropriate point in time, and ultimately improves the convenience of the AI ​​conversation service.

[0147] Furthermore, as described above, in the above embodiment, the server 20 presents the administrator with information about the interlocutor, multiple video data related to the dialogue, and question themes included in the dialogue. The server 20 associates the question themes included in the dialogue with the playback position in the first video data or the second video data where the question themes start, and presents the question themes together with the playback positions to the administrator. This makes it easier for the administrator (employer) to understand the flow of the video.

[0148] When server 20 receives a specification from the administrator regarding a question theme included in the dialogue, it plays either the first or second video data from the playback position associated with that question theme. This improves the convenience of video playback for the administrator (the person who adopted the system).

[0149] <8. Variation> A modified example of the above embodiment will be described.

[0150] <8.1. Variation 1> Server 20 may evaluate the interlocutor based on the first video data and the second video data. Specifically, the control unit 203 inputs a prompt to the generating AI system 30 via the communication unit 201, which includes the first video data, the second video data, and instructions for evaluating the interlocutor. For example, if the dialogue is a job interview, the instructions might be: "Evaluate the abilities, qualities, suitability, etc., of the applicant interlocutor based on the first video data and the second video data. Evaluate the interlocutor based on the question themes included in the first video data and the second video data (including the question themes at the time the interlocutor left)." The control unit 203 receives the evaluation of the interlocutor output from the generating AI system 30.

[0151] <8.2. Modification 2> In the above embodiment, in step S1005, the server 20 marks the question topic at the time the interlocutor left the dialogue for reference when resuming, in step S1102, it presents the interlocutor with a message regarding resuming, in step S1103, the server 20 receives a request for resuming from the interlocutor, and in step S1104, upon receiving the request for resuming, it resumes the dialogue session from the beginning of the question topic at the time the interlocutor left.

[0152] However, the server 20 may mark the point in time when the interlocutor left the dialogue for reference purposes, present the interlocutor with a message regarding resumption, and, upon receiving a request from the interlocutor to resume the dialogue session, restart the dialogue session from the point in time when the interlocutor left the dialogue. Alternatively, the server 20 may mark the question items that were in progress when the interlocutor left the dialogue for reference purposes, present the interlocutor with a message regarding resumption, and, upon receiving a request from the interlocutor to resume the dialogue session, restart the dialogue session from the beginning of the question items that were in progress when the interlocutor left the dialogue. Alternatively, the server 20 may not mark the point for reference purposes, present the interlocutor with a message regarding restarting the dialogue, and, upon receiving a request from the interlocutor to restart the dialogue session, start the dialogue session from the beginning, i.e., restart the dialogue session from the beginning. This increases the options available for returning to a dialogue in the dialogue service.

[0153] Furthermore, regarding a dialogue from which a participant has withdrawn, Server 20 only accepts requests from the participant to resume the dialogue, and does not need to accept requests from the participant to restart the dialogue. This prevents unfairness arising from restarting a dialogue when the dialogue forms the basis for evaluation, hiring, compensation, etc.

[0154] Furthermore, the server 20 may pre-set an upper limit on the number of times a dialogue can be restarted if a participant has left the dialogue. For example, an administrator can pre-set an upper limit on the number of times a dialogue can be restarted via the terminal device 40. This prevents unfairness caused by restarting dialogues when the dialogue is the basis for evaluation, hiring, compensation, etc.

[0155] <8.3. Modification 3> In the above embodiment, during the repetition of the processing from step S1002 to step S1004, the control unit 203 associates the question theme with the playback position in the video where the question theme begins. However, after step S1006, the control unit 203 may associate the question theme with the playback position in the video where the question theme begins for the first video data. In this case, it is not necessary to associate the question theme at the time of departure with the playback position in which that theme begins. Furthermore, after step S1106, the control unit 203 may associate the question theme with the playback position in the video where the question theme begins for the second video data.

[0156] <9. Basic Computer Hardware Configuration> Figure 13 is a block diagram showing the basic hardware configuration of computer 90. Computer 90 comprises at least a processor 901, main memory 902, auxiliary memory 903, and a communication interface IF991. These are electrically connected to each other by a communication bus 921.

[0157] The processor 901 is hardware for executing the instruction set written in a program. The processor 901 consists of an arithmetic unit, registers, peripheral circuits, etc.

[0158] Main memory 902 is used to temporarily store programs and data processed by programs, etc. For example, it is a volatile memory such as DRAM (Dynamic Random Access Memory).

[0159] Auxiliary storage device 903 refers to a storage device for saving data and programs. Examples include flash memory, HDD (Hard Disc Drive), magneto-optical disk, CD-ROM, DVD-ROM, and semiconductor memory.

[0160] The IF991 communication interface is an interface for inputting and outputting signals for communication with other computers via a network using wired or wireless communication standards. A network consists of various mobile communication systems, such as the Internet, LANs, and wireless base stations. For example, a network includes 3G, 4G, and 5G mobile communication systems, LTE (Long Term Evolution), and wireless networks that can connect to the Internet via designated access points (e.g., Wi-Fi®). When connecting wirelessly, communication protocols include, for example, Z-Wave®, ZigBee®, and Bluetooth®. When connecting via a wired connection, the network also includes connections made directly via USB (Universal Serial Bus) cables, etc.

[0161] Furthermore, by distributing all or part of each hardware configuration across multiple computers 90 and connecting them to each other via a network, a computer 90 can be virtually realized. Thus, the concept of computer 90 includes not only a computer 90 housed in a single enclosure or case, but also a virtualized computer system.

[0162] <Basic Functional Configuration of Computer 90> The functional configuration of the computer realized by the basic hardware configuration of computer 90 is described below. The computer comprises at least one functional unit: a control unit, a memory unit, and a communication unit.

[0163] Furthermore, the functional units of computer 90 can also be realized by distributing all or part of each functional unit across multiple computers 90 interconnected via a network. The concept of computer 90 includes not only a single computer 90 but also a virtualized computer system.

[0164] The control unit is realized when the processor 901 reads various programs stored in the auxiliary storage device 903, loads them into the main memory device 902, and executes processing according to those programs. The control unit can realize various functional units that perform information processing depending on the type of program. In this way, the computer is realized as an information processing device that performs information processing.

[0165] The memory unit is implemented by the main memory 902 and the auxiliary memory 903. The memory unit stores data, various programs, and various databases. The processor 901 can also reserve memory areas corresponding to the memory unit in the main memory 902 or the auxiliary memory 903 according to the program. The control unit can also cause the processor 901 to perform operations such as adding, updating, and deleting data stored in the memory unit according to the various programs.

[0166] A database, specifically a relational database, is used to manage and link together tabular data sets called masters, which are structurally defined by rows and columns. In a database, tables are called tables, masters are called masters, the columns of tables are called columns, and the rows of tables are called records. In a relational database, relationships can be established and linked between tables and masters. Typically, each table and master has a primary key column to uniquely identify records, but setting a primary key column is not mandatory. The control unit can instruct the processor 901 to add, delete, or update records in specific tables and masters stored in the memory unit, according to various programs. Furthermore, by storing data, various programs, and various databases in the memory unit, the information processing device and information processing system related to this disclosure can be considered to have been manufactured.

[0167] Furthermore, the databases and masters in this disclosure may include any data structures (lists, dictionaries, associative arrays, objects, etc.) in which information is structurally defined. Data structures also include data that can be considered as data structures by combining data with functions, classes, methods, etc., written in any programming language.

[0168] The communication unit is implemented by the communication IF991. The communication unit provides the functionality to communicate with other computers 90 via the network. The communication unit can receive information transmitted from other computers 90 and input it to the control unit. The control unit can cause the processor 901 to perform information processing on the received information according to various programs. The communication unit can also transmit information output from the control unit to other computers 90.

[0169] Furthermore, each of the above-mentioned configurations, functions, processing units, processing means, etc., may be implemented in hardware, either partially or entirely, by designing them as integrated circuits, for example. The present invention can also be implemented by software program code that realizes the functions of the embodiment. In this case, a storage medium on which the program code is recorded is provided to a computer, and the processor of that computer reads the program code stored in the storage medium. In this case, the program code read from the storage medium itself realizes the functions of the embodiment described above, and the program code itself and the storage medium on which it is stored constitute the present invention. Examples of storage media used to supply such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, SSDs, optical disks, magneto-optical disks, CD-Rs, magnetic tapes, non-volatile memory cards, ROMs, and the like.

[0170] Furthermore, the program code that implements the functions described in this embodiment can be implemented in a wide range of programming or scripting languages, such as assembler, C / C++, Perl, Shell, PHP, and Java (registered trademark).

[0171] Furthermore, the program code for the software that implements the functions of the embodiment may be distributed via a network and stored in a storage means such as a computer's hard disk or memory, or in a storage medium such as a CD-RW or CD-R, and the computer's processor may read and execute the program code stored in the storage means or storage medium.

[0172] The functions realized by the components described herein may be implemented in a circuit or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), CPUs (a Central Processing Unit), conventional circuits, and / or combinations thereof, programmed to realize the functions described herein. A processor is considered to be a circuit or processing circuitry, including transistors and other circuits. A processor may be a programmed processor that executes a program stored in memory. In this specification, circuitry, unit, and means are hardware programmed to perform or execute the functions described herein. Such hardware may be any hardware disclosed herein, or any hardware known to be programmed to perform or execute the functions described herein. If the hardware is a processor that is considered to be a type of circuitry, then the circuitry, means, or unit is a combination of hardware and software used to constitute the hardware and / or processor.

[0173] While several embodiments of this disclosure have been described above, these embodiments can be implemented in a variety of other forms, and various omissions, substitutions, and modifications are permitted without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents.

[0174] (Note) The details described in each of the above embodiments are noted below.

[0175] (Note 1) A program for operating a computer comprising a processor and memory, wherein the program is configured to operate the processor, The steps include: storing a video of the dialogue between the interlocutor and the generating AI as first video data; and, if the interlocutor leaves the dialogue midway through and then returns to the dialogue, restarting the dialogue from a point related to the question topic at the time of departure; The steps include: storing the video of the dialogue after its resumption as a second video data; A program that executes the command. (Note 2) The program described in (Appendix 1) wherein, in the step of resuming the dialogue, the dialogue is resumed from the point in time when the dialogue participant left the dialogue. (Note 3) The program described in (Appendix 1) or (Appendix 2), wherein in the step of resuming the dialogue, the dialogue is resumed from the beginning of the question topic that was in progress at the time the dialogue participant withdrew. (Note 4) In the step of resuming the dialogue, the program described in any of (Appendix 1) to (Appendix 3) resumes the dialogue from the beginning of the question items that were in progress when the dialogue participant withdrew. (Note 5) A program as described in any of (Appendix 1) to (Appendix 4), wherein in the step of resuming the dialogue, a message relating to the resumption of the dialogue is presented to the dialogue participant, a request for the resumption of the dialogue is received from the dialogue participant, and the dialogue is resumed. (Note 6) In the step of restarting, the program presents the dialogue participant with a message regarding restarting the dialogue, receives a request from the dialogue participant to restart the dialogue, and restarts the dialogue from the beginning, as described in any of the notes (1) to (5). (Note 7) The aforementioned processor, A step of presenting the administrator with information about the interlocutor, multiple video data related to the interlocutor, and the question themes included in the interlocutor. A program described in any of (Appendix 1) to (Appendix 6) that executes the above. (Note 8) The aforementioned processor, The steps include associating the question theme included in the dialogue with the playback position on the first video data or the playback position on the second video data in which the question theme started, The step of presenting the aforementioned question topic, along with the playback position, to the administrator. The program described in (Appendix 7) that executes the above. (Note 9) The program described in (Appendix 8) wherein, in the associated step, the association between the question topic that was in progress at the time the interlocutor left and the playback position on the first video data is invalidated. (Note 10) The aforementioned processor, When the administrator specifies a question theme included in the dialogue, the first video data or the second video data is played from the playback position associated with that question theme. The program described in (Appendix 8) or (Appendix 9) that causes the execution of the above. (Note 11) The aforementioned processor, Steps to evaluate the interlocutor based on the first video data and the second video data. A program described in any of (Appendix 1) to (Appendix 10) that executes the above. (Note 12) A method to be performed on a computer comprising a processor and memory, wherein the processor performs all steps performed in any of the inventions described in (Appendix 1) to (Appendix 11). (Note 13) An information processing apparatus comprising a control unit and a storage unit, wherein the control unit performs all steps performed in any of the inventions described in (Appendix 1) to (Appendix 11). (Note 14) A system comprising means for performing all steps performed in any of the inventions described in (Appendix 1) to (Appendix 11). [Explanation of symbols]

[0176] 1... System 10…Terminal device 12…Communication IF 13…Input device 14…Output device 15…Memory 16…Storage 19… Processor 20... Server 22...Communication IF 23…Input / Output Interface 25…Memory 2 hours… storage 29… Processor 80…Network

Claims

1. A program for operating a computer comprising a processor and memory, wherein the program is configured to operate the processor, The steps include: storing a video of the person speaking with the AI ​​as the first video data; A step of detecting that the person speaking to the conversation has left the conversation midway through the conversation, In response to the detection of the aforementioned withdrawal, the steps include marking a point in time for reference to resume, A step of detecting that the person speaking has returned to the conversation, The steps include: referring to the marked time, and resuming the dialogue from the marked time; The steps include storing a video of the person speaking during the conversation between the person speaking and the generating AI after the conversation has resumed as second video data, The steps include associating the question theme included in the dialogue with the playback position on the first video data or the playback position on the second video data where the question theme is initiated, The steps include presenting the administrator with information about the interlocutor, an instruction input object for the first video data, an instruction input object for the second video data, and a question theme associated with the playback position on the first video data or the playback position on the second video data, on the same screen. A program that executes the command.

2. The program according to claim 1, wherein in the step of resuming the dialogue, the dialogue is resumed from the point in time when the dialogue participant left the dialogue.

3. The program according to claim 1, wherein in the step of resuming the dialogue, the dialogue is resumed from the beginning of the question topic that was in progress at the time the dialogue participant left.

4. The program according to claim 1, wherein in the step of resuming the dialogue, the dialogue is resumed from the beginning of the question items that were in progress at the time the dialogue participant withdrew.

5. The program according to claim 1, wherein in the step of resuming the dialogue, a message relating to the resumption of the dialogue is presented to the dialogue participant, a request for the resumption of the dialogue is received from the dialogue participant, and the dialogue is resumed.

6. The program according to claim 1, wherein in the step of restarting, a message relating to restarting the dialogue is presented to the dialogue participant, a request for restarting the dialogue is received from the dialogue participant, and the dialogue is restarted from the beginning.

7. The program according to claim 1, wherein, in the step of detecting the departure, when the departure is detected, the association between the question topic that was in progress at the time the interlocutor left and the playback position on the first video data is deleted.

8. The aforementioned processor, When the administrator specifies a question theme included in the dialogue, the first video data or the second video data is played from the playback position associated with that question theme. The program according to claim 1, which causes to execute.

9. The aforementioned processor, The steps include inputting a prompt to the generating AI that includes the first video data, the second video data, and instructions for evaluating the interlocutor based on the first and second video data, and obtaining the evaluation of the interlocutor output from the generating AI. The program according to claim 1, which causes to execute.

10. A method to be performed on a computer comprising a processor and memory, wherein the processor performs all steps performed in the invention according to any one of claims 1 to 9.

11. An information processing apparatus comprising a control unit and a storage unit, wherein the control unit performs all steps performed in the invention according to any one of claims 1 to 9.

12. A system comprising means for performing all steps performed in the invention according to any one of claims 1 to 9.

Images