Information processing system and its program

Abstract

This invention relates to an information processing technology that optimizes presented information in response to the participant's reactions during a dialogue session and continuously improves it based on the outcome information. Conventionally, it was possible to provide advice based on the participant's emotions, but it was difficult to optimize the content and timing of presentation in conjunction with past performance results. In this invention, the participant's reaction features are calculated from audio, video, text, or operation history, and the information to be presented is determined based on these features and past outcome information, and is presented in synchronization with the point in time when the reaction changes or the end of a topic. Furthermore, the selection rules are updated based on target events such as application, passing the selection process, or accepting a job offer, which are obtained after the dialogue. This enables the presentation of appropriate information according to the dialogue situation and its learning-based optimization, improving the certainty of goal achievement and the reproducibility of dialogue support.

Description

Technical Field

[0001] The present invention relates to an information processing system that controls presentation information based on reaction features of a target person in a dialogue session and updates an information selection rule using post-event result information.

Background Art

[0002] In recent years, with the decrease in the working population, securing human resources has become an important management issue for companies, and there is a demand for improving efficiency and filling rate in recruitment activities. In the field of conventional recruitment screening, analog and personal activities that rely on the experience and intuition of interviewers have been central, and it has been difficult to objectively capture the detailed reactions of candidates and provide optimal information and attraction. On the other hand, a conversation analysis system that extracts the mood of the other party from voice and video during a dialogue and gives advice and proposals in real time, such as the technology described in Patent Document 1, is known. However, a mechanism for controlling the presentation content and timing by associating the reaction features of the target person during the dialogue session with past result information, and further updating the information selection rule using target events such as success or failure obtained after the dialogue has not been sufficiently established, and a technology for scientifically optimizing the dialogue based on objective data is desired.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In conventional dialogue support technologies, for example, in the system described in Patent Document 1, it was possible to extract the mood of the interlocutor and give proposals. However, in a scenario aiming for a specific result by a dialogue session, it has been difficult to reflect past result information and reaction tendencies of similar target persons in detail up to the selection of information to be presented and its presentation timing.

[0005] This invention has been made in view of the above problems, and its objective is to provide a means for dynamically controlling the type and timing of information presentation based on response features during dialogue and past outcome information. Furthermore, a major objective is to improve the reproducibility and suitability of dialogue support by updating selection rules using target events after the dialogue session. [Means for solving the problem]

[0006] The information processing system of the present invention calculates response features of the first subject based on data such as audio, video, text, or operation history obtained from a dialogue session between the first subject and the second subject. Based on these response features and past performance information, it determines information to be presented during the dialogue, such as talk scripts and candidate dialogue partners. It also generates presentation control data to present the information in synchronization with the point in time when the response features change or when a topic is divided. Furthermore, it updates selection rules using information about target events such as application, passing the selection process, or accepting a job offer, obtained after the dialogue. In this way, by configuring a dynamic presentation control and learning loop that combines response features and past performance data, the system realizes the presentation of optimal information in accordance with the dialogue subject's responses, improving the suitability of dialogue support and the certainty of achieving goals. [Effects of the Invention]

[0007] According to the present invention, the optimal type and timing of information presentation are dynamically controlled based on the real-time responses of the person being spoken to and their past performance, thereby improving the certainty of goal achievement without relying on individual experience. In particular, by presenting information in sync with changes in responses and breaks in topics, the motivation of the person being spoken to is effectively drawn out, increasing the success rate of recruitment and other outcomes, and the quality of dialogue support can be automatically optimized through continuous learning that reflects subsequent results. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows the configuration of an information processing system according to an embodiment of the present invention. [Figure 2] This is a block diagram showing the functional configuration of an information processing device. [Figure 3] This flowchart shows the overall procedure for determining the information to be presented and controlling the timing of its presentation. [Figure 4] This flowchart shows the details of the process for calculating the response features of the first subject. [Figure 5] This diagram illustrates the concept of selecting information to present based on past performance data and the attributes of similar target individuals. [Figure 6] This timeline diagram illustrates an example of a process that controls the presentation timing in synchronization with the point in time when the response features change or the end of a topic. [Figure 7] This figure shows an example of a user interface for displaying the target information on the terminal of the second target person. [Figure 8] This flowchart shows the steps for the learning process that updates the selection rules after the occurrence of a target event. [Figure 9] This diagram shows the structure of a filtering process that excludes items unrelated to aptitude or ability from the list of candidates. [Figure 10] This figure shows an example of a data structure for managing potential conversation partners and talk scripts as information to be presented. [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described in detail below with reference to the drawings.

[0010] Figure 1 shows the configuration of an information processing system 10 according to an embodiment of the present invention.

[0011] The information processing system 10 is configured as a system to support a dialogue session conducted between the first subject and the second subject.

[0012] In this embodiment, a dialogue session refers to a unit of communication in which the first subject and the second subject exchange information with each other.

[0013] The dialogue session includes online dialogues conducted via a communication network and offline dialogues conducted in the same space.

[0014] In addition, the dialogue session may include not only synchronous dialogues conducted in real time but also asynchronous dialogues conducted via recorded information or the like.

[0015] The information processing system 10 includes an information processing device 100.

[0016] The information processing system 10 also includes a first user terminal 110.

[0017] Furthermore, the information processing system 10 includes a second user terminal 120. [[ID=2l]]

[0018] The information processing device 100 and the first user terminal 110 are communicably connected to each other via a network 130.

[0019] The information processing device 100 and the second user terminal 120 are also communicably connected to each other via the same network 130.

[0020] The first user terminal 110 and the second user terminal 120 are communicably connected to each other via the information processing device 100 or directly via the network 130.

[0021] The network 130 is constituted by the Internet, an intranet, a local area network, a wide area network, or a combination thereof.

[0022] The network 130 includes not only a wired communication network but also a wireless communication network.

[0023] As the wireless communication network, for example, a mobile phone communication network or a wireless local area network is applicable.

[0024] Network 130 supports various communication protocols for data communication.

[0025] In the information processing system 10, the information processing device 100 functions as a server computer that controls the entire system (see Figure 2).

[0026] The information processing device 100 may consist of a single computer device, or it may be configured as a distributed computing system in which multiple computer devices work together.

[0027] In a cloud computing environment, it is also possible for each function of the information processing device 100 to be implemented in a distributed manner across multiple virtual servers.

[0028] The information processing device 100 is responsible for the main data processing, information storage, and communication control in the information processing system 10 (see Figures 3 to 10).

[0029] The first participant terminal 110 is a terminal device used by the first participant who is participating in the dialogue session (see Figure 1).

[0030] The first target terminal 110 can be any type of computing device, such as a personal computer, smartphone, tablet, or wearable device.

[0031] The first target terminal 110 is equipped with an input device that accepts operation input from the first target.

[0032] Furthermore, the first target terminal 110 is equipped with an output device that outputs images and audio to the first target.

[0033] The second participant terminal 120 is a terminal device used by the second participant who is participating in the dialogue session (see Figure 1).

[0034] Similar to the first target terminal 110, various devices such as personal computers, smartphones, and tablet devices are used as the second target terminal 120.

[0035] The second target terminal 120 is equipped with an input device that receives operation input from the second target and an output device that outputs images and sounds.

[0036] The hardware configuration of the information processing device 100 will be described (see Figure 1).

[0037] The information processing device 100 is equipped with a central processing unit that is responsible for the primary control.

[0038] The central processing unit (CPA) is a processor that interprets programs and performs various arithmetic operations.

[0039] The information processing device 100 is equipped with a main memory that functions as a work area for the central processing unit.

[0040] The main memory uses volatile memory such as random access memory.

[0041] The information processing device 100 is equipped with an auxiliary storage device for permanently storing programs and data.

[0042] Auxiliary storage devices include non-volatile storage media such as hard disk drives, solid-state drives, and flash memory.

[0043] The auxiliary storage device stores the operating system and various application programs necessary to run the information processing system 10.

[0044] Furthermore, the auxiliary storage device stores a dedicated program to support the dialogue session (see Figures 2 to 10).

[0045] The information processing device 100 is equipped with a communication interface for data communication with other devices via the network 130.

[0046] The communication interface includes wired communication modules and wireless communication modules, and transmits and receives data according to a predetermined communication protocol.

[0047] The information processing device 100 may be equipped with an input interface that accepts input from administrators or other personnel as needed.

[0048] Furthermore, the information processing device 100 may be equipped with an output interface for displaying information to an administrator or the like.

[0049] The hardware configuration of the first target user terminal 110 will be described (see Figure 1).

[0050] The first target terminal 110 is equipped with a processor that controls the entire terminal.

[0051] The first target terminal 110 is equipped with memory that temporarily holds programs and data for the processor to execute.

[0052] The first target terminal 110 is equipped with a storage device that stores the operating system and application programs.

[0053] The first target terminal 110 is equipped with a communication module for connecting to the network 130.

[0054] The communication module has functions for performing wireless local area network communication and mobile data communication.

[0055] The first subject terminal 110 is equipped with an imaging device that acquires video of the first subject.

[0056] The imaging device is, for example, a camera, which captures the face and upper body of the first subject, as well as the surrounding environment.

[0057] The first subject terminal 110 is equipped with a sound collection device that acquires the voice of the first subject.

[0058] A sound collection device, such as a microphone, converts the speech content of the first subject and ambient sounds into electrical signals.

[0059] The first target terminal 110 is equipped with a display device that visually presents information.

[0060] Display devices include, for example, liquid crystal displays and organic electroluminescent displays.

[0061] The first target terminal 110 is equipped with an audio output device that outputs sound.

[0062] Audio output devices include, for example, speakers or earphone jacks, which output the voice of the person being spoken to or notification sounds from the system.

[0063] The first target terminal 110 is equipped with an input device that receives instructions from the first target.

[0064] Input devices include touch panels, keyboards, mice, and pointing devices.

[0065] The hardware configuration of the second target user terminal 120 will be described (see Figure 1).

[0066] The hardware configuration of the second target terminal 120 is basically the same as that of the first target terminal 110.

[0067] In other words, the second target terminal 120 is equipped with a processor, memory, storage device, and a communication module for connecting to the network 130.

[0068] The communication module, like the first target terminal 110, is equipped with functions for performing wireless local area network communication and mobile data communication.

[0069] The second subject terminal 120 is equipped with imaging devices such as a camera to acquire video of the second subject, and sound collection devices such as a microphone to acquire audio of the second subject.

[0070] The second subject terminal 120 is equipped with a display device for displaying video and information of the first subject, and an audio output device for outputting the first subject's voice, etc.

[0071] The second subject terminal 120 is equipped with an input device for the second subject to operate (see Figure 7).

[0072] In the information processing system 10, the information processing device 100, the first target terminal 110, and the second target terminal 120 operate in cooperation with each other via the network 130 (see Figure 1).

[0073] As the dialogue session begins, video and audio data transmission and reception start between the first participant terminal 110 and the second participant terminal 120.

[0074] The transmission and reception of video and audio data may be carried out via the information processing device 100, or it may be carried out directly between terminals.

[0075] The information processing device 100 has the function of collecting various types of data acquired during a dialogue session in real time.

[0076] The data collected by the information processing device 100 includes audio and video data of the first subject transmitted from the first subject terminal 110.

[0077] The data collected by the information processing device 100 may also include operation history data and text data from the first target terminal 110.

[0078] The information processing device 100 performs various information processing operations to support the dialogue session based on the collected data (see Figure 3).

[0079] Figure 2 is a block diagram showing the functional configuration of the information processing device 100.

[0080] The information processing device 100 includes a reaction feature calculation unit 200 as a functional block (see Figure 4).

[0081] The information processing device 100 includes a function block, which is a unit for determining the information to be presented (see Figures 5 and 9).

[0082] The information processing device 100 includes a presentation control data generation unit 220 as a functional block (see Figure 6).

[0083] The information processing device 100 includes a selection rule update unit 230 as a functional block (see Figure 8).

[0084] The information processing device 100 includes a storage unit 240 as a functional block (see Figure 10).

[0085] These functional blocks are realized by the processor of the information processing device 100 executing a program stored in memory.

[0086] Furthermore, some or all of each functional block may be implemented by dedicated hardware circuits.

[0087] The memory unit 240 is implemented by the auxiliary storage device or main memory of the information processing device 100 in the information processing system 10 shown in Figure 1.

[0088] The memory unit 240 has the function of permanently or temporarily storing various types of information necessary to support the dialogue session.

[0089] The memory unit 240 stores the first target attribute information 500 and history information.

[0090] The memory unit 240 stores information about the second subject and various content information to be used in the dialogue session.

[0091] Furthermore, the memory unit 240 stores a history of past dialogue sessions and past results information 510.

[0092] The memory unit 240 stores selection rules 520 for determining the information to be presented, parameters of the learning model, and the like, as shown in the data configuration in Figure 10.

[0093] The role of the reaction feature calculation unit 200 will be explained with reference to the block diagram in Figure 2.

[0094] The response feature calculation unit 200 monitors the data acquired during the dialogue session between the first subject and the second subject.

[0095] As shown in Figure 4, the reaction feature calculation unit 200 performs a reaction feature calculation step 300 to calculate features that indicate the reaction of the first subject from the acquired data.

[0096] The calculated response features are indicators that quantitatively represent the first participant's level of interest, understanding, emotional changes, or level of engagement during the dialogue session.

[0097] The response feature calculation unit 200 has the function of analyzing the acoustic characteristics of the audio data received from the first target terminal 110.

[0098] The reaction feature calculation unit 200 has the function of analyzing the image features of the video data received from the first target terminal 110.

[0099] The reaction feature calculation unit 200 also analyzes the features of text data and operation history data when they are acquired.

[0100] The reaction features calculated by the reaction feature calculation unit 200 are passed on to other functional blocks within the information processing device 100.

[0101] The role of the information to be presented determination unit 210 will be explained with reference to Figure 2 as appropriate.

[0102] As shown in Figure 3, the information to be presented determination unit 210 executes an information to be presented determination step 310, which determines the information to be presented during or before / after the dialogue session.

[0103] The information presentation determination unit 210 uses the reaction features calculated by the reaction feature calculation unit 200 as the basis for its determination.

[0104] The information presentation determination unit 210 also uses past performance information 510 stored in the storage unit 240 as the basis for its determination (see Figure 5).

[0105] The information presentation determination unit 210 selects the optimal information to present by comparing the current response state of the first subject with past successful cases.

[0106] The information presentation unit 210 refers to the selection rules 520 and the learning model stored in the memory unit 240 to determine the optimal one from among multiple candidates.

[0107] The information to be presented, as determined, includes various supplementary information that the filtering unit 910 has excluded from the presentation candidates based on the exclusion list 900, which is irrelevant to the suitability or abilities of the first subject (see Figure 9).

[0108] Furthermore, the information to be presented may also include a talk script 1000 and potential conversation partners 1010 to be displayed on the second target terminal 120 (see Figure 10).

[0109] The decision result made by the information to be presented unit 210 is notified to the presentation control data generation unit 220 (see Figure 2).

[0110] The role of the presentation control data generation unit 220 will now be explained.

[0111] The presentation control data generation unit 220 executes a presentation timing control step 320 to control the presentation timing of the information to be presented, which was determined in the presentation target information determination step 310 (see Figure 3).

[0112] The presentation control data generation unit 220 generates data for transmitting the presentation target information to a target device such as the second target terminal 120.

[0113] In addition to simply transmitting information, the presentation control data generation unit 220 generates presentation control data that includes instruction information for controlling the presentation timing 620.

[0114] The presentation control data generation unit 220 monitors the progress of the dialogue, the timing of changes in response features 600, and the topic division 610, and controls the presentation so that information is presented at the most effective presentation timing 620 (see Figure 6).

[0115] The presentation control data generated by the presentation control data generation unit 220 is transmitted to the second target terminal 120 via the network 130 (see Figure 1).

[0116] The second target terminal 120 outputs the information to be presented from the presentation information display area 710 of the dialogue screen 700 at the specified presentation timing 620, in accordance with the received presentation control data (see Figure 7).

[0117] The role of the selection rule update unit 230 will be explained.

[0118] The selection rule update unit 230 executes the learning process 810 using retrospective information obtained after the end of the dialogue session, etc. (see Figure 8).

[0119] The selection rule update unit 230 obtains the target event information 800 acquired after the dialogue session.

[0120] Target Event Information 800 is an outcome indicator that shows what results the dialogue session ultimately led to.

[0121] The selection rule update unit 230 associates the acquired target event information 800 with the reaction features calculated in the reaction feature calculation step 300 (see Figure 4) and the presented information.

[0122] The selection rule update unit 230 uses this associated performance data to perform a process to update the selection rule 520 used by the presentation target information determination unit 210.

[0123] The selection rule update unit 230 updates the selection rule 520, allowing the system to learn from past results information 510 and determine more appropriate information to present (see Figure 5).

[0124] The updated selection rules 520 and learning model parameters are stored in the memory unit 240 and used in the next dialogue session.

[0125] In the information processing system 10, the effectiveness of information presentation can be continuously improved through the coordination of each functional block.

[0126] The information processing system 10 is configured as a collection of programs that run on a computer.

[0127] The processor of the information processing device 100 executes a series of steps according to instructions written in the program.

[0128] As shown in Figure 3, the basic flow of a program running on a computer will be explained.

[0129] First, the program causes the information processing device 100 to perform a step that detects the start of an interactive session.

[0130] Once the dialogue session begins, the program executes a step that continuously receives data transmitted from the first participant terminal 110 and buffers it in memory.

[0131] Next, the program executes a reaction feature calculation step 300, which calculates the reaction features of the first subject from the received data (see Figure 4).

[0132] This response feature calculation step 300 is performed repeatedly at predetermined intervals during the progress of the dialogue session.

[0133] Next, the program executes a presentation target information determination step 310, which determines the information to be presented based on the calculated response features and past performance information 510.

[0134] In this information to be presented determination step 310, the storage unit 240 is accessed and the selection rule 520 is applied to perform calculations to select the optimal information (see Figure 5). At this time, the filtering unit 910 shown in Figure 9 may exclude information based on the exclusion list 900.

[0135] Next, the program causes the presentation control data generation unit 220 to execute the step of generating presentation control data for outputting the determined presentation target information.

[0136] In this generation step, control commands are constructed that define the destination of the information, the content to be sent, such as the talk script 1000 or the candidate conversation partner 1010, and the presentation timing 620 (see Figures 6 and 10).

[0137] The program performs the process of sending the generated presentation control data to the second target terminal 120 via the communication interface.

[0138] On the second target terminal 120, based on the received presentation control data, operations such as displaying information in the presentation information display area 710 of the dialogue screen 700 shown in Figure 7 are performed.

[0139] After the dialogue session ends, the program executes a step to obtain target event information 800 from an external system or administrator.

[0140] When target event information 800 is obtained, the program performs a process to compare it with past session data.

[0141] The program then causes the selection rule update unit 230 to execute a learning process 810 that updates the selection rule 520 using the target event information 800 (see Figure 8).

[0142] The updated selection rule 520 is written to storage and prepared for processing the next interaction session.

[0143] Thus, the program has a flow that automatically handles the loop from data acquisition to feature calculation, information determination and presentation, and subsequent learning and updating.

[0144] As shown in Figures 1 and 2, the integration of the hardware and software configurations in the information processing system 10 enables a series of dialogue support functions to be executed without delay.

[0145] It is preferable that the communication protocol between the information processing device 100 and each target terminal employs a real-time communication standard that enables low-latency transmission and reception.

[0146] To address bandwidth fluctuations and delays in the network 130, the information processing device 100 may have functions for compressing transmitted and received data and adjusting the resolution.

[0147] Even though the first target terminal 110 and the second target terminal 120 operate on different platforms and operating systems, interoperability is ensured by standardized communication protocols.

[0148] The information processing device 100 (see Figure 2) is equipped with multithreading and load balancing functions for processing multiple dialogue sessions in parallel.

[0149] The memory unit 240 is built with a database management system that enables high-speed searching of large amounts of historical data and past achievement information 510.

[0150] The reaction feature calculation unit 200 may use dedicated computing units or graphics processing units to speed up processing in real time, as it processes a large amount of data.

[0151] The information presentation unit 210 implements optimization algorithms for quickly evaluating complex selection rules 520 and large-scale learning models.

[0152] The presentation control data generation unit 220 has a function to dynamically adjust the format of the presentation control data according to the screen resolution and display state of the second target terminal 120.

[0153] The selection rule update unit 230 can also perform learning as a batch process asynchronous to the processing of the dialogue session, so as not to degrade the overall system performance.

[0154] Through the coordination of the above hardware configuration (see Figure 1) and functional blocks (see Figure 2), the information processing system 10 provides a basic framework for optimizing information presentation in dialogue sessions.

[0155] Figure 4 is a flowchart detailing the process for calculating the response features of the first subject. The response feature calculation step 300 will be explained in detail below with reference to Figure 4.

[0156] This process is executed in real time during the dialogue session, or repeatedly at predetermined short intervals.

[0157] Upon commencement of processing, the information processing device 100 first executes a multimodal analysis process 400.

[0158] The multimodal analysis process 400 is a process for quantitatively and continuously capturing the state and reactions of the first subject from various aspects.

[0159] Specifically, the analysis will target at least one of the following: audio, video, text, or device operation history.

[0160] By combining multiple modalities, it becomes possible to precisely grasp subtle emotional changes and levels of interest in the primary subject that cannot be captured by a single source of information.

[0161] In the multimodal analysis process 400, each acquired input data is distributed to a separate analysis engine for processing according to the modality.

[0162] These analysis engines may be implemented as internal modules of the information processing device 100, or they may utilize a group of external, dedicated analysis servers via the network 130.

[0163] To optimize processing load and achieve low-latency analysis that does not disrupt the progress of the dialogue session, it is preferable that analysis for each modality be processed in parallel.

[0164] First, I will explain in detail the procedure for analyzing audio data.

[0165] The audio data is a continuous series of acoustic signals acquired from the microphone or other equipment of the terminal used by the first subject.

[0166] As shown in Figure 4, the multimodal analysis process 400 preprocesses this acoustic signal to extract basic acoustic features.

[0167] As a preprocessing step, processes such as background noise removal and volume normalization are performed.

[0168] By applying adaptive noise cancellation technology, it is possible to extract only the subject's pure voice components even in communication environments where ambient noise is prone to change.

[0169] One of the main indicators of acoustic characteristics is the time-series change in the pitch, or fundamental frequency, of the sound.

[0170] A rapid increase in pitch serves as an indicator of the subject's level of interest, surprise, or excitement.

[0171] On the other hand, a gradual decrease or stabilization of pitch can be interpreted as a response that suggests the subject is calmer and has solidified their understanding of the content.

[0172] Furthermore, the multimodal analysis process 400 continuously analyzes the fluctuations in voice volume, or amplitude, from the audio data.

[0173] A significant increase in vocal volume is likely to indicate enthusiasm or strong emotional engagement with a particular topic.

[0174] Conversely, a sudden decrease in volume is interpreted as indicating hesitation, lack of confidence, or stagnation in the person's thinking.

[0175] Furthermore, the multimodal analysis process 400 also quantitatively calculates the speech rate, or speech velocity.

[0176] Specifically, speech speed is quantified by counting the number of morae or words contained within a certain time frame.

[0177] The phenomenon of speaking faster than usual indicates that the person is well-versed in the topic or has a strong desire to communicate.

[0178] A noticeable decrease in speaking speed reflects a state of mental reflection or careful search for the right words.

[0179] Changes in intonation, or prosody patterns, during speech are also extremely important subjects of analysis.

[0180] A flat, unchanging intonation indicates a lack of interest in the topic, while a rich, dynamic intonation indicates a strong immersion in the topic.

[0181] In addition, the multimodal analysis process 400 precisely measures the length of silence between utterances.

[0182] The length of the silence period is strongly correlated with the length of time the subject spends processing the given information in their brain.

[0183] A silence of appropriate length indicates deep thought or rumination, while an unnaturally long and excessive silence is a sign of stagnation or confusion in understanding.

[0184] Furthermore, non-verbal audio components such as laughter and interjections can be independently detected from the waveform pattern of the audio signal.

[0185] Detecting laughter objectively indicates that the participants are relaxed and a positive atmosphere is being fostered during the dialogue session.

[0186] The frequency and timing of verbal affirmations (nodding or acknowledging what the other person is saying) quantitatively reflect the degree to which a relationship has been built with the other person and the attitude of actively listening to what the other person is saying.

[0187] Furthermore, by analyzing the distribution of formant frequencies from the audio, a method is used to quantify the resonance and clarity of the voice. Clear and resonant vocalization is a characteristic that indicates the subject's positive attitude and confident psychological state.

[0188] These diverse audio analysis results are sequentially stored as a time-series numerical data set in the temporary storage area of ​​the information processing device 100.

[0189] Next, with reference to Figure 4, the procedure for analyzing the video data will be explained in detail.

[0190] The video data consists of high-frame-rate video footage acquired from the camera device of the first subject terminal 110.

[0191] The multimodal analysis process 400 detects the face region of the subject in real time from each frame image of the video data.

[0192] For stable detection of facial regions, pre-trained, advanced facial recognition algorithms and deep learning models are used.

[0193] From within the detected facial region, feature points, or landmarks, that represent the contours of major facial features such as the eyes, nose, mouth, and eyebrows are extracted.

[0194] By tracking the temporal changes in the relative positions of these numerous feature points, we can estimate subtle changes in the subject's facial expressions.

[0195] For example, characteristic movements such as the upward curve of the corners of the mouth or the formation of wrinkles at the corners of the eyes can be used to detect expressions of joy or goodwill, while conversely, the formation of wrinkles between the eyebrows or strong tension around the mouth can be used to detect negative expressions such as doubt, anxiety, or dissatisfaction.

[0196] The results of the facial analysis are output as a vector of classification scores for multiple basic emotion categories such as joy, sadness, anger, and surprise.

[0197] The multimodal analysis process 400 tracks the eye movements of the subject and precisely analyzes the direction of their gaze.

[0198] The system determines, frame by frame, which area of ​​the screen the user is focusing on, or whether they are looking at the camera lens.

[0199] Frequent eye movements or looking away from the screen can indicate decreased concentration, loss of interest, or high levels of anxiety, while maintaining a steady gaze on a camera or specific material on the screen indicates a very high level of interest and concentration.

[0200] Furthermore, the movement of the subject's head is spatially tracked throughout the entire video data.

[0201] By detecting up-and-down head movements, i.e., nodding, the degree of the subject's agreement and understanding of the content is measured, and the depth and speed of repetition are also parameterized as important indicators of the strength of the response and the degree of empathy.

[0202] Shaking the head from side to side or tilting the head can be immediately interpreted as a reaction suggesting a negative view or lack of understanding (see Figure 4).

[0203] The multimodal analysis process 400 includes changes in the posture of the entire upper body of the subject, not just the head, as part of its analysis.

[0204] Leaning forward towards the screen is highly valued as an indication of strong interest in the topic and a willingness to participate.

[0205] Leaning back deeply against the backrest indicates a decreased level of interest or a state of complete relaxation.

[0206] The frequency and speed of blinking are also features that can be easily extracted from video data by the feature extraction process 410.

[0207] An extreme increase in blinking frequency is considered a physiological response indicating extreme tension or high cognitive load.

[0208] In video analysis, the angles of the face's orientation, namely the yaw angle, pitch angle, and roll angle, are also calculated continuously.

[0209] If the subject frequently looks down, it means they are reading notes or documents in their hands, or they are averting their gaze.

[0210] Furthermore, advanced image processing techniques that capture minute changes in pupil dilation and constriction may be introduced to estimate the response of the autonomic nervous system.

[0211] By automatically correcting for changes in screen brightness, the system prevents a decrease in the accuracy of facial expression analysis due to fluctuations in the lighting environment.

[0212] These vast amounts of video analysis results are also aggregated and quantified on a frame-by-frame or time window basis.

[0213] Next, we will explain in detail the procedure for analyzing text data.

[0214] The text data consists of highly accurate transcription results from the speech recognition engine during the dialogue session, as well as text strings entered using the chat function.

[0215] The multimodal analysis process 400 executes the latest natural language processing algorithms on the acquired text data.

[0216] First, the entire text is divided into morphemes or tokens, and the part of speech and syntactic role of each word are determined.

[0217] The frequency of occurrence of a predefined set of keywords is counted from a collection of divided word groups.

[0218] The appearance of specialized terminology related to specific technical fields, as well as positive words that demonstrate a forward-looking attitude, are highly valued as characteristics that indicate proactiveness.

[0219] In natural language processing, thesauruses and vector space models are used to absorb variations in expression and synonyms.

[0220] This allows us to accurately grasp the essential intentions behind the words used by the subject, even if they are different.

[0221] The multimodal analysis process 400 also simultaneously performs the process of determining the emotional polarity of the entire text.

[0222] The overall tone of the participant's speech is expressed using a continuous score, indicating whether it has a positive, negative, or neutral connotation (see Figure 4).

[0223] If the text contains many positive or approving expressions, it serves as strong evidence of a favorable response from the target audience.

[0224] Furthermore, the clarity of logical development and contextual consistency in the participants' spoken texts will also be included in the analysis.

[0225] The degree of a person's logical thinking ability is estimated by examining the use of conjunctions such as those indicating cause and effect, and the logical connections between sentences.

[0226] The multimodal analysis process 400 (see Figure 2) also evaluates the semantic relationship between the utterance text of the dialogue partner and the utterance text of the target.

[0227] Determine whether you are responding appropriately and directly to the intent of the other person's question, or whether you are intentionally changing the subject.

[0228] By analyzing the text data over time, natural transitions between topics and the starting points of topic separators (see Figure 6).

[0229] If a participant spontaneously introduces new topics or asks questions, it is considered to demonstrate an extremely high level of initiative.

[0230] Furthermore, the length of characters in a single utterance and the amount of information contained within are also important indicators that can be obtained from text analysis.

[0231] If the responses consist only of extremely short answers such as "yes" or "no," it is highly likely that the person is passive or uninterested.

[0232] In contrast, detailed and informative utterances that include specific examples demonstrate a strong engagement with and enthusiasm for the topic.

[0233] For in-depth contextual analysis, large-scale language models can be used to perform advanced semantic understanding and summary extraction (see Figure 10).

[0234] The results derived from these text analyses are stored as utterance-level or sentence-level features in parallel with data from other modalities by the feature extraction process 410.

[0235] Finally, we will explain in detail the procedure for analyzing the operation history data shown in Figure 4.

[0236] The operation history data is a record of physical operations performed via input devices such as a mouse, keyboard, or touch panel on the first subject's terminal 110 used by the first subject (see Figure 1).

[0237] The movement trajectory of the pointing device, the location of click events, and the amount of screen scrolling are recorded in chronological order.

[0238] The multimodal analysis process 400 analyzes the frequency, timing, and speed of these operation events.

[0239] The act of diligently scrolling up and down the materials presented in the information display area 710 of the dialogue screen 700 (see Figure 7) indicates a strong desire to explore information and a high level of interest in its content.

[0240] Clicking or pinching to enlarge a specific screen element or text portion within the information display area 710 reflects a very strong interest in that element.

[0241] The dwell time of the pointer in the information display area 710 is also measured, and areas with longer dwell times suggest that the subject is carefully reading the content.

[0242] The operation history includes detailed information such as the speed of character input from the keyboard and the history of input corrections made using the backspace key (see Figure 4).

[0243] Significant delays in input or frequent rewriting indicate hesitation, caution, or psychological conflict regarding the content of the answer.

[0244] Additionally, the history of when focus shifts to other applications or browser tabs besides the dialogue screen 700 is also recorded as an important event.

[0245] A high frequency of losing focus from the dialogue screen 700 indicates a lapse in concentration or simultaneous work on other tasks.

[0246] If a smartphone or tablet is being used, unique touch events such as taps, swipes, and flicks are analyzed.

[0247] Information obtained from pen pressure and accelerometer sensors during touch operations is also incorporated as part of the subject's response features, if available.

[0248] This operation history data serves as powerful auxiliary data for inferring the subject's potential state that is not apparent in audio or video.

[0249] As described above, the multimodal analysis processing 400 by the reaction feature calculation unit 200 (see Figure 2) extracts raw data from multiple perspectives, including audio, video, text, and operation history, which are all completely different dimensions.

[0250] Once the extraction process for various data is complete and all the data is available, the main processing shifts to the feature extraction process 410 shown in Figure 4.

[0251] The feature extraction process 410 is a process for integrating the diverse raw and intermediate data obtained in the multimodal analysis process 400 into a single metric.

[0252] Data obtained from different modalities often differs in terms of sampling rate and acquisition timing.

[0253] The feature extraction process 410 accurately maps this asynchronous data onto a common system time axis to achieve temporal synchronization.

[0254] Synchronized multimodal data is summarized and statistically processed for each time frame of a predetermined length, i.e., a sliding window.

[0255] For example, the average pitch, the percentage of smiles, and the number of times positive words appear within a window of the last few seconds are all calculated simultaneously.

[0256] In situations where the pace of the dialogue is fast, the window size is dynamically shortened to sensitively capture even subtle changes in responses.

[0257] Conversely, in situations where lengthy explanations are given continuously, the window size should be set longer to stably grasp the overall response trends.

[0258] The feature extraction process 410 normalizes these calculated statistical values ​​and concatenates them into a single multidimensional vector for representation.

[0259] The vast set of extracted features can be reduced to only the most important components with a high information content using dimensionality reduction techniques such as principal component analysis.

[0260] This dimensionality reduction significantly reduces the system load in subsequent computational processing while maintaining real-time presentation control.

[0261] Furthermore, by calculating the difference between the feature history in the previous time frame and the current value, the rate of change in the state is added as a new feature.

[0262] The presence of components with large rates of change indicates a decisive moment when the subject's emotions or interests fluctuated significantly (see Figure 4).

[0263] This integrated multidimensional vector is ultimately calculated and determined as the response feature of the first subject.

[0264] The calculated response features serve as universal indicators for objectively and quantitatively handling the psychological or cognitive state of the subject within the system.

[0265] In multimodal integrations, it is also possible to weight data from specific modalities according to the nature of the session.

[0266] For example, in situations where subtle changes in facial expression should be prioritized over nuances of language, the weight of video-derived features is dynamically increased.

[0267] The feature extraction process 410 also handles normalization processes such as interpolating missing values ​​due to temporary sensor malfunctions or communication delays, and removing noise from outliers (see Figure 2).

[0268] Even if video frames are temporarily lost due to a deterioration in the communication environment, the gaps can be filled in using audio and text features.

[0269] This strongly ensures the overall stability and accuracy of the system in calculating reaction features.

[0270] The calculated reaction features are sequentially stored and accumulated as time-series data in the high-speed memory area, the memory unit 240, within the information processing device 100 (see Figure 1).

[0271] This makes it possible to track the changes in participants' responses throughout the entire session as a continuous graph.

[0272] Next, referring to Figure 5, we will explain in detail the processing concept for optimally determining the information to be presented based on the aforementioned reaction features.

[0273] Figure 5 is a diagram that visually explains the concept of selecting information to present based on past performance information 510 and the attributes of similar subjects.

[0274] In the information processing system 10, the information to be presented refers to useful content and potential dialogue partners 1010 that should be presented to the first target during the dialogue session (see Figure 10).

[0275] As the first step in determining the information to be presented, the information processing device 100 acquires or references the first target person attribute information 500 (see Figure 3).

[0276] The first participant attribute information 500 is profile information that is already registered in the system database before or at the start of the dialogue session.

[0277] Specifically, this includes the applicant's age, work history, area of ​​expertise, desired occupation, place of residence, hobbies, qualifications, or the results of a pre-submitted aptitude questionnaire (see Figure 9).

[0278] The information processing device 100 uses this first subject attribute information 500 to search for similar individuals from a vast amount of subject data accumulated in the past.

[0279] Identifying similar past subjects increases the likelihood of applying information presentation approaches that were effective for those subjects to the current primary subject (see Figure 8).

[0280] Within the search process, the similarity between the first target attribute information (500) and the attribute information of past targets is calculated in a multidimensional space.

[0281] Mathematical metrics such as calculating the cosine similarity between vectors and nearest neighbor search based on Euclidean distance are widely used as methods for calculating similarity (see Figures 6 and 7).

[0282] By assigning different weights to each attribute, it becomes possible to perform a precise similarity calculation that best matches the purpose of the session (see Figure 5).

[0283] For example, in the case of a dialogue session that highly values the degree of match of specialized skills, large weighting factors are set for attributes such as work history and held qualifications.

[0284] A group of past subjects whose calculated similarity exceeds a predetermined threshold is extracted as a cluster of similar subjects.

[0285] Furthermore, past achievement information 510 associated with the extracted cluster of similar subjects is read from the database and referred to.

[0286] The past achievement information 510 is a definite record indicating what actions and results a similar subject brought about during or after a past dialogue session.

[0287] For example, short-term intermediate achievements such as the similar subject making a positive statement immediately after presenting specific information or the quantity of questions increasing are recorded.

[0288] The past achievement information 510 naturally also includes a long-term result label indicating whether the ultimate goal, which is the original purpose of the dialogue session, was achieved.

[0289] The long-term result label may also include the result of the final decision-making that was found several days or months after the end of the dialogue session.

[0290] The information processing apparatus 100 statistically analyzes what types of information were presented to a similar subject in the past and what past achievement information 510 was obtained as a result.

[0291] It is reasonably judged that the information presented in cases where the achievements were extremely good has a high probability of acting effectively on a first subject with similar attributes as well.

[0292] Conversely, the information presented in cases where the achievements were poor is classified negatively as information to be avoided because it may dampen motivation.

[0293] Selection Rule 520 plays a central role in the advanced reasoning and decision-making involved here.

[0294] The selection rule 520 is a logical or mathematical model that takes the most recent response features of the first subject, the first subject attribute information 500, and past performance information 510 as input, and outputs the most suitable target information to present.

[0295] The structure of selection rule 520 may consist of a rule-based decision tree that returns a specific action when certain conditions are met.

[0296] Alternatively, it may consist of machine learning algorithms such as neural network models or gradient boosting models that have been trained on vast amounts of historical data.

[0297] The information processing device 100 applies the latest reaction feature quantities, which are continuously calculated in the process shown in Figure 4, to the input layer of the selection rule 520.

[0298] For example, consider a situation where the calculated response features indicate an extremely strong interest in and positive feelings towards a particular topic from the subject.

[0299] Selection rule 520 refers to a database of past performance information 510 and searches for additional presentation information that was most effective for similar target audiences who have similar attributes and showed a similarly strong response.

[0300] This extensive search process lists several candidates that should be presented next within the system.

[0301] Each of the listed candidates is then scored precisely based on Selection Rule 520.

[0302] The score is a continuous numerical value that reflects the predicted probability of achieving a favorable outcome if the candidate information is presented at this time.

[0303] In the scoring process, not only the content of the information but also the format of the medium to be presented is considered as an important factor.

[0304] At the same time, judgments on media selection are also made, such as whether a presentation in concise text is appropriate or whether a presentation in a video appealing to the visual sense is appropriate.

[0305] As a result of the calculation, the candidate with the highest score or the upper candidate group that meets the predetermined quality criteria based on the exclusion list 900 is determined as the final presentation target information (see Figure 9).

[0306] The determination of the presentation target information is not a one-time process but is dynamically and continuously updated in accordance with the transition of the context and the change of the situation in the dialogue session.

[0307] For example, when the reaction feature amount of the target person suddenly decreases and shows signs of boredom, the selection rule 520 immediately determines new information that promotes a topic change.

[0308] Also, when signs that the target person has specific doubts or anxieties can be read from the reaction feature amounts of video and audio, information for resolving those doubts in advance is preferentially selected.

[0309] In this way, by multiplying the first target person attribute information 500 and the past achievement information 510 multidimensionally, information presentation completely personalized for an individual target person is realized (see Figure 5).

[0310] Rather than simply presenting pre-determined uniform information sequentially, it becomes possible to make a very flexible and adaptable response according to the instantaneous reaction of the other party.

[0311] In the scoring in the selection rule 520, it is also effective to apply a reinforcement learning method such as the context bandit algorithm to optimize the balance between exploration and exploitation (see Figure 8).

[0312] Multiple types of selection rule 520 are available within the system and can be dynamically selected and used depending on the target audience's attribute category and the specific purpose of the session.

[0313] The information processing device 100 immediately passes the determined information to be presented to the next system processing block and displays it on the dialogue screen 700 of the second user terminal 120 (see Figure 7).

[0314] The data structure that is handed over includes not only the entities and identifiers of the information to be presented, such as the talk script 1000 and the candidate conversation partner 1010, but also scores and attribute factors that explain why that information was selected (see Figure 10).

[0315] After the information to be presented is determined, frequency information—how often that information has been selected and presented in the past—is also recorded as an internal log.

[0316] This series of decision-making steps constructs a completely data-driven and objective information processing system 10 that does not rely excessively on human experience or intuition (see Figure 1).

[0317] By closely coordinating the calculation of response features in real time using multimodal analysis with the application of selection rules 520 based on historical data, extremely sophisticated information presentation control is established (see Figure 6).

[0318] Figure 3 is a flowchart showing the overall procedure for determining the information to be presented and controlling the timing of its presentation.

[0319] The information processing system 10 is configured to execute each of these steps in real time and continuously, following the progress of the dialogue session (see Figure 2).

[0320] When the dialogue session begins, the information processing system 10 first initiates the response feature calculation step 300.

[0321] In the response feature calculation step 300, the stream data transmitted from the first subject terminal 110 is sequentially acquired and buffered in memory (see Figure 4).

[0322] The buffered data is divided into segments based on a certain frame rate or time window and subjected to multimodal analysis processing 400 (see Figure 4).

[0323] In the multimodal analysis process 400, the feature extraction process 410 extracts quantitative values ​​such as changes in speech pitch and facial expression.

[0324] The extracted numerical data is integrated and recorded as multidimensional vector data in a time series.

[0325] This time-series data is sequentially written to the shared memory or other internal memory of the information processing device 100 (see Figures 1 and 2) as the response characteristics of the first subject.

[0326] The response feature calculation step 300 continues to run continuously in the background until the dialogue session ends.

[0327] Next, the information processing system 10 (see Figure 3) executes the step 310 for determining the information to be presented based on the written response features.

[0328] In the step 310 for determining the information to be presented, a process is performed to compare and match the latest response features with past trend data such as past performance information 510 stored in the memory unit 240 (see Figure 5).

[0329] This comparison and matching process extracts information that is most appropriate to the ongoing conversation situation as a candidate for presentation (for example, talk script 1000 or candidate conversation partner 1010 (see Figure 10)).

[0330] When extracting candidates for presentation, the filtering unit 910 may perform a process to exclude inappropriate information based on the exclusion list 900 (see Figure 9).

[0331] If multiple candidates exist, a score indicating their suitability is calculated, and the information with the highest ranking is determined to be the final information presented.

[0332] The step 310 for determining the information to be presented is executed based on the selection rules 520 updated by the learning process 810 (see Figure 8), at the timing when a change occurs in the response features, etc.

[0333] Once the information to be presented is determined, the information processing system 10 immediately proceeds to the presentation timing control step 320.

[0334] In the presentation timing control step 320, the timing of when to display the determined information on the second target terminal 120 is calculated based on the response feature change time 600 and the topic division 610 (see Figure 6).

[0335] Rather than simply displaying information instantly, the presentation timing 620 is controlled to take into account the context of the dialogue and the psychological state of the other party.

[0336] The information processing system 10 has a function to temporarily suspend the output processing of the information until the calculated optimal presentation time is reached.

[0337] Then, at the moment the optimal presentation time arrives, the presentation control data generation unit 220 generates presentation control data and transmits it via the network 130.

[0338] With the transmission of this presentation control data, one unit of the series of processing cycles shown in Figure 3 is completed.

[0339] In subsequent dialogue sessions, the system rapidly repeats the process from the response feature calculation step 300 to the presentation timing control step 320.

[0340] This minimizes the time lag between information determination and presentation, while enabling information delivery at the most effective moment in the information display area 710 of the dialogue screen 700 (see Figure 7).

[0341] The elimination of delay in the presentation timing control step 320 is supported by the parallel processing architecture of the information processing device 100.

[0342] The information processing system 10 (see Figure 1) employs a structure in which data acquisition, analysis, and output are processed asynchronously using separate threads.

[0343] This prevents the accuracy of presentation timing 620 from being compromised even if the computational load temporarily increases.

[0344] The presentation control data generated in presentation timing control step 320 (see Figure 3) includes a drawing instruction command along with an information identifier.

[0345] The drawing instructions also specify parameters in detail regarding the on-screen placement coordinates and display duration.

[0346] According to this parameter, the display layout in the interactive screen 700 (see Figure 7), which will be described later, is dynamically rearranged.

[0347] The information processing device 100 (see Figure 2) stores the execution history of the presentation timing control step 320 as log data in the storage unit 240.

[0348] This log data serves as an important indicator for evaluation after the end of a dialogue session and for parameter tuning during the learning process 810 (see Figure 8).

[0349] Thus, the procedure shown in Figure 3 plays a fundamental role in achieving both appropriate information selection and control of the optimal presentation timing 620.

[0350] Figure 6 is a timeline diagram showing an example of a process that controls the presentation timing 620 in synchronization with the response feature change time 600 or topic division 610.

[0351] The horizontal axis represents the elapsed time of the dialogue session, and the vertical axis represents the amount of change in the response features extracted by the response feature calculation unit 200 (see Figure 4) and the occurrence of events.

[0352] The information processing system 10 has the function of monitoring a time-series data sequence of reaction features and accurately identifying the time point 600 at which the reaction features change.

[0353] The response feature change time point 600 corresponds, for example, to the moment when the tone of voice suddenly rises or the moment when the facial expression softens significantly.

[0354] Based on past performance information 510 (see Figure 5), the information processing system 10 interprets such moments as times when the first subject showed strong interest in or empathy for a particular topic.

[0355] The information processing system 10 uses mathematical methods such as moving averages and first derivatives to detect sharp rises and falls in the data and records these as response feature change time points 600.

[0356] Furthermore, the information processing system 10, including analysis by the filtering unit 910 (see Figure 9), simultaneously detects topic segmentation 610 not only through physical changes in response features but also through linguistic analysis of the utterance content.

[0357] Topic break 610 refers to a point, for example, when an answer to a question has been completed and a few seconds of silence have occurred.

[0358] Alternatively, a topic segment 610 is also determined when the trend of the keyword group associated with the talk script 1000 extracted by natural language processing (see Figure 10) shifts to a different category.

[0359] Using clues such as the frequency of utterances and interjections, the information processing system 10 divides the dialogue into logical paragraphs in real time.

[0360] The information processing system 10 determines the presentation timing 620 using either the reaction feature change time 600 or the topic division 610 as a trigger.

[0361] For example, in synchronization with the moment 600 when the response feature changes and the interest of the first subject reaches its peak, the information processing system 10 outputs related in-depth information as presentation timing 620.

[0362] Providing additional information at the moment of heightened interest can further stimulate engagement in the dialogue (see Figure 1).

[0363] On the other hand, information intended to transition to a new topic is presented at a timing 620 that is synchronized with the topic break 610, when the intensity of the conversation has subsided, as shown in Figure 6.

[0364] This allows the second participant using the second participant terminal 120 to be guided to the next topic in a natural flow without unintentionally interrupting the ongoing conversation (see Figure 5).

[0365] There are also cases where the reaction feature change point 600 and the topic break point 610 occur consecutively in very close time intervals.

[0366] In such cases, the information processing device 100 selects which event to use as the basis for the presentation timing 620 according to the procedure shown in Figures 3 and 8, based on priority rules such as the pre-set selection rule 520.

[0367] When the presentation timing 620 arrives, the information processing device 100 immediately generates the aforementioned presentation control data and sends it to the second target terminal 120 via the network 130.

[0368] Even considering the delay in network 130, the transmitted presentation control data reaches the terminal in the order of a few milliseconds to tens of milliseconds.

[0369] This extremely low-latency data transfer enables synchronous presentation control to function precisely, allowing the second participant to incorporate the provided information into the dialogue without interrupting their own thinking.

[0370] The response feature change point 600 can be treated not only as a single event, but also as the starting point of a trend over a certain period.

[0371] During periods when a trend persists, it is also possible to control the information processing system 10 to continuously provide information of the same category intermittently in accordance with the presentation timing 620.

[0372] To improve the accuracy of detecting topic segments 610, the information processing device 100 uses a complex judgment logic (see Figure 2) that combines the intonation of the utterance with the sentence-ending expression of the text, as shown in Figure 4.

[0373] This reduces the risk that the system might mistake a simple hesitation or silence for a clear conclusion of the topic.

[0374] Controlling the presentation timing 620 also serves as a kind of pacemaker in the progress of the dialogue session.

[0375] Figure 7 shows an example of a user interface for displaying target information on the second target terminal 120.

[0376] On the device's display, the interactive screen 700 is shown in full screen or in a window via a browser or dedicated application.

[0377] The majority of the dialogue screen 700 is allocated to display the real-time camera feed of the first person, who is the other party in the dialogue.

[0378] Status information, such as a microphone volume indicator and icons showing the communication status, is placed around the video.

[0379] When the information processing device 100 receives presentation control data, a presentation information display area 710, including the configuration illustrated in Figure 9 or Figure 10, appears at a specific location within the dialogue screen 700.

[0380] The information display area 710 is displayed as a semi-transparent overlay on top of the video so as not to impair the visibility of the dialogue screen 700.

[0381] Alternatively, a layout may be adopted in which the video area of ​​the dialogue screen 700 is temporarily reduced and a separate information display area 710 is provided on the right or bottom side of the screen.

[0382] In all layouts, the information display area 710 is positioned so that the second subject can read the information without significantly shifting their gaze away from the first subject's facial expression.

[0383] When the information display area 710 appears on the screen in synchronization with the presentation timing 620, the system applies a visual animation effect.

[0384] For example, effects such as a smooth slide-in from the edge of the screen or a fade-in effect where the opacity gradually increases are used.

[0385] Such animations prevent information from appearing too abruptly and allow the attention of the secondary audience to be naturally drawn in.

[0386] Within the information display area 710, not only text information but also visual elements such as graphs and charts are appropriately arranged.

[0387] The system dynamically controls the font size and color scheme of the displayed text according to the importance and urgency of the information.

[0388] For example, information that needs to be checked urgently can be highlighted by changing the background color to a different color to encourage quick recognition by the second target audience.

[0389] The information display area 710 is equipped with an auto-hide function that automatically erases or minimizes the information after a certain period of time has elapsed in accordance with the progress of the conversation.

[0390] This prevents the dialogue screen 700 from being filled with outdated information, maintaining a clean state where only the latest and necessary information is always presented.

[0391] The second target user can also manually expand or collapse the information display area 710 using a mouse or touch panel.

[0392] Furthermore, when multiple pieces of information are to be presented simultaneously, the information display area 710 can either display the information side-by-side as a card-like UI or allow switching between them using a carousel-style swipe.

[0393] The information display area 710 may also include a brief explanation of how the presented information appeared due to changes in response features.

[0394] This explanation allows the second target individual to instantly understand the system's intention in presenting that information at that particular time.

[0395] The coordinated operation between the dialogue screen 700 and the information display area 710 is executed extremely smoothly by front-end scripting.

[0396] The presentation control data arriving via the network is written in a lightweight format such as JSON, minimizing the load required for screen redrawing.

[0397] The layout and color scheme of the information display area 710 can be pre-configured to match the preferences and visual characteristics of the second target user.

[0398] Figure 10 shows an example of a data structure for managing potential conversation partners and talk scripts as information to be presented.

[0399] The system stores a wide variety of 1000 talk scripts in internal or external databases and manages them centrally.

[0400] Talk Script 1000 is not merely a collection of predefined phrases, but rather has a hierarchical structure that is finely categorized according to context and the attributes of the target audience.

[0401] Each of the 1000 talk scripts is tagged with multiple metadata elements, such as the topic category, difficulty level, and corresponding emotion status (see Figure 10).

[0402] In step 310, which determines the information to be presented, these tags are matched with the response features of the first target person, as shown in Figures 3 to 5, and the optimal talk script 1000 is searched for.

[0403] For example, if the first participant shows a strong interest in technical topics, the system will extract 1000 talk scripts that include in-depth technical questions.

[0404] The extracted 1000 talk scripts may be presented as is, or they may be dynamically adjusted using natural language processing to match the context of the current conversation, with changes to word endings and expressions.

[0405] Furthermore, the information processing system 10 (see Figure 1) manages not only the talk script 1000 but also information on candidate conversation partners 1010, indicating which conversation partner should be assigned next.

[0406] The data for the 1010 potential dialogue partners includes each person's area of ​​expertise, success rate in past dialogue sessions, and current schedule availability.

[0407] At topic break 610, where a discussion session becomes heated and requires specific expertise, the system immediately identifies the corresponding candidate conversation partner 1010 (see Figure 6).

[0408] The identified candidate conversation partner 1010 is displayed in the information display area 710 on the conversation screen 700 of the second participant terminal 120 as an additional invitation to join the current conversation session (see Figure 7).

[0409] Alternatively, candidate 1010 may be recommended as the most suitable person to lead a subsequent dialogue session to be held at a later date.

[0410] When a potential conversation partner (1010) is presented, the reasons why that person is suitable and a brief profile of that person are also presented in the form of a talk script (1000).

[0411] This allows the second participant to persuasively describe the appeal of their next conversation partner to the first participant.

[0412] The talk script 1000 and the candidate conversation partner 1010 are linked to each other, and the content of the talk script 1000 to be spoken next automatically changes depending on the selected candidate conversation partner 1010.

[0413] The system uses speech recognition to detect whether the provided talk script 1000 was actually spoken and feeds this usage data back into a database such as the memory unit 240.

[0414] This usage data serves as important evidence for evaluating which talk script 1000 was effective in the learning process 810 (see Figure 8) performed by the information processing device 100 (see Figure 2).

[0415] Similarly, the history of whether the second target person has taken an action to accept the recommendation of candidate 1010 will also be used to improve the recommendation logic.

[0416] The Talk Script 1000 database is updated daily by multiple users within the organization, with success stories continuously being added.

[0417] The newly added 1000 talk scripts will be immediately indexed and made available as potential presentation options for the next dialogue session.

[0418] The schedule information for candidate conversation partners 1010 is constantly synchronized to the latest state, down to the second, by integrating with the company's internal calendar system via API.

[0419] This prevents the system from mistakenly suggesting a person who is unavailable as a potential conversation partner (1010).

[0420] The Talk Script 1000 may include not only the content to be spoken but also prohibited items and points to be aware of based on the Exclusion List 900 (see Figure 9), which also functions from a risk management perspective.

[0421] This detailed information is provided in the dialogue screen 700 (see Figure 7) in sync with changes in response features (see Figure 4) (see Figure 6), thereby raising the overall quality of the dialogue across the entire organization without relying on individual skills.

[0422] In this way, the information processing system 10 (see Figure 1) highly integrates and dynamically manages the talk script 1000, which is the concrete entity of the presented content, and the candidate dialogue partner 1010 (see Figure 10).

[0423] Referring to Figure 8, the mechanism by which the information processing device 100 (see Figure 2) updates the selection rules 520 in the overall procedure of the dialogue session (see Figure 3) will be explained.

[0424] In this embodiment, a mechanism for updating the selection rule 520 (see Figure 5) for determining the information to be presented, while taking into account the restrictions imposed by the filtering unit 910 (see Figure 9), using information about the target event acquired after the dialogue session, will be described in detail.

[0425] The information processing system 10 evaluates the results of the information presented through the dialogue session, using the target event information 800 as an indicator.

[0426] Target event information 800 is often obtained not immediately after the dialogue session, but after a certain period of time has elapsed. This delayed information on outcomes is used as an outcome delay label.

[0427] By establishing a feedback loop based on outcome delay labels, the proposal accuracy of the information processing system 10 gradually improves.

[0428] The learning process 810 shown in Figure 8 is at the core of this feedback loop and is executed by the selection rule update unit 230.

[0429] The learning process 810 is triggered by the input of target event information 800, which enables the selection of more appropriate information to present in the next dialogue session.

[0430] Target event information 800 is data that indicates the final or intermediate goal set according to the objective of the dialogue session.

[0431] Specifically, it is recorded as historical data indicating that a candidate or other target individual has taken a particular action.

[0432] This target event information (800) is linked to the target audience's attribute information and past interaction history.

[0433] The information processing system 10 acquires target event information 800 by linking with external recruitment management databases and human resources systems.

[0434] The timing of data acquisition can be either periodic collection via batch processing or real-time notification when an event occurs.

[0435] Target event information 800 can include both positive labels indicating a positive outcome and negative labels indicating a negative outcome.

[0436] For example, the fact that the next step was not taken is also treated as important training data.

[0437] If multiple dialogue sessions are held consecutively, it is possible to allocate the contribution of each session and link it to the target event information 800.

[0438] This allows for a detailed analysis of which presented information in each dialogue session contributed to the outcome, based on the concept shown in Figure 5.

[0439] Accurate acquisition of target event information 800 is an essential element in ensuring the quality of the learning process 810 shown in Figure 8.

[0440] One specific type of target event information (800 items) is information regarding the completion of applications.

[0441] This indicates that the candidate has decided to proceed to the formal selection process after participating in initial dialogue sessions such as information sessions or casual interviews.

[0442] Completing the application process serves as proof that your interest in and motivation for the company have exceeded a certain level.

[0443] The information processing system 10 records the date and time when the applicant submits the application form and the content entered as target event information 800.

[0444] If a success story of a specific young employee is presented during a dialogue session, the subsequent application completion rate will be tracked.

[0445] If a high probability of application completion is observed after the presentation of the relevant episode, then a strong correlation can be established between the presented information and the outcome of application completion.

[0446] The completion of applications is often observed relatively shortly after the dialogue session.

[0447] Therefore, it functions as an important feedback signal in the initial stages of the learning process 810.

[0448] The information processing system 10 compares the response features of those who completed the application process with those who did not.

[0449] Cases in which the response features to the presented information are good and the application is completed will be given the highest evaluation.

[0450] Conversely, cases where the response features were good but the application was not completed will be analyzed as false positives.

[0451] This makes it possible to learn the gap between superficial excitement and genuine action prompts.

[0452] The target event information 800 related to application completion contributes to optimizing the selection rule 520 at the top of the recruitment funnel.

[0453] Optimization at this stage allows for more efficient population formation.

[0454] Application completion is an extremely useful indicator for measuring the initial engagement of potential candidates.

[0455] Next, we will explain information regarding participation in the interview.

[0456] Participation in an interview is a target event information 800 that indicates the applicant actually showed up for the selection process after completing their application.

[0457] In recent years, a declining conversion rate has become a challenge, with many applicants dropping out between the application and interview stages.

[0458] Therefore, it is important to consider whether information was presented during the dialogue session that motivated participants to take part in the interview.

[0459] The system detects whether the subject logged into the system or checked in at the offline venue on the scheduled interview date and time.

[0460] This detection result is stored as target event information 800 related to participation in the interview.

[0461] If FAQs and talk scripts that address candidates' anxieties are presented during the dialogue session, the effect will be reflected in the interview participation rate.

[0462] The learning process 810 evaluates which talk script most strongly elicited the action of participating in the interview.

[0463] If participation in the interview is confirmed, the system will look back at records of past dialogue sessions and compare them with the generation history of the presentation control data.

[0464] By analyzing the timing of presentation and changes in the respondent's response characteristics, we identify the decisive moments that contributed to their participation in the interview.

[0465] For example, this could be the case where a video conveying the workplace atmosphere was presented, immediately generating a strong response, and subsequently leading to confirmation of participation in an interview.

[0466] The goal event information of participating in interviews (800) helps in building selection criteria to maintain the candidate's continued interest.

[0467] Whether or not a participant attends the interview is given to the learning model as a binary label.

[0468] The system updates its logic for determining which information to present to maximize the probability of encouraging participation in the interview.

[0469] This helps prevent candidates from dropping out during the selection process and improves the retention rate.

[0470] Furthermore, information regarding passing the selection process also constitutes 800 important target event pieces of information.

[0471] Passing the selection process indicates that the candidate has met the company's criteria and has been able to move on to the next step.

[0472] This demonstrates the individual's abilities and aptitudes, and is also a result of effective communication during the dialogue session.

[0473] Let's consider a scenario where, during a dialogue session, potential in-depth questions designed to draw out the participant's strengths are presented.

[0474] If a question reveals the candidate's true abilities and ultimately leads to their selection, then the value of that question is highly regarded.

[0475] The system imports selection result data from the personnel evaluation system and stores it as target event information 800 related to passing the selection process.

[0476] Passing the selection process is a unique achievement label because, unlike simply participating or applying, it involves an external criterion: evaluation by the company.

[0477] In learning process 810, the focus is on whether essential dialogues that would lead to success in the selection process were elicited.

[0478] The information processing system 10 analyzes the trends in the information commonly presented to the group of candidates who passed the selection process (see Figure 1).

[0479] This analysis leads to the formation of selection rules 520 for identifying excellent candidates and effectively appealing to them.

[0480] By utilizing the target event information of 800, which is passing the selection process, the system learns presentation patterns that enable high-quality matching (see Figure 5).

[0481] Furthermore, it is possible to record 800 pieces of target event information at each stage of the selection process, such as passing the first interview or passing the second interview.

[0482] By utilizing the information obtained at each stage, the selection rule update unit 230 can update the selection rules 520 in detail according to the selection phase (see Figure 2).

[0483] Information regarding the success of the selection process can also be used as an indirect indicator of whether the interviewer's support information functioned correctly.

[0484] As a result, dialogue support that improves the accuracy of evaluations is promoted.

[0485] Among the 800 target event pieces of information, the most decisive performance indicator is information regarding acceptance of job offers.

[0486] Accepting a job offer signifies that the applicant has made a firm decision to join the company in question.

[0487] The primary objective of these dialogue sessions is often to secure acceptance of the job offer.

[0488] Information regarding acceptance of job offers is a typical example of a delayed outcome, as it is obtained several weeks to several months after the initial interview session.

[0489] The information processing system 10 acquires this information when the subject submits a consent form or presses the consent button on the dialogue screen 700 (see Figure 7).

[0490] If the candidate accepts the job offer, the information processing device 100 comprehensively analyzes the history of all dialogue sessions the candidate has experienced.

[0491] In particular, if the candidate showed a strong reaction to the interview with candidate 1010 or to the explanation of specific business activities, it is presumed that this information was the deciding factor in their acceptance of the job offer (see Figure 10).

[0492] The learning process 810 adjusts the weights of the selection rule 520 to maximize the positive feedback of accepting the job offer (see Figure 8).

[0493] Even cases where a candidate does not accept a job offer, i.e., declines it, become extremely important learning data.

[0494] By analyzing the correlation between the reason for refusal and the reaction feature change time 600 calculated in reaction feature calculation step 300, the factors causing the attraction failure can be identified (see Figures 4 and 6).

[0495] For example, the filtering unit 910 learns how to improve its explanation method from cases where the response features decreased during the explanation of the reward, ultimately leading to the participant declining the offer (see Figure 9).

[0496] Target event information 800 related to acceptance of job offers is often given the highest weight when updating selection rule 520.

[0497] This delayed label-incorporating step 310 for determining the information to be presented allows for the selection of information that will influence the final decision-making process, rather than simply generating temporary excitement (see Figure 3).

[0498] By optimizing the process with the acceptance of job offers as the objective function, it becomes possible to directly increase the ultimate success rate of recruitment activities.

[0499] The information processing system 10 (see Figure 1) will evolve to the point where it can predict the probability of acceptance of the job offer in real time and control the presentation of information during the conversation.

[0500] Furthermore, as a long-term performance indicator, information related to new hires is included in the 800 target event information (see Figures 3 and 8).

[0501] Information regarding joining the company indicates that the person who accepted the job offer has actually started working and become a member of the organization.

[0502] Since there is a risk of the candidate declining the offer even between accepting the offer and actually joining the company, the fact of joining the company serves as the final confirmation of their acceptance.

[0503] The information processing device 100 acquires target event information 800 related to joining the company upon completion of employee registration in the personnel system (see Figure 2).

[0504] Furthermore, it is possible to incorporate the retention and performance levels of employees during a certain period after joining the company as 800 expansive target event information.

[0505] The learning process 810, which uses labels related to joining the company, contributes to the construction of selection rules 520 to prevent mismatches after joining the company (see Figure 8).

[0506] In dialogue sessions, cases where information, including the negative aspects of the company, is honestly presented via the dialogue screen 700 (see Figure 7), and where the candidate subsequently decides to join the company, are highly valued.

[0507] Presenting information in this way is effective in helping candidates understand realistic job requirements without creating unrealistic expectations.

[0508] The selection rule update unit 230 uses the target event information 800, which is joining the company, to learn selection rules 520 related to determining the information to be presented (such as talk scripts 1000 and potential conversation partners 1010) that will produce the best matching from a long-term perspective (see Figures 5, 9, and 10).

[0509] By analyzing the pre-employment conversation history of candidates who have joined the company, it is possible to extract information useful for onboarding.

[0510] Since information regarding joining the company has the longest delay in acquisition, it is learned hierarchically in combination with 800 other target event information.

[0511] For example, the series of events from application completion to joining the company are treated as a chronological path, and the model is adjusted to increase the completion rate of the entire path (see Figure 6).

[0512] A feedback loop based on the outcome of hiring provides a foundation for extending the value proposition of the system beyond the recruitment area to the area of ​​organizational adaptation after joining the company.

[0513] Advanced analysis that correlates employee engagement scores after joining the company with response features during dialogue sessions is also envisioned.

[0514] Thus, the 800 target event information items related to joining the company serve as the most profound source of learning.

[0515] Here, we will detail the characteristics of target event information 800 as an outcome delay label.

[0516] The reaction features obtained in step 300 (see Figure 4), etc., are merely immediate indicators.

[0517] Even if the person being discussed smiles or nods in interest at the time, it doesn't necessarily mean that it will lead to a final action.

[0518] There is a temporal gap and causal uncertainty between the immediate response features and the final target event information 800.

[0519] The learning process 810 is a mechanism to bridge this gap (see Figure 2).

[0520] The information processing system 10 (see Figure 1) calculates a provisional evaluation score based on immediate response features at the end of the dialogue session.

[0521] Subsequently, when the target event information 800 is obtained several days or months later, the provisional evaluation score is overwritten with the true outcome label.

[0522] This process enables a fundamental update of selection rule 520 that is not deceived by superficial reactions.

[0523] A key challenge in handling delay labels is how to isolate the influence of external factors that occur over time.

[0524] The information processing device 100 extracts the true correlation between the presented information and the target event information 800 by statistically processing large amounts of data obtained from multiple subjects.

[0525] A storage unit 240 is provided for accumulating delay labels for results, and data is continuously integrated.

[0526] The confidence score of a particular piece of presented information increases when it is consistently associated with positive target event information (800).

[0527] Conversely, information that elicits a good immediate response but does not lead to the target event information (800) will have its confidence score lowered.

[0528] By utilizing these delay labels, the information processing system 10 can discover patterns that are difficult to notice through human intuition or short-term observation.

[0529] Latent label-based reinforcement learning approaches are a core technology that ensures long-term performance improvements in dialogue support systems.

[0530] In executing the learning process 810 shown in Figures 3 and 8, the information processing device 100 undergoes several preparatory steps.

[0531] First, a matching process is performed to accurately link the acquired target event information 800 with the historical data of the corresponding dialogue session.

[0532] This matching process is performed using the subject's unique identifier as the key.

[0533] Next, the time-series reaction feature data calculated by the reaction feature calculation unit 200 (see Figure 4) is formatted as a feature vector.

[0534] Simultaneously, historical data (see Figure 10) such as the identifier of the information to be presented, the presentation timing 620 (see Figure 6), and the presentation duration, which were presented on the dialogue screen 700 (see Figure 7), are organized.

[0535] These data sets are processed by the filtering unit 910 (see Figure 9) and, together with the first target attribute information 500 (see Figure 5) and past achievement information 510, are constructed as a training data set for learning.

[0536] In the constructed training dataset, the target event information 800 functions as either the dependent variable or the reward signal.

[0537] The system performs data cleansing to remove outliers and missing values ​​that would otherwise be considered noise.

[0538] For example, data from sessions where response features could not be properly acquired due to communication problems will be excluded from the training.

[0539] During the preparation phase, sampling adjustments are also made to prevent data bias.

[0540] In some cases, the data may be adjusted to have an equal ratio of data with positive target event information (800) to data with negative information.

[0541] The refined training data is then ready to be fed into the learning algorithm.

[0542] For the learning process 810 to function effectively, the continuous construction of large, high-quality datasets is essential.

[0543] The system autonomously performs this preparation process in the background.

[0544] Once preparations are complete, the system will begin executing learning process 810 in earnest.

[0545] Machine learning algorithms and statistical inference models are applied to the learning process 810.

[0546] The system models how the combination of the subject's attributes and response features, along with the presented target information, affects the probability of the occurrence of target event information 800.

[0547] For example, when using a deep learning model, the weights of the multi-layered neural network are updated by backpropagation.

[0548] The network parameters are adjusted to minimize the error between the target event information (800) and the predicted values.

[0549] When using decision tree-based ensemble learning, the branching conditions for features that maximize information gain are recalculated.

[0550] The learning process 810 highlights which audience segment is most effective for specific presented information.

[0551] Furthermore, the appropriateness of the presentation timing is also learned.

[0552] The study compares two approaches to determine which yields a higher success rate in achieving the target event information (800 items): presenting the information at the end of a topic or presenting it at the moment when the response features change rapidly.

[0553] The learning process 810 could be run as a batch process, either daily or weekly, during low-load periods such as late at night.

[0554] Alternatively, an online learning model could be adopted in which the model is updated sequentially each time new target event information 800 is acquired.

[0555] Adopting online learning allows companies to quickly adapt to changing candidate trends and developments in the recruitment market.

[0556] As a result of the learning process, an expected outcome score is calculated for each piece of presented information.

[0557] This expected performance score serves as a direct indicator when selecting the information to be presented in the next dialogue session in the information processing system 10 (see Figure 1).

[0558] The learning process 810 also identifies patterns of meaningless or counterproductive information presentation (see Figures 8 and 9).

[0559] Presentation patterns strongly associated with negative target event information (800) will be penalized and demoted from future selection options.

[0560] This continuously reduces the risk of the information processing system 10 providing inappropriate support.

[0561] The execution log of the learning process 810 is saved as audit information to monitor the algorithm's operation.

[0562] Human administrators can monitor the convergence status and performance indicators of the learning process 810.

[0563] The learning process 810 elevates the information processing system 10 from a mere rule-based program into an adaptive system that grows from experience.

[0564] Based on the results of the learning process 810, an automatic update process for the selection rules 520 is executed. The selection rules 520 are a set of internal logic and parameters that the information processing system 10 uses to determine the information to be presented in a new dialogue session.

[0565] The initial state selection rule 520 consists of a simplified model based on heuristic rules and general trends set by human experts.

[0566] The automated update process replaces or adjusts this initial model into a data-driven, advanced model.

[0567] The information processing device 100 (see Figure 2) safely reflects the new set of parameters obtained in the learning process 810 into the selection rules 520 of the actual operating environment.

[0568] The updates may be applied uniformly to the entire information processing system 10, or they may be updated as individualized selection rules 520 for specific companies or departments.

[0569] For example, selection rule 520 for technical candidates and selection rule 520 for sales candidates are updated independently.

[0570] During automatic updates, the information processing device 100 maintains a backup of the old selection rule 520 so that it can be rolled back in the event of any performance degradation.

[0571] The updated selection rule 520 is immediately used to determine the information to be presented in the next dialogue session.

[0572] The more frequently updates are performed, the better the information processing system 10 will be able to adapt to even minute environmental changes.

[0573] Updates to selection rule 520 often manifest as changes to the coefficients of the information display ranking algorithm.

[0574] When the score calculation coefficient of a particular talk script 1000 is increased by the backing of the target event information 800 (see Figure 10), that talk script 1000 will be ranked as a more preferred candidate for presentation in the next dialogue session.

[0575] Additionally, updates to Selection Rule 520 may automatically generate new rules (see Figure 1).

[0576] For example, conditional rules are automatically extracted from the data, such as presenting a specific set of 1000 talk scripts or a list of suggested questions (see Figure 10) when the speaking rate of the first participant (see Figure 4) decreases.

[0577] The automated update process is designed to complete autonomously without human intervention and is executed by the selection rule update unit 230 (see Figures 2 and 8).

[0578] However, to prevent abrupt changes in selection rule 520, a stabilization mechanism is incorporated that sets an upper limit on the amount of parameter change per update.

[0579] In this way, the selection rule 520 is continuously refined through the accumulation of daily dialogue sessions and target event information 800 (see Figure 3).

[0580] The automatic update process is fundamental to ensuring that the information processing system 10 always provides the latest and most optimal support.

[0581] Each time the selection rule 520 is updated, the quality of the dialogue support provided by the information processing system 10 continues to improve.

[0582] The automatic updating of selection rule 520 leads to the final optimization process.

[0583] The optimization process goes beyond mere parameter adjustment and includes a broader control mechanism to maximize the overall system performance of the information processing device 100.

[0584] The information processing system 10 may incorporate exploratory approaches, such as A / B testing, to verify the effectiveness of the updated selection rules 520.

[0585] The process is probabilistically divided into an application phase, where known optimal selection rules 520 are applied, and an exploration phase, where new possibilities are sought.

[0586] This prevents getting stuck in a local optimum and allows for the discovery of previously unknown, superior presentation patterns.

[0587] The new data obtained through the exploratory approach is then returned to the update loop based on the learning process 810 and target event information 800.

[0588] This cycle puts selection rule 520 on a path to permanent optimization.

[0589] The optimization process has the effect of comprehensively increasing the achievement rate of the 800 target event information items.

[0590] The conversion rate at each stage of the funnel, from application completion to onboarding, is improved (see Figure 5).

[0591] The information processing system 10 will be able to make a comprehensive judgment on which information to present in which dialogue session will contribute most to the ultimate goal of hiring the employee (see Figure 9).

[0592] As optimization progresses, the information processing system 10 will be able to provide highly individualized support for each primary target individual.

[0593] By precisely referencing data from groups that previously possessed similar primary target attribute information (500) and recorded positive target event information (800), optimal presentation control is performed (see Figure 6).

[0594] As a result, interviewers and recruiters, who are the other parties in the dialogue, can reproduce high performance simply by following the support of the information processing system 10 via the dialogue screen 700 (see Figure 7).

[0595] The know-how of dialogue, which previously relied on individual experience and intuition, is now accumulated as data within the organization and returned as optimized selection rules 520.

[0596] The loop starting from the learning process 810 shown in Figure 8 represents the framework for autonomous knowledge acquisition in the information processing system 10.

[0597] By learning by waiting for the delayed truth signal of target event information 800, a robust system that is resistant to noise is achieved.

[0598] This optimization process becomes more powerful the longer the system is in operation and the larger the amount of data.

[0599] The accuracy with which the system determines the information to present evolves to a level unimaginable from its initial rule-based approach.

[0600] For the dynamic challenge of supporting dialogue sessions, goal-event-based feedback learning offers the most rational solution.

[0601] With the above steps completed, the process of acquiring target event information 800, executing the learning process 810, and updating and optimizing the selection rule 520 is finished.

[0602] This continuous loop structure is the source of the sustained improvement in the value of the information processing system 10 according to this embodiment.

[0603] Figure 9 shows the configuration of a filtering process that excludes items unrelated to aptitude or ability from the list of candidates.

[0604] In the information processing system 10 that supports the dialogue session, guardrail processing is required to remove certain inappropriate information from the presented information, including the talk script 1000 and the candidate dialogue partner 1010 shown in Figure 10.

[0605] In particular, during the recruitment process, there is a compliance requirement that prohibits the collection or use of information unrelated to a candidate's suitability or abilities in order to conduct a fair selection.

[0606] Guardrail processing provides system control to meet these compliance requirements.

[0607] As shown in Figure 9, this filtering process consists of an exclusion list 900 and a filtering unit 910.

[0608] The exclusion list 900 is a set of data defining information unrelated to suitability or ability.

[0609] The exclusion list of 900 includes various categories of matters related to respecting the fundamental human rights of candidates.

[0610] For example, items related to one's living environment, such as registered domicile and place of birth, are included in the exclusion list of 900.

[0611] Furthermore, family-related matters such as family members' occupations, income, or health status are also included in the exclusion list of 900.

[0612] Furthermore, matters relating to religion, political affiliation, or ideology are also included in the exclusion list of 900 items.

[0613] This is because this information is not inherently related to a candidate's ability to perform the job or their suitability for the role.

[0614] The exclusion list of 900 may not be merely a list of keywords, but rather a dictionary of conceptual categories and related terms.

[0615] The exclusion list of 900 (see Figure 9) is regularly updated and managed to ensure compliance with the latest laws and guidelines.

[0616] The information processing device 100 (see Figure 2), which constitutes the information processing system 10 (see Figure 1), can obtain the latest exclusion list 900 from an external compliance information provision server and store it in the storage unit 240.

[0617] As part of the procedure shown in Figure 3, the filtering unit 910 performs filtering on the determined information to be presented using the exclusion list 900.

[0618] The filtering unit 910 acquires text data, video metadata, or audio data, etc., which have been determined to be presented as information to be analyzed.

[0619] The filtering unit 910 compares the acquired information to be presented with the exclusion list 900 to determine whether it contains any items unrelated to suitability or ability.

[0620] In addition to keyword searches using exact matches, semantic similarity calculations using natural language processing can also be employed as a method for determining similarity.

[0621] The filtering unit 910 analyzes the context of the information to be presented and scores whether or not it belongs to an inappropriate category intended by the exclusion list 900.

[0622] If the calculated score exceeds a predetermined threshold, the filtering unit 910 determines that the information is inappropriate.

[0623] If the filtering unit 910 determines that the information is inappropriate, it executes a process to remove the information from the list of candidates to be presented.

[0624] One method of exclusion is to completely remove the entire information in question from the list of potential suggestions.

[0625] Another method of exclusion involves masking only the parts deemed inappropriate and retaining the remaining parts as potential suggestions.

[0626] If the filtering unit 910 excludes the information to be presented, the information processing device 100 performs a process to re-determine alternative information to be presented based on the concepts shown in Figure 5.

[0627] This prevents the depletion of information during a dialogue session and enables continuous dialogue support, as shown in the timeline in Figure 6.

[0628] Furthermore, the filtering unit 910 may perform real-time filtering not only on the information to be presented but also on the content of the interviewer's speech, who is the second target.

[0629] The interviewer's spoken text is converted into text in real time using speech recognition technology (see Figure 4) and sequentially input into the filtering unit 910.

[0630] The filtering unit 910 monitors whether the interviewer's past statements or the talk script 1000 (see Figure 10) that they are about to speak falls under the exclusion list 900.

[0631] If it is determined that the interviewer is attempting to ask questions unrelated to aptitude or ability, the information processing system 10, which reflects the results of the learning process 810 (see Figure 8), executes a warning process.

[0632] This warning process is carried out by displaying an alert on the screen of the second target terminal 120, which displays a dialogue screen 700 as shown in Figure 7.

[0633] The alert message may include the reason why the comment is inappropriate and suggestions for improvement.

[0634] This type of system control can reduce the risk of interviewers unintentionally asking inappropriate questions.

[0635] This will ensure fair recruitment and selection processes, and will also have the concrete effect of improving compliance awareness throughout the organization.

[0636] Furthermore, the information processing system 10 also includes a logging function to meet governance requirements for AI decision-making and processing.

[0637] The filtering unit 910's judgment results and the content of the excluded information are stored in the storage unit 240 along with the dialogue session log (see Figure 9).

[0638] This log can later be output as an audit report and used to verify whether the information processing system 10 was functioning correctly.

[0639] The information processing system 10 may also include a bias monitoring function to monitor whether candidates are being unfairly discriminated against based on specific attributes.

[0640] This proves that the AI-based information processing system 10 is being operated in accordance with laws and ethical standards.

[0641] The information processing system 10 can also be applied as a human-intervention type processing system that incorporates final verification by a human.

[0642] In other words, instead of automatically excluding information that the filtering unit 910 has determined to be inappropriate, the system is configured to first send an approval request to the administrator's terminal (see Figure 9).

[0643] Information will only be removed from the list of potential submissions if the administrator reviews the content and approves its exclusion.

[0644] This type of presentation control with human approval prevents the loss of valuable information due to misjudgments by the information processing system 10.

[0645] Up to this point, we have mainly explained guardrail processing using the scenario of a job interview as an example, but the information processing system 10 can be applied to purposes other than recruitment.

[0646] The information processing system 10 can also be used for various dialogue sessions, such as the onboarding process after joining the company, regular one-on-one meetings, or goal-setting meetings (see Figure 1).

[0647] In the onboarding process, new employees are the primary target group, while HR personnel and training staff are the secondary target group.

[0648] During the dialogue sessions in the onboarding process, appropriate explanatory information about the company's culture and systems is presented based on the response characteristics of the new employee.

[0649] For example, if a new employee shows interest in employee benefits, relevant company regulations and usage examples are presented to the second target terminal 120 in sync with that response (see Figures 6 and 7).

[0650] Furthermore, it is possible to transfer and utilize the response features and past performance information 510 acquired during the hiring interview and stored in the memory unit 240 during the onboard process.

[0651] Data on topics and values ​​that candidates strongly responded to during the hiring process will be used in onboarding training plans and placement selection.

[0652] This enables consistent, individually optimized communication support from recruitment to post-employment.

[0653] In regular one-on-one meetings, the subordinate is the primary person to be interviewed, and the supervisor is the secondary person to be interviewed.

[0654] Based on the subordinate's response characteristics during the one-on-one meeting, the second participant terminal 120 is presented with suggestions on how to provide feedback and the next work-related issues to discuss (see Figure 7).

[0655] If it is determined that the subordinate's response has declined, 1000 talk scripts and success stories designed to improve motivation will be selected as the information to be presented (see Figure 10).

[0656] Similarly, in goal-setting interviews, appropriate goal proposals and career path examples are presented, taking into account the subordinate's growth aspirations and past performance.

[0657] By switching the types of information presented and the selection rules 520 according to the purpose of the dialogue session, it is possible to support a variety of human resource policies (see Figures 2, 3, and 5).

[0658] For non-recruitment applications, the information processing system 10 can also be applied to sales and customer success interviews.

[0659] In a sales meeting, the customer is the primary target, and the sales representative is the secondary target.

[0660] Based on customer response characteristics and past performance information such as past order results and contract renewal results, optimal sales materials and FAQs are dynamically presented.

[0661] Similarly, in customer support call centers and consultation desks, potential answers and instructions for transferring the user to a different representative are presented based on the user's response. In this case, processing may be performed by the filtering unit 910 based on the exclusion list 900 (see Figure 9).

[0662] In this case, the problem resolution rate and customer satisfaction survey results are used as target event information 800 to update the selection rule 520 (see Figure 8).

[0663] This makes it possible to produce highly reproducible results in any dialogue situation.

[0664] The format of the dialogue session is not limited to online video calls.

[0665] The information processing system 10 also functions effectively in face-to-face offline interviews and meetings (see Figure 1).

[0666] In offline environments, data is acquired using microphone arrays and cameras installed in the conference room, or using participants' smartphones.

[0667] When there are multiple participants, beamforming and speaker separation technologies are used to individually extract the speech and video of each participant.

[0668] Response features are calculated from the extracted data of each individual, and the overall conversation situation is analyzed (see Figure 4).

[0669] Even in offline interviews, the second participant can receive the presented information through the second participant terminal 120 (see Figure 7).

[0670] This allows us to capture nonverbal information unique to face-to-face interactions while providing system support equivalent to online systems.

[0671] Furthermore, the information processing system 10 can also be applied to one-to-many dialogue sessions, such as briefing sessions or roundtable discussions, in which multiple primary target participants are involved.

[0672] In one-to-many sessions, response features obtained from multiple participants are aggregated or averaged to calculate the overall level of interest.

[0673] Based on the calculated overall response index, the order of the next slides to be displayed, as well as the topic breaks 610 and presentation timings 620, are determined (see Figure 6).

[0674] This makes it possible to conduct presentations and group discussions that are more likely to capture the interest of all participants.

[0675] Furthermore, an embodiment is also possible in which an asynchronous video interview, in which the candidate unilaterally records the interview, is linked with the information processing system 10.

[0676] The response features extracted from the asynchronous video interview data that the candidate answered in advance (see Figure 4) are stored in the memory unit 240 as prior information.

[0677] By referring to this preliminary information during subsequent live dialogue sessions, it becomes possible to determine with greater accuracy which information to present.

[0678] The information processing system 10 can also operate in conjunction with external systems.

[0679] For example, the system automatically retrieves 500 pieces of primary target attribute information (see Figure 5) and evaluation history from recruitment management systems and talent management systems, and uses this information to determine which information to present.

[0680] After the dialogue session ends, the system exports the generated summary data and trend reports of response indicators to an external system.

[0681] Furthermore, it can be integrated with another AI agent system that handles scheduling and automatically assigns interviewers.

[0682] The information processing system 10 specializes in presentation control and outcome learning, and enhances the overall flexibility of the system by delegating peripheral tasks to an external AI agent.

[0683] It's also conceivable to integrate with the company's knowledge base and document management system to dynamically search for and present necessary documents during the dialogue session.

[0684] Thus, an API-based delivery model that assumes integration with other systems is also one of the effective embodiments of this information processing system 10.

[0685] The information processing device 100 does not need to be implemented by a single physical server; it may be implemented in a cloud computing environment where multiple computers cooperate via a network 130, as shown in Figures 1 and 2.

[0686] Some processing, such as multimodal analysis processing 400 (e.g., speech recognition and video analysis), which is computationally intensive, will be executed using powerful computing resources on the cloud.

[0687] On the other hand, an edge computing configuration may be adopted in which some processes, such as the calculation of reaction features by the reaction feature calculation unit 200 and the processing by the filtering unit 910, are performed on the terminal side.

[0688] By performing processing on the edge side, network latency 130 can be reduced, enabling more real-time presentation control.

[0689] Furthermore, it is possible to have the terminal store a cache of selection rules 520 and information to be presented in preparation for failures or temporary disconnections of the communication network 130.

[0690] This has the concrete effect of enabling the continuation of basic dialogue support even in unstable network 130 environments.

[0691] As shown in Figure 9, the scope of application of the exclusion list 900 can be flexibly customized according to the laws and regulations of the company and region.

[0692] When using the information processing system 10 in a multinational corporation, multiple exclusion lists 900 that comply with the employment laws and privacy protection laws of each country are maintained and switched for each session.

[0693] For example, if a question that is legal in one country is excluded in another, the system will apply the appropriate list based on the device's location and settings.

[0694] Furthermore, the selection rule update unit 230 (see Figure 8) may have a function to separate the learning data for each specific tenant and update the selection rules 520 so that data from other companies is not mixed in.

[0695] Tenant separation allows for the secure construction of a logic for determining the information to be presented that is optimized for the company's own unique recruitment criteria and values, within the information processing system 10 (see Figure 1).

[0696] Furthermore, even within the same tenant, selection rule 520 (see Figure 5) can be branched for each department or job type, allowing for support tailored to the respective areas of expertise.

[0697] The data formats handled by the information processing system 10 are not limited to video, audio, and text, but may also include input from biosensors.

[0698] For example, in this embodiment, physiological data such as heart rate and sweat volume acquired from a wearable device are used as part of the reaction features by the reaction feature calculation unit 200 (see Figure 2) in the reaction feature calculation step 300 (see Figure 4).

[0699] By combining physiological data, it is possible to estimate the unconscious state of tension and excitement of the person being interviewed in more detail.

[0700] However, since the handling of physiological data requires stricter privacy protection and explicit consent from the subjects, this is also controlled as part of the processing by the filtering unit 910 (see Figure 9).

[0701] Furthermore, the information processing system 10 visualizes the progress of the dialogue session as a timeline (see Figure 6) and provides an interface for retrospective analysis.

[0702] In this interface, the dialogue screen 700 (see Figure 7), the changes in response features, along with what information was presented at what timing and how the user responded to it, are displayed in a list in the presented information display area 710, etc.

[0703] HR personnel and managers can use this interface to reflect on their own communication skills and identify areas for improvement.

[0704] Thus, the information processing system 10 not only provides real-time support during the conversation, but also plays a role in providing continuous skill improvement support after the conversation through the presentation timing control step 320 (see Figure 3), etc.

[0705] When a candidate for a dialogue partner, 1010 (see Figure 10), is selected as the information to be presented, the selection criteria consider not only compatibility with similar candidates in the past but also their current operational status.

[0706] For example, if an employee who has previously received high evaluations is unable to participate due to scheduling conflicts, the information selection unit 210 will present another employee with the next highest suitability as a potential alternative conversation partner candidate 1010.

[0707] Furthermore, dynamic assignments are possible, where employees with specific skills are temporarily invited to interactive sessions to answer essential questions.

[0708] This dynamic selection of dialogue partners allows for the effective use of limited human resources in the most opportune situations.

[0709] The information processing system 10 may operate in conjunction with an automatic translation function when the languages ​​used by the first target terminal 110 and the second target terminal 120 are different.

[0710] One utterance is translated in real time, and based on the translated text, decisions are made in step 310 (see Figure 3) regarding the information to be presented.

[0711] Since the information presented is also translated and displayed on the second user's terminal 120, dialogue support that transcends language barriers becomes possible.

[0712] In all these variations, guardrail treatment is consistently applied, maintaining its role in preventing inappropriate information transmission.

[0713] The exclusion list 900 may also have a self-extending function that automatically detects and adds new inappropriate expressions using the learning process 810 (see Figure 8).

[0714] The system learns the context of statements that have been manually excluded by humans in the past, and if an unknown expression is estimated to belong to an inappropriate category, it is automatically added to the exclusion list of 900.

[0715] This will enable a rapid response to changes in language trends and new harassment terminology.

[0716] However, it is desirable to configure the system so that automatically added items undergo an administrator review process before being formally applied.

[0717] Furthermore, the presentation control data generated by the presentation control data generation unit 220 may not only include data for visually presenting information, but also data for reading the information aloud using speech synthesis (see Figure 7).

[0718] In situations where the interviewer cannot see the screen, the presentation control data generation unit 220 controls the system so that the interviewer can receive advice from the system via earphones.

[0719] As shown in Figure 6, in the presentation timing control step 320, playback is started at an appropriate presentation timing 620 using the response feature change time 600 and topic division 610 so as not to interrupt the conversation (see Figure 3).

[0720] These controls improve the usability of the information processing system 10 (see Figure 1) and enable more natural dialogue support.

[0721] The information processing device 100 has a function to output logs in a format that can respond to information disclosure requests from the subject.

[0722] This is because, from the perspective of protecting individual rights, there are cases where individuals are required to disclose what data was used to determine their evaluation.

[0723] In this respect as well, the record of the guardrail processing by the filtering unit 910 functioning effectively serves as proof that the company is operating the system properly (see Figure 9).

[0724] The update of the selection rules 520 by the selection rule update unit 230 may be performed periodically as a batch process, or it may be performed sequentially each time the target event information 800 occurs (see Figures 2 and 8).

[0725] Continuous updates allow insights gained from the most recent dialogue session to be immediately reflected in the next one.

[0726] Conversely, batch processing for updates is suitable when a sufficient amount of data has been accumulated and statistically stable updates are to be performed.

[0727] System administrators can set the optimal timing for learning updates according to the nature of the target business and the amount of data involved.

[0728] The information processing system 10 can also utilize data obtained by quantifying qualitative survey results using natural language processing as output information (see Figure 5).

[0729] For example, we can perform sentiment analysis on the free-response comments included in candidate questionnaires after interviews and use the degree of positivity as a success label.

[0730] This enables more nuanced learning based on performance indicators with a gradient, rather than just binary events such as pass / fail or rejection.

[0731] Alternatively, a federated learning method may be applied, which involves sharing the parameters of anonymized pre-trained models among multiple information processing systems 10.

[0732] This allows for the mutual refinement of the 520 selection rules within each company's system while avoiding direct data sharing between companies.

[0733] The information processing system 10 is designed not only to provide support for a single conversational session, but also to support the building of long-term relationships through multiple interviews and meetings.

[0734] The results calculated by the response feature calculation unit 200 during the first interview, along with the results of the presented information, are used as important context in determining the information to be presented in subsequent interviews (see Figures 4 and 10).

[0735] With each subsequent interview, the information processing system 10 gains a deeper understanding of the candidate's aptitudes and interests, continuously improving the accuracy of the information it presents.

[0736] This continuous feedback loop is a core function that maximizes the value of the information processing system 10.

[0737] Each of the functional units and processing steps described so far is typically implemented as a software program module (see Figures 1 to 10).

[0738] These programs are loaded into the computer's memory and executed sequentially by the CPU or other processor to perform their functions.

[0739] Furthermore, it is technically possible to implement some or all of the functions of the information processing device 100 using dedicated hardware circuits, ASICs, or FPGAs.

[0740] The degree to which functions are distributed and implemented in hardware is a design matter that should be appropriately modified according to the requirements and operating environment of the information processing system 10.

[0741] Furthermore, the type of network 130 used in the information processing system 10 is not limited to the internet; it may also be an intranet, a closed network, or a dedicated line.

[0742] The forms of the first target terminal 110 and the second target terminal 120 are not limited to personal computers or smartphones; smart displays and dedicated communication terminals can also be used.

[0743] The storage unit 240 of the information processing device 100 may be an internal hard disk or SSD, as well as network storage or an external database server.

[0744] Furthermore, the specific numerical values, thresholds, and category names presented in this specification are merely examples and do not preclude the use of other appropriate values ​​or names.

[0745] The order of processes shown in flowcharts such as Figure 3, Figure 4, or Figure 8 can be rearranged or executed in parallel as appropriate, as long as no logical inconsistencies arise.

[0746] It is permissible, within the scope of the concept of this information processing system 10, to omit certain steps or to add new steps.

[0747] Although the above explanation used a server as an example, the information processing device 100 can also be a server that provides chat services.

[0748] In this case, the new message notification can be a notification to a pre-registered chat service, and the reception by the information processing system 10 can be the reception of a notification generated by the server providing the chat service.

[0749] Furthermore, please note that unless the word "only" is explicitly stated, such as in phrases like "based only on," "according only to," or "in the case only," it is assumed in this specification that additional information may also be considered.

[0750] Also, please note that the statement "If a, then do b" does not necessarily mean "if a, then always do b" unless explicitly stated otherwise.

[0751] Furthermore, for the sake of clarity, even if there are aspects in some method, program, first target terminal 110, second target terminal 120, information processing device 100, or information processing system 10 that perform operations different from those described herein, each aspect of the present invention is intended to be the same as any of the operations described herein.

[0752] It should be added that the existence of operations different from those described herein does not mean that such methods, etc., fall outside the scope of each aspect of the present invention. [Explanation of symbols]

[0753] 10 Information Processing Systems 100 Information Processing Devices 110 Primary Target Terminal 120 Second Target User Terminal 130 Networks 200 Reaction Feature Calculation Unit 210 Information to be presented determination unit 220 Presentation control data generation unit 230 Selection Rule Update Section 240 Storage section 300 Response Feature Calculation Steps 310 Steps for determining the information to be presented 320 Presentation timing control step 400 Multimodal Analysis Processing 410 Feature Extraction Process 500 Target Audience Attribute Information 510 Past results information 520 Selection Rules 600 Response Feature Change Point 610 Topic break 620 Presentation timing 700 Dialogue screen 710 Presentation information display area 800 Target Event Information 810 Learning Process 900 Exclusion List 910 Filtering section 1000 Talk Scripts 1010 Potential dialogue partners

Claims

1. A means for calculating response features, which are indicators that quantitatively represent the degree of interest, understanding, emotional changes, or level of engagement of the first subject to the dialogue session, by analyzing data obtained in a dialogue session between the first subject and the second subject, including at least one of the following: audio, video, text, or terminal operation history. A means for determining information to be presented to the terminal of a second subject during or before / after a dialogue session, based on the response features and past outcome information, which is information indicating the outcome of a dialogue session, including information on a target event, which is an event indicating that the subject performed a predetermined action, obtained after the dialogue session for a past dialogue session, the means for determining information to be presented to the terminal of a second subject during or before / after a dialogue session, the means for determining information to be presented to the terminal of a second subject, based on the response features and past outcome information, which is information indicating the outcome of a dialogue session, including information on a target event, which is an event indicating that the subject performed a predetermined action, obtained after the dialogue session for a past dialogue session, the means for determining information to be presented to the terminal of a second subject, the means for determining information to be presented, the means for determining information to be presented, the means for which the score is calculated for each of the multiple presentation candidates, and the information to be presented includes at least one of a talk script, a question candidate, or a dialogue partner candidate displayed on the terminal of the second subject, the means for which the information to be presented includes, A means for generating presentation control data to present the information to be presented during the dialogue session from the determined information to be presented, to the terminal of the second target, at a timing synchronized with the point in time when the response features change as detected from the time-series data of the response features, or with the topic division as detected by analysis of the content of the utterances in the dialogue session. A means for updating the selection rules for determining the information to be presented, using the information regarding the target event for the first subject, obtained after the dialogue session, as training data or reward signals; An information processing system equipped with the following features.

2. The information processing system according to claim 1, The means for making the determination comprises calculating the similarity between the attribute information of the first subject and the attribute information of past subjects, and determining the information to be presented by calculating a score based on outcome information that shows the actions or results shown by past subjects in or after past dialogue sessions, provided that the similarity meets a predetermined standard. Information processing system.

3. The information processing system according to claim 2, The information relating to the aforementioned target event comprises information relating to at least one of the following for the first target person: completion of application, participation in an interview, passing the selection process, acceptance of a job offer, or joining the company. Information processing system.

4. The information processing system according to claim 1, The means for making the determination comprises a configuration that determines the information to be presented by excluding information unrelated to the suitability or ability of the first subject from the candidates for presentation. Information processing system.

5. A computer, The steps include: analyzing data obtained during a dialogue session between a first subject and a second subject, which includes at least one of audio, video, text, or terminal operation history, to calculate a response feature, which is an index that quantitatively represents the first subject's level of interest, understanding, emotional changes, or level of engagement during the dialogue session; A step of determining information to be presented to the terminal of the second subject during or before the dialogue session, based on the response features and past outcome information, which is information indicating the outcome of a dialogue session, including information on a target event, which is an event indicating that the subject performed a predetermined action, obtained after the dialogue session for a past dialogue session, the steps of generating a plurality of presentation candidates by referring to the past outcome information and searching for presentation information that is similar in attributes to the first subject and shows a response similar to the response indicated by the response features, and for which a good outcome was obtained for the past subject, the score is calculated for each of the plurality of presentation candidates using a selection rule constructed based on the past outcome information, which outputs a score indicating the predicted probability of obtaining a good outcome when the presentation information is presented with the presentation information as input, and the information to be presented is determined based on the score, wherein the information to be presented includes at least one of a talk script, a question candidate, or a dialogue partner candidate to be displayed on the terminal of the second subject. The steps include generating presentation control data to present the information to be presented during the dialogue session from the determined information to be presented, to the terminal of the second target person at a timing synchronized with the point in time when the response features change as detected from the time-series data of the response features, or with the topic division as detected by analysis of the content of the utterances in the dialogue session, The steps include updating the selection rules for determining the information to be presented, using the information regarding the target event for the first subject, obtained after the dialogue session, as training data or reward signals; A program to execute.

Images