Information processing system, information processing method, and information processing program

Abstract

The aim is to appropriately support users' introspection while taking into account the differences in values ​​that users hold regarding each topic, as well as the changes in their emotional state at any given time. [Solution] The information processing system includes a processor, which acquires user dialogue data, extracts topics related to the user based on the dialogue data, estimates the user's emotional state, stores a baseline emotional state for each topic in a database for each user based on the topics and the emotional state, and when new dialogue data is acquired through dialogue with the user, the baseline state associated with the topics extracted based on the newly acquired dialogue data is acquired from the database, and feedback is provided to the user according to the user's emotional state estimated based on the newly acquired dialogue data and the baseline state.

Description

[[TECHNICAL FIELD]]

[0001] The present disclosure relates to an information processing system, an information processing method, and an information processing program. [[BACKGROUND ART]]

[0002] Conventionally, information processing techniques have been known that estimate the user's emotions, interests, concerns, or the like based on dialogue data with the user, and present information or adjust response content in accordance with the estimation results. For example, Patent Document 1 describes a technique that analyzes information related to emotions and values based on information acquired from a user, and presents the analysis results as visualization or advice. Further, Patent Document 2 describes a technique for adjusting conversation content and topics for subsequent conversations based on conversation history with the user, user reactions, or the user's emotions and conditions. [[PRIOR ART DOCUMENTS]] [[PATENT DOCUMENTS]]

[0003] [[Patent Document 1]] Japanese Unexamined Patent Publication No. 2025-047177 [[Patent Document 2]] Japanese Unexamined Patent Publication No. 2025-058998 [[SUMMARY OF THE INVENTION]] [[Problem to be Solved by the Invention]]

[0004] With conventional techniques, it is possible to estimate a user's emotion based on data acquired during dialogue, and change the content to be output in accordance with the estimation result. However, it has not been easy to interpret the user's emotional state at the relevant time point and provide information conducive to self-reflection, while taking into account what values the user holds with respect to a certain topic. That is, even for the same user, emotional tendencies may differ depending on the topic, and even for the same topic, different emotions may be exhibited depending on the situation, so there has been a problem that it is difficult to provide appropriate self-reflection support.

[0005] This disclosure is made in light of these issues and aims to provide an information processing system, information processing method, and information processing program that can appropriately support users' introspection while taking into account the differences in values ​​that users have regarding each topic and the changes in their emotional state at any given time. [Means for solving the problem]

[0006] An information processing system according to a first aspect of this disclosure includes a processor, which acquires user dialogue data, extracts topics related to the user based on the dialogue data, estimates the user's emotional state, stores a baseline emotional state for each topic in a database for each user based on the topics and the emotional state, and when new dialogue data is acquired through dialogue with the user, it acquires the baseline state associated with the topics extracted based on the newly acquired dialogue data from the database, and provides the user with feedback corresponding to the user's emotional state estimated based on the newly acquired dialogue data and the baseline state.

[0007] An information processing system according to a second aspect of this disclosure, in an information processing system according to a first aspect, the processor determines the discrepancy between the user's emotional state estimated based on newly acquired dialogue data and the reference state, and provides the feedback to the user if the discrepancy satisfies predetermined conditions.

[0008] An information processing system according to a third aspect of this disclosure, in an information processing system according to a second aspect, the processor stores in the database an importance level indicating how important each topic is to the user for each user, and performs the discrepancy determination when the importance level associated with a topic extracted based on newly acquired dialogue data exceeds a predetermined threshold.

[0009] An information processing system according to a fourth aspect of this disclosure, in an information processing system according to any one of the first to third aspects, the processor acquires data as dialogue data which includes linguistic information indicating the content of the user's utterance and non-linguistic information indicating at least one of the user's facial expression and voice.

[0010] An information processing system according to a fifth aspect of this disclosure, in an information processing system according to a fourth aspect, wherein the processor estimates a first emotional state of the user based on the linguistic information and estimates a second emotional state of the user based on the non-linguistic information, and determines the content of the feedback according to the discrepancy between the first emotional state and the second emotional state.

[0011] An information processing method relating to a sixth aspect of this disclosure includes a computer acquiring user dialogue data, extracting topics related to the user based on the dialogue data and estimating the user's emotional state, storing a baseline emotional state for each topic in a database for each user based on the topics and the emotional state, and, when new dialogue data is acquired through dialogue with the user, acquiring the baseline state associated with the topics extracted based on the newly acquired dialogue data from the database, and providing the user with feedback corresponding to the user's emotional state estimated based on the newly acquired dialogue data and the baseline state.

[0012] An information processing program according to the seventh aspect of this disclosure causes a computer to perform the following: a process of acquiring user dialogue data; a process of extracting topics related to the user based on the dialogue data and estimating the user's emotional state; a process of storing a standard emotional state that the user has for each topic in a database for each user, based on the topics and the emotional state; a process of acquiring the standard emotional state associated with the topics extracted based on the newly acquired dialogue data from the database when new dialogue data is acquired through dialogue with the user; and a process of providing the user with feedback corresponding to the user's emotional state estimated based on the newly acquired dialogue data and the standard emotional state.

[0013] The information processing system, information processing method, and information processing program described herein can appropriately support the user's introspection while taking into account the differences in the user's values ​​regarding each topic and the changes in their emotional state at any given time. [Brief explanation of the drawing]

[0014] [Figure 1] This figure shows a schematic configuration of the information processing system 1 according to this embodiment. [Figure 2] This figure shows an example of the hardware configuration of the information processing device 100 according to this embodiment. [Figure 3] This is a block diagram showing an example of the functional configuration of the information processing device 100 according to this embodiment. [Figure 4] This flowchart shows an example of information processing during the learning phase performed by the information processing device 100 according to this embodiment. [Figure 5] This figure shows an example of the information stored in database 50. [Figure 6] This flowchart shows an example of the processing in the usage phase according to this embodiment. [Figure 7] This figure shows an example of the feedback display according to this embodiment. MODE FOR CARRYING OUT THE INVENTION

[0015] Hereinafter, one embodiment of the present disclosure will be described with reference to the drawings. The configuration shown in the drawings is an example for convenience of explanation, and the present disclosure is not limited thereto. In addition, the same or corresponding parts are denoted by the same reference symbols in the drawings.

[0016] Fig. 1 is a diagram showing a schematic configuration of the information processing system 1 according to the present embodiment. The information processing system 1 includes a terminal 10, a server 30, a database 50, an AI model 70, and a network 90. Here, a case where there is one each of the terminal 10, the server 30, the database 50, the AI model 70, and the network 90 is shown as an example. However, the information processing system 1 may include two or more of at least any one of the terminal 10, the server 30, the database 50, the AI model 70, and the network 90.

[0017] Hereinafter, a case where the server 30 is the information processing apparatus 100 that is the主体 that executes the information processing according to the present disclosure will be described as an example. However, the present disclosure is not limited to this, and the terminal 10 may be the information processing apparatus 100 according to the present disclosure. Furthermore, both the terminal 10 and the server 30 may be the information processing apparatus 100 according to the present disclosure. That is, the information processing according to the present disclosure may be processed on the terminal 10 side, may be processed on the server 30 side, or may be processed in a distributed manner on the terminal 10 side and the server 30 side.

[0018] The terminal 10 is an apparatus operated by a user, and is configured by, for example, a smartphone, a tablet terminal, a laptop computer, a personal computer, or the like. The terminal 10 may input user's dialogue data and transmit the dialogue data to the server 30. In addition, the terminal 10 may output various kinds of information provided from the server 30.

[0019] The server 30 is a device that executes various types of information processing based on dialogue data acquired from the terminal 10. The server 30 may be configured by, for example, one or a plurality of computers. Note that part or all of the functions of the server 30 may be implemented in a cloud computing environment.

[0020] The database 50 is a storage unit that stores various types of information used for information processing executed by the server 30. The database 50 may be built into the server 30, or may be configured as a separate storage device or storage system from the server 30.

[0021] The AI model 70 is a model that executes at least part of analysis processing based on dialogue data. The AI model 70 may be configured as a natural language processing model, an emotion estimation model, a generative AI model, or a model combining these.

[0022] The network 90 is a communication network that communicably connects the terminal 10, the server 30, the database 50, and the AI model 70. The network 90 may include, for example, the Internet, a local area network, a mobile communication network, a wireless communication network, or the like.

[0023] In the information processing according to the present disclosure, user dialogue data is acquired, a topic related to the user is extracted based on the dialogue data, the emotional state of the user is estimated, and information processing based on the topic and the emotional state is executed. The information processing apparatus 100 that executes such information processing will be described in detail.

[0024] Figure 2 shows an example of the hardware configuration of the information processing device 100 according to this embodiment. The information processing device 100 may be a computer. More specifically, the information processing device 100 comprises a processor 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a storage 104, and an interface 105, and these components may be connected to each other so as to be able to communicate with each other via a bus 109.

[0025] The processor 101 executes various programs and controls each component. The processor 101 may be, for example, a CPU (Central Processing Unit).

[0026] ROM 102 stores the boot program and the like. RAM 103 temporarily stores programs or data as a working area. Storage 104 consists of an HDD (Hard Disk Drive) or SSD (Solid State Drive) and stores various programs and data, including the operating system. Note that storage 104 is not limited to a physical storage device and may be configured as cloud storage accessible via network 90.

[0027] In the information processing device 100, the information processing program 106 is stored in either the ROM 102 or the storage 104. Here, the case where the information processing program 106 is stored in the storage 104 is shown as an example. In this case, the processor 101 reads the information processing program 106 from the storage 104 and executes it using the RAM 103 as a working area, thereby performing control of each configuration and various calculation processes according to the information processing program 106.

[0028] Interface 105 is an input / output unit for the information processing device 100 to exchange information with the outside world. Interface 105 may include a communication interface for communicating with other devices and a user interface for exchanging information with a user.

[0029] Figure 3 is a block diagram showing an example of the functional configuration of the information processing device 100 according to this embodiment. Each functional block shown in this figure is a logical function realized by the processor 101 executing the information processing program 106 in the information processing device 100, and does not necessarily need to be configured independently in hardware. The information processing device 100 includes an acquisition unit 110, an extraction unit 120, an estimation unit 130, a storage control unit 140, an acquisition control unit 150, a determination unit 160, a generation unit 170, and a presentation unit 180.

[0030] The acquisition unit 110 acquires user dialogue data. In this case, the acquisition unit 110 may acquire data as dialogue data that includes linguistic information indicating the content of the user's utterances and non-linguistic information indicating at least one of the user's facial expressions and voice.

[0031] The extraction unit 120 extracts topics related to the user based on the dialogue data acquired by the acquisition unit 110.

[0032] The estimation unit 130 estimates the user's emotional state based on the dialogue data acquired by the acquisition unit 110. In this process, the estimation unit 130 may estimate the user's first emotional state based on verbal information and estimate the user's second emotional state based on nonverbal information.

[0033] The memory control unit 140 stores in the database 50, for each user, a baseline emotional state for each topic, based on the topics extracted by the extraction unit 120 and the emotional state estimated by the estimation unit 130. At this time, the memory control unit 140 may also store in the database 50 for each user an importance level indicating how important each topic is to the user.

[0034] When new dialogue data is acquired through interaction with the user, the acquisition control unit 150 acquires a reference state from the database 50 that is associated with a topic extracted based on the newly acquired dialogue data.

[0035] The determination unit 160 determines the discrepancy between the user's emotional state estimated by the estimation unit 130 based on the newly acquired dialogue data and the reference state acquired by the acquisition control unit 150. In this case, the determination unit 160 may determine a discrepancy if the importance associated with the topics extracted based on the newly acquired dialogue data exceeds a predetermined threshold.

[0036] The generation unit 170 generates feedback based on the user's emotional state estimated by the estimation unit 130 based on the newly acquired dialogue data, and the reference state acquired by the acquisition control unit 150. In this case, the generation unit 170 may determine the content of the feedback according to the discrepancy between the first emotional state and the second emotional state.

[0037] The presentation unit 180 presents the feedback generated by the generation unit 170 to the user. In this case, the presentation unit 180 may provide feedback to the user if the deviation determined by the determination unit 160 satisfies predetermined conditions.

[0038] The information processing performed by the information processing device 100 equipped with such functional components will be explained in detail using flowcharts, divided into a learning phase and a utilization phase.

[0039] Figure 4 is a flowchart showing an example of information processing in the learning phase performed by the information processing device 100 according to this embodiment. In the learning phase, the information processing device 100 learns the baseline state and importance of the emotions the user has for each topic based on the user interaction data, and stores them in the database 50.

[0040] In step S11, the processor 101 acquires user dialogue data. For example, the processor 101 may acquire dialogue data that includes linguistic information indicating the content of the user's speech input via the terminal 10, and non-linguistic information indicating at least one of the user's facial expressions and voice. As an example, the processor 101 may acquire video data including the user's speech and facial expressions by using the microphone and camera provided in the terminal 10 to photograph and record the user, and acquire said video data as dialogue data.

[0041] In step S12, the processor 101 extracts topics relevant to the user based on the dialogue data acquired in step S11. For example, the processor 101 may use the AI ​​model 70 to extract topics included in the dialogue data based on words, context, or topical units contained in the linguistic information. As an example, the processor 101 may extract topics such as "grandchildren," "work," "health," or "weather" from words or contexts contained in the user's utterances.

[0042] In step S13, the processor 101 estimates the user's emotional state based on the dialogue data acquired in step S11. For example, the processor 101 may use the AI ​​model 70 to estimate the user's first emotional state based on linguistic information and also estimate the user's second emotional state based on non-linguistic information. As an example, the processor 101 may estimate an emotional state such as "calm," "reassured," "neutral," or "happy" from expressions contained in the user's utterances. The processor 101 may also estimate an emotional state such as "lovable," "proud," "fulfilled," or "sad" from facial expressions or voices contained in the video data.

[0043] In step S14, the processor 101 stores in the database 50, for each user, a baseline emotional state for each topic, based on the topics extracted in step S12 and the emotional state estimated in step S13. At this time, the processor 101 may also store in the database 50 an importance score indicating how important each topic is to the user. For example, the processor 101 may store in the database 50 an importance score of "90" and a baseline state of "lovable, happy" for the topic "grandchild".

[0044] Here, the baseline state is information that indicates the baseline emotional state for a user when they engage in a conversation about a particular topic. For example, the baseline state may indicate the emotional state that the user typically feels about the topic, or the emotional state that the user ideally feels about the topic. As an example, if the topic is "grandchildren," and the user repeatedly shows high levels of positive emotion when they engage in a conversation about the topic, the baseline state may be set to "lovable" or "happy," etc.

[0045] Furthermore, importance refers to information indicating how important each topic is to the user in question. For example, importance may be calculated based on the intensity and frequency of emotions expressed in relation to the topic in past conversations. As an example, if conversations with a large absolute value of positive or negative emotions are repeatedly observed regarding a particular topic, that topic may be assigned a high importance rating.

[0046] For example, if the user's past dialogue data includes an utterance such as "My grandchild came to visit the other day..." and brightness is detected from the tone of voice or facial expression at the time of the utterance, the processor 101 may assign a high positive value to the topic "grandchild". As an example, the processor 101 may calculate an emotion score of "+0.9" based on the results of the analysis of verbal and nonverbal information.

[0047] The processor 101 may then determine a baseline state based on multiple sentiment scores calculated for the topic in question. For example, the processor 101 may determine a baseline state for a topic based on the mean, median, mode, or representative value of the sentiment scores corresponding to that topic over a predetermined period. As an example, if consistently high positive sentiment scores are obtained for the topic "grandchildren," the processor 101 may determine the baseline state for that topic to be "lovable, happy."

[0048] Furthermore, the processor 101 may calculate importance based on the intensity and frequency of emotions in past conversations and normalize said importance to a predetermined range of values. For example, the processor 101 may represent the importance as a numerical value between 0 and 100. The importance may be an integer or a decimal value. As an example, if high positive emotions are frequently observed regarding the topic "grandchild," the processor 101 may set the importance to "90."

[0049] Thus, for each topic, the processor 101 may determine a baseline state based on the emotional tendencies that have appeared in the user's past conversations, and may also determine the importance based on the intensity and frequency of those emotions. This allows the emotional state of newly acquired conversation data to be interpreted in the usage phase described later, taking into account the user's normal emotional tendencies and the importance of the topic.

[0050] Figure 5 shows an example of information stored in database 50. As shown in Figure 5, database 50 may store importance and baseline status associated with each topic. For example, for the topic "grandchildren," importance "90" and baseline status "lovable, happy" may be stored. Similarly, for the topic "work," importance "80" and baseline status "proud, fulfilled" may be stored. The baseline status may be represented by an emotion label such as "lovable" or "happy," and may also be stored in database 50 along with an emotion score associated with that emotion label.

[0051] These importance levels and baseline states are stored in database 50, associated with each user and each topic. As a result, a user-specific values ​​map is formed in database 50 for each user, associating the importance level and baseline state for each topic.

[0052] Furthermore, when new dialogue data is acquired, the processor 101 may update at least one of the importance level and the baseline state based on the dialogue data. The update may be performed each time dialogue data is acquired, or it may be performed in batches after accumulating multiple dialogue data sets. As a result, the user-specific values ​​map formed in the database 50 is updated in accordance with the accumulation of the user's dialogues, and may also change to reflect the user's growth, accumulation of experience, changes in living environment, or changes in interpersonal relationships.

[0053] Figure 6 is a flowchart showing an example of the processing in the usage phase according to this embodiment. In the usage phase, the information processing device 100 grasps topics and emotional states related to the user based on newly acquired dialogue data, obtains the reference state and importance corresponding to the topic from the database 50, and generates and presents feedback to the user according to the results.

[0054] In step S21, the processor 101 acquires user dialogue data. The process for acquiring dialogue data in step S21 may be the same as the process for acquiring dialogue data in step S11 described above, so a redundant explanation will be omitted here. As an example, the processor 101 may acquire dialogue data including the user's utterance, "I haven't been able to see my grandchildren lately, but that can't be helped," the tone of voice at the time of the utterance, and facial expressions.

[0055] In step S22, the processor 101 extracts topics related to the user based on the dialogue data acquired in step S21. The topic extraction process in step S22 may be the same as the topic extraction process in step S12 described above, so a redundant explanation is omitted here. As an example, the processor 101 may extract the topic "grandchild" based on linguistic information indicating the user's utterance, "I haven't been able to see my grandchild lately, but that can't be helped."

[0056] In step S23, the processor 101 estimates the user's emotional state based on the dialogue data acquired in step S21. The emotional state estimation process in step S23 may be the same as the emotional state estimation process in step S13 described above, so redundant explanations are omitted here. As an example, the processor 101 may estimate the first emotional state as "calm" based on linguistic information indicating the utterance, "I haven't been able to see my grandchildren lately, but that can't be helped." The processor 101 may also estimate the second emotional state as "sad" based on nonverbal information indicating the tone of voice and facial expression at the time of the utterance.

[0057] In step S24, the processor 101 retrieves the baseline state and importance associated with the topic extracted in step S22 from the database 50. For example, the processor 101 may access the database 50 to refer to the user-specific values ​​map corresponding to the user and retrieve the baseline state and importance associated with the extracted topic using the extracted topic as a search key. As an example, if the topic "grandchild" is extracted, the processor 101 may retrieve the importance "90" and baseline state "beloved, happy" associated with the topic "grandchild" from the values ​​map corresponding to the user.

[0058] In step S25, the processor 101 determines whether the importance score obtained in step S24 exceeds a predetermined threshold. For example, if the extracted topic is sufficiently important to the user, the processor 101 may perform a subsequent deviation determination. As an example, if the processor 101 obtains an importance score of "90" for the topic "grandchild", it may determine that the importance score exceeds the threshold of "50". On the other hand, if the importance score is below the threshold, the processor 101 may terminate processing without generating or presenting feedback regarding the topic.

[0059] In step S26, the processor 101 determines whether the discrepancy between the emotional state estimated in step S23 and the baseline state obtained in step S24 satisfies predetermined conditions. For example, the processor 101 may calculate the degree of discrepancy based on the difference between the emotional score associated with the baseline state and the emotional score associated with the emotional state estimated based on newly acquired dialogue data. As an example, the processor 101 may obtain an emotional score of "+0.9" associated with the baseline state "beloved" corresponding to the topic "grandchild". The processor 101 may also calculate an emotional score of "-0.8" associated with the current emotional state "sad" based on the utterance "I haven't been able to see my grandchild lately, but that can't be helped," and the facial expression or voice at the time of the utterance. The processor 101 may also calculate the degree of discrepancy as |+0.9-(-0.8)|=1.7. The processor 101 may then determine that the deviation satisfies a predetermined condition based on the fact that the deviation of 1.7 exceeds the threshold of 1.0. On the other hand, if the deviation is below the threshold, the processor 101 may terminate processing without generating or presenting feedback.

[0060] In step S27, the processor 101 generates feedback corresponding to the emotional state estimated in step S23 and the baseline state obtained in step S24. For example, the processor 101 may generate a prompt for generating feedback based on the emotional state estimated in step S23, the baseline state and importance obtained in step S24, and the content of the deviation determined in step S26. As an example, the processor 101 may generate a prompt that includes the topic "grandchild", the baseline state corresponding to the topic "lovable" or "happy", the importance "90", the first emotional state based on linguistic information "calm", the second emotional state based on non-linguistic information "sad", and a prompt to generate gentle and empathetic feedback for the user.

[0061] In this case, processor 101 may generate a prompt by embedding information representing the topic, baseline state, importance, emotional state, and discrepancy into a predetermined template. For example, processor 101 may generate a prompt by embedding each piece of information into a template that reads, "The user has a high importance level for [topic], and [baseline state] is registered as the baseline state. In this dialogue, the emotional state based on verbal information is [first emotional state], and the emotional state based on nonverbal information is [second emotional state]. Taking these discrepancies into account, generate gentle and supportive feedback that will help the user realize their true feelings." Then, processor 101 may input this prompt into AI model 70. As a result, if AI model 70 has a high importance level set for the topic "grandchild," and the baseline state is "lovable" or "happy," while in this dialogue the user outwardly appears calm, but sadness can be inferred from their facial expression and voice, AI model 70 may output gentle and supportive feedback that will help the user realize their true feelings. The processor 101 may generate feedback using the AI ​​model 70 in this manner, for example.

[0062] In step S28, the processor 101 presents the feedback generated in step S27 to the user. For example, the processor 101 may display the feedback on the monitor of terminal 10 and output the feedback as audio from the speaker of terminal 10. As an example, the processor 101 may display feedback consisting of kind and supportive text on a screen viewable by the user, and may also read the feedback aloud.

[0063] Figure 7 shows an example of feedback display according to this embodiment. As shown in Figure 7, the processor 101 may present feedback regarding topics of high importance to the user, taking into account the discrepancy between the baseline state and the newly estimated emotional state. For example, the processor 101 may present feedback regarding the topic "grandchild" that includes the fact that the topic is important to the user, that while the user is feigning calmness in words, loneliness can be seen in their facial expression or voice, and that the user should be encouraged to become aware of their true feelings.

[0064] In this way, the processor 101 can interpret the emotional state estimated based on newly acquired dialogue data by referring to the baseline state and importance included in the value map formed for each user and topic, and can provide feedback according to the result. This allows for appropriate support of the user's introspection.

[0065] As described above, the information processing device 100 according to this embodiment acquires user dialogue data, extracts topics related to the user based on the dialogue data, estimates the user's emotional state, stores a standard emotional state for each topic in the database 50 for each user based on the topics and emotional state, and when new dialogue data is acquired through dialogue with the user, it acquires the standard state associated with the topics extracted based on the newly acquired dialogue data from the database 50, and provides the user with feedback corresponding to the user's emotional state estimated based on the newly acquired dialogue data and the standard state. Thus, according to the information processing device 100 according to this embodiment, it is possible to provide feedback that corresponds to the user's emotional state at that time, taking into account the standard emotional state that the user has for each topic, and can appropriately support the user's introspection.

[0066] Furthermore, the information processing device 100 according to this embodiment determines the discrepancy between the user's emotional state estimated based on newly acquired dialogue data and the baseline state, and provides feedback to the user if the discrepancy satisfies predetermined conditions. As a result, the information processing device 100 according to this embodiment can provide feedback when the discrepancy between the user's emotional state and the baseline state exceeds a predetermined level, making it easier to encourage meaningful awareness for the user.

[0067] Furthermore, the information processing device 100 according to this embodiment stores an importance level for each user in the database 50, indicating how important each topic is to the user. When the importance level associated with a topic extracted based on newly acquired dialogue data exceeds a predetermined threshold, a discrepancy is determined. As a result, the information processing device 100 according to this embodiment can prioritize topics of high importance to the user for evaluation, making it easier to provide feedback that focuses on topics of high importance to that user.

[0068] Furthermore, the information processing device 100 according to this embodiment acquires data as dialogue data, which includes linguistic information indicating the content of the user's utterances and non-linguistic information indicating at least one of the user's facial expressions and voice. As a result, the information processing device 100 according to this embodiment can use dialogue data that includes non-linguistic information in addition to linguistic information, making it easier to understand the user's state from multiple perspectives.

[0069] The information processing device 100 according to this embodiment estimates the user's first emotional state based on linguistic information and estimates the user's second emotional state based on non-linguistic information, and determines the content of the feedback according to the discrepancy between the first emotional state and the second emotional state. As a result, the information processing device 100 according to this embodiment can adjust the content of the feedback based on the difference between the emotions expressed in linguistic information and the emotions expressed in non-linguistic information, making it easier to provide feedback that corresponds to the discrepancy between the user's superficial statements and their inner emotions.

[0070] Thus, unlike technologies that merely estimate the user's emotional state and change the output according to the estimation result, the information processing device 100 according to this embodiment refers to a baseline emotional state formed for each user and topic, and provides feedback based on the relationship between the current emotional state and the baseline state. As a result, according to the information processing device 100 according to this embodiment, even if the same emotional state is estimated, different feedback can be provided if the topic or user is different, and even for the same user and the same topic, feedback can be provided according to the discrepancy between the emotional state at that time and the baseline state. Therefore, it is possible to realize more appropriate introspective support that takes into account the user's unique values.

[0071] It should be noted that this disclosure is not limited to the foregoing, and can be implemented in various modified forms without departing from its intent.

[0072] For example, the above embodiment shows an example where a deviation is determined when the importance exceeds a predetermined threshold, but it is not limited to this. Feedback may be generated and presented even when the importance does not exceed the threshold.

[0073] Furthermore, while the above embodiment shows an example of providing feedback when the discrepancy between the estimated emotional state and the baseline state satisfies predetermined conditions, it is not limited to this. Feedback may be generated and presented even when the discrepancy does not satisfy those conditions.

[0074] Furthermore, while the above embodiment shows an example in which non-verbal information indicating at least one of facial expressions and voice is acquired in addition to linguistic information as dialogue data, the embodiment is not limited to this. Dialogue data may include only linguistic information.

[0075] Furthermore, while the above embodiment shows an example where feedback is generated and presented based on dialogue data each time new dialogue data is acquired, the system is not limited to this. Feedback may be generated and presented all at once after accumulating data from multiple dialogues. For example, feedback on multiple topics may be presented all at once after the end of a day's dialogue, after a predetermined period of time has elapsed, or after a predetermined number of dialogues have been completed.

[0076] Furthermore, while the above embodiment shows an example where the baseline state and importance are associated and stored for each user, the system is not limited to this. The baseline state and importance may also be associated and stored for each group of multiple users. For example, a values ​​map may be formed on a family basis, sibling basis, spouse basis, or predetermined community basis.

[0077] Furthermore, while the above embodiment shows examples of using the AI ​​model 70 to extract topics, estimate emotional states, or generate feedback, it is not limited to these examples. At least some of these processes may be performed by rule-based processing, dictionary-based processing, statistical processing, or a combination thereof.

[0078] Furthermore, while the above embodiment shows an example where the baseline state is represented by emotion labels and emotion scores, it is not limited to this. The baseline state may also be represented by emotion categories, vector representations, ranking information, or a combination of multiple emotion labels.

[0079] Furthermore, while the above embodiments show examples of presenting feedback by display or audio output, the system is not limited to these. Feedback may be presented by text, audio, images, animation, avatar speech, or a combination thereof.

[0080] In the embodiments described above, the term "processor" refers to a processor in a broad sense, including general-purpose processors (e.g., CPUs) and dedicated processors (e.g., GPUs: Graphics Processing Units, ASICs: Application Specific Integrated Circuits, FPGAs: Field Programmable Gate Arrays, programmable logic devices, etc.).

[0081] Furthermore, the operation of the processor in the above-described embodiment may not be performed by a single processor, but may be performed by multiple processors located in physically separate locations working together. Also, the order of each processor operation is not limited to the order described in the above-described embodiment, but may be changed as appropriate.

[0082] Furthermore, although the information processing device 100 in this embodiment is described as being composed of a single device, it may be composed of multiple devices.

[0083] Furthermore, the processing performed by the information processing device 100 according to the above embodiment may be performed by software, by hardware, or by a combination of both. In addition, the processing performed by each part of the information processing device 100 may be stored as a program on a storage medium and distributed. [Explanation of Symbols]

[0084] 1. Information Processing System 10 devices 30 servers 50 Databases 70 AI Models 90 Networks 100 Information Processing Devices 101 Processors 102 ROM 103 RAM 104 storage 105 Interfaces 106 Information Processing Programs 109 Bus 110 Acquisition Department 120 Extraction part 130 Estimation part 140 Memory Control Unit 150 Acquisition Control Unit 160 Judgment section 170 Generation part 180 Presentation section

Claims

1. The processor comprises, Obtain user interaction data, Based on the aforementioned dialogue data, topics related to the user are extracted, and the user's emotional state is estimated. Based on the aforementioned topics and emotional states, the database stores for each user a baseline emotional state for each topic, as well as an importance level indicating how important each topic is to the user. When dialogue data is newly acquired through the interaction with the user, the reference state and importance level associated with the topic extracted based on the newly acquired dialogue data are obtained from the database. If the importance level associated with a topic extracted based on the newly acquired dialogue data exceeds a predetermined threshold, the discrepancy between the user's emotional state estimated based on the newly acquired dialogue data and the baseline state is determined. If the aforementioned discrepancy satisfies predetermined conditions, the system provides the user with feedback corresponding to the user's emotional state estimated based on the newly acquired dialogue data and the baseline state. Information processing system.

2. comprising a processor, the processor is As user dialogue data, data is acquired that includes linguistic information indicating the content of the user's utterances and non-linguistic information indicating at least one of the user's facial expressions and voice. Based on the aforementioned dialogue data, topics related to the user are extracted, and the user's emotional state is estimated. Based on the aforementioned topics and emotional states, the baseline emotional state that the user has for each topic is stored in a database for each user. When dialogue data is newly acquired through the interaction with the user, the reference state associated with the topic extracted based on the newly acquired dialogue data is obtained from the database. In providing the user with feedback corresponding to the user's emotional state estimated based on the newly acquired dialogue data and the reference state, Based on the linguistic information, the first emotional state of the user is estimated, and based on the non-linguistic information, the second emotional state of the user is estimated. The content of the feedback is determined according to the discrepancy between the first emotional state and the second emotional state. Information processing system.

3. Computers Obtaining user interaction data, Based on the aforementioned dialogue data, topics related to the user are extracted, and the user's emotional state is estimated. Based on the aforementioned topics and emotional states, the database stores for each user the baseline emotional state that the user has for each topic, as well as the importance level indicating how important each topic is to the user. When dialogue data is newly acquired through the interaction with the user, the reference state and importance level associated with the topic extracted based on the newly acquired dialogue data are obtained from the database. If the importance level associated with a topic extracted based on the newly acquired dialogue data exceeds a predetermined threshold, the discrepancy between the user's emotional state estimated based on the newly acquired dialogue data and the baseline state is determined. If the aforementioned discrepancy satisfies predetermined conditions, the system includes providing the user with feedback corresponding to the user's emotional state estimated based on the newly acquired dialogue data and the aforementioned baseline state, Information processing methods.

4. A computer, The system acquires user dialogue data that includes linguistic information indicating the content of the user's speech and nonverbal information indicating at least one of the user's facial expressions and voice. Based on the aforementioned dialogue data, topics related to the user are extracted, and the user's emotional state is estimated. Based on the aforementioned topics and emotional states, the standard emotional state that the user has for each topic is stored in a database for each user, When dialogue data is newly acquired through the interaction with the user, the reference state associated with the topic extracted based on the newly acquired dialogue data is acquired from the database, In providing the user with feedback corresponding to the user's emotional state estimated based on the newly acquired dialogue data and the reference state, The first emotional state of the user is estimated based on the linguistic information, and the second emotional state of the user is estimated based on the non-linguistic information. This includes determining the content of the feedback in accordance with the discrepancy between the first emotional state and the second emotional state, Information processing methods.

5. On the computer, The process of obtaining user interaction data, Based on the aforementioned dialogue data, the process extracts topics related to the user and estimates the user's emotional state. Based on the aforementioned topics and emotional states, the process involves storing in a database, for each user, the baseline emotional state that the user has for each topic, and the importance level indicating how important each topic is to the user. When dialogue data is newly acquired through the interaction with the user, the process involves acquiring the reference state and importance level from the database, which are associated with the topics extracted based on the newly acquired dialogue data. If the importance level associated with a topic extracted based on the newly acquired dialogue data exceeds a predetermined threshold, the process determines the discrepancy between the user's emotional state estimated based on the newly acquired dialogue data and the baseline state. If the aforementioned discrepancy satisfies predetermined conditions, the system will perform a process to provide the user with feedback corresponding to the user's emotional state estimated based on the newly acquired dialogue data and the baseline state. Information processing program.

6. A computer, A process for acquiring data as user dialogue data, which includes linguistic information indicating the content of the user's speech and nonverbal information indicating at least one of the user's facial expressions and voice. Based on the aforementioned dialogue data, the process extracts topics related to the user and estimates the user's emotional state. Based on the aforementioned topics and emotional states, the process involves storing in a database the baseline emotional state that the user has for each topic, for each user. When dialogue data is newly acquired through the interaction with the user, the process involves acquiring the reference state associated with the topic extracted based on the newly acquired dialogue data from the database, In providing the user with feedback corresponding to the user's emotional state estimated based on the newly acquired dialogue data and the reference state, A process for estimating the user's first emotional state based on the linguistic information and estimating the user's second emotional state based on the non-linguistic information, The system performs a process that determines the content of the feedback according to the discrepancy between the first emotional state and the second emotional state. Information processing program.

Images