Information processing system, program, and information processing method

Abstract

[Problem] To provide an information processing system and the like capable of facilitating the enhancement of accuracy in the estimation result of the QOL of a user. [Solution] One aspect of the present invention provides an information processing system comprising a processor, wherein the processor: in a first acquisition step, acquires correlation information having a correlation with the quality of life (QOL) of a user, the correlation information including two or more types of information that can be acquired in association with an operation at the time of utterance of the user; and, in a first output step, outputs a score indicating the QOL of the user on the basis of the acquired correlation information.

Description

Information processing system, program, and information processing method

[0001] The present invention relates to an information processing system, a program, and an information processing method.

[0002] Patent Document 1 discloses a technology that analyzes voice input via a voice input device to calculate emotional parameters, outputs a predetermined voice via a voice output device based on a speech scenario that is predetermined according to each emotional parameter, and stores the speech scenario in an updatable manner.

[0003] Japanese Patent Application Laid-Open No. 2008-125815

[0004] It is said that it is important to encourage mentally fatigued people to self-disclose, including negative content, and for this purpose, talking to a machine with a low social presence is effective. However, if the same monotonous behavior is repeated, people will become bored with interacting with the machine, and the effect of encouraging self-disclosure will be reduced. In order to add variety to the conversation, it is important to understand the other person's state.

[0005] In view of the above circumstances, the present invention provides an information processing system and the like that can easily improve the accuracy of the estimation result of a user's QOL.

[0006] According to one aspect of the present invention, there is provided an information processing system including a processor, in which, in a first acquisition step, the processor acquires correlation information that correlates with a user's QOL (Quality of life), the correlation information including two or more types of information that can be acquired in accordance with the user's actions when speaking, and in a first output step, the processor outputs a score indicating the user's QOL based on the acquired correlation information.

[0007] According to this aspect, it is possible to increase the accuracy of the estimation result of the user's QOL.

[0008] 1 is a diagram showing the overall configuration of an automated dialogue system 1. FIG. 2 is a diagram showing the hardware configuration of a server device 10. FIG. 3 is a diagram showing the hardware configuration of a user terminal 20. FIG. 4 is a diagram showing the hardware configuration of a robot 30. FIG. 5 is a diagram showing an example of the functional configuration of a control unit of each device. FIG. 6 is an activity diagram showing an example of automated dialogue processing. FIG. 7 is an activity diagram showing an example of information acquisition processing. FIG. 8 is a diagram showing an example of utterance rules. FIG. 9 is a diagram showing an example of a dialogue screen. FIG. 10 is a diagram showing an example of an error in a QOL score. FIG. 11 is a diagram showing another example of experimental results. FIG. 12 is a diagram showing another example of experimental results. FIG. 13 is a diagram showing another example of experimental results. FIG. 14 is a diagram showing another example of a rule table. FIG. 15 is a diagram showing an example of a score table. FIG. 16 is a diagram showing an example of an input probability table.

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below with reference to the accompanying drawings. Various features shown in the following embodiments can be combined with each other.

[0010] Incidentally, a program for realizing the software appearing in one embodiment may be provided as a non-transitory computer-readable recording medium, or may be provided so as to be downloadable from an external server, or may be provided so that the program is started on an external computer and its functions are realized on a client terminal (so-called cloud computing).

[0011] Furthermore, various information processing according to an embodiment may realize input and output corresponding to the input. Here, the form of information referenced in such information processing (hereinafter referred to as reference information) is not limited as long as an output is obtained as a result of the input. The reference information may be, for example, rule-based information such as a database, a lookup table, or a predetermined function (including a decision formula such as a regression formula constructed using a statistical method), a trained model that has previously trained the correlation between input and output, or a large-scale language model that can output a desired result by inputting a prompt.

[0012] In one embodiment, the term "unit" may include, for example, a combination of hardware resources implemented by a circuit in the broad sense and software information processing that can be specifically realized by these hardware resources. In one embodiment, various information is handled, and this information is represented, for example, by physical values ​​of signal values ​​representing voltage or current, high or low signal values ​​as a binary bit set consisting of 0 or 1, or quantum superposition (so-called quantum bits), and communication and calculations can be performed on the circuit in the broad sense.

[0013] Furthermore, a circuit in a broad sense is a circuit realized by at least an appropriate combination of a circuit, circuitry, a processor, a memory, etc. The processor may be a general-purpose processor or a dedicated circuit. That is, it includes application specific integrated circuits (ASICs), programmable logic devices (e.g., simple programmable logic devices (SPLDs), complex programmable logic devices (CPLDs), and field programmable gate arrays (FPGAs)), etc.

[0014] 1. Hardware Configuration In this section, the hardware configuration of the automated dialogue system according to this embodiment will be described.

[0015] Fig. 1 is a diagram showing the overall configuration of an automated dialogue system 1. Fig. 1 shows an overview of each device included in the automated dialogue system 1, users who use each device, and information processed by each device. Each overview will be explained as needed, with reference to other figures.

[0016] The automated dialogue system 1 is an information processing system that executes automated dialogue processing to automatically engage in dialogue with a user. The automated dialogue system 1 includes a communication line 2, a server device 10, a user terminal 20, and a robot 30. The communication line 2 includes the Internet or the like and mediates data exchange between devices connected to the communication line. Each device is connected to the communication line 2 by wire or wirelessly. The server device 10 is an information processing device that executes main processes of the automated dialogue processing.

[0017] The user terminal 20 is a terminal used by a user, such as a smartphone, tablet terminal, or laptop computer. The robot 30 is a machine that can imitate the movements of a specific living creature and output human-like sounds. Both the user terminal 20 and the robot 30 are machines that serve as interfaces for dialogue with the user. The user terminal 20 and the robot 30 are controlled by the server device 10 and communicate with the user by displaying images or outputting sounds.

[0018] 2 is a diagram showing the hardware configuration of the server device 10. The server device 10 includes a control unit 11, a storage unit 12, a communication unit 13, and a bus 14. The bus 14 electrically connects the various units included in the server device 10.

[0019] (Control Unit 11) The control unit 11 is, for example, a central processing unit (CPU) not shown. The control unit 11 is a computer that realizes various functions related to the automated dialogue system 1 by reading out predetermined programs stored in the storage unit 12. In other words, information processing by software stored in the storage unit 12 is specifically realized by the control unit 11, which is an example of hardware, and can be executed as each functional unit included in the control unit 11. These will be described in more detail in the next section. Note that the control unit 11 is not limited to being single, and the system may be implemented with multiple control units 11 for each function. A combination of these may also be used.

[0020] (Memory Unit 12) The memory unit 12 stores various pieces of information defined above. This can be implemented, for example, as a storage device such as a solid state drive (SSD) that stores various programs and the like related to the automated dialogue system 1 executed by the control unit 11, or as a memory such as a random access memory (RAM) that stores temporarily required information (arguments, arrays, etc.) related to program calculations. The memory unit 12 stores various programs, variables, etc. related to the automated dialogue system 1 executed by the control unit 11.

[0021] (Communication Unit 13) The communication unit 13 is configured to be able to transmit various electrical signals from the server device 10 to external components. The communication unit 13 is also configured to be able to receive various electrical signals from the external components to the server device 10. More preferably, the communication unit 13 has a network communication function, which may enable communication of various information between the server device 10 and external devices via the communication line 2.

[0022] 3 is a diagram showing the hardware configuration of the user terminal 20. The user terminal 20 includes a control unit 21, a storage unit 22, a communication unit 23, an input unit 24, an output unit 25, and a bus 26. The bus 26 electrically connects the various units included in the user terminal 20. The control unit 21, the storage unit 22, and the communication unit 23 are similar hardware to the control unit 11, the storage unit 12, and the communication unit 13 shown in FIG. 2, although the specifications, model, etc. may differ.

[0023] (Input Unit 24) The input unit 24 has one or more input interfaces such as keys, buttons, a touch screen, and a mouse, and receives input from the user. The input unit 24 also has an audio input interface such as a microphone, and receives audio input from the user, performing so-called recording. The input unit 24 also has a lens, an image sensor, and the like, and receives input of an image formed by light incident through the lens, performing so-called photography.

[0024] (Output unit 25) The output unit 25 has an output device such as a display, and displays visual information generated in a manner visible to the user, such as a screen, an image, an icon, text, etc. on the display surface. The output unit 25 also has an output device for sound (including human voice and non-voice sounds), such as a speaker, and outputs these sounds.

[0025] 4 is a diagram showing the hardware configuration of the robot 30. The robot 30 includes a control unit 31, a storage unit 32, a communication unit 33, an input unit 34, an output unit 35, a drive unit 36, and a bus 37. The bus 37 electrically connects the various units included in the robot 30. The control unit 31, the storage unit 32, and the communication unit 33 are similar hardware to the control unit 11, the storage unit 12, and the communication unit 13 shown in FIG. 2, although the specifications, model, etc. may differ.

[0026] (Input Unit 34) The input unit 34 has one or more input interfaces such as buttons and a touch screen, and accepts input from the user. The input unit 34 also has an audio input interface such as a microphone, and accepts audio input from the user, performing so-called recording. The input unit 34 also has a lens, an image sensor, and the like, and accepts input of an image formed by light that passes through the lens and enters, performing so-called photography.

[0027] (Output unit 35) The output unit 35 has an output device such as a small display, and displays visual information generated in a manner that is visible to the user, such as a screen, an image, an icon, text, etc. on the display surface. The output unit 35 also has an output device for sound (including human voice and non-voice sounds), such as a speaker, and outputs these sounds.

[0028] (Drive unit 36) The robot 30 has a shape that imitates a specific living creature and has movable parts (hands, feet, eyes, etc.) that move in a manner that imitates the living creature. The drive unit 36 ​​has a power source such as a battery, a motor, gears, etc., and drives these movable parts.

[0029] 2. Functional Configuration This section describes the functional configuration of this embodiment. As described above, information processing by software stored in the storage unit of each device is specifically realized by a control unit, which is an example of hardware, and each functional unit included in the control unit can be executed.

[0030] 5 is a diagram showing an example of the functional configuration of the control unit of each device. The control unit 11 of the server device 10 includes a server display unit 111, a user authentication unit 112, an information acquisition unit 113, a utterance recognition unit 114, a first artificial intelligence unit 115, a second artificial intelligence unit 116, a QOL estimation unit 117, and a dialogue processing unit 118. The control unit 21 of the user terminal 20 includes a user display unit 211, an operation acceptance unit 212, a sound processing unit 213, and an image processing unit 214. The robot 30 includes a user display unit 311, an operation acceptance unit 312, a sound processing unit 313, an image processing unit 314, and a motion processing unit 315.

[0031] The server display unit 111 executes processing for displaying a system screen related to the automated dialogue system 1 on the user terminal 20. The server display unit 111 performs processing such as generating and transmitting an HTML (Hyper Text Markup Language) file, and displays a web page showing the system screen on the user terminal 20. The server display unit 111 may also perform processing such as generating and transmitting display data for an application for using the automated dialogue system 1.

[0032] The user authentication unit 112 authenticates a user who logs in via the user terminal 20 and the robot 30 based on authentication information such as a registered user ID (Identification) and password. This authentication enables the server device 10 to process data exchanged with the user terminal 20 and the robot 30 in association with the user associated with the data.

[0033] The information acquisition unit 113 acquires predetermined information to be used in the automatic dialogue processing by the automatic dialogue system 1. For example, the information acquisition unit 113 acquires correlation information that correlates with the user's QOL (Quality of Life). QOL is a term that refers to the quality of life including physical, mental, social, and economic aspects. QOL is indicated by eight scores (scales), for example, physical functioning (PF), role physical functioning (RP), bodily pain (BP), general health perceptions (GH), vitality (VT), social functioning (SF), role emotional functioning (RE), and mental health (MH).

[0034] The correlation information includes multiple types of information, such as text indicating what the user said (first correlation information), features of the user's voice (second correlation information), features of the user's facial expression (third correlation information), features of the user's gaze pattern (fourth correlation information), and features of the user's head shaking (fifth correlation information).

[0035] These correlation information are known to be correlated with QOL. For example, the higher the QOL (the higher the score indicating QOL), the more positive the statements tend to be. Furthermore, the higher the QOL, the higher the voice tends to be and the more cheerful the facial expression tends to be. Furthermore, the higher the QOL, the more calm the eye movements tend to be and the less the head moves than necessary. Each piece of correlation information is information that can be used to determine these tendencies. Note that the tendencies described here are merely examples and are not limited to these.

[0036] The information acquisition unit 113 also acquires utterance information indicating the content of the user's utterance. The utterance recognition unit 114 recognizes the content of the user's utterance based on the utterance information acquired by the information acquisition unit 113.

[0037] The first artificial intelligence unit 115 and the second artificial intelligence unit 116 are modules that are adjusted (tuned) using AI (Artificial Intelligence) technology to obtain a predetermined output for a predetermined input. The first artificial intelligence unit 115 is, for example, an AI module having a dedicated learning model related to QOL. This learning model is, for example, a model that learns the correspondence between multimodal information including the above-mentioned first correlation information to fifth correlation information and QOL scores corresponding to the multimodal information. Multimodal information refers to input information that includes multiple types of information. When multimodal information is input, the first artificial intelligence unit 115 outputs a corresponding QOL score.

[0038] The second artificial intelligence unit 116 may be, for example, an AI module having a natural language processing model whose accuracy has been improved by machine learning using a large data set called LLM (Large Language Models), or may be an AI module having a dedicated learning model related to QOL (for example, a model that has learned the association between utterance information and QOL scores and response content). When the utterance information acquired by the information acquisition unit 113 and the QOL score output by the first artificial intelligence unit 115 are input, the second artificial intelligence unit 116 generates and outputs a response content to the utterance content indicated by the utterance information.

[0039] The information input to the first artificial intelligence unit 115 and the second artificial intelligence unit 116 and the information output by the first artificial intelligence unit 115 and the second artificial intelligence unit 116 are not fixed but have a certain degree of freedom. The QOL estimation unit 117 controls the input and output to the first artificial intelligence unit 115 and estimates the user's QOL. The QOL estimation unit 117 inputs, for example, one or more pieces of correlation information out of multiple types of correlation information to the first artificial intelligence unit 115 and instructs the first artificial intelligence unit 115 on the type of score to output.

[0040] The dialogue processing unit 118 controls input and output to the second artificial intelligence unit 116 and executes processing for responding to a user's question. For example, the dialogue processing unit 118 inputs utterance information acquired by the information acquisition unit 113 and one or more scores among the multiple types of scores output from the first artificial intelligence unit 115 to the second artificial intelligence unit 116, and instructs the second artificial intelligence unit 116 on rules for the response to be output. The response rules include, for example, rules for praising, paraphrasing the user's utterance, and repeating the user's utterance, and will be described in detail later.

[0041] The user display unit 211 executes display-related processing. For example, the user display unit 211 controls the display of its own device to display a web page or an application screen transmitted from the server device 10. The user display unit 211 also displays text indicating the content of a user's statement, or displays text indicating the content of a response transmitted from the server device 10. The operation accepting unit 212 accepts user operations on the user terminal 20.

[0042] The sound processing unit 213 performs processing related to sound. For example, the sound processing unit 213 converts the user's voice into data and transmits the data to the server device 10, or outputs a sound represented by the sound data transmitted from the server device 10. The image processing unit 214 performs processing related to images. For example, the image processing unit 214 controls the input unit 24 to photograph the user and transmits the photographed image of the user to the server device 10.

[0043] The user display unit 311 performs processing related to display. For example, the user display unit 311 controls the display on a small display, which is a display means possessed by the robot 30, and displays text transmitted from the server device 10. The operation reception unit 312 receives user operations. The sound processing unit 313 performs processing related to sound. For example, the sound processing unit 313 converts the user's voice into data and transmits it to the server device 10, or outputs sound indicated by sound data transmitted from the server device 10. The image processing unit 314 controls the input unit 34 to capture an image of the user and transmits the captured image of the user to the server device 10. The movement processing unit 315 controls the drive unit 36 ​​to perform processing related to the movement of the robot 30.

[0044] 6, the flow of information processing executed by the automated dialogue system 1 will be outlined below. The automated dialogue system 1 includes a processor capable of executing a program to perform each step (each process) described below. The automated dialogue system 1 executes the above-described automated dialogue process (processing for automatically interacting with a user).

[0045] Fig. 6 is an activity diagram showing an example of automatic dialogue processing. The example in Fig. 6 illustrates a case where a user is having a dialogue with the user terminal 20. First, the user thinks of something to say (A10). Next, the user inputs the thoughtful utterance into the user terminal 20. The user operates, for example, a keyboard or the like to input text indicating the utterance (A11). In this case, the user terminal 20 accepts input of the text indicating the utterance via the operation acceptance unit 212 (A21).

[0046] The user can also make a sound indicating the content of the statement (A12). In this case, the user terminal 20 records the sound indicating the content of the statement using the sound processing unit 213 (A22). Whether inputting text or making a sound, it is difficult for the user to keep their body completely still, and they always make some kind of movement while inputting the content of the statement (A13). Body movement includes facial movement, eye movement, and head movement. The user terminal 20 uses the image processing unit 214 to capture images indicating these user movements (A23).

[0047] The operation reception unit 212 transmits input data indicating the content of the text input received in A21 to the server device 10. The sound processing unit 213 transmits audio data indicating the audio recorded in A22 to the server device 10. The image processing unit 214 transmits image data of the image showing the user's movements captured in A23 to the server device 10. The server device 10 executes information acquisition processing by the information acquisition unit 113 to acquire the above-mentioned utterance information and related information based on the transmitted input data, audio data, and image data (A30).

[0048] 7 is an activity diagram showing an example of information acquisition processing. When a statement is made by text input, the information acquisition unit 113 acquires the text indicated by the input data transmitted in A21 as expression information (A31). Furthermore, when a statement is made by text input, the information acquisition unit 113 also acquires the text indicated by the input data transmitted in A21 as first related information (A32).

[0049] If the utterance is made by voice, the information acquisition unit 113 recognizes the voice indicated by the voice data transmitted in A22 using a well-known voice recognition technology and acquires text indicating the recognition result as utterance information (A33). Furthermore, if the utterance is made by voice, the information acquisition unit 113 acquires features of the voice indicated by the voice data transmitted in A22 as second related information (A34). The features of the voice are, for example, information indicating prosodic features (tone of voice, rhythm, strength, high and low, length, etc.).

[0050] The information acquisition unit 113 acquires, as third related information, feature quantities indicative of the user's facial expression based on the image of the user's facial part indicated by the image data transmitted in A23 and a well-known technique for estimating emotions from facial images (A35). The feature quantities indicative of the facial expression include, for example, the rate of change in the distance between specific points on the face and the rate of change in the area of ​​the part surrounded by specific points on the face.

[0051] The information acquisition unit 113 acquires, as fourth related information (A36), a feature of the user's gaze pattern based on the image of the user's eyes shown in the image data transmitted in A23 and a well-known technique for estimating emotions from gaze patterns. The feature of the gaze pattern is, for example, the frequency of gaze jumps (microsaccades).

[0052] The information acquisition unit 113 acquires, as fifth related information, feature quantities of the user's head movement based on the image of the user's head shown in the image data transmitted in A23 and a well-known technique for estimating emotions from head movement (A37). The feature quantities of the head movement include, for example, the proportion of time during which the head movement is detected, the amount of change in head tilt, and the amount of change in head rotation.

[0053] The information acquisition unit 113 supplies the utterance information acquired in A31 or A33 to the dialogue processing unit 118. The information acquisition unit 113 also supplies the first related information (text indicating what the user said), second related information (feature amounts of the user's voice), third related information (feature amounts of the user's facial expression), fourth related information (feature amounts of the user's gaze pattern), and fifth related information (feature amounts of the user's head shaking) acquired in A32, A34, A35, A36, and A37 to the QOL estimation unit 117.

[0054] Next, the server device 10 executes a first input process (A41) in which the QOL estimation unit 117 inputs the acquired related information to the first artificial intelligence unit 115. For example, the QOL estimation unit 117 inputs multimodal information including the first correlation information, the second correlation information, the third correlation information, the fourth correlation information, and the fifth correlation information to the first artificial intelligence unit 115, and instructs the first artificial intelligence unit 115 to output a QOL score corresponding to the input multimodal information.

[0055] The first artificial intelligence unit 115 outputs eight scores (PF, RP, BP, GH, VT, SF, RE, MH) corresponding to the input multimodal information. The eight scores output based on the multimodal information are hereinafter referred to as "integrated scores." The server device 10 estimates the user's QOL using the QOL estimation unit 117, regarding the output integrated score as a score indicating the user's QOL (A42). The QOL estimation unit 117 supplies the estimated integrated score to the dialogue processing unit 118.

[0056] The server device 10 executes a second input process (A43) in which the dialogue processing unit 118 inputs the utterance information supplied from the information acquisition unit 113 and the integrated score supplied from the QOL estimation unit 117 to the second artificial intelligence unit 116. The dialogue processing unit 118 inputs the integrated score and instructs the second artificial intelligence unit 116 to output the utterance content based on predetermined utterance rules. The predetermined utterance rules will be described with reference to FIG. 8.

[0057] Fig. 8 is a diagram showing an example of utterance rules. Fig. 8 shows a rule table TB11 in which utterance rules are associated with the therapist's empathy level and frequency of use. The rule table TB11 shows six utterance rules: "paraphrasing," "agreement," "echoing," "praising," "deep dive," and "self-disclosure."

[0058] "Rephrasing" is restating what the other person said. "Agreeing" is agreeing with what the other person said. "Parroting" is repeating exactly what the other person said. "Praising" is praising the other person. "Digging deeper" is finding out more details about what the other person said. "Self-disclosure" is talking about your own thoughts.

[0059] A therapist is someone who uses specialized knowledge and techniques to heal and treat the mind and body of a client. The degree of empathy of a therapist is the degree to which a client feels that the therapist has accepted the client's opinions, feelings, etc. as they are, and is the result of a questionnaire conducted by the inventors in which actual clients were asked to evaluate statements made in accordance with each statement rule. The therapist's empathy was rated "high" for "paraphrasing" and "agreement," "medium" for "echoing" and "praising," and "low" for "digging deeper" and "self-disclosure."

[0060] In the rule table TB11, the therapist empathy levels of "high," "medium," and "low" are associated with usage frequencies of "high," "medium," and "low." The usage frequencies refer to the frequency with which the dialogue processing unit 118 uses the utterance rules in giving instructions to the second artificial intelligence unit 116. As a result, the dialogue processing unit 118 gives instructions to the second artificial intelligence unit 116 by using, for example, the utterance rules "paraphrase" and "agree" frequently and the utterance rules "dig deeper" and "self-disclosure" only infrequently.

[0061] The number of utterance rules used when issuing an instruction may be limited to one, or two or more may be allowed to be used simultaneously. If two or more utterance rules are used simultaneously, an upper limit may be set. The second artificial intelligence unit 116 generates and outputs a response content to the utterance content indicated by the utterance information, based on the input of the utterance information and the integrated score and the instruction of the utterance rule.

[0062] The second artificial intelligence unit 116 generates a response content based on the utterance rule using various methods. For example, when the utterance rule is "rephrasing," the second artificial intelligence unit 116 translates the utterance content indicated by the expression information from Japanese to English, and then translates the translation result back into Japanese and makes minor corrections to create a "rephrased" sentence that has the same meaning but a different sentence structure. Furthermore, the second artificial intelligence unit 116 stores phrases that express agreement, such as "I like it," "I think so," "That's right," and "That's true," and when the utterance rule is "agree," the second artificial intelligence unit 116 can create a sentence of "agreement" by including the phrases that express agreement in the response content.

[0063] Even when the utterance rule is "praise," the second artificial intelligence unit 116 can similarly create a "praise" sentence by storing phrases that express praise (such as "That's wonderful!", "That's good," and "You're good at it.") Furthermore, when the utterance content is "echoing," the second artificial intelligence unit 116 can create a "echoing" sentence by cutting out a part of the utterance content indicated by the utterance information and including it in the response content.

[0064] When the utterance content is "deep dive," the second artificial intelligence unit 116 can create an "deep dive" sentence by including a part of the utterance content indicated by the utterance information and a phrase such as "Please tell me more about it." In addition, when the utterance content is "self-disclosure," if the utterance content indicated by the utterance information contains a phrase indicating a positive emotion or a negative emotion, the second artificial intelligence unit 116 can create a "self-disclosure" sentence by including a phrase close to that emotion in the response content. Note that the second artificial intelligence unit 116 may generate a response content based on utterance rules using any other well-known method, not limited to the above-described method.

[0065] The server device 10 generates, via the dialogue processing unit 118, response data indicating the response content output from the second artificial intelligence unit 116 (A44). If expression information has been acquired in A31, i.e., if the user has input text indicating the utterance content, the dialogue processing unit 118 generates text data indicating the response content in text as the response data. If expression information has been acquired in A33, i.e., if the user has uttered a voice indicating the utterance content, the dialogue processing unit 118 generates voice data indicating the response content in voice as the response data. The dialogue processing unit 118 transmits the generated response data to the user terminal 20.

[0066] The user terminal 20 outputs the transmitted response data. If the response data is text data, the user terminal 20 outputs the response content indicated by the response data in text form using the user display unit 211 (A51). If the response data is audio data, the user terminal 20 outputs the response content indicated by the response data in audio form using the sound processing unit 213 (A52). The dialogue screen on which the dialogue described above takes place will now be described with reference to FIG. 9.

[0067] Fig. 9 is a diagram showing an example of a dialogue screen. In the example of Fig. 9, the user display unit 211 displays an automatic dialogue screen G1. The automatic dialogue screen G1 includes a utterance image C11, a response image C12, a robot image D11, a key input button B11, and a voice input button B12. The utterance image C11 is an image showing the content of a utterance made by the user of the user terminal 20. The response image C12 is an image showing the response content indicated by the response data generated by the above-mentioned automatic dialogue processing.

[0068] The robot image D11 is an image that makes it easier to imagine the conversation partner, and represents the appearance of a virtual conversation partner. The key input button B11 is a button that is operated when manually inputting the content of a statement using a keyboard. The voice input button B12 is a button that is operated when inputting the content of a statement by voice. In the example of FIG. 9 , the user looks at the response content shown in the response image C12 (A61) and thinks about the next content of a statement (A10). Furthermore, if the response content is spoken, the user listens to the response content indicated by the spoken voice (A62) and thinks about the next content of a statement (A10).

[0069] 6 are repeated, the user interacts with the user terminal 20 and the server device 10. The user interacts with the user terminal 20 and the server device 10 by repeating the activities A10 to A51 or A52 shown in FIG.

[0070] As described above, in the automated dialogue system 1, the information acquisition unit 113 functions as an example of a first acquisition unit that acquires correlation information that correlates with the user's QOL. The correlation information includes two or more types of information that can be acquired in accordance with the user's actions when speaking. The actions when speaking include actions of uttering words, actions of speaking, actions of moving the eyes, and actions of moving the head.

[0071] When the action of speaking is performed, it becomes possible to obtain text indicating what the user said, which is first correlation information. When the action of speaking is performed, it becomes possible to obtain features of the user's voice, which is second correlation information. When the action of making a facial expression is performed, it becomes possible to obtain features indicating the user's facial expression, which is third correlation information. When the action of moving the gaze is performed, it becomes possible to obtain features of the user's gaze pattern, which is fourth correlation information. When the action of moving the head is performed, it becomes possible to obtain features of the user's head shaking, which is fifth correlation information.

[0072] The QOL estimation unit 117 functions as an example of a first output unit that outputs a score indicating the user's QOL based on the correlation information acquired by the information acquisition unit 113. In the example of Fig. 6, the QOL estimation unit 117 supplies, i.e., outputs, the integrated score (an example of a score indicating QOL) estimated in A42 to the dialogue processing unit 118. In this way, the QOL of the user is estimated by the integrated score output based on the user's correlation information.

[0073] The inventors conducted an experiment to compare the errors between the QOL scores estimated for each of a plurality of users and the QOL measurement results obtained by conducting a questionnaire survey for the users, using a comparison AI module having a model that learned the association between each of the first to fifth correlation information and the QOL scores, and the first artificial intelligence unit 115. The results of the experiment are shown in FIG.

[0074] FIG. 10 is a diagram showing an example of an error in a QOL score. FIG. 10 shows a graph in which the vertical axis represents the magnitude of the error in the QOL score and the horizontal axis represents the type of correlation information. In this graph, the range from the upper limit to the lower limit of the error is indicated by a solid line, and the range in which the error is concentrated is indicated by a rectangle. As shown in FIG. 10 , when multimodal information is used, the accuracy of the estimated QOL is improved compared to when the first, third, fourth, and fifth related information are used individually. Furthermore, no significant difference was observed between when multimodal information was used and when the second related information was used alone.

[0075] From the above results, when the correlation information includes only one type of information, unless that information is second correlation information, the accuracy of the estimated QOL is lower than when multimodal information is used, and even when second correlation information is used, there is no difference in the accuracy of the estimated QOL compared to when multimodal information is used. In other words, the accuracy of the user's QOL estimated by the automated dialogue system 1 is equal to or higher than the accuracy of the QOL estimated when the correlation information includes only one type of information. Therefore, according to the automated dialogue system 1, it is possible to more easily obtain a high accuracy in the estimation result of the user's QOL compared to when the correlation information includes only one type of information.

[0076] Next, other experimental results will be described. FIG. 11 is a diagram showing another example of the experimental results. FIG. 11 shows a table showing changes in scores indicating QOL resulting from an experiment in which the robot 30 included in the automated dialogue system 1 and an elderly person aged 65 or older conversed every day for one week. For example, the scores for the above-mentioned PF, RP, BP, GH, VT, SF, RE, and MH changed to "0," "+4.55," "+7.83," "+0.65," "+4.01," "+4.41," "+2.19," and "+5.51," respectively.

[0077] Furthermore, the three highest scores for "Physical," "Mental," and "Role-social" changed to "+0.33," "+4.42," and "+1.67," respectively. These highest scores can be calculated from the eight subscales using an existing algorithm. In the example of Figure 11, all scores improved except for PF.

[0078] Furthermore, the regression coefficients for PF, RP, BP, GH, VT, SF, RE, MH, Physical, Mental, and Role-social were "0", "0.65", "1.12", "0.092", "0.58", "0.63", "0.31", "0.79", "0.048", "0.63", and "0.24", respectively, and the 95% confidence intervals of the regression coefficients were "-1.03 to 1.03", "-0.15 to 1.45", "-0.30 to 2.53", "-0.47 to 0.65", "-0.19 to 1.35", "-0.37 to 1.63", "-0.39 to 1.02", "0.023 to 1.55", "-0.91 to 1.01", "0.029 to 1.23", and "-0.39 to 0.87", respectively.

[0079] Furthermore, for PF, RP, BP, GH, VT, SF, RE, MH, Physical, Mental, and Role-social, the p-values ​​in the paired t-test were "0.500," "0.012," "0.033," "0.313," "0.013," "0.077," "0.102," "0.005," "0.447," "0.011," and "0.215." From these results, based on a significance level of 0.05 at the 95% confidence interval, it was shown that for RP, BP, VT, MH, and Mental, where the p-value was smaller than the significance level, there was a significant improvement in scores.

[0080] FIG. 12 shows another example of experimental results. In an experiment similar to that shown in FIG. 11 , participants were asked to rate the intensity of their negative emotions using a numerical value between 0 and 100 (100 being the most negative emotion) at the beginning and end of each daily conversation, and the numerical values ​​were then compiled. Before the conversation, the maximum value was 100, the first quartile was 90, the second quartile was 80, the third quartile was 60, and the minimum was 20. After the conversation, the maximum value was 90, the first quartile was 50, the second quartile was 40, the third quartile was 20, and the minimum was 0. When the significance level was set to 0.1, the p-value was smaller than the significance level, indicating that the intensity of negative emotions significantly decreased before and after the conversation.

[0081] Fig. 13 shows another example of experimental results. Fig. 13 shows a graph representing changes in BDI (Beck Depression Inventory) scores before and after the one-week experiment, similar to that shown in Fig. 11. This graph indicates that a BDI score of less than 11 is "within the normal range," a score of 11 or more but less than 17 is "mild depression," a score of 17 or more but less than 21 is "approaching clinical depression," and a score of 21 or more is "moderate depression."

[0082] In the example of Figure 13, of the 20 subjects, 9 subjects showed an improvement in BDI score, 5 subjects showed no change in BDI score, and 6 subjects showed a worsening of BDI score. From these results, the number of subjects whose BDI score improved was greater than the number whose BDI score worsened. In addition, 4 subjects showed a significant improvement in BDI score (5 points or more), which was significantly more than the 1 subject whose BDI score significantly worsened.

[0083] Fig. 14 shows another example of experimental results. Instead of the BDI scores shown in Fig. 13, Fig. 14 shows a graph representing changes in K6 (Kessler Psychological Distress Scale) scores. This graph indicates that a K6 score of less than 9 is within the "normal range," while a K6 score of 9 or greater is in the "requiring special attention" range.

[0084] In the example of Figure 14, of the 12 subjects, 8 subjects showed an improvement in K6 score, 2 subjects showed no change in K6 score, and 2 subjects showed a worsening of K6 score. From these results, the number of subjects whose K6 score improved (8 subjects) was significantly greater than the number whose K6 score worsened (2 subjects). In addition, there were 4 subjects whose K6 score improved significantly (3 points or more), which was significantly greater than the number of subjects whose K6 score worsened significantly (1 subject).

[0085] The above experimental results show that using the automated dialogue system 1 to conduct dialogue can improve the user's QOL.

[0086] Furthermore, in the automated dialogue system 1, the information acquisition unit 113 functions as an example of a second acquisition unit that acquires utterance information indicating the content of a utterance made by a user. The information acquisition unit 113 acquires the utterance information at A31 or A33 shown in Fig. 7, for example. The QOL estimation unit 117 functions as an example of an input unit that inputs the utterance information acquired by the information acquisition unit 113 and two or more scores output from the first artificial intelligence unit 115 to the response module.

[0087] The response module here is a module that performs learning based on a data group that uses a combination of a statement content and a score indicating QOL as input data and includes a response content corresponding to the statement content as output data, and the second artificial intelligence unit 116 is an example of such a module. The above-mentioned PF, RP, BP, GH, VT, SF, RE, and MH are each examples of scores indicating QOL. The dialogue processing unit 118 functions as an example of a second output unit that outputs the response content generated by the response module (second artificial intelligence unit 116) based on the input from the QOL estimation unit 117.

[0088] 6, the dialogue processing unit 118 transmits, i.e., outputs, response data indicating the response content generated in A44 to the user terminal 20. According to this mode, response content that also corresponds to the user's QOL is output, so that an appropriate response can be made to the user's utterance, compared to when response content is output regardless of the user's QOL.

[0089] Furthermore, in the automated dialogue system 1, rules are predefined for the method of generating a response content. The utterance rules ("paraphrasing," "agreement," "echoing," "praising," "deep dive," and "self-disclosure") shown in FIG. 8 are examples of predefined rules. Note that, while six utterance rules are predefined in the example of FIG. 8, the number of utterance rules is not limited to this, and one or more rules may be predefined. The dialogue processing unit 118 functions as an example of an instruction unit that instructs the response module (first artificial intelligence unit 115) to generate a response content based on one or more predefined rules when an input is made by the QOL estimation unit 117. According to this aspect, the response content can be made closer to a specific content.

[0090] Furthermore, in the example of Figure 8, two or more rules are defined, and as described in the explanation of Figure 8, the two or more rules differ in the degree of empathy felt by the therapist by the client. For example, "rephrasing" and "agreement" are perceived by the client with a high degree of empathy, while "deep dive" and "self-disclosure" are perceived by the client with a low degree of empathy. In general, the more a client feels that their thoughts are being empathized with, the more likely their QOL is to improve. In other words, the above two or more rules differ in the degree to which they contribute to improving QOL.

[0091] In this case, in the example of Fig. 8, by using the rule table TB11, the dialogue processing unit 118 (an example of an instruction unit) is configured to use a rule with a higher probability of instructing the user, the more likely the rule contributes to improving the user's QOL. According to this configuration, a response based on a rule that contributes more to improving the user's QOL is output, compared to when all rules are used with the same probability of instructing the user. As a result, the user's QOL can be improved significantly while preventing responses to the user from becoming uniform.

[0092] <Other embodiments> When the rules for generating response content include two or more rules (six rules in the example of Figure 8), the dialogue processing unit 118 (an example of an instruction unit) may vary the rule to be used for instructions from among the two or more rules depending on the content of the utterance indicated by the utterance information acquired by the information acquisition unit 113.

[0093] FIG. 15 is a diagram showing another example of a rule table. FIG. 15 shows a rule table TB21 in which utterance rules, utterance contents, and keywords are associated with each other. Similar to FIG. 8 , the rule table TB21 shows six utterance rules: "paraphrase," "agree," "echo," "praise," "dig deeper," and "self-disclosure." Each utterance rule is associated with utterance contents of "Topic A," "Topic B," "Topic C," "Topic D," "Topic E," and "Topic F," respectively, and keywords of "K11, K12, ...," "K21, K22, ...," "K31, K32, ...," "K41, K42, ...," "K51, K52, ...," and "K61, K62, ...."

[0094] If the utterance content indicated by the output utterance information contains any of the keywords, the dialogue processing unit 118 determines that the utterance content is associated with the keyword, and uses the utterance rule associated with the utterance content to issue instructions to the second artificial intelligence unit 116. According to this aspect, different utterance rules are used depending on the utterance content, which can improve the diversity of responses compared to when the same utterance rule is always used.

[0095] Also, there may be certain utterance rules that are appropriate for responding depending on the content of the utterance. For example, if a user makes a utterance about their hobbies, digging deeper will make the user more likely to join in the conversation, and if the user makes a utterance expressing their opinion, agreeing will make the user more likely to continue the conversation. By associating appropriate utterance rules with the content of the utterance in this way, users can be encouraged to talk more actively than if the same utterance rules were always used.

[0096] In addition, in the embodiment, the score indicating QOL includes eight types of scores: PF, RP, BP, GH, VT, SF, RE, and MH. However, this is not limiting, and for example, only some of the eight types of scores may be included. However, since only one type of score can represent only one aspect of QOL, it is desirable that the score indicating QOL include two or more types of scores. In this case, the dialogue processing unit 118 (an example of an input unit) may vary the type of score to be input to a response module (e.g., the second artificial intelligence unit 116) from among the two or more types of scores, depending on the acquired utterance information.

[0097] The dialogue processing unit 118 uses a score table that associates utterance content with the type of score to be input. Fig. 16 is a diagram showing an example of the score table. Fig. 16 shows a score table TB31 that associates utterance content, keywords, and score types. In the score table TB31, utterance content such as "Topic A," "Topic B," "Topic C," "Topic D," "Topic E," and "Topic F" are associated with keywords such as "K11, K12, ...," "K21, K22, ...," "K31, K32, ...," "K41, K42, ...," "K51, K52, ...," and "K61, K62, ...," respectively.

[0098] In addition, in the score table TB31, each utterance content is associated with two or more score types from the above eight score types. When the utterance content indicated by the output utterance information contains any of the keywords, the dialogue processing unit 118 determines that the utterance content is associated with the keyword, and inputs the score type associated with the utterance content to the second artificial intelligence unit 116 (an example of a response module).

[0099] In the embodiment, the second artificial intelligence unit 116 has a model that has learned to associate statement information and QOL scores (eight types of scores) with response content as a dedicated learning model for QOL. However, when the score table TB31 is used, it is desirable that the second artificial intelligence unit 116 also has a model that has learned to associate the eight types of scores by replacing them with the combinations of score types shown in the score table TB31.

[0100] Scores indicating a user's QOL may be easily or not affected by the content of their comments. For example, if a user talks about their physical condition (e.g., their leg has been hurting recently), physical functioning (PF), role physical functioning (RP), and bodily pain (BP) are likely to be affected, while social functioning (SF), role mental functioning (RE), and mental health (MH) are less likely to be affected. Conversely, if a user talks about their mental condition (e.g., they have been feeling depressed recently), SF, RE, and MH are likely to be affected, while PF, RP, and BP are less likely to be affected.

[0101] Therefore, by having the dialogue processing unit 118 input the to the second artificial intelligence unit 116, the response content that is output corresponds to a score that is easily influenced by the content of the statement, and therefore a more appropriate response content can be output compared to when the same type of score is always used.

[0102] In the embodiment, the eight scores (scales) defined by SF-36v2 (Short-Form 36-Item Health Survey version 2) are used as scores indicating QOL, but the present invention is not limited thereto. For example, scores defined by EQ-5D (EuroQol 5 Dimension), HUI (Health Utilities Index), WHO QOL, etc. may also be used. In either case, it is sufficient to use one or more scores that are correlated with the above-mentioned correlation information, and preferably two or more scores.

[0103] In the score table TB31, the content of the utterance indicated by the utterance information is associated with the type of score, but this is not limiting, and for example, the length of the utterance indicated by the utterance information or the time of the utterance may be associated with the type of score. In such cases, by inputting a score that is likely to be affected by the length or time of the utterance into the second artificial intelligence unit 116, it is possible to output a more appropriate response content than when the same type of score is always used.

[0104] Furthermore, the dialogue processing unit 118 (an example of an input unit) may be configured to input, with a higher probability, a type of score that is more highly related to the utterance information acquired by the information acquisition unit 113, among the two or more types of scores to the response module (e.g., the second artificial intelligence unit 116). In this case, the dialogue processing unit 118 uses, for example, an input probability table that associates the utterance content and keywords indicated by the utterance information with the input probability for each type of score.

[0105] 17 is a diagram showing an example of an input probability table. Fig. 17 shows an input probability table TB41 in which comment content, keywords, and input probabilities for each score type are associated with each other. In the input probability table TB41, comment content such as "Topic A," "Topic B," "Topic C," "Topic D," "Topic E," and "Topic F" are associated with keywords such as "K11, K12, ...," "K21, K22, ...," "K31, K32, ...," "K41, K42, ...," "K51, K52, ...," and "K61, K62, ...," respectively.

[0106] Furthermore, in the input probability table TB41, each utterance content is associated with an input probability (high, medium, low) for each of the eight types of scores described above. For example, if "Topic A" is a topic about exercise, the physical score has a higher relevance than the mental score. Therefore, the utterance content of "Topic A" is associated with an input probability of "high" if the score type is PF, RP, or BP, an input probability of "medium" if the score type is GH or VT, and an input probability of "low" if the score type is SF, RE, or MH.

[0107] Furthermore, if "Topic B" is a topic related to feelings, mental scores have a higher relevance than physical scores. Therefore, the utterance content of "Topic B" is associated with an input probability of "high" if the score type is SF, RE, or MH, an input probability of "medium" if the score type is GH or VT, and an input probability of "low" if the score type is PF, RP, or BP. For other utterance content (Topic C, etc.), an input probability is associated with each score type.

[0108] If the utterance content indicated by the output utterance information contains any of the keywords, the dialogue processing unit 118 determines that the utterance content is associated with the keyword, and inputs each score to the second artificial intelligence unit 116 (an example of a response module) with the input probability associated with the utterance content. For example, if the utterance content is "Topic A," PF, RP, VT, and MH are input with a high probability, BP and SF are input with a medium probability, and GH and RE are input with a low probability. Note that an upper limit, a lower limit, or both, may be set for the number of types of scores to be input.

[0109] In the input probability table TB41, the utterance content indicated by the utterance information is associated with the input probability, but this is not limiting, and for example, the utterance length or utterance time indicated by the utterance information may be associated with the input probability. In such cases, by inputting a type of score that is more highly related to the utterance length or utterance time to the second artificial intelligence unit 116 with a higher probability, it is possible to output a more appropriate response content than when the input probability is always the same.

[0110] Furthermore, the correlation information is not limited to the multimodal information (including the first to fifth correlation information) described in the embodiment. For example, the information may be multimodal information or may include two to four types of correlation information among the first to fifth correlation information. Furthermore, each piece of correlation information is not limited to the first to fifth correlation information, i.e., text indicating what the user said, features of the user's voice, features indicating the user's facial expression, features of the user's gaze pattern, and features of the user's head shaking.

[0111] The correlation information may be, for example, information indicating the movement characteristics of a first part of the user's body and the movement characteristics of a second part of the body when the user speaks. The first and second parts of the body may be, in addition to the face, eyes, and head, for example, the hands, arms, torso, and legs. The correlation information may be information indicating two or more of the information indicating the movement characteristics, the voice characteristics of the user's speech, or the characteristics of the content of the user's speech. According to this aspect, the QOL at the time of speech can be estimated with higher accuracy than when the correlation information is one type of information.

[0112] Furthermore, the QOL scores are not limited to the eight scores described in the embodiment. The QOL scores may be scores for one or more of functions related to physical health (examples include PF, RP, and BP), functions related to mental health (examples include RE and MH), functions related to daily living (example includes VT), and functions related to sociality (example includes SF). According to this aspect, the user's condition can be understood from more diverse perspectives than when there is only one QOL score.

[0113] Furthermore, in the embodiment, the correlation information includes two or more types of information that can be acquired in accordance with the user's actions when speaking. However, the correlation information may include one or more types of information that can be acquired in accordance with the user's actions when speaking. For example, the correlation information may be one of the first to fifth correlation information. Even in this case, the dialogue processing unit 118 (an example of an input unit) inputs the utterance information acquired by the information acquisition unit 113 and the output score to a response module (an example of which is the second artificial intelligence unit 116), and the response module outputs a response content generated based on the input. This makes it possible to provide an appropriate response to the user's utterance, compared to when a response content is output regardless of the user's QOL.

[0114] In the embodiment, automatic dialogue with the user is performed via the user terminal 20, but automatic dialogue with the user may also be performed via the robot 30. In this case, since the display of the robot 30 is small, it is preferable that the dialogue is mainly performed via voice. Furthermore, the robot 30 may emit voices and move various parts of its body to imitate the movements of a living creature using the movement processing unit 315.

[0115] The configurations (overall configuration, hardware configuration, functional configuration, etc.) shown in FIG. 1 and elsewhere are merely examples, and other configurations may be used as long as they are not inconvenient for implementation. For example, one device may be distributed across two or more devices, or may be replaced by a cloud computing system. Furthermore, the functions of one device may be distributed across two or more devices, or the functions of two or more devices may be centralized and realized in one device. Furthermore, the operation performed by one function may be distributed across two or more functions, or two or more functions may be integrated into one function. In short, as long as each function required by the automated dialogue system 1 as a whole is realized, any configuration may be used to realize those functions.

[0116] The output destination of information or data (hereinafter referred to as "information, etc.") may be another device, a display, a memory unit (including an internal memory unit and an external memory unit), etc. Acquisition of information, etc. includes acquiring information, etc. transmitted from another device, as well as acquiring information, etc. generated by the device itself. The table or database, etc. in which parameters are associated is not limited to the illustrated table, etc., and the number of parameters may be reduced or increased. Furthermore, information, etc. corresponding to parameters may be obtained using a mathematical formula, a conditional formula, etc., without using a table, etc.

[0117] The above-described embodiment is an information processing system such as the automated dialogue system 1 that includes an information processing device such as the server device 10, but may also be an information processing method. The information processing method includes steps (processes) executed by the information processing system. The above-described embodiment may also be a program. The program causes a computer to execute steps (processes) executed by a similar information processing system.

[0118] <Additional Notes> Furthermore, the present invention may be provided in the following aspects.

[0119] (1) An information processing system comprising a processor, wherein in a first acquisition step, the processor acquires correlation information having a correlation with a user's QOL (Quality of life), the correlation information including two or more types of information that can be acquired in accordance with the user's actions when speaking, and in a first output step, outputs a score indicating the user's QOL based on the acquired correlation information.

[0120] According to this aspect, it is possible to increase the accuracy of the estimation result of the user's QOL.

[0121] (2) In the information processing system described in (1) above, in the second acquisition step, the processor acquires utterance information indicating the content of the utterance made by the user, and in the input step, inputs the acquired utterance information and the two or more output scores to a response module, the response module is a module that uses a combination of the utterance content and a score indicating QOL as input data and has performed learning based on a data group that includes response content corresponding to the utterance content as output data, and in the second output step, outputs the response content generated by the response module based on the input.

[0122] According to this aspect, it is possible to give an appropriate response to a user's comment.

[0123] (3) In the information processing system described in (2) above, the scores include two or more types of scores, and in the input step, the processor varies the type of score to be input into the response module from the two or more types of scores based on the acquired utterance information.

[0124] According to this aspect, it is possible to output more appropriate response content.

[0125] (4) In the information processing system described in (3) above, in the input step, the processor inputs into the response module with a higher probability the type of score that is more highly relevant to the acquired utterance information among the two or more types of scores.

[0126] According to this embodiment, a more accurate response can be made.

[0127] (5) In the information processing system described in any one of (2) to (4) above, rules are defined in advance for the method of generating the response content, and in the instruction step, the processor instructs the response module to generate the response content based on the rules when the input is made.

[0128] According to this embodiment, the response content can be made closer to a specific content.

[0129] (6) In the information processing system described in (5) above, the rules include two or more rules, and in the instruction step, the processor varies the rule to be used for the instruction from among the two or more rules depending on the content of the utterance indicated by the acquired utterance information.

[0130] According to this embodiment, it is possible to improve the diversity of responses.

[0131] (7) In the information processing system described in (6) above, the two or more rules each have a different degree of contribution to improving the QOL, and in the instruction step, the processor uses a rule with a higher degree of contribution with a higher probability in the instruction.

[0132] According to this aspect, it is possible to prevent responses to the user from becoming uniform, and to further improve the user's QOL.

[0133] (8) In the information processing system described in any one of (1) to (7) above, the correlation information is information indicating two or more of the following characteristics: characteristics of the movement of a first part of the user's body when the user speaks; characteristics of the movement of a second part of the body; characteristics of the voice in the utterance; or characteristics of the content of the utterance in the utterance.

[0134] According to this aspect, the QOL during speech can be estimated with higher accuracy.

[0135] (9) In the information processing system described in any one of (1) to (8) above, the score is a score for one or more of functions related to physical health, functions related to mental health, functions related to daily living, and functions related to sociality.

[0136] According to this embodiment, the user's condition can be grasped from more diverse angles.

[0137] (10) An information processing system including a processor, wherein the processor, in a first acquisition step, acquires correlation information having a correlation with a user's QOL (Quality of life), the correlation information including information that can be acquired in accordance with the user's actions when speaking, in a first output step, outputs a score indicating the user's QOL based on the acquired correlation information, in a second acquisition step, acquires utterance information indicating the content of the utterance made by the user, in an input step, inputs the acquired utterance information and the output score to a response module, the response module is a module that has performed learning based on a data group that uses a combination of the utterance content and the score indicating QOL as input data and includes response content corresponding to the utterance content as output data, and in a second output step, outputs the response content generated by the response module based on the input.

[0138] According to this aspect, it is possible to give an appropriate response to a user's comment.

[0139] (11) A program that causes a computer to execute each step of the information processing system described in any one of (1) to (10) above.

[0140] (12) An information processing method comprising the steps of the information processing system according to any one of (1) to (10) above. Of course, this is not a limitation. Furthermore, the above-described embodiments and modifications may be implemented in any combination.

[0141] Finally, while various embodiments of the present invention have been described, these are presented by way of example only and are not intended to limit the scope of the invention. The novel embodiments may be embodied in various other forms, and various omissions, substitutions, and modifications may be made without departing from the spirit of the invention. The embodiments and their modifications are intended to be included within the scope and spirit of the invention, as well as within the scope of the inventions and their equivalents as defined in the appended claims.

[0142] 1: Automated dialogue system, 2: Communication line, 10: Server device, 11: Control unit, 20: User terminal, 21: Control unit, 30: Robot, 31: Control unit, 111: Server display unit, 112: User authentication unit, 113: Information acquisition unit, 114: Speech recognition unit, 115: First artificial intelligence unit, 116: Second artificial intelligence unit, 117: QOL estimation unit, 118: Dialogue processing unit, 211: User display unit, 212: Operation reception unit, 213: Sound processing unit, 214: Image processing unit, 311: User display unit, 312: Operation reception unit, 313: Sound processing unit, 314: Image processing unit, 315: Motion processing unit

Claims

1. An information processing system comprising a processor, wherein in a first acquisition step, the processor acquires correlation information correlating with a user's QOL (Quality of life), the correlation information including two or more types of information that can be acquired in accordance with the user's actions when speaking, and in a first output step, outputs a score indicating the user's QOL based on the acquired correlation information.

2. An information processing system as described in claim 1, wherein the processor, in a second acquisition step, acquires utterance information indicating the content of the utterance uttered by the user; in an input step, inputs the acquired utterance information and the two or more output scores to a response module, the response module being a module that has performed learning based on a data group that takes a combination of the utterance content and a score indicating QOL as input data and includes a response content corresponding to the utterance content as output data; and in a second output step, outputs the response content generated by the response module based on the input.

3. An information processing system as described in claim 2, wherein the scores include two or more types of scores, and in the input step, the processor varies the type of score to be input to the response module from among the two or more types of scores depending on the acquired utterance information.

4. An information processing system according to claim 3, wherein in the input step, the processor inputs into the response module with a higher probability the type of score that is more highly related to the acquired utterance information, among the two or more types of scores.

5. An information processing system according to any one of claims 2 to 4, wherein the method of generating the response content has predefined rules, and in the instruction step, the processor instructs the response module to generate the response content based on the rules when the input is made.

6. An information processing system according to claim 5, wherein the rules include two or more rules, and wherein in the instruction step, the processor varies a rule to be used for the instruction among the two or more rules depending on the content of the utterance indicated by the acquired utterance information.

7. An information processing system according to claim 6, wherein the two or more rules each have a different degree of contribution to improving the QOL, and in the instruction step, the processor uses a rule with a higher degree of contribution with a higher probability in the instruction.

8. An information processing system according to any one of claims 1 to 7, wherein the correlation information is information indicating two or more of the movement characteristics of a first part of the user's body when the user speaks, the movement characteristics of a second part of the body, the voice characteristics of the utterance, or the characteristics of the content of the utterance.

9. An information processing system according to any one of claims 1 to 8, wherein the score is a score for one or more of a function related to physical health, a function related to mental health, a function related to daily living, and a function related to sociality.

10. An information processing system comprising a processor, wherein in a first acquisition step, the processor acquires correlation information having a correlation with a user's QOL (Quality of life), the correlation information including information that can be acquired in accordance with the user's actions when speaking, in a first output step, a score indicating the user's QOL based on the acquired correlation information, in a second acquisition step, acquires utterance information indicating the content of the utterance made by the user, in an input step, the acquired utterance information and the output score are input to a response module, the response module being a module that has performed learning based on a data group including a combination of the utterance content and the score indicating QOL as input data and a response content corresponding to the utterance content as output data, and in a second output step, outputs the response content generated by the response module based on the input.

11. A program causing a computer to execute each step of the information processing system according to any one of claims 1 to 10.

12. An information processing method comprising the steps of the information processing system according to any one of claims 1 to 10.

Images