Dynamic traits in a conversational agent

Abstract

A human-computer interaction method and a computer-based dialog system between a human user (1) and a virtual agent (2) in an environment (3), wherein the virtual agent (2) comprises static characteristics (5), dynamic characteristics (6), and a dynamic knowledge base (8). The virtual agent (2) is configured to continuously update both the dynamic characteristics (6) and the dynamic knowledge base (8) through interactions with its environment (3) and, upon receiving a user input (16), determine adjusted dynamic characteristics (7) and an agent output (17) based on both said static characteristics (5), said adjusted dynamic characteristics (7) and said dynamic knowledge base (8).

Description

[0001] DYNAMIC TRAITS IN A CONVERSATIONAL AGENT

[0002] TECHNICAL FIELD

[0003] The disclosure relates to the field of communications technologies, and in particular, to a humancomputer interaction method and a computer-based dialog system between a human user and a virtual agent.

[0004] BACKGROUND

[0005] The use of conversational agents, chatbots, and intelligent virtual assistants are growing in popularity. There is a lot of research going on in this area and the quality of these services is improving at a fast pace. The same can be said for the video game industry, which will reportedly continue to grow as an economic sector throughout the next decade.

[0006] In video games as well as other virtual simulation environments one can encounter entities implemented as Non-Playable Characters (NPC) with limited conversational abilities. This means that a character in a video game can currently only say what the developers of the game have written and prepared for that character.

[0007] There are some existing technologies that allow a conversational agent (eg: video game character) to discuss any topic using chat-bots made with large scale neural models and Natural Language Processing (NLP) techniques. Furthermore, efforts have been carried out to make these conversational agents more human-like by giving them personalities, human-like traits, varying tones and even memory graphs to track the conversation style and spoken relationship with a given user.

[0008] US20200395008A1 describes personality-based conversational agents and chatbots that the user can interact with and operate. The chatbot can be based upon a real person or fictitious character and includes human-like personality traits, and can adjust its tone based on the situation. The agents can use personality modules, natural language understanding modules and natural language generation modules all interconnected together.

[0009] US10250532B2 describes a personality consistent chatbot that tracks the relationship with the user by means of a memory graph which is maintained and updated. US11361166B2 describes chatbots wherein the conversational style may be characterized by the features of characters.

[0010] US20190205382A1 describes techniques to make natural language generation more personal and human-like, wherein the conversational agent can express opinions using a consistent style or personality.

[0011] Although these implementations indeed help improve the quality of the conversation with the Al and make the agent more human-like, appealing and relatable, these prior art technologies fail to fully capture the essence of individualism in Al agents and realistic interactions between entities. To reach higher levels of realism in conversations and interactions there are a series of additional technologies that should be included into conversational agents.

[0012] SUMMARY

[0013] It is an object to provide an improved human-computer interaction method and a computer-based dialog system which overcomes or at least reduces the problems mentioned above.

[0014] The foregoing and other objects are achieved by the features of the independent claims. Further implementation forms are apparent from the dependent claims, the description and the figures.

[0015] According to a first aspect, there is provided a human-computer interaction method using a conversational agent with both static and dynamic traits, historical awareness, a personalized dynamic knowledge base and interactive abilities beyond just conversations, enabled by dynamic characteristics which are adjusted based on a received user input and are used in generating an agent output, thereby allowing the agent’s emotions to vary dynamically based on the input of the user. The agent output is generated based on both the static characteristics, adjusted dynamic characteristics and the dynamic knowledge base.

[0016] Such an agent is capable of carrying on a conversation as well as sensing and performing actions in an environment and reacting to actions performed by the user or other agents.

[0017] In a possible implementation form of the first aspect the method comprises providing a virtual agent comprising static characteristics, dynamic characteristics, and a dynamic knowledge base; the virtual agent being configured to continuously interact with an environment and, based on the interactions, update both the dynamic characteristics and the dynamic knowledge base; receiving a user input from the environment; determining, based on the user input, adjusted dynamic characteristics for the virtual agent; and generating an agent output to the environment; wherein generating the agent output is based on both the static characteristics, the adjusted dynamic characteristics and the dynamic knowledge base.

[0018] The method allows an improved quality of conversation with a virtual agent and making the virtual agent more human-like, appealing and relatable, while also enabling individualism and realistic interactions between entities by ensuring that the interactions have an actual impact where what the user says has an emotional or psychological effect on the virtual agent.

[0019] In a possible implementation form of the first aspect generating the agent output is based on historical context awareness of the virtual agent; wherein the interactions between the virtual agent and the environment comprise historical interactions predating the user input with at least one of the human users, other human users, other virtual agents provided within the environment, and further variables of the environment. This improves interactivity, understanding and knowledge, as well as relatability, human likeness, and realness of the Al. It further adds novelty for the users, as conversational agents have never before lied, been deceitful, or had become forgetful.

[0020] In particular, an aspect that is problematic in current chatbot implementations is the lack of historical context awareness. Conversational agents have a lot of trouble understanding the inputs they receive because they lack context. Even if the context is explained to them, the information is oftentimes not saved or processed. This is not just a problem because of the impediment to understanding and comprehension but also it means that the agent has no form of memories, and it cannot discuss complex topics, mention past occurrences or evolve his opinions based on events. According to this implementation form in a simulation a virtual agent is able to have historic context awareness stemming from its environment and all possible kinds of inputs and events regardless of the personal previous interaction of the agent with the user itself.

[0021] In an embodiment the interactions between the virtual agent and its environment can be generated using Artificial Intelligence thus saving costs and time and allowing for a robust model.

[0022] In possible embodiments, for providing the historical context awareness for the virtual agent, separate knowledge bases, knowledge graphs or other knowledge systems can be used. In a possible implementation form of the first aspect the virtual agent is an Al agent capable of sensing its environment and taking actions to achieve a goal, the Al agent using a machine learning model trained using the interactions with the environment and being configured to improve its own performance in achieving the goal.

[0023] In an embodiment the machine learning model is trained through Reinforcement Learning guided by a reward and / or punishment system within the environment.

[0024] In an embodiment the reward function that teaches or guides the reinforcement learning algorithm is based on human user input, using a human-in-the-loop strategy.

[0025] In a possible embodiment the human user input involves receiving input from multiple human users simultaneously using Federated Learning. Having many people give inputs simultaneously can greatly speed up the training process.

[0026] In an embodiment the machine learning model is an artificial neural network.

[0027] In a possible implementation form of the first aspect the virtual agent is implemented on a server, wherein the server is configured to update the dynamic characteristics and / or the dynamic knowledge base via a communications interface. Having more sources of data income means more varied types of data. Furthermore, since the Al constantly interacts with many people at once on their own platforms or simulations it has the possibility to use federated learning data on such interactions between human users and virtual agents within other environments.

[0028] In a possible implementation form of the first aspect the static characteristics comprise basic metadata related to at least one of an age, profession, residence, background or relationships of the virtual agent with other virtual agents.

[0029] In a possible implementation form of the first aspect the dynamic characteristics comprise individual metadata related to short-term states comprising at least one of an emotional state such as ‘happy’, ‘sad’; mood; or instant reaction such as ‘surprise’, ‘fear’, or ‘excitement’, and longterm states comprising at least one of a core personality, identity, religious belief, political belief, culture, lifestyle, philosophy, mannerisms, quirks, habits, likes and dislikes, opinions, feelings or other long-term personal characteristics, thereby allowing the agent to have different states that vary dynamically based on the input of the user. In an embodiment the basic metadata and / or the individual metadata can be generated using Artificial Intelligence, allowing for a robust model.

[0030] In a possible implementation form of the first aspect the virtual agent is further configured to update at least one of the static characteristics, dynamic characteristics, and the dynamic knowledge base through manual settings, thus allowing manual adjustments.

[0031] In a possible implementation form of the first aspect the virtual agent further comprises human behaviour variables related to the dynamic knowledge base and affecting the use of information from the dynamic knowledge base by the virtual agent, the human behaviour variables comprising knowledge-dependent human behaviour variables including at least one of an ignorant variable resulting in partial occlusion of information from the dynamic knowledge base; deceitfulness variable resulting in obscuring or altering information from the dynamic knowledge base; forgetfulness variable resulting in temporary or permanent deletion of information from the dynamic knowledge base; learning variable resulting in temporary or permanent addition of true information to the dynamic knowledge base; or beliefs variable resulting in temporary or permanent altering of information and / or addition of false information to the dynamic knowledge base. This makes the virtual agent more human-like, appealing and relatable, while also enabling individualism and realistic interactions.

[0032] In a possible implementation form of the first aspect the virtual agent further comprises a feelings variable related to at least one of the human user, other human users, or other virtual agents within the environment; the feelings variable being configured to imitate the development of human feelings and opinions based on the dynamic characteristics of the virtual agent, the user input and historical interactions predating the user input with at least one of the human user, other human users , and other virtual agents respectively; wherein generating the agent output is at least partially based on the feelings variable. This allows mimicking feelings such as the evolution of romantic or hatred feelings towards things or people by the machine, making the virtual agent even more human-like.

[0033] In a possible implementation form of the first aspect the environment is a text-based virtual environment, wherein the virtual agent is implemented as a chat bot, and wherein receiving the user input from the environment and generating the agent output to the environment comprises a text-based dialogue.

[0034] In a possible implementation form of the first aspect the environment is a simulated virtual environment, wherein the virtual agent is implemented as a Non-Playable Character configured to perform actions and interact with an avatar of a human user within the simulated virtual environment.

[0035] In a possible implementation form of the first aspect the simulated virtual environment is generated by a simulation module as part of a video game architecture, the simulated virtual environment being defined by simulation parameters, wherein the simulation parameters are configured to be updated through at least one of the interactions between the virtual agent and the environment, the user input, and manual settings.

[0036] In a possible implementation form of the first aspect the environment is a physical environment, wherein the virtual agent is implemented in a computer-based system configured to sense changes within the physical environment via at least one sensor, perform actions, and interact with the human user via an input-output interface.

[0037] In a possible implementation form of the first aspect receiving the user input from the environment comprises receiving natural language input in the form of speech or written text, and transforming the natural language input using a Natural Language Processing (NLP) model into input data for the virtual agent; wherein generating the agent output to the environment comprises receiving output data from the virtual agent, and transforming the output data using the NLP model into natural language output in the form of speech or written text.

[0038] In a possible implementation form of the first aspect transforming the natural language input into input data for the virtual agent comprises interpreting the speech or written text using a Natural Language Understanding (NLU) module of the NLP model, and inferring the input data using a Natural Language Inference (NLI) module of the NLP model; wherein transforming the output data from the virtual agent into natural language output comprises transforming the output data into speech or written text using a Natural Language Generation (NLG) module of the NLP model. In a possible implementation form of the first aspect receiving the user input from the environment comprises receiving an input action from the user, interpreting the input action using an Action Understanding module of an Action Processing model, and inferring the input data using an Action Inference module of the Action Processing model; wherein generating the agent output to the environment comprises receiving output data from the virtual agent, and transforming the output data into output action using an Action Generation module of the Action Processing model.

[0039] According to a second aspect, there is provided computer-based dialog system between a human user and a virtual agent, the system comprising an input-output interface, a memory, and a processor; the input-output interface and the memory being coupled to the processor, the memory being configured to store computer program code, the computer program code comprising computer instructions; and the processor being configured to read the computer instructions from the memory and cause the system to perform the steps of a method according to any one of the possible implementation forms of the first aspect.

[0040] Such a system enables virtual conversational agents showing emotions and generating responses that vary dynamically based on the input of a user.

[0041] These and other aspects will be apparent from and the embodiment(s) described below.

[0042] BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In the following detailed portion of the present disclosure, the aspects, embodiments and implementations will be explained in more detail with reference to the example embodiments shown in the drawings, in which:

[0044] Fig. 1 shows a block diagram of a human-computer interaction method according to an example of the embodiments of the disclosure;

[0045] Fig. 2 shows a flow diagram of a human-computer interaction method according to an example of the embodiments of the disclosure;

[0046] Fig. 3 shows an illustration in the form of a block diagram of a virtual agent according to an example of the embodiments of the disclosure;

[0047] Fig. 4 shows an illustration of a text-based virtual environment according to an example of the embodiments of the disclosure; Fig. 5 shows an illustration of a physical environment according to an example of the embodiments of the disclosure;

[0048] Fig. 6 shows a combined block diagram and flow chart of a human-computer interaction method according to an example of the embodiments of the disclosure;

[0049] Fig. 7 shows a block diagram of a video game architecture using a simulation module according to an example of the embodiments of the disclosure;

[0050] Fig. 8 shows an illustration of a Non-Playable Character in a simulated virtual environment according to an example of the embodiments of the disclosure;

[0051] Fig. 9 shows a block diagram of a simulated virtual environment according to an example of the embodiments of the disclosure; and

[0052] Fig. 10 shows a block diagram of a computer-based system according to an example of the embodiments of the disclosure.

[0053] DETAILED DESCRIPTION

[0054] In the following detailed description, numerous specific details are set forth by way of examples in order to provide a thorough understanding of the relevant disclosure. However, it should be apparent to those skilled in the art that the present disclosure may be practiced without such details. In other instances, well known methods, procedures, systems, components, and / or circuitry have been described at a relatively high-level, without detail, in order to avoid unnecessarily obscuring aspects of the present disclosure.

[0055] GENERAL DEFINITIONS

[0056] Natural Language Processing (NLP): The area of Computer Science and Artificial Intelligence that deals with the representation and manipulation of human natural language from a computational perspective.

[0057] Natural Language Understanding (NLU): Subfield of Natural Language Processing that deals with the problems of interpreting the meaning of natural language and representing it in a computational format. Natural Language Generation (NLG): The subfield of Natural Language Processing that deals with the problems of generating fluent natural language utterances from formal representations of information.

[0058] Non-Playable Character (NPC): Character in a simulation or a game that is not the avatar of a user.

[0059] Conversational Agent: A computer system intended to converse with a human.

[0060] Al Agent: anything capable of sensing its environment and taking actions to achieve goal, that may improve its own performance with learning or knowledge. Agents interacting and learning from its environment is the core idea from Reinforcement Learning.

[0061] Reinforcement Learning: A form of Al learning whereby a machine learns by trial and error within an environment, usually in a simulation. The learning is guided by a reward and / or punishment system for achieving goals.

[0062] Federated Learning (also known as collaborative learning): A machine learning technique that trains an algorithm across multiple decentralized edge devices or servers holding local data samples, without exchanging them. This approach stands in contrast to traditional centralized machine learning techniques where all the local datasets are uploaded to one server, as well as to more classical decentralized approaches which often assume that local data samples are identically distributed.

[0063] DESCRIPTION OF DRAWINGS

[0064] Fig. 1 illustrates human-computer interaction method, wherein the method may be applied to a computer-based dialog system between a human user 1 and a virtual agent 2. The user 1 and the virtual agent 2 are both in an environment 3, and the virtual agent 2 is configured to continuously interact with the environment 3.

[0065] In case of the environment 3 being a simulated virtual environment 23 as illustrated further in Figs. 7 to 9, the virtual agent 2 may be configured to perform actions and interact with an avatar 4 of a human user 1 within the simulated virtual environment 23 instead of the actual user 1. In case of the environment 3 being physical environment 20 as illustrated in Fig. 5, the virtual agent 2 may be configured to sense changes within the physical environment 20 via sensors 22, and to perform actions and interact with the human user 1 via an input-output interface 40.

[0066] The virtual agent 2 comprises at least the following: static characteristics 5, dynamic characteristics 6, and a dynamic knowledge base 8. The virtual agent 2 is further configured to continuously update both the dynamic characteristics 6 and the dynamic knowledge base 8 based on interactions with the environment and the user 1.

[0067] In particular, as shown in the flowchart of Fig. 2, the agent 2 can receive a user input 16 from the environment 3 and determine based on this user input 16, adjusted dynamic characteristics 7 for the virtual agent 2 and an agent output 17.

[0068] As shown in Fig. 2, generating this agent output 17 is based on at least the static characteristics 5, the adjusted dynamic characteristics 7 and the dynamic knowledge base 8.

[0069] As illustrated in both Fig. 1 and 2, generating the agent output 17 can be further based on historical context awareness 9 of the virtual agent 2.

[0070] Current chatbots lack such historical context awareness and thus conversational agents have a lot of trouble understanding the inputs they receive because they lack context. Even if the context is explained to them, the information is oftentimes not saved or processed. This is not just a problem because of the impediment to understanding and comprehension but also it means that the agent has no form of memories. It cannot discuss complex topics, mention past occurrences or evolve his opinions based on events. The emotional state memory graph mentioned in the prior art is a step in this direction as it allows the emotional state of the chat bot to change or evolve over time based on the chat bot’s interactions with the user and models the relationship closeness with the user based on the user’s engagement and conversation history with the chatbot. However, it is restricted to the conversation with the user 1. In a simulation as illustrated in this disclosure, a virtual agent 2 can have historical context awareness 9 stemming from its environment 3 (changes, actions, settings, etc.) and all possible kinds of inputs and events regardless of the agent’s 2 personal previous interaction with the user 1 itself. This allows creating a virtual agent 2 capable of carrying on a conversation as well as sensing / performing actions in an environment 3 and reacting to actions performed by the user or other agents 2.

[0071] The agents 2 can also have their own identity, emotional state, moods, core personality, religious or political beliefs, culture, lifestyle, philosophy, mannerisms, quirks, habits, likes and dislikes, opinions, feelings and other personal characteristics, as will be described below in more detail.

[0072] Another advantage of the method is that it enables using constant data feedback from (a) manual settings, (b) user interactions and (c) the environment 3 as shown in Fig. 1 and 2 to teach, fine-tune and adapt the virtual agents 2. Since the Al can constantly interact with many people on their own platforms or simulations it has the possibility to use Federated Learning.

[0073] Having a virtual agent 2 with such dynamic traits as described above means that its characteristics and features are able to change and vary depending on the user input 16, historic context 9 and environment 3. Achieving this also allows for short term states 12 such as emotional state, moods and instant reactions as well as long term states 13 such as personality, religious or political beliefs, culture and other long-term personal characteristics, as illustrated in more detail in Fig. 3.

[0074] In some embodiments the interactions between the virtual agent 2 and its environment 3 can be generated using Artificial Intelligence, e.g. as part of creating a simulated virtual environment 23 wherein the virtual agent 2 is implemented as a Non-Playable Character (NPC) 46, such as illustrated in Figs. 8 and 9.

[0075] In possible embodiments, for providing the historical context awareness 9 for the virtual agent 2, separate knowledge bases, knowledge graphs or other knowledge systems can also be used.

[0076] Fig. 3 illustrates a virtual agent 2 implemented according to an example of the disclosure. Steps and features that are the same or similar to corresponding steps and features previously described or shown herein are denoted by the same reference numeral as previously used for simplicity.

[0077] The virtual agent 2 can be implemented in some cases as an Al agent capable of sensing its environment 3 and taking actions to achieve a goal. Such an Al agent can be based on a machine learning model trained using the interactions with the environment 3 and being configured to improve its own performance in achieving the set goal. The machine learning model can be trained through Reinforcement Learning guided by a reward and / or punishment system within the environment 3. The machine learning model is an artificial neural network (ANN).

[0078] The virtual agent 2 can be implemented on a server 18 as shown in Fig. 4 and 10, wherein the server 18 is configured to update the dynamic characteristics 6 and / or the dynamic knowledge base 8 via a communications interface 44. This update can at least partially be based on federated learning data on interactions between human users 1 and virtual agents 2 within other environments 3.

[0079] As shown in Fig. 3, apart from having historical context awareness 9 as described before, the static characteristics 5 of the virtual agent 2 may comprise basic metadata 10 related to at least one of an age, profession, residence, background or relationships of the virtual agent 2 with other virtual agents 2.

[0080] On the other hand, the dynamic characteristics 6 of the virtual agent 2 may comprise individual metadata 11 related to either or both:

[0081] - short-term states 12 which can be an emotional state such as ‘happy’, ‘sad’; mood; or instant reaction such as ‘surprise’, ‘fear’, or ‘excitement’, and

[0082] - long-term states 13 which can relate to a core personality, identity, religious belief, political belief, culture, lifestyle, philosophy, mannerisms, quirks, habits, likes and dislikes, opinions, feelings or other long-term personal characteristics.

[0083] Both the basic metadata 10 and / or the individual metadata 11 can be generated using Artificial Intelligence.

[0084] The virtual agent 2 may be further configured so that it is possible to adjust and-or update at least one of the static characteristics 5, dynamic characteristics 6, and the dynamic knowledge base 8 through manual settings.

[0085] As further illustrated in Fig. 3, the virtual agent 2 may also comprise knowledge-dependent human behavior variables 14 related to the dynamic knowledge base 8 and affecting the use of information from the dynamic knowledge base 8 by the virtual agent 2. These human behavior variables 14 may be one or multiple of the following examples (the below being a non-exhaustive list):

[0086] - ignorant variable resulting in partial occlusion of information from the dynamic knowledge base 8;

[0087] - deceitfulness variable resulting in obscuring or altering information from the dynamic knowledge base 8;

[0088] - forgetfulness variable resulting in temporary or permanent deletion of information from the dynamic knowledge base 8;

[0089] - learning variable resulting in temporary or permanent addition of true information to the dynamic knowledge base 8; or

[0090] - beliefs variable resulting in temporary or permanent altering of information and / or addition of false information to the dynamic knowledge base 8.

[0091] As further illustrated in Fig. 3, the virtual agent 2 may further comprise a feeling’s variable 15 which influences the agent output 17 with respect to the human user 1, other human users 1, and / or other virtual agents 2 within the environment 3.

[0092] This feelings variable 15 is configured to imitate the development of human feelings and opinions based on the dynamic characteristics 6 of the virtual agent 2, the user input 16 and historical interactions predating the user input 16 with at least one of the human user 1 , other human users 1, and other virtual agents 2 respectively.

[0093] All together results in virtual agents 2 with their own identity, emotional state, moods, core personality, religious or political beliefs, culture, lifestyle, philosophy, mannerisms, quirks, habits, likes and dislikes, opinions, feelings and other personal characteristics which make them more human-like and engaging in both interaction and conversations.

[0094] Figs. 4 and 5 illustrate different environments 3 the virtual agent 2 can be implemented in.

[0095] Fig. 4 shows an example where the environment 3 is a text-based virtual environment 19, and wherein the virtual agent 2 is implemented as a chat bot. In this case receiving the user input 16 from the environment 3 and generating the agent output 17 to the environment 3 comprises a textbased dialogue displayed e.g. on a smartphone display which functions as the input-output interface 40 as mentioned before.

[0096] On the other hand, Fig. 5 shows an example where the environment 3 is a physical environment 20, and wherein the virtual agent 2 is implemented in a computer-based system 21 configured to sense changes within the physical environment 20 via at least one sensor 22, and perform actions, and interact with the human user 1 via an input-output interface 40 which can also be a display or any other interface capable of human-computer interaction.

[0097] Fig. 6 illustrates the underlying architecture used for generating different types of agent output 17 based on different types of user input 16.

[0098] In particular, as shown in the top part of Fig. 6, in some cases the user input 16 from the environment 3 means receiving natural language input 29 in the form of speech or written text. In such cases the natural language input 29 is transformed using a Natural Language Processing (NLP) model 30 into input data 27 for the virtual agent 2; and at the end of the flow generating the agent output 17 to the environment 3 includes receiving output data 28 from the virtual agent 2 and transforming this output data 28 using the NLP model 30 into natural language output 41 in the form of speech or written text, which can then be heard or read by the user 1.

[0099] Going more into detail, transforming the natural language input 29 into input data 27 for the virtual agent 2 may involve interpreting the speech or written text using a Natural Language Understanding (NLU) module 31 of the NLP model 30, and inferring the input data 27 using a Natural Language Inference (NLI) module 32 of the NLP model 30. Similarly, transforming the output data 28 from the virtual agent 2 into natural language output 41 may comprise transforming the output data 28 into speech or written text using a Natural Language Generation (NLG) module 33 of the NLP model 30.

[0100] In other cases, as also shown in Fig. 6, the user input 16 from the environment 3 means receiving an input action 34 from the user 1. In such cases the virtual agent 2 interprets the input action 34 using an Action Understanding module 37 of an Action Processing model 36 and infers the input data 27 using an Action Inference module 38 of the Action Processing model 36. At the end of the flow, generating the agent output 17 to the environment 3 includes receiving output data 28 from the virtual agent 2 and transforming the output data 28 into output action 35 using an Action Generation module 39 of the same Action Processing model 36.

[0101] Fig. 7 illustrates yet another example for implementing the virtual agent 2, wherein the environment 3 is a simulated virtual environment 23, and wherein the virtual agent 2 is implemented as a Non- Playable Character (NPC) 46 configured to perform actions and interact with an avatar 4 of a human user 1 within the simulated virtual environment 23.

[0102] This embodiment is useful in video games as well as other virtual simulation environments where one can accordingly encounter entities which include artificial intelligence and conversational abilities. As an example: a character in a video game can currently only say what the developers of the game have written and prepared for that character. With a conversational agent 2 as illustrated, the user 1 (player) could ask the NPC 46 (agent 2) virtually anything from their opinion on other characters to their childhood memories.

[0103] Within a video game a video game engine updates or draws from a series of game objects. From these modules the discussed method affects the game objects, and in particular, where the video game architecture includes a module for simulation 24 and (within it) NPCs, the method directly impacts this simulation module 24.

[0104] As shown in Fig. 7, the simulated virtual environment 23 can be generated by the simulation module 24 as part of a video game architecture 25, the simulated virtual environment 23 being defined by simulation parameters 26, wherein the simulation parameters 26 are configured to be updated through the interactions between the virtual agent 2 and the environment 3, the user input 16, and also manual settings.

[0105] Figs. 8 and 9 illustrate examples of embodiments of a simulated virtual environment 23 with an NPC 46 implementation of the virtual agent 2.

[0106] Disclaimer: all images inside of these diagrams are used to exemplify the architecture and purpose of the discussed methods. They belong to the video game: “The Witcher 3: Wild Hunt” by CD Projekt RED.

[0107] In Fig. 8 the first thing we wish to show is how the method would be applied in the software as observed from the systems inputs and outputs. In Fig. 8 we use the example of “the Pellar”, a character in the video game “The Witcher 3”. This character has a series of metadata (information connected to him) such as his backstory, his home and his pet. All NPC players in the Witcher games (and other games) have written dialogue that has been prepared by the developers beforehand. Our goal is for the conversations and interactions to be interactive. Rather than choosing from a pre-determined set of conversation options, the user 1 can say or do what he / she wishes and receive a genuine realistic response from the NPC character 46 according to the method described before.

[0108] Fig. 9 illustrates an example of how the virtual agent 2 according to the discussed embodiments could be used in the generation of dialogue (or actions) using a simplified architecture where different modules are described in oversimplified terms for easier understanding. In the illustrated example the dialogue between the NPC 46 (“the Pellar”) and the avatar 4 of the user 1 is triggered by an input text 29 and is affected by the environment 3, the basic metadata 10 and individual metadata 11 of the agent 2, as well as an “Individualistic Knowledge System” based on a dynamic knowledge base 8, historical context awareness 9 and further possible factors as described before.

[0109] Finally, Fig. 10 shows a schematic diagram of a hardware structure of a computer-based dialog system between a human user 1 and a virtual agent 2 according to an implementation. The system comprises an input-output interface 40, a memory 42, and a processor 43, as well as a communications interface 44 for enabling interaction between the human user 1 and the virtual agent 2, and an optional storage unit 45 for long-term data storage. All these components are in data connection via either a universal bus implemented as part of a computer, or a wired or wireless data connection to each other and / or other components implemented as part of a server 18 through the communications interface 44.

[0110] The processor 43 may be a general-purpose central processing unit (Central Processing Unit, CPU), a microprocessor, an application-specific integrated circuit (Application-Specific Integrated Circuit, ASIC), or one or more integrated circuits configured to control program execution of the solutions in this application. The processor 43 may alternatively include a plurality of CPUs, and the processor 43 may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. The processor 43 herein may be one or more devices, circuits, and / or processing cores configured to process data (for example, a computer program instruction). The memory 42 may be a read-only memory (read-only memory, ROM) or another type of static storage device that can store static information and an instruction; or a random access memory (random access memory, RAM) or another type of dynamic storage device that can store information and an instruction; or may be an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a compact disc read-only memory (Compact Disc Read-Only Memory, CD-ROM) or another compact disc storage medium, optical disc storage medium (including a compact disc, a laser disc, an optical disc, a digital versatile disc, a Blu-ray disc, or the like), a magnetic disk storage medium, another magnetic storage device, or any other medium that can be used to carry or store expected program code in a form of an instruction or a data structure and that can be accessed by a computer. This is not limited. The memory 42 may exist independently or connected to the processor 43 through a bus. The memory 42 may alternatively be integrated with the processor 43. The memory 42 is configured to store application program code for executing the solutions in this application, and the processor 43 controls the execution. The processor 43 is configured to execute the computer program code stored in the memory 42, to implement the human-computer interaction method as described in the embodiments of this application.

[0111] The communications interface 44 may be configured to communicate with another device or communications network, such as Ethernet or a wireless local area network (wireless local area networks, WLAN).

[0112] Generally, the word “module”, “unit,” or “block,” as used herein, refers to logic embodied in hardware or firmware, or to a collection of software instructions. A module, a unit, or a block described herein may be implemented as software and / or hardware and may be stored in any type of non-transitory computer-readable medium or other storage device. In some embodiments, a software module / unit / block may be compiled and linked into an executable program. It will be appreciated that software modules can be callable from other modules / units / blocks or from themselves, and / or may be invoked in response to detected events or interrupts. Software modules / units / blocks configured for execution on computing devices (e.g., processor 43 as illustrated in Fig. 10) may be provided on a computer readable medium, such as a compact disc, a digital video disc, a flash drive, a magnetic disc, or any other tangible medium, or as a digital download (and can be originally stored in a compressed or installable format that needs installation, decompression, or decryption prior to execution). Such software code may be stored, partially or fully, on a storage device of the executing computing device, for execution by the computing device. Software instructions may be embedded in a firmware, such as an EPROM. It will be further appreciated that hardware modules / units / blocks may be included of connected logic components, such as gates and flip-flops, and / or can be included of programmable units, such as programmable gate arrays or processors. The modules / units / blocks or computing device functionality described herein may be implemented as software modules / units / blocks, but may be represented in hardware or firmware. In general, the modules / units / blocks described herein refer to logical modules / units / blocks that may be combined with other modules / units / blocks or divided into sub- modules / sub-units / sub-blocks despite their physical organization or storage.

[0113] It will be understood that when a unit, module or engine is referred to as being “on”, “connected to” or “coupled to” another unit, module, or engine, it may be directly on, connected or coupled to, or communicate with the other unit, module, or engine, or an intervening unit, module, or engine may be present, unless the context clearly indicates otherwise. As used herein, the term “and / or” includes any and all combinations of one or more of the associated listed items.

[0114] The various aspects and implementations have been described in conjunction with various embodiments herein. However, other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed subject-matter, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measured cannot be used to advantage. A computer program may be stored / distributed on a suitable medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems.

[0115] The reference signs used in the claims shall not be construed as limiting the scope.

Claims

CLAIMS1. A human-computer interaction method, wherein the method may be applied to a computer-based dialog system between a human user (1) and a virtual agent (2), the method comprising: providing a virtual agent (2) comprising static characteristics (5), dynamic characteristics (6), and a dynamic knowledge base (8); the virtual agent (2) being configured to continuously interact with an environment (3) and, based on said interactions, update both said dynamic characteristics (6) and said dynamic knowledge base (8); receiving a user input (16) from said environment (3); determining, based on the user input (16), adjusted dynamic characteristics (7) for said virtual agent (2); and generating an agent output (17) to said environment (3); wherein generating said agent output (17) is based on both said static characteristics (5), said adjusted dynamic characteristics (7) and said dynamic knowledge base (8).

2. The method according to claim 1 , wherein generating said agent output (17) is based on historical context awareness (9) of said virtual agent (2); wherein said interactions between the virtual agent (2) and said environment (3) comprise historical interactions predating said user input (16) with at least one of said human user (1), other human users (1), other virtual agents (2) provided within said environment (3), and further variables of said environment (3).

3. The method according to any preceding claim, wherein said virtual agent (2) is an Al agent capable of sensing its environment (3) and taking actions to achieve a goal, the Al agent using a machine learning model trained using said interactions with the environment (3) and being configured to improve its own performance in achieving said goal.

4. The method according to any preceding claim, wherein the virtual agent (2) is implemented on a server (18), wherein the server (18) is configured to update said dynamic characteristics (6) and / or said dynamic knowledge base (8) via a communications interface (44) at least partially based onfederated learning data on interactions between human users (1) and virtual agents (2) within other environments (3).

5. The method according to any preceding claim, wherein said static characteristics (5) comprise basic metadata (10) related to at least one of an age, profession, residence, background or relationships of the virtual agent (2) with other virtual agents (2).

6. The method according to any preceding claim, wherein said dynamic characteristics (6) comprise individual metadata (11) related to- short-term states (12) comprising at least one of an emotional state such as ‘happy’, ‘sad’; mood; or instant reaction such as ‘surprise’, ‘fear’, or ‘excitement’, and- long-term states (13) comprising at least one of core personalities, identity, religious belief, political belief, culture, lifestyle, philosophy, mannerisms, quirks, habits, likes and dislikes, opinions, feelings or other long-term personal characteristics.

7. The method according to any preceding claim, wherein the virtual agent (2) is further configured to update at least one of said static characteristics (5), dynamic characteristics (6), and said dynamic knowledge base (8) through manual settings.

8. The method according to any preceding claim, wherein the virtual agent (2) further comprises human behavior variables (14) related to the dynamic knowledge base (8) and affecting the use of information from the dynamic knowledge base (8) by the virtual agent (2), said human behavior variables (14) comprising at least one of an- ignorant variable resulting in partial occlusion of information from the dynamic knowledge base (8);- deceitfulness variable resulting in obscuring or altering information from the dynamic knowledge base (8);- forgetfulness variable resulting in temporary or permanent deletion of information from the dynamic knowledge base (8);- learning variable resulting in temporary or permanent addition of true information to the dynamic knowledge base (8); or- beliefs variable resulting in temporary or permanent altering of information and / or addition of false information to the dynamic knowledge base (8).

9. The method according to any preceding claim, wherein the virtual agent (2) further comprises a feelings variable (15) related to at least one of said human user (1), other human users (1), or other virtual agents (2) within said environment (3); said feelings variable (15) being configured to imitate the development of human feelings and opinions based on said dynamic characteristics (6) of the virtual agent (2), said user input (16) and historical interactions predating said user input (16) with at least one of said human user (1), other human users (1), and other virtual agents (2) respectively; and wherein generating said agent output (17) is at least partially based on said feelings variable (15).

10. The method according to any of claims 1 to 9, wherein the environment (3) is a text-based virtual environment (19), wherein the virtual agent (2) is implemented as a chat bot, and wherein receiving said user input (16) from said environment (3) and generating said agent output (17) to said environment (3) comprises a text-based dialogue.

11. The method according to any of claims 1 to 9, wherein the environment (3) is a simulated virtual environment (23), and wherein the virtual agent (2) is implemented as a Non-Playable Character, NPC (46) configured to perform actions and interact with an avatar (4) of a human user (1) within said simulated virtual environment (23).

12. The method according to claim 11, wherein the simulated virtual environment (23) is generated by a simulation module (24) as part of a video game architecture (25), the simulated virtualenvironment (23) being defined by simulation parameters (26), wherein said simulation parameters (26) are configured to be updated through at least one of said interactions between the virtual agent (2) and said environment (3), said user input (16), and manual settings.

13. The method according to any of claims 1 to 9, wherein the environment (3) is a physical environment (20), and wherein the virtual agent (2) is implemented in a computer-based system (21) configured to sense changes within the physical environment (20) via at least one sensor (22), perform actions, and interact with the human user (1) via an input-output interface (40).

14. The method according to any preceding claim, wherein receiving said user input (16) from said environment (3) comprises receiving natural language input (29) in the form of speech or written text, and transforming said natural language input (29) using a Natural Language Processing, NLP, model (30) into input data (27) for said virtual agent (2); and wherein generating said agent output (17) to said environment (3) comprises receiving output data (28) from said virtual agent (2), and transforming said output data (28) using said NLP model (30) into natural language output (41) in the form of speech or written text.

15. The method according to claim 14, wherein transforming said natural language input (29) into input data (27) for said virtual agent (2) comprises interpreting said speech or written text using a Natural Language Understanding, NLU, module (31) of said NLP model (30), and inferring said input data (27) using a Natural Language Inference, NLI, module (32) of said NLP model (30); and wherein transforming said output data (28) from said virtual agent (2) into natural language output (41) comprises transforming said output data (28) into speech or written text using a Natural Language Generation, NLG, module (33) of said NLP model (30).

16. The method according to any preceding claim, whereinreceiving said user input (16) from said environment (3) comprises receiving an input action (34) from said user (1), interpreting said input action (34) using an Action Understanding module (37) of an Action Processing model (36), and inferring said input data (27) using an Action Inference module (38) of said Action Processing model (36); and wherein generating said agent output (17) to said environment (3) comprises receiving output data (28) from said virtual agent (2), and transforming said output data (28) into output action (35) using an Action Generation module (39) of said Action Processing model (36).

17. A computer-based dialog system between a human user (1) and a virtual agent (2), the system comprising an input-output interface (40), a memory (42), and a processor (43); the input-output interface (40) and the memory (42) being coupled to the processor (43), the memory (42) being configured to store computer program code, the computer program code comprising computer instructions; and the processor (43) being configured to read the computer instructions from the memory (42) and cause the system to perform the steps of the method according to any one of claims 1 to 16.

Images