Dialogue system, method of operating such system, software service system, computer software and computer readable medium

Abstract

The present invention relates to a dialog system (e.g., an avatar-based system) with an improved user experience, a method of operating the system, a software service system, computer software, and a computer readable medium. The invention first shows means that enable machines, such as dialog systems, in particular avatar-based systems, to cover their defects and / or to be incapable and to hide their technical deficiencies. The objective is to maintain tracking of users and / or clients in a virtual and / or enhanced world while waiting for the end of data processing and / or response of an avatar-based system. Application of a progress bar is not always useful in this aspect, as it increasingly causes intolerance rather than endurance. According to the invention, the dialogue system appears as if it would be completely busy with a session with the user, while the dialogue system actually works and processes the data, and the output module gives only the so-called "fill".

Description

[0001] The present invention relates to a dialogue system (e.g., an avatar-based dialogue system) with an improved user experience, a method for operating the dialogue system, a software service system, computer software, and a computer-readable medium.

[0002] Conversational systems like chatbots are becoming an increasingly popular online method for product and service providers to interact with their customers and / or users. An avatar is a form of software agent and can be referred to as a chatbot, chat program, artificial conversational entity, conversational agent, intelligent assistant, or other such terms. An avatar-based conversational system is a system used as a manipulation tool and / or to mimic a conversation with a user through voice, language, and / or text-based input in order to provide them with information and services, and also to receive information from the user.

[0003] Previously, machines (such as robots in industry) were operated by web-based controller tools, and / or users may have performed this interaction via applications (such as on mobile devices or websites).

[0004] Meanwhile, cloud software architectures are largely based on the heavy processing of massive amounts of data. When interacting with avatars in the industrial metaverse, clients, machines, and / or users must be patient to receive answers and / or the results of data processing. Consider a game where the screen freezes and a progress bar appears during idle periods until processing is complete. However, this is not a proper user experience and fails to emphasize human-computer interaction by making users feel comfortable when interacting with a dialogue system.

[0005] Furthermore, there are issues with workflow and human focus. Once the dialogue system freezes the screen and / or displays a progress bar, the user is thrown back to reality, realizing they are waiting in the real world, for example, in their dirty chair. Therefore, during such an interruption, some, or even many, of the soft skills that enable the user to interact smoothly with the dialogue system will be lost in the workflow. After the interruption, the user must readjust to the demands of being mentally in a virtual or augmented world and / or interacting with the dialogue system again.

[0006] Additionally, if the user is represented by an avatar in virtual or augmented reality, such an interruption will cause the user to lose some or all of the emotions associated with their identification with the avatar. This is a detrimental effect, undermining many highly specialized techniques designed to help users identify with their avatar and use the dialogue system correctly, efficiently, and effectively. After the interruption, the dialogue system may take a considerable amount of time to regain those emotional and identification levels. This results in the dialogue system becoming significantly slower and less energy-efficient overall. Furthermore, this will consume a significant amount of time that could have been used for other purposes.

[0007] This becomes even more problematic when avatars and / or avatars within the industrial metaverse are represented in a 3D environment like a 3D industrial metaverse, because avatar-based systems will exhibit lag. In such cases, the user will have to wait, the workflow will be halted, and this waiting position will be visualized and made apparent by freezing the screen or displaying a progress bar. Once this occurs, the user is thrown back into the real world, into their actual environment, and temporarily loses much of the mental preparation for engaging with the machine. This skill must be rebuilt after the interruption, which is costly, time-consuming, laborious, and disruptive to the workflow.

[0008] Therefore, the purpose of this invention is to keep human users in a virtual and / or augmented world while waiting until the dialogue system has processed, generated, stored, received, and / or otherwise obtained the results and is able to provide a specific answer. Even more so, because it has been proven and found that staring at a progress bar completely destroys immersion.

[0009] Therefore, one subject of the present invention is to provide a dialogue system including means of an output module or means connected to an output module to issue fillers during the dialogue system’s operation and / or processing of data in response to user input and / or output belonging to a session.

[0010] In short, the present invention replaces progress bars and / or frozen screens indicating the progress of a machine (especially a dialogue system) with some filler actions, gestures and / or words from the output module.

[0011] The output modules of a dialogue system can be robots, chatbots, screens, speakers, robots represented by Barbie or some other icon, and any technically feasible combination thereof.

[0012] According to a preferred aspect of the invention, the dialogue system is, for example, an avatar-based dialogue system, wherein the dialogue system includes means connected to or integrated into an output module (such as a speaker, a screen with an avatar, a talking robot, a 3D avatar, a chatbot, etc.), which emit or provide fillers, such as approving sounds, general statements, words somewhat related to the ongoing conversation, questions, groans, sighs, head scratching, ear rubbing, laughter, or some movement of the whole image or part (such as an arm), while the dialogue system is processing data and / or disposing of output results belonging to the conversation.

[0013] Furthermore, another subject of the present invention is to provide a method for operating a dialogue system, the method comprising the following steps: -Initiate a conversation between the user and the dialogue system. - The dialogue system receives user input through the input module. - Read user input from the dialogue system via processor and / or network-based and / or via Internet of Things (IoT) and / or via Artificial Intelligence (AI). - Process data through a dialogue system - Send out filler text as output of the dialogue system. - By processing the results generated by the dialogue system - The output module provides the output calculated and processed by the dialogue system as the result of AI data processing and processor operations.

[0014] According to a preferred embodiment of the method for operating a dialogue system, the method includes the following steps: - Start the conversation between the user and the dialogue system through the input module. - Dialogue systems receive user input via input modules (such as microphones). - The dialogue system reads user input through a processor and / or via a network-based and / or via the Internet of Things (IoT) and / or via artificial intelligence (AI). - Dialogue system processes data -Optionally, the dialogue system estimates the duration of data processing. - Optionally, the dialog system selects an appropriate filler from the filler list—suitable for the following (optionally individually or in combination). -user -Time required -The sessions involved, -The location where the conversation took place -When the conversation took place (winter-summer-Christmas...) - After selection, the output module emits filler, whichever is foreseeable and / or appropriate. - After processing, the dialogue system displays the results and / or provides outputs as the results calculated by the dialogue system through output modules (e.g., avatars).

[0015] A software service system comprising a dialogue system, computer software, and a computer-readable medium for operating and executing the computer software is a further subject of the present invention.

[0016] The general idea of ​​this invention is to maintain an emotional connection between the user and the dialogue system through "buying time" fillers, as long as the dialogue system is in contact with the user while it is working or processing data. According to a preferred embodiment of the invention, the filler can be implemented to fit into the interruption in a timely manner. The general idea of ​​this invention is to improve the emotional user experience of human-computer interaction in dialogue systems (e.g., avatar-based systems like chatbot systems) by enabling chatbots to react with appropriate fillers, much like humans, when they have nothing to say at some point during a conversation in which the dialogue system is engaged. Essentially, the advantage of this invention is that it creates dialogue systems, particularly avatar-based dialogue systems, including fillers provided by tools, methods and / or features of the dialogue system's output modules, to draw the user's attention to things other than the fact that the dialogue system is working and the user is waiting (e.g., for data to process).

[0017] The term "dialogue system" as used in this specification can refer to a conversational system, a chatbot system, and / or a conversational agent "CA". This is a computer-based system designed to converse with humans. Dialogue systems can communicate on both input and output channels using one or more of the following modes: text, voice, graphics, haptic feedback, gestures, and / or others. Modules of a dialogue system are—for example, but not limited to—[…]. - Input and output modules, processing modules, storage modules, access and connectivity to the Internet of Things (IoT), a computer with neural networks that apply artificial intelligence during sessions, and optional connectivity to various applications, smartphone applications, and / or cloud applications. One or more processors are part of the aforementioned "processor module". - A connection to a device configured to work with and / or process artificial intelligence (AI), - One or more automatic speech recognizers / generators - One or more pose recognizers / generators - One or more handwriting recognition / generators - One or more natural language understanding / generation units - One or more text-to-speech engines - One or more heads or avatars that speak like talking robots.

[0018] According to an advantageous method of operating the dialogue system according to the invention, the dialogue system is capable of and can be configured to estimate the duration of data processing.

[0019] According to the method disclosed above, it is advantageous if the method further includes a process step in which the dialogue system preselects an appropriate filler from one or more lists of fillers stored in a memory unit.

[0020] The memory unit stores lists of fillers within the dialogue system and / or its output modules. Each list contains different fillers rated and / or sorted under different topics or items, representing attributes of the fillers, such as "this filler took 1.7 minutes." These lists can be pre-selected objects implemented by the dialogue system or combined with, for example, a web-based dialogue system.

[0021] Each indenter may be subject to one or more constraints from these lists, thus focusing on different properties of the corresponding indenter.

[0022] There are many ways to design the "filler" according to the invention. Essentially, the filler is the engine of the dialogue system, maintaining interaction with the user while the dialogue system is processing user input and lacks the capacity to receive further input. The filler keeps the output module responsive to the user, keeping it busy in virtual or augmented reality, but it does not provide useful results from the dialogue system's data processing. The filler emits any content stored in its memory and selected by the dialogue system and its corresponding output module, either planned or simultaneous.

[0023] Some examples of filler text: An avatar—for example, a tool in the output module of a dialogue system—is a chatbot. - Basically, yes, but... Let me see... -let me see…… -I understand what you mean, um... What can we do? -I know, How could that be? Isn't that great? I can't believe it.

[0024] According to the present invention, output systems such as avatars use session fillers and gestures to replace progress bars. What the user says is irrelevant. The present invention attempts to cover the processing time required for the backend loading system's response output (such as the avatar's statement). For example, if an avatar is to recite a Shakespearean sonnet, its loading takes several seconds, and then it needs to be converted to an audio file, and then the system needs to calculate the right-lip synchronization of the poem, and all of this must happen before the avatar can actually recite the poem.

[0025] Therefore, there will be a noticeable waiting period for the user. For this use case, the system according to the invention will introduce so-called "filler" statements, noise, and / or gestures, which are stored or retrieved, for example, in or from a simple memory within the output module of a processor such as an avatar, such as "Hmm, let me see if I remember correctly, yes, basically, ... no—what is that, oh yes, I see, ... accompanied by gestures such as scratching one's head—through this simple but effective technique of the output module, the human user will feel that the session is uninterrupted. For example, without the involvement of AI, there is only a series of noises and simple phrases emitted by the avatar to fill the time between the user's last input and / or the actual output of the system (the corresponding avatar).

[0026] While the system is calculating and processing data, another infill can change the mode of the output module to another mode, characterized in that the "other" mode includes a simple talking robot whose voice output includes, for example, meaningless utterances.

[0027] If a chatbot system has an avatar acting as the user's interactor, it will buy time just as would happen with human interaction. For example, simply by using fillers such as "humph" and / or "haha," and using words similar to those listed above. Such a system is, for example, called "avatar-based."

[0028] An avatar is a user's online alteration of themselves; in other words, it is a digital representation of a human being in a virtual environment.

[0029] Based on higher-level fillers, the avatar might ask simple questions, such as giving the impression that some further input is lacking, for example: Could you repeat that specific item in more detail? Did you immediately realize that it was a damage? -What are other people's comments on this action? -Any questions about dogs? - Can you believe this? Isn't that great? According to an advantageous embodiment of the invention, the filler displays the posture of the avatar, such as... a) Regarding head and face posture - Scratch its head Looking up at the sky - Move your eyes, - Indicates a posture of reflection and thought. -imitate - Movement, such as the movement of the hands, tongue, lips, eyes, etc. - Touch its nose - Use its hair to do things, such as the hair backwards or upwards. - Quickly type something on a device (such as a smartphone) - Write something down - Shake your head left and right, shake your head up and down - Nodding and grinning -Regarding body posture, such as stretching, moving arms and legs, and moving shoulders, b) Regarding hand and nail posture c) Moving around in the environment.

[0030] According to another aspect of the invention, the filler includes speech, gestures, groans, sighs, laughter, movements and / or imitations, and any combination of the aforementioned outputs.

[0031] According to another advantageous embodiment of the invention, the incorporation is technically represented by a list of different infills, which are rated according to specific attributes. Those attributes may relate to how much time the infill is gaining and / or what tool the infill is operating, such as speech only or speech combined with head movement.

[0032] The attributes used to select fillers can also involve tools for avatars related to time, date, weather, location, and / or the user's cultural / religious background.

[0033] A list with fillers can display a series of fillers that can be selected based on the topic of the conversation. Therefore, the fillers can contain statements about general topics related to the input data, such as: - Did I understand correctly? Your goal is to get the right food and park the car in the shade. Another series of fillers can be displayed to entertain the user's attributes. Therefore, fillers can select some keywords from the aforementioned conversation or user input, and tell interesting stories, jokes, puns, and / or riddles related to them. In this embodiment of the invention, the fillers can have connectivity to the Internet of Things (IoT).

[0034] Another filler can have attributes that allow it to talk in general about weather, politics, sports, kindergarten, school, general children's education, current events, cultural and / or religious events (such as Christmas), seasons, plants, pets, fashion, etc.

[0035] Another filler can speak by combining the actions of the avatar, such as... - Gestures, such as nodding or shaking your head, looking up and saying "The weather is nice in Auckland today" or something similar. - Wave your arms, scratch your nose or rub your ears, look up and down, show some pictures or objects. -Dancing, playing music, singing, playing musical instruments - Imitation, like a smile, tells you what you might theoretically be supposed to do next... For all infills that involve the location, time, and environment or place where a user stays, the infill can be connected to sensors and / or systems to identify those locations and / or environments, as well as cloud applications and / or IoT to obtain contextual information about it.

[0036] Another preferred embodiment could depict the avatar meeting other people in the surrounding area. Therefore, an avatar representing a chatbot or user might encounter other people in virtual or augmented reality, asking them questions that just fill the time, as shown in the example below: Don't you feel hot today? How did the basketball team play last night? Do you have children in school? Oh my god, Christmas is always so exciting! The dialogue system allows other avatars in the virtual / augmented reality to respond to the user's avatar in a comparable manner.

[0037] Therefore, users are entertained by conversations between avatars and other avatars during the processing time of the dialogue system, without losing track of the virtual or augmented reality.

[0038] According to one embodiment of the invention, a dialogue system implemented as a chatbot system selects fillers, for example, based on the amount of time they are "gaining". Technically, with a baseline regarding the waiting time for outputs, results, and / or data points, the system can automatically pre-select fillers based on how much time they will "gain".

[0039] In this embodiment of the invention, before selecting a filler, the operation module and / or method of the dialogue system roughly estimates the time required for a specific processing procedure of a particular session.

[0040] In order for the system to select an appropriate filler, according to a specific embodiment of the invention, the dialogue system can analyze whether the user is quite aggressive, patient, or in other emotional states.

[0041] This analysis can be performed in different ways, such as by analyzing the sound of the voice, the number of keys used on the keyboard during a specific period of time, the pressure applied to the keys, etc. Therefore, the dialogue system of this embodiment of the invention does indeed have means for analyzing the user's emotions and feelings during a conversation, such as gestures, voice, and tone recognition, which are integrated into, for example, a microphone, especially when combined with a processor configured to analyze the user's voice using NLP (Natural Language Processing).

[0042] One of the main advantages of this invention is that it can replace and / or complete the progress bar in an avatar-based system by using one or more fillers.

[0043] The term "filler" as used in this specification can be a set of data and / or software code that causes a dialogue system (especially its output module, preferably having an avatar) to emit the following: -General statement, -The words spoken are somehow related to the ongoing conversation. -question, - Groans, sighs, - Postures without a specific meaning, such as scratching the head, rubbing the ears, grinning, laughing, etc.

[0044] This invention demonstrates for the first time a means to enable machines (such as dialogue systems, especially avatar-based systems) to cover up their defects and / or incompetence and conceal their technical deficiencies. The aim is to maintain the user and / or client's tracking in the virtual and / or augmented world while awaiting the completion of data processing and / or the response from the avatar-based system. The application of progress bars is not always useful in this regard, as it increasingly induces impatience rather than patience. According to the invention, the dialogue system behaves as if it were completely busy with the conversation with the user, while the dialogue system actually works and processes data, and the output module only provides so-called "filler".

[0045] Individuals with male, female, or other gender identities are included in this term, independent of the use of grammatical terms.

Claims

1. A dialogue system, comprising means of an output module or means connected to an output module, for issuing fillers during the time when the dialogue system is working and / or processing data in response to user input and / or output belonging to a session.

2. The dialogue system according to claim 1, having a means for issuing filler material integrated in the output module.

3. The dialogue system according to claim 1 or 2, having means for dispensing filler connected to the output module.

4. The dialogue system according to any one of the preceding claims, including a chatbot system.

5. The dialogue system according to any one of the preceding claims, comprising a session agent "CA".

6. The dialogue system according to any one of the preceding claims is avatar-based.

7. The dialogue system according to any one of the preceding claims, wherein the filler includes any combination of speech, gestures, groans, sighs, laughter, movements and / or imitations, and the aforementioned outputs.

8. The dialogue system according to any one of the preceding claims, wherein the avatar is technically a list of different fillers, which are listed and / or rated according to specific attributes.

9. The dialogue system according to any one of the preceding claims, having a baseline for the waiting period for output.

10. The dialogue system of claim 9, using a baseline to preselect a suitable filler that takes or gains the required amount of time.

11. A method for operating a dialogue system, the method comprising the following steps: -Initiate a conversation between the user and the dialogue system. - The dialogue system receives user input through the input module. - Read user input from the dialogue system via processor and / or network-based and / or via Internet of Things (IoT) and / or via Artificial Intelligence (AI). - Process data through a dialogue system - Send out filler text as output of the dialogue system. - By processing the results generated by the dialogue system - The output module provides the output calculated and processed by the dialogue system as the result of AI data processing and processor operations.

12. The method of operating a dialogue system according to claim 11, wherein the dialogue system is capable of and configured to estimate the duration of data processing.

13. The method of operating a dialogue system according to any one of claims 11 or 12, comprising the process step of pre-selecting an appropriate filler from a list of fillers stored in a memory unit via the dialogue system.

14. A software service system comprising a dialogue system according to any one of claims 1 to 10.

15. Computer software, when executed by a computer, causes the computer to provide a dialog system using fillers through an output module as described in any one of claims 1 to 10.

16. A computer software that, when executed by a computer, causes the computer to perform the method according to any one of claims 11 to 14.

17. A computer-readable medium comprising the computer software as described in claim 15 or 16.