Computer-implemented method for communicative interaction and communication system

Abstract

A computer-implemented method for communicative interaction of a human user with the virtual world through an output device that comprises a visual appearance (e.g. an avatar) outputting the results o

Description

Technical Field The invention relates to a computer-implemented method for communicative interaction and communication system, e.g., an avatar-based-system for interaction with human user and / or human customer with an improved user experience. Background Art Computer-implemented methods for communicative interaction of a human user with the virtual world regularly use dialogue-systems, like chatbots. These are becoming an increasingly popular online method for product and / or service providers to interact with their human customers and / or human users. Any visual appearance of the input and / or output-giving device like an avatar builds the communication bridge or the “interface” between the human user and the Al driven machine. Those avatars are a form of software agent and may be referred to as chatbots, chatterbots, artificial conversational entities, conversational agents, smart assistants, or other similar terms. An avatar-based-dialogue-system is a system that serves as a steering tool and / or mimics conversation, either by voice-, language- and / or text-based input with a user so as to provide them with information and / or services and also receive information and / or data from the user. Previously, machines, for example robots in industry, were steered by a web-based controllertool and / or user may have carried out this interaction via an app, for example on a mobile device or a website. Meanwhile, cloud software architecture is mostly based on heavy processing of enormous amounts of data. While interacting with the avatar in -e.g. industrial metaverse-, the clients, machines, and / or users do have to become familiar and comfortable with the reactions they experience from the machine. The so called “uncanny valley” effect is a hypothesized psychological and aesthetic relation between an object’s degree of resemblance to a human being and the emotional response to the object. Examples of the phenomenon exist among robotics, 3D computer animations and lifelike dolls. The uncanny valley is the region of negative emotional response towards robots that seem “almost” human. Movement, as the avatars usually show, amplifies the emotional response. The uncanny valley phenomenon can be described as an eerie and / or unsettling feeling that some people experience in response to not-quite-human figures like humanoid robots and lifelike computer-generated characters. Ultra-realistic avatars, androids and animated characters appear stuck in a disturbing inbetween world. They are so lifelike and yet they are not “right”. This void of strangeness is known as the uncanny valley. In the recently -12 Mar 2024 - updated IEEE Spectrum publication “What is the Uncanny Valley” by Rina Diane Caballar, she states that watching a lifelike digital being such as an avatar can be both fascinating and disconcerting. Strategies to avoid the uncanny valley effect in human-machine-interaction are, for example, not creating robots or computer animated characters that combine features on different sides of a boundary - for example, human and nonhuman, living and nonliving or real and artificial. Summary of Invention To accelerate digitalization in general and especially a smooth overall human machine interaction it is a common effort of all developers of chatbots, robotics and / or avatars to give the human client and / or user a feeling of a relaxed atmosphere during interaction. All efforts, of course aim to achieve that goal with as less of data packets and energy for processing as possible. The invention discloses a method and a chatbot communication system that ensures a good understanding and comfortable feeling for the human user, because it overcomes the uncanny valley effect at least partially by mirroring the human user’s behavior, gestures and attitude giving him an unconscious feeling to talk to a similar personality or at least to a person of the same cultural and social background, a person who simply understands. It has been proven that it is crucial for a successful human-machine-interaction or human-virtual-world-interaction that the human part feels comfortable in the interaction-situation. This also applies to effectiveness with regards to time and / or energy savings. The uncanny valley effect counteracts this effort. For - among others - economic and environmental purposes, it is therefore important to make human-machine-interaction as effective as possible, especially in regard to energy efficiency, sustainability, and other green and / or economic topics. For that, there is a need to improve the emotional aspects of human-machine-interaction and / or human-virtual-world-interaction as much as possible. However, there is still a gap between a comfortable and relaxed situation and the realistic feeling of humans interacting with avatars that are as realistic as possible, due to effects like the uncanny valley phenomenon. To get true and quick conversation and / or machine human interaction in - for example -chatbot communication system applications in general and the industrial metaverse in particular, it would be highly appreciated if interacting humans feel comfortable and calm while being in conversation with a machine and therefor it is an object of the present invention that in a chatbot communication system application, such as an avatar, the uncanny valley effect is as low as possible. It is accordingly the object of the present invention to provide a chatbot and / or avatar-based communication system application with a visual appearance of the machine in form of a human having an improved feel-good effect of visual counteraction of the machine while being in communication with a human user. It is therefore subject of the present invention to provide a computer-implemented method for communicative interaction of a human user with the virtual world through an output device that comprises a visual appearance outputting the results of the artificial intelligence within the virtual world, wherein the following process steps are realized: Tracking by real-world modules, sensors and / or recording devices personality features of the human user, - Generating data packets from tracking, - Computer-implemented-processing of the data packets with Al to analyze the tracked personality features, Generating results by the Al enabling the visual appearance of the output device to realize the same, corresponding and / or suitable personality features, Implementing these results into the visual appearance of the output-device of the communication system so that it mirrors and / or reflects the personality features of the human user. Besides, a chatbot communication system to carry out the above described computer-implemented method of communicative interaction of a computer using Al with a human user and / or human client is subject of the present invention; the system comprising devices, modules, tools and features that enable the system to record, recognize, analyze, reproduce and equip any kind of visual appearance of the input and / or output-giving device to mirror the body language, gestures, style, habit, speech and / or behavior of the human user and / or the human customer of the system. The general understanding of the present invention is that a visual appearance of an Al works smoother with a human user if the system pretends the human user would be in communication and / or interaction with at least a humanoid communication and / or interaction partner. To achieve that feeling for the human user the system comprises devices and computers to enable the system to present to the human user a humanoid visual appearance of the input and / or output-giving device that mirrors the human user’s behavior, style, or other characteristics. It is a competent knowledge that communication between communication partners has a better, more efficient, and smoother flow if the communication partners have less barriers to be open with one another. It is a basic recognition of the present invention to realize that creating a kind of similarity leads to a more open-minded human user. It has been found out that a visual appearance of the input and / or output-giving device of the communication system, like an avatar, will be more accepted as being a partner to be open with, if the avatar is deemed to be somehow of the same cultural background, the same manner, the same taste, social class, education, age, and sex. If the avatar arrives to appear similar to the human user, the latter one will be open-minded for any kind of communication and overcome the uncanny valley effect, since that will be small compared to the huge similarities. The invention warms up the human user because the avatar adapts to and / or reflects him during communication. This may be executed by mirroring the gestures, attitude, speech style and voice characteristics as well as by analyzing the style of small accessories on the human side. This may be consciously recognized by the human user or not. It is therefore an important improvement for all machine-communication using chatbots, avatars, etc. as visual appearance of the humanoid communication partner to be able to mirror and / or reflect the human communication partner in some way or another. Description of Embodiments Independent of the grammatical term usage, individuals with male, female or other gender identities are included within the term. Avatar is used as generally describing all kinds of visual appearance of the input and / or output-giving device of the communication system. According to an example of an embodiment of the present invention the communication system comprises a facial recognition tool like Face ID® developed by Apple Inc. for the iPhone. The tool allows biometric recognition of the face as well as head tracking, eye tracking and gesture recognition. The data generated by that tool will be fed to the Al of the communication system, e.g. the chatbot, in order to allow it to generate a humanoid avatar following the user’s gaze. According to another example of an embodiment of the present invention the communication system comprises one or more camera(s). These recording tools enable the communication system to track the body language of the human user, to mirror that body language or -alternatively - respond to the body language by the visual appearance of the input and / or output-giving device of the communication system. According to another example of an embodiment of the present invention the communication system comprises one or more microphones. These recording tools not only enable the communication system to record the words and text of the spoken communication of the human user - for e.g. natural language processing NLP - but also to track the non-verbal noises of the human user during the communication like sighing, laughing, breathing deeply, crepitation of fingers, crossing legs and so on. These unwanted and normally unconsciously expressed noises of the human user may give a perfect background sound of the virtual world for the communication. According to another embodiment of the present invention the system has a special camera including a software to analyze the user’s jewelry, clothing make-up, age and / or hairstyling. This means style, quality, style of appearance meaning how much makeup, how natural, how sportive and / or how stylish looking is the human user? Does he represent a rather romantic style, rather business style or without any style just blank without certain features of a specific personality hints? The same questions apply to the speech, gestures, attitude, attentiveness. Does the human user respond quickly or is he a rather reflecting type of person? All these questions are to be answered by the system through analyzing the human user. With the results of these analysis the system will adapt the humanoid visualization of the chatbot. According to another embodiment of the present invention the system will be trained during the interaction by analyzing not only the personality features referring to the style of the human user as it is, but also analyzing the reaction of the human user once being confronted with the avatar representing his style. This may be not always positive, because like looking in a mirror, what you see not always pleases you. So, the user’s reaction of this mirroring, the communication system tracks whether the user is delighted, e.g. whether he smiles, raises his head, opens his eyes wider than before, and so on and makes the avatar adapt to that response. The thereby tracked personality features do not necessarily have to be visual ones, maybe it is one or more wanted or unwanted noise(s) the user makes like “hugh” “ehm” all the time, or pulls up his nose etc. According to one embodiment an exemplary communication system according to the invention recognizes and / or analyzes certain signals of looks, like trendy looks in certain colors, business outfits etc. These small accessories as well as the type of clothing in general, personal styling, referring to makeup, hairstyling, haircoloring, jewelry may be recorded, analyzed, and compared to typical accessories. One exemplary communication system will equip the avatar, being the visual appearance of a chatbot - preferably in real-time - with some or all of these style features. This does not have to be exactly the same but could be belonging to the same type and thus showing to the human user that they do have same cultural background, e.g. similar behavior, similarities in the way of speaking, performance, clothing, gestures, humor, speech, habit, behavior and / or dialect. An exemplary avatar according to the present invention may for example stimulate the human counterpart to be open in communication and / or interaction and to show some personality features that are non-obvious. One example may be that the human user starts the communication for getting the information about movies downtown tonight, and the avatar promptly asks whether he is serious about that or whether he does not remember being angry about that stupid movie lately? So, the normal human user laughs showing his teeth and giving an example of his way to laugh about a joke. This is another example for an embodiment of the present invention where the communication system tracks and analyzes not only the human user’s personality features but also tracks and analyzes the human user’s reaction of the adapted and / or correspondingly styled avatar. If the reaction is positive, the avatar will go ahead showing this personality, behaviors and / or styling features, if the human user reacts negative, for example wrinkles his forehead, the avatar stops representing and / or showing those features. To get all data the system needs for the above-described analysis, the system shows several modules, components, and devices. For example, the face recognition device being incorporated into a camera tracking the overall body movements and attitude of the human user. The data packets being generated out of these camera videos will be sent and transferred to some computer, server and / or processor and some device with a neutral network running one or more integrated artificial intelligence(s). This may be the same processor as the chatbot of the communication system is running on, but it may also be a different one, gathering the data packets from the results of the Al processing of the data packets analysis. Finally, the processor or computer on which the chatbot runs gets the result from the Al analysis and incorporates these results while creating the avatar or another visual appearance of the input and / or outputgiving device. For example, recording just head and face of the human user, once the human user scratches his head, the avatar will scratch its head as well. Otherwise, the human user laughs, tells a joke, the avatar’s laughter will be created accordingly. Another example the human user breathes deeply, the avatar will do the same. Or the human user sighs, so the avatar will sigh as well. Instead of breathing deeply or sighing a nervous rubbing of fingers and / or lifting the eyebrows may be reflected by the avatar. For another example, regarding the body attitude, the human user may be attentive, relaxed, nervous, depressed, alert, aggressive, ironic, in a good mood etc... All these moods may correlate to certain features of body, face, speech, voice etc.... of the avatar which mirrors the human user’s body attitude and / or body language this way. The above-described conceivable features of embodiments of the present invention are all exemplary and may be realized supplementary one to the other or alternatively one or the other feature in different embodiments. Brief Description of the Drawings Fig. 1 shows the human user interacting with the chatbot communication system. Fig. 2 shows the monitor of the chatbot communication system representing an avatar acting like the human user and being dressed like him. Figure 1 shows a typical human user of a chatbot communication system 1. The user sits in front of a small desk 3, having a computer of the chatbot communication system with a monitor 2 on it. The small desk 3 has some devices incorporated, like sensors 4 (invisible) measuring the pressure and / or frequency of the user touching it. Besides, the monitor 2 comprises a camera, microphone and / or a facial recognition tool 5 to track the human user’s mimics and gestures generating thereby data packets which are send, stored, processed by a processor to enable an Al (not shown) to analyze the human user’s cultural background, mood, etc. The transfer of the data packets is not shown, it may be done by cable lines or wireless, for example by Bluetooth®. Some other recording devices may be somewhere in the room, around the desk and / or at the ceiling. All data packets generated are transferred to the Al in order to analyze what the user’s typical personal features are. As may be seen, the user 1 is sitting relaxed but attentive looking at the monitor 2 of his device in front of him. He is sitting in a casual way with one leg 6 stretched out and the other leg 7 being angled. The system has enough recoding devices spread around the working space of the human user 1 to capture all acoustic and / or visible signals the human user 1 is sending during his interaction with the chatbot communication system and generate data packets out of it. These data packets are suitable to be processed by an Al which is connected to the chatbot communication system to equip e.g. an avatar to show the same acoustic and / or visible signals while interacting with this human user 1. The data are used to equip a visual appearance of the chatbot answer, like an avatar with typical features that mirror the human user’s behavior, gestures, style, mood, cultural background, clothing, personal style etc... Figure 2 shows the visual appearance of the input and / or output-giving device of the chatbot communication system presenting the result of the human user’s interaction with the chatbot communication system. This visual appearance of the input and / or output-giving device of the communication system is present in the monitor 2 mirroring the scene of the human user sitting in figure 1. Obviously, the monitor 2 of figure 2 shows an avatar T sitting in front of a small desk 3’ like the human user 1 of figure 1 and mirroring his behavior attitude and clothing. The avatar T sits in front of a desk of the same kind as figure 1 shows, is dressed, styled, and equipped the same way as the human user 1 of figure 1. The avatar 1 ’ has the same chair as the human user in figure 1 and behaves like the human user of figure 1, with one leg 6 being stretched out and the other leg 7 being angled. The avatar sits as relaxed and attentive as the human user 1 of figure 1 does. The avatar mirrors the situation of the human user 1 of figure 1. In this case as shown, the avatar 1’ supports his elbow on the same side of the table as the human user 1 does, however, for another embodiment of mirroring the sides of the supported elbow of the avatar T may be changed. It may be easy realized that the mirroring of the human user’s gestures creates sympathy. According to psychological studies a person will always sympathize with a person with shared behaviorisms. Therefore, the chatbot communication system - according to one embodiment -will firstly begin the interaction with the user by intentionally triggering certain behaviors. For example, the chatbot communication system, dressed and styled matching to the human user may start telling a joke to find out how the human user laughs in order to recreate the same or similar kind of laughter on the avatar. Another example of the chatbot communication system would be that it starts with a sad anecdote to see how the user sympathizes and record the user’s reaction and mirror back same reaction when the human user explains a problem. The present invention takes the animation of an avatar, or a chatbot communication system one step further and actually mirrors and / or reflects and / or responds to the human user’s gestures or other personality features themselves, thus creating sympathy. Sympathy helps the system to establish a comfortable atmosphere for the human user while interacting with the communication system. This atmosphere enables the human user to communicate freely with the communication system. Such a freer human-machine-interaction makes the results of the communication more solid, more reliable, shortens the time of human-machine-communication and makes the interaction more effective and less time and / or energy - consuming.

Claims

1. A computer-implemented method for communicative interaction of a human user with the virtual world through an output device that comprises a visual appearance outputting the results of the artificial intelligence within the virtual world, wherein the following process steps are realized:Tracking by real-world modules, sensors and / or recording devices one or more personality features of the human user,- Generating data packets from tracking,- Computer-implemented-processing of the data packets with Al to analyze the tracked personality features, Generating results by the Al enabling the visual appearance of the input and / or output device to reproduce, produce and / or mirror one or more of the tracked, corresponding and / or suitable personality features, Implementing these results into the visual appearance of the input and / or output-device of the communication system so that it mirrors and / or responds to the personality features of the human user.

2. Computer-implemented method according to claim 1, wherein the visual appearance of the input and / or output device of the communication system is an avatar.

3. Computer-implemented method according to one of claims 1 or 2, further comprising the step of stimulating the human user to show personality features by the visual appearance of the input and / or output device of the communication system.

4. Computer-implemented method according to one of claims 1 or 2, wherein one or more personality features are tracked by a facial recognition tool.

5. Computer-implemented method according to one of the previous claims, wherein one or more personality features are tracked by one or more camera(s).

6. Computer-implemented method according to one of the previous claims, wherein one or more personality features are tracked by one or more microphone(s).

7. Computer-implemented method according to one of the previous claims, wherein the visual appearance shows one or more personality features by mirroring the human user’s gestures.

8. Computer-implemented method according to one of the previous claims, wherein the visual appearance shows one or more personality features by mirroring the human user’s way of speaking.

9. Computer-implemented human-machine- and / or human-virtual-world-communication system suitable to carry out the method according to one of claims 1 to 8, using artificial intelligence Al with a visual appearance of the input and / or output device, comprising one or more recording device(s) and / or software tools to record and / or recognize personality features of a human user while using the system, further comprising one or more device(s) to analyze, reproduce and equip visual appearance of the input and / or output device of the system to mirror, respond or correspond to the body language, gestures, style, habit, speech and / or behavior of the human user and / or the human client of the system.

10. Computer-implemented communication system according to claim 9, wherein at least one record device is a camera equipped with software for facial recognition.

Images