Embodied dialog and embodied speech authoring tools for use with an expressive social robot

Abstract

A social robot provides more believable, spontaneous, and understandable expressive communication via embodied communication capabilities by which a robot can express one or more of: paralinguistic audio expressions, sound effects or audio / vocal filters, expressive synthetic speech or pre-recorded speech, body movements and expressive gestures, body postures, lighting effects, aromas, and on-screen content, such as graphics, animations, photos, videos. These are coordinated with produced speech to enhance the expressiveness of the communication and non-verbal communication apart from speech communication.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS[0001]This application claims the benefit of priority of U.S. Provisional Patent Application No. 62 / 422,217, titled “EMBODIED DIALOG AND EMBODIED SPEECH AUTHORING TOOLS FOR USE WITH AN EXPRESSIVE SOCIAL ROBOT,” filed on Nov. 15, 2016, which is hereby incorporated by reference in its entirety.BACKGROUND[0002]A number of challenges exist for managing dialog between a social robot and a human. One of these is the difficulty in causing a robot to deliver expressions that convey emotion, tone, or expression in a way that seems authentic, believable and understandable, rather than what is commonly called “robotic.” By contrast, humans often convey speech together with non-language sounds, facial expressions, gestures, movements, and body postures that greatly increase expressiveness and improve the ability of other humans to understand and pay attention. A need exists for methods, devices, and systems that allow a social robot to convey these other e...

AI Technical Summary

Benefits of technology

This patent describes a technology that helps a social robot communicate more effectively by combining spoken words with other non-spoken cues like facial expressions and body language. This technology helps the robot produce a believable and emotionally rich speech output, making it easier to understand and connect with the robot on a social level. Overall, the technology described in this patent is called "embodied speech" and it helps to make social robots more effective at expressing their emotions and intentions through spoken words.

Problems solved by technology

A number of challenges exist for managing dialog between a social robot and a human.

One of these is the difficulty in causing a robot to deliver expressions that convey emotion, tone, or expression in a way that seems authentic, believable and understandable, rather than what is commonly called “robotic.” By contrast, humans often convey speech together with non-language sounds, facial expressions, gestures, movements, and body postures that greatly increase expressiveness and improve the ability of other humans to understand and pay attention.

Another challenge lies in the difficulty in causing a robot to convey expression that is appropriate for the context of the robot, such as based on the content of a dialog, the emotional state of a human, the state of an activity performed between human and robot, an internal state of the robot (e.g., related to the hardware state or software / computational state), or the current state of the environment of the robot.

Given this, an additional challenge is enabling a social robot to decide or learn how to lend expressive attributes that are synchronized with dialog with a person, or enabling a developer to author lines of expressive natural language utterances with coordinated multi-modal paralinguistic expressions.

Hence, another challenge is the development of such an authoring environment.

Note than an additional challenge exists if such pre-authored multi-modal expressive utterances must work in conjunction with a social robot that may also be making real-time decisions how to express an utterance.

Images