Presenting facial expressions in virtual meetings

Abstract

Embodiment systems and methods for presenting facial expressions in a virtual meeting can include detecting a user facial expression of a user based on information received from a sensor of a computing device, determining whether the detected user facial expression is approved to be presented on an avatar in the virtual meeting, generating an avatar exhibiting a facial expression consistent with the detected user facial expression in response to determining that the detected user facial expression is approved to be presented on the avatar in the virtual meeting, generating an avatar exhibiting an approved presented facial expression in response to determining that the detected user facial expression is not approved to be presented on the avatar in the virtual meeting, and presenting the generated avatar in the virtual meeting.

Description

[0001] Related applications

[0002] This application claims the benefit of priority to U.S. Patent Application No. 17 / 320,627, filed May 14, 2021, entitled “Presenting A Facial Expression In AVirtual Meeting”, the entire contents of which are incorporated herein by reference.

[0003] background

[0004] Communication networks enable the development of applications and services for online meetings and gatherings. Some systems provide virtual environments that present visual representations of participants, known as "avatars," ranging from simple or cartoonish images to realistic depictions. Some of these systems include virtual reality (VR) devices (such as VR headsets or other VR equipment) that record the user's movements and speech. Such systems can generate facial expressions for the user's avatar based on the user's movements and speech.

[0005] However, the facial expressions, speech, and actions of virtual meeting participants may be irrelevant to the meeting. Users may react to any number of things happening in their real-world environment, such as disturbances from children, pets, or other people, external noise, phone calls, and other distractions. Furthermore, the system may detect and display expressions that users do not wish to show to others in the online meeting, such as anger, annoyance, frustration, etc. Additionally, the system may not accurately capture and display expressions, resulting in a mismatch between the expression a user wants to convey and the displayed expression (which may be awkward, confused, or offensive). Current systems do not provide users with a mechanism to review or approve facial expressions displayed on their avatars.

[0006] Overview

[0007] The aspects may include methods for presenting facial expressions in a virtual meeting and computing devices configured to perform these methods. The aspects may include: detecting a user's facial expressions based on information received from sensors of the computing device; determining whether the detected user facial expressions are approved for presentation in an avatar in the virtual meeting; generating an avatar displaying facial expressions consistent with the detected user facial expressions in response to determining that the detected user facial expressions are approved for presentation in an avatar in the virtual meeting; generating an avatar displaying facial expressions approved for presentation in an avatar in the virtual meeting but different from the detected user facial expressions in response to determining that the detected user facial expressions are not approved for presentation in an avatar in the virtual meeting; and presenting the generated avatar in the virtual meeting. Some aspects may further include: continuing to present the currently presented avatar in response to determining that the detected user facial expressions are not approved for presentation in an avatar in the virtual meeting. Some aspects may further include: generating an avatar displaying facial expressions approved for presentation in an avatar in the virtual meeting but different from the detected user facial expressions in response to determining that the detected user facial expressions are not approved for presentation in an avatar in the virtual meeting.

[0008] In some aspects, generating an avatar displaying facial expressions approved for presentation in the virtual meeting but different from the detected user facial expressions, in response to determining that the detected user facial expressions are not approved for presentation in the avatar, may include generating an avatar displaying the recently approved facial expressions. In some aspects, detecting a user's facial expressions based on information received from sensors of a computing device may include detecting user facial expressions based on information received from an image sensor of the computing device.

[0009] In some aspects, determining whether a detected user facial expression is approved for presentation in an avatar in a virtual meeting may include: determining whether the user facial expression was previously approved for presentation in an avatar in a virtual meeting; and presenting the generated avatar in the virtual meeting may include: in response to determining that the user facial expression was previously approved for presentation in an avatar in a virtual meeting, presenting the generated avatar displaying the previously approved facial expression in the virtual meeting. In some aspects, determining whether a detected user facial expression is approved for presentation in an avatar in a virtual meeting may include: displaying an avatar displaying a facial expression consistent with the detected user facial expression on a user interface configured to receive user approval or rejection; and determining that the detected user facial expression is approved in response to receiving input on the user interface of a computing device indicating that the user facial expression is approved for presentation in the virtual meeting.

[0010] Some aspects may include: determining that a detected user facial expression is approved for presentation in an avatar during a virtual meeting if there is no responsive input to the user interface within a threshold time period. Other aspects may include: storing in memory, in response to receiving input on the user interface of a computing device indicating whether a detected user facial expression is approved or not approved for presentation in an avatar during the virtual meeting, an indication that an avatar displaying a facial expression consistent with the detected user facial expression is approved or not approved for presentation in the virtual meeting.

[0011] In some aspects, determining whether a detected user facial expression is approved for display on an avatar in a virtual meeting may include: determining whether the detected user facial expression is stored in a preset list as approved or not approved; displaying an avatar with a facial expression consistent with the detected user facial expression on a user interface configured to receive user approval or rejection; and updating the preset list in response to receiving input that indicates, but is different from the preset list, that the user facial expression is approved or not approved for display on an avatar in the virtual meeting.

[0012] In some aspects, determining whether a detected user facial expression is approved for presentation in an avatar during a virtual meeting may include: determining whether a detected user facial expression is approved for presentation in an avatar during a virtual meeting based on the user's expressive voice. In some aspects, presenting a generated avatar in a virtual meeting may include: displaying a representation of the user's expressive voice in the virtual meeting in conjunction with presenting the generated avatar in the virtual meeting.

[0013] A further aspect may include a computing device including a memory and a processor coupled to the memory and configured with processor-executable instructions for performing operations of any of the methods described above. A further aspect may include a processor-readable storage medium storing the processor-executable instructions thereon, the instructions being configured to cause a controller of the computing device to perform operations of any of the methods described above. A further aspect may include a computing device including means for performing functions of any of the methods described above. Brief description of the attached diagram

[0015] The accompanying drawings, which are incorporated herein and form part of this specification, illustrate various exemplary embodiments and, together with the general description given above and the detailed description given below, are used to explain the features of some embodiments.

[0016] Figure 1 This is a system block diagram illustrating an example communication system suitable for implementing any of the various embodiments.

[0017] Figure 2This is a component block diagram of an example computing device suitable for implementing any of the embodiments in the various embodiments.

[0018] Figure 3 This is a component block diagram illustrating an example computing system architecture suitable for implementing any of the various embodiments.

[0019] Figure 4 These are conceptual diagrams illustrating various aspects of methods for presenting facial expressions in a virtual meeting, according to various embodiments.

[0020] Figure 5 This is a flowchart illustrating a method for presenting facial expressions in a virtual meeting according to various embodiments.

[0021] Figure 6A and 6B Operations that can be performed as part of a method for presenting facial expressions in a virtual meeting, according to various embodiments, are explained.

[0022] Detailed description

[0023] Various embodiments will be described in detail with reference to the accompanying drawings. Where possible, the same reference numerals will be used throughout the drawings to refer to the same or similar parts. References to specific examples and implementations are for illustrative purposes and are not intended to limit the scope of the various embodiments or claims.

[0024] Various embodiments provide methods for displaying facial expressions deemed appropriate by a participant on a participant avatar in a virtual meeting, methods that can be implemented in a device such as a mobile computing device. These embodiments enable the computing device to learn facial expressions that the user approves and disapprove of being displayed on a user avatar in a virtual meeting. These embodiments eliminate the need for additional bulky and expensive peripheral equipment (such as virtual reality headsets and similar devices). These embodiments improve the operation of the computing device and the virtual meeting system by implementing automatic filtering of facial expressions displayed on the participant avatar, thereby improving the conduct of the virtual meeting.

[0025] The terms “component,” “module,” “system,” and similar terms are intended to include computer-related entities such as, but not limited to, hardware, firmware, combinations of hardware and software, software, or software in execution configured to perform a particular operation or function. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and / or a computer. By extension, both an application running on a computing device and the computing device itself can be referred to as a component. One or more components may reside within a process and / or a thread of execution, and components may be localized on a single processor or core and / or distributed across two or more processors or cores. Furthermore, these components may execute from various non-transitory computer-readable media on which various instructions and / or data structures are stored. Components may communicate via local and / or remote processes, function or procedure calls, electronic signals, data packets, memory reads / writes, and other known computer, processor, and / or process-related communication methods.

[0026] The term “computing device” is used herein to refer to any or all of the following: cellular phones, smartphones, portable computing devices, personal or mobile multimedia players, laptops, tablets, smartbooks, ultrabooks, handheld computers, email receivers, Internet-enabled multimedia cellular phones, router devices, medical devices and equipment, biosensors / devices, wearable devices (including smartwatches, smart clothing, smart glasses, smart wristbands, smart jewelry (e.g., smart rings, smart bracelets, etc.)), entertainment devices (e.g., game controllers, music and video players, satellite radios, etc.), Internet of Things (IoT) devices (including smart meters / sensors, industrial manufacturing equipment, large and small machines and appliances for home or business use), computing devices within autonomous and semi-autonomous vehicles, mobile devices attached to or incorporated into various mobile platforms, GPS devices, and similar electronic devices including memory and programmable processors.

[0027] The term "System-on-a-Chip" (SOC) is used herein to refer to a single integrated circuit (IC) chip containing multiple resources and / or processors integrated on a single substrate. A single SOC may contain circuitry for digital, analog, mixed-signal, and radio frequency functions. A single SOC may also include any number of general-purpose and / or special-purpose processors (digital signal processors, modem processors, video processors, etc.), memory blocks (e.g., ROM, RAM, flash memory, etc.), and resources (e.g., timers, voltage regulators, oscillators, etc.). Each SOC may also include software for controlling the integrated resources and processors, as well as software for controlling peripheral devices.

[0028] The term "System-in-Package" (SIP) may be used herein to refer to a single module or package containing multiple resources, computing units, cores and / or processors on two or more IC chips, a substrate, or a System-on-a-Chip (SoC). For example, a SIP may comprise a single substrate on which multiple IC chips or semiconductor dies are stacked in a vertical configuration. Similarly, a SIP may comprise one or more multi-chip modules (MCMs) on which multiple ICs or semiconductor dies are packaged into a unified substrate. A SIP may also comprise multiple independent SoCs coupled together and packaged adjacently (e.g., on a single motherboard or in a single wireless device) via high-speed communication circuitry. The proximity of the SoCs facilitates high-speed communication and the sharing of memory and resources.

[0029] As used herein, the terms “network,” “system,” “wireless network,” “cellular network,” and “wireless communication network” can be used interchangeably to refer to part or all of an operator’s wireless network associated with a wireless device and / or a subscription on that wireless device. The techniques described herein can be used in a variety of wireless communication networks, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), FDMA, Orthogonal FDMA (OFDMA), Single Carrier FDMA (SC-FDMA), and others. Generally, any number of wireless networks can be deployed in a given geographic area. Each wireless network can support at least one radio access technology, which can operate on one or more frequencies or frequency ranges. For example, a CDMA network can implement Universal Terrestrial Radio Access (UTRA) (including the Wideband Code Division Multiple Access (WCDMA) standard), CDMA2000 (including the IS-2000, IS-95, and / or IS-856 standards), etc. In another example, a TDMA network can implement Global System for Mobile Communications (GSM), GSM Evolution Enhanced Data Rate (EDGE). In another example, OFDMA networks can implement Evolved UTRA (E-UTRA) (including the LTE standard), IEEE 802.11 (WiFi), IEEE 802.16 (WiMAX), IEEE 802.20, and... References to wireless networks using the LTE standard may be made, and therefore the terms "Evolved Universal Terrestrial Radio Access," "E-UTRAN," and "eNodeB" may be used interchangeably herein to refer to wireless networks. However, such references are provided merely as examples and are not intended to exclude wireless networks using other communication standards. For example, while various third-generation (3G), fourth-generation (4G), and fifth-generation (5G) systems are discussed herein, those systems are cited merely as examples and may be replaced by future-generation systems (e.g., sixth-generation (6G) or higher) in various examples.

[0030] Some online meeting and party systems offer virtual environments that present participants as visual "avatars" rather than simple name tags or live images. These avatars can range from simple or cartoonish images to lifelike depictions. Some systems supporting virtual meetings include VR devices that record a user's movements and speech, and then generate facial expressions on the user's avatar based on those movements and words.

[0031] As mentioned above, in many situations, the facial expressions, speech, and actions of virtual meeting participants may be irrelevant to the meeting. For example, users may react to any number of things happening in their real-world environment, such as disturbances from children, pets, or other people, external noise, phone calls, and other distractions. Furthermore, participants in virtual meetings may sometimes react negatively or unprofessionally to what is said or presented, and these participants should ideally avoid displaying inappropriate facial expressions that they do not want to be shown to others in the online meeting. For example, meeting participants may not want their meeting avatars to display fleeting expressions such as anger, annoyance, or frustration. Additionally, virtual meeting software may sometimes misunderstand or inaccurately detect participants' facial expressions, leading to a mismatch between the expression a participant wants to convey in the meeting and the expression displayed on their avatar (which may be embarrassing, confusing, or insulting to other meeting participants).

[0032] Various embodiments include methods for displaying facial expressions on participant avatars in a virtual meeting and computing devices configured to perform the methods, which include filtering out or avoiding displaying expressions that the participant does not wish to display on the participant avatar. In various embodiments, the computing device may detect user facial expressions or emotions based on information received from one or more sensors (e.g., a camera) of the computing device. The computing device may determine whether the detected user facial expressions are approved for display on participant avatars in the virtual meeting. In response to determining that the detected user facial expressions are approved for display on avatars in the virtual meeting, the computing device may generate an avatar displaying facial expressions consistent with the detected user facial expressions. In response to determining that the detected user facial expressions are not approved for display on avatars in the virtual meeting, the computing device may avoid displaying facial expressions consistent with the detected user facial expressions on the participant avatar, such as by generating an avatar displaying approved facial expressions and / or not changing the expressions currently displayed on the participant avatar. In some embodiments, in response to determining that the detected user facial expressions are not approved for display on avatars in the virtual meeting, the computing device may generate an avatar displaying recent or most recent approved facial expressions. In some embodiments, the computing device may generate an avatar that displays expressions consistent with recently or most recently approved facial expressions. The computing device may present the avatar and the generated representation of the user's facial expressions in a virtual meeting. In various embodiments, the facial expressions presented on the avatar may include expressions that affirmatively convey meaning or emotion (e.g., joy, sadness, surprise, anger, contempt, disgust, fear, etc.), expressive gestures (e.g., laughing, sighing, blinking, smiling, panting, etc.), neutral expressions, "resting" faces, or "no" expressions.

[0033] In some embodiments, the computing device may detect a user's facial expressions based on images and / or information received from the computing device's image sensors. In some embodiments, the computing device may process images of the user's face (which may be a single image or video) to detect facial expressions. In some embodiments, the computing device may include a multi-mode sensor or a group of sensors, including an image sensor to capture images or videos of the user's face. In various embodiments, the system does not use or requires additional devices, such as VR devices.

[0034] In some embodiments, the computing device may also store and process speech and / or other meaningful sounds, and determine speech, phrases, and sounds that accompany or correspond to certain facial expressions. In some embodiments, the computing device may use a microphone or another suitable audio sensor to capture a user's speech and other expressive sounds (sighs, laughter, etc.). In some embodiments, the computing device may detect a user's facial expressions based on, for example, expressive sounds received via the user's microphone. The term "expressive sounds" is used herein to refer collectively to speech and phrases (i.e., language) as well as meaningful sounds. Examples of meaningful sounds include laughter, sighs, hums, coughs, hesitant noises (e.g., "um," "uh," "oh," "mm," etc.), panting, or another sound that conveys meaning to another person. In some embodiments, the computing device may correlate or associate virtual facial expressions with a user's expressive sounds. In some embodiments, the computing device may learn speech, phrases, and / or sounds that are associated with or related to a user's various facial expressions.

[0035] In some embodiments, the computing device may automatically determine whether a generated virtual facial expression should be displayed on a user avatar presented in a virtual meeting. In some embodiments, the system may compare the generated virtual facial expression with a database, list, or other suitable data structure of permitted or acceptable expressions (e.g., smiling, laughing, working, etc.) and / or a database of unacceptable expressions (e.g., anger, frowning, disgust, etc.), and if the generated virtual facial expression is on the list of permitted or acceptable expressions, the computing device may display that expression on the user avatar. The lists of permitted or acceptable expressions and / or unacceptable expressions may be pre-configured in the computing device, such as in a default or preset list provided by the manufacturer or in a user-selected or generated list established during setup procedures (e.g., in software related to virtual meetings).

[0036] In some embodiments, the computing device may learn and / or adjust a database, list, or other suitable data structure of permitted or acceptable facial expressions and / or a list of prohibited or unacceptable facial expressions through user interaction. For example, the computing device may determine that a detected facial expression is neither approved nor unapproved (e.g., in a preset list or a database or list of previously approved or rejected facial expressions). In response to determining that a detected user facial expression is neither approved nor unapproved (in other words, no decision has been made regarding the acceptability of the detected facial expression), the computing device may present an avatar on the user interface of the computing device that displays an expression consistent with the user facial expression detected in a user interface configured to receive approval or disapproval user input. In some embodiments, the computing device may display on the user interface of the computing device an avatar that displays an expression consistent with the detected user facial expression, along with an invitation or prompt to the user to indicate whether the detected user facial expression is approved for presentation. In response to receiving input on the user interface of the computing device indicating that a user facial expression is approved for presentation in a virtual meeting, the computing device may determine that the detected user facial expression is approved. In some embodiments, the computing device may store instructions regarding whether an avatar displaying a user's facial expressions is approved, such as listing the avatar's facial expressions in a database or list of approved facial expressions or in a database or list of rejected facial expressions.

[0037] In some embodiments, the computing device may display virtual facial expressions in the user interface to allow the user to input whether or not to approve the displayed virtual facial expressions, and may give the user a short period of time (e.g., 3-5 seconds) to respond to the user interface with an approval or rejection selection. This preview opportunity allows the user to filter or screen facial expressions before they appear on the avatar in the virtual meeting. To allow the user to approve or reject the displayed virtual facial expressions without input each time they make a facial expression on the user interface, the system may take a default action if the user does not respond within a short period of time (such as before the expiration of a threshold time period). The default decision to approve or reject the virtual facial expressions presented in the user interface may be user-defined (e.g., the user can access settings in the configuration interface to choose whether to ignore user interface prompts as approval or rejection).

[0038] In some embodiments, the computing device may determine whether a generated virtual facial expression should be displayed on the user's avatar based on the user's expressive voice. For example, the user's expressive voice may be associated with unacceptable expressions, such as disdainful and contemptuous facial expressions, or laughter and surprised or suspicious facial expressions. In some embodiments, the computing device may determine whether a detected user facial expression is approved for display on the user's avatar based on the user's expressive voice. For example, the computing device may determine that the user's expressive voice is associated with unacceptable expressions. In response, instead of an unapproved facial expression, the computing device may generate an avatar displaying an approved (or previously approved) facial expression.

[0039] In some embodiments, the computing device may be configured to: learn over time (e.g., through a learning algorithm) user facial expressions that the user wishes to display and expressions that the user does not wish to display (i.e., acceptable and unacceptable expressions). The computing device may also be configured to: learn acceptable or suitable user facial expressions when the image of the user's face is blurred. The computing device may also be configured to: learn expressive sounds associated with approved and unapproved facial expressions. In some embodiments, the computing device may be configured to: determine whether a detected user facial expression has previously been approved for presentation on an avatar in a virtual meeting (e.g., based on a database, list, etc.), and in response to determining that the detected user facial expression has previously been approved for presentation on an avatar in a virtual meeting, the computing device may automatically generate an avatar displaying facial expressions consistent with the detected user facial expressions. As mentioned above, the computing device may confirm and / or update a preset or default facial expression in response to receiving input that differs from a preset or default facial expression. In this way, the computing device may learn acceptable or unacceptable user facial expressions to update preset facial expressions or update the database of acceptable and unacceptable facial expressions, and to add new facial expressions.

[0040] In this way, computing devices can learn to automatically determine whether to display virtual facial expressions without prompting the user to respond. Over time, such as in response to several consistent user decisions, the computing device can be configured to reduce the number or frequency of prompts regarding approval or disapproval. Ultimately, the computing device will stop prompting the user for expressions that are always accepted or never accepted, and will immediately display accepted expressions without disturbing the user with prompts to approve or disapprove of specific virtual avatar expressions.

[0041] Various embodiments improve the operation of computing devices and virtual conferencing systems by filtering or preventing the generation of unacceptable avatar facial expressions in virtual meetings. Various embodiments also improve the operation of computing devices and virtual conferencing systems by learning various approved and unapproved facial expressions presented in virtual meetings, and thus operating in an increasingly seamless manner requiring less and less user prompting over time. Various embodiments further improve the operation of computing devices and virtual conferencing systems by improving their speed and efficiency. Various embodiments further improve the operation of computing devices and virtual conferencing systems by improving the speed and efficiency of their operability and user interactivity.

[0042] Figure 1 This is a system block diagram illustrating an example communication system 100. Communication system 100 can be a 5G New Radio (NR) network, or any other suitable network (such as a Long Term Evolution (LTE) network). Although Figure 1 The 5G network has been explained, but subsequent networks may include the same or similar elements. Therefore, the references to 5G networks and 5G network elements in the following description are for illustrative purposes and are not intended to be limiting.

[0043] Communication system 100 may include a heterogeneous network architecture, which includes a core network 140 and various wireless devices. Figure 1 The components are referred to as wireless devices 120a-120e. The communication system 100 may also include several base stations (referred to as BS110a, BS110b, BS110c, and BS110d) and other network entities. A base station is an entity that communicates with the wireless devices and may also be referred to as a B-node, an LTE evolved B-node (eNodeB or eNB), an access point (AP), a radio headend, a transmit / receive point (TRP), a new radio base station (NR BS), a 5G B-node (NB), a next-generation B-node (g B-node or gNB), and so on. Each base station provides communication coverage for a specific geographic area. In 3GPP, the term "cell" can refer to the coverage area of ​​a base station, or a base station subsystem serving that coverage area, or a combination thereof, depending on the context in which the term is used. The core network 140 can be any type of core network, such as an LTE core network (e.g., an evolved packet core (EPC) network), a 5G core network, etc.

[0044] Base stations 110a-110d can provide communication coverage for macrocells, picocells, femtocells, another type of cell, or combinations thereof. Macrocells can cover a relatively large geographic area (e.g., a radius of several kilometers) and allow unrestricted access by wireless devices with service subscriptions. Picocells can cover a relatively small geographic area and allow unrestricted access by wireless devices with service subscriptions. Femtocells can cover a relatively small geographic area (e.g., a residential area) and allow restricted access by wireless devices associated with that femtocell (e.g., wireless devices in a closed subscriber group (CSG)). A base station used for a macrocell may be referred to as a macro BS. A base station used for a picocell may be referred to as a pico BS. A base station used for a femtocell may be referred to as a femtocell BS or a home BS. Figure 1 In the example described, base station 110a can be a macro BS for macro cell 102a, base station 110b can be a pico BS for pico cell 102b, and base station 110c can be a femto BS for femto cell 102c. Base stations 110a-110d can support one or more (e.g., three) cells. The terms “eNB,” “base station,” “NR BS,” “gNB,” “TRP,” “AP,” “B node,” “5G NB,” and “cell” are used interchangeably herein.

[0045] In some examples, the cell may not be stationary, and the geographical area of ​​the cell may move depending on the location of the mobile base station. In some examples, base stations 110a-110d may be interconnected with each other and to one or more other base stations or network nodes (not described) in the communication system 100 via various types of backhaul interfaces (such as direct physical connections, virtual networks, or combinations thereof using any suitable transport networks).

[0046] Base stations 110a-110d can communicate with the core network 140 via wired or wireless communication link 126. Wireless devices 120a-120e can communicate with base stations 110a-110d via wireless communication link 122.

[0047] The wired communication link 126 can use various wired networks (such as Ethernet, TV cable, telephone, fiber optic, and other forms of physical network connection) that can use one or more wired communication protocols, such as Ethernet, point-to-point protocol, high-level data link control (HDLC), advanced data communication control protocol (ADCCP), and transmission control protocol / internet protocol (TCP / IP).

[0048] The communication system 100 may also include relay stations (such as relay BS110d). A relay station is an entity capable of receiving data transmissions from an upstream station (e.g., a base station or wireless device) and transmitting those data transmissions to a downstream station (e.g., a wireless device or base station). A relay station can also be a wireless device capable of relaying transmissions for other wireless devices. Figure 1 In the example described, relay station 110d can communicate with macro base station 110a and wireless device 120d to facilitate communication between base station 110a and wireless device 120d. A relay station can also be referred to as a relay base station, relay, relay, etc.

[0049] The communication system 100 can be a heterogeneous network comprising different types of base stations (e.g., macro base stations, pico base stations, femto base stations, relay base stations, etc.). These different types of base stations may have different transmit power levels, different coverage areas, and different effects on interference in the communication system 100. For example, macro base stations may have high transmit power levels (e.g., 5 to 40 watts), while pico base stations, femto base stations, and relay base stations may have lower transmit power levels (e.g., 0.1 to 2 watts).

[0050] Network controller 130 can be coupled to a set of base stations and can provide coordination and control over these base stations. Network controller 130 can communicate with the base stations via backhaul. Base stations can also communicate with each other directly or indirectly, for example, via wireless or wired backhaul.

[0051] Wireless devices 120a, 120b, and 120c can be distributed throughout the communication system 100, and each wireless device can be stationary or mobile. Wireless devices may also be referred to as access terminals, terminals, mobile stations, subscriber units, stations, user equipment (UE), etc.

[0052] Macro base station 110a can communicate with communication network 140 on wired or wireless communication link 126. Wireless devices 120a, 120b, and 120c can communicate with base stations 110a-110d on wireless communication link 122.

[0053] Wireless communication links 122 and 124 may include multiple carrier signals, frequencies, or frequency bands, each of which may include multiple logical channels. Wireless communication links 122 and 124 may utilize one or more radio access technologies (RATs). Examples of RATs that can be used in wireless communication links include: 3GPP LTE, 3G, 4G, 5G (such as NR), GSM, Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Microwave Access Global Interoperability (WiMAX), Time Division Multiple Access (TDMA), and other mobile phone communication technology cellular RATs. Other examples of RATs that can be used in one or more of the various wireless communication links within communication system 100 include mid-range protocols (such as Wi-Fi, LTE-U, LTE-Direct, LAA, MuLTEfire) and relatively short-range RATs (such as ZigBee, Bluetooth, and Bluetooth Low Energy (LE)).

[0054] Some wireless networks (e.g., LTE) utilize Orthogonal Frequency Division Multiplexing (OFDM) on the downlink and Single-Carrier Frequency Division Multiplexing (SC-FDM) on the uplink. OFDM and SC-FDM divide the system bandwidth into multiple (K) orthogonal subcarriers, often referred to as frequency modulation, frequency slots, etc. Each subcarrier can be modulated with data. Generally, modulation symbols are transmitted in the frequency domain for OFDM and in the time domain for SC-FDM. The spacing between adjacent subcarriers can be fixed, and the total number of subcarriers (K) can depend on the system bandwidth. For example, the subcarrier spacing could be 15 kHz, and the minimum resource allocation (called a "resource block") could be 12 subcarriers (or 180 kHz). Therefore, for system bandwidths of 1.25, 2.5, 5, 10, or 20 MHz, the nominal Fast Fourier Transform (FFT) size could be 128, 256, 512, 1024, or 2048, respectively. The system bandwidth can also be divided into subbands. For example, a subband can cover 1.08 MHz (i.e., 6 resource blocks), and for system bandwidths of 1.25, 2.5, 5, 10, or 20 MHz, there can be 1, 2, 4, 8, or 16 subbands, respectively.

[0055] While some implementations may use terminology and examples associated with LTE technology, some implementations are applicable to other wireless communication systems, such as New Radio (NR) or 5G networks. NR can utilize OFDM with a cyclic prefix (CP) on both the uplink (UL) and downlink (DL) and includes support for half-duplex operation using Time Division Duplex (TDD). A single component carrier bandwidth of 100 MHz can be supported. NR resource blocks can span 12 subcarriers with a subcarrier bandwidth of 75 kHz over a duration of 0.1 milliseconds (ms). Each radio frame can include 50 subframes with a length of 10 ms. Therefore, each subframe can have a length of 0.2 ms. Each subframe can indicate the link direction for data transmission (i.e., DL or UL), and the link direction of each subframe can be dynamically switched. Each subframe can include DL / UL data as well as DL / UL control data. Beamforming can be supported, and beam direction can be dynamically configured. Precoded multiple-input multiple-output (MIMO) transmission can also be supported. The MIMO configuration in DL can support up to 8 transmit antennas (with up to 8 streams of multilayer DL transmission) and up to 2 streams per wireless device. Multilayer transmission with up to 2 streams per wireless device is supported.

[0056] Up to eight serving cells can be used to support the aggregation of multiple cells. Alternatively, NR can support different air interfaces in addition to the OFDM-based air interface.

[0057] Some wireless devices can be considered Machine-Type Communication (MTC) or Evolved or Enhanced Machine-Type Communication (eMTC) wireless devices. MTC and eMTC wireless devices include, for example, robots, drones, remote devices, sensors, meters, monitors, location tags, etc., which can communicate with a base station, another device (e.g., a remote device), or some other entity. Wireless computing platforms can provide connectivity to or to a network (e.g., a wide area network, such as the Internet) or a cellular network, for example, via wired or wireless communication links. Some wireless devices can be considered Internet of Things (IoT) devices, or can be implemented as NB-IoT (Narrowband Internet of Things) devices. Wireless devices 120a-120e can be included within a housing that houses the components of wireless devices 120a-120e, such as processor components, memory components, similar components, or combinations thereof.

[0058] Generally, any number of communication systems and wireless networks can be deployed in a given geographical area. Each communication system and wireless network can support a specific Radio Access Technology (RAT) and can operate on one or more frequencies. A RAT can also be referred to as a radio technology, air interface, etc. A frequency can also be referred to as a carrier, frequency channel, etc. Each frequency can support a single RAT in a given geographical area to avoid interference between communication systems using different RATs. In some cases, 4G / LTE and / or 5G / NR RAT networks can be deployed. For example, a 5G Non-Self-Aggressive (NSA) network can use a 4G / LTE RAT on the 4G / LTE RAN side of a 5G NSA network, and simultaneously use a 5G / NR RAT on the 5G / NR RAN side of the 5G NSA network. The 4G / LTE RAN and 5G / NR RAN can be interconnected and connected to the 4G / LTE core network (e.g., an evolved packet core (EPC) network) in the 5G NSA network. Other example network configurations may include a 5G self-reliant (SA) network, in which the 5G / NR RAN is connected to the 5G core network.

[0059] In some implementations, two or more wireless devices (e.g., referred to as wireless device 120a and wireless device 120e) may communicate directly using one or more sidelink channels (e.g., without using base stations 110a-110d as an intermediary). For example, wireless devices 120a-120e may communicate using peer-to-peer (P2P) communication, device-to-device (D2D) communication, vehicle-to-everything (V2X) protocols (which may include vehicle-to-vehicle (V2V) protocols, vehicle-to-infrastructure (V2I) protocols, or similar protocols), mesh networks, or similar networks, or combinations thereof. In this scenario, wireless devices 120a-120e may perform scheduling operations, resource selection operations, and other operations described elsewhere herein as performed by base stations 110a-110d.

[0060] Figure 2 This is a component block diagram of an example computing device 200 suitable for implementing any of the various embodiments. (Refer to...) Figure 1 and 2The computing device 200 may include a first system-on-a-chip (SOC) processor 202 (such as an SOC-CPU) coupled to a second SOC 204 (such as a 5G-capable SOC). The first and second SOCs 202, 204 may be coupled to internal memories 206, 216, a display 212, and a speaker 214. Additionally, the computing device 200 may include an antenna 218 for transmitting and receiving electromagnetic radiation that can be connected to a wireless data link, and / or a wireless transceiver 208 coupled to one or more processors in the first and / or second SOCs 202, 204. The one or more processors may be configured to determine the signal strength level of the signal received by the antenna 218. The computing device 200 may also include menu selection buttons or a joystick switch 220 for receiving user input. Additionally, soft virtual buttons may be displayed on the display 212 for receiving user input.

[0061] The computing device 200 may also include a sound codec (CODEC) circuit 210 that digitizes sound received from a microphone into data packets suitable for wireless transmission and decodes the received sound data packets to generate an analog signal provided to a speaker to produce sound. Furthermore, one or more of the processors in the first and second SOCs 202, 204, the wireless transceiver 208, and the CODEC 210 may include digital signal processor (DSP) circuitry (not shown separately). The computing device 200 may also include one or more optical sensors 222, such as a camera. The optical sensors 222 may be coupled to one or more processors in the first and / or second SOCs 202, 204 to control the operation of the optical sensors 222 and receive information (e.g., images, video, etc.) from the optical sensors 222.

[0062] The processor of computing device 200 (e.g., SOC 202, 204) can be any programmable microprocessor, microcomputer, or one or more multiprocessor chips that can be configured via software instructions (applications) to perform various functions, including those described in the various embodiments below. In some wireless devices, multiple processors may be provided, such as one processor within SOC 204 dedicated to wireless communication functions and another within SOC 202 dedicated to running other applications. Typically, software applications including processor-executable instructions may be stored in a non-transient processor-readable storage medium (such as memory 206, 216), which are then accessed and loaded into the processor. Processors 202, 204 may include internal memory sufficient to store application software instructions. Mobile devices 102a-102e may also include optical sensors, such as cameras (not shown).

[0063] Figure 3This is a component block diagram illustrating an example computing system 300 architecture suitable for implementing any of the various embodiments. (Refer to...) Figure 1-3 Various embodiments may be implemented on several single-processor and multi-processor computer systems, including system-on-a-chip (SOC) or system-in-package (SIP) systems. The computing system 300 may include two SOCs 302 and 304, a clock 306, and a voltage regulator 308. In some embodiments, the first SOC 302 operates as the central processing unit (CPU) of a wireless device, executing instructions by performing arithmetic, logic, control, and input / output (I / O) operations specified by instructions from a software application. In some embodiments, the second SOC 304 may operate as a dedicated processing unit. For example, the second SOC 304 may operate as a dedicated 5G processing unit responsible for managing high-capacity, high-speed (e.g., 5Gbps) and / or ultra-high frequency shortwave length (e.g., 28GHz millimeter-wave (mmWave) spectrum) communications.

[0064] The first SOC 302 may include a digital signal processor (DSP) 310, a modem processor 312, a graphics processor 314, an application processor 316, one or more coprocessors 318 (e.g., vector coprocessors) connected to one or more of these processors, memory 320, a custom circuit system 322, system components and resources 324, an interconnect / bus module 326, one or more temperature sensors 330, a thermal management unit 332, and a thermal power envelope (TPE) component 334. The second SOC 304 may include a 5G modem processor 352, a power management unit 354, an interconnect / bus module 364, multiple millimeter-wave transceivers 356, memory 358, and various additional processors 360 (such as application processors, packet processors, etc.).

[0065] Each processor 310, 312, 314, 316, 318, 352, 360 may include one or more cores, and each processor / core may perform operations independently of other processors / cores. For example, the first SOC 302 may include a processor running a first type of operating system (e.g., FreeBSD, LINUX, OS X, etc.) and a processor running a second type of operating system (e.g., MICROSOFT WINDOWS 10). Additionally, any or all of processors 310, 312, 314, 316, 318, 352, 360 may be included as part of a processor cluster architecture (e.g., synchronous processor cluster architecture, asynchronous or heterogeneous processor cluster architecture, etc.).

[0066] The first and second SOCs 302 and 304 may include various system components, resources, and custom circuitry for managing sensor data, analog-to-digital conversion, wireless data transmission, and performing other specialized operations, such as decoding data packets and processing encoded audio and video signals for display in a web browser. For example, the system components and resources 324 of the first SOC 302 may include power amplifiers, voltage regulators, oscillators, phase-locked loops, peripheral bridges, data controllers, memory controllers, system controllers, access ports, timers, and other similar components used to support processors and software clients running on wireless devices. System components and resources 324 and / or custom circuitry 322 may also include circuitry for interfacing with peripheral devices, such as cameras, electronic displays, wireless communication devices, external memory chips, etc.

[0067] The first and second SOCs 302 and 304 can communicate via interconnect / bus module 350. Various processors 310, 312, 314, 316, and 318 can be interconnected via interconnect / bus module 326 to one or more memory elements 320, system components and resources 324, custom circuitry 322, and thermal management unit 332. Similarly, processor 352 can be interconnected via interconnect / bus module 364 to power management unit 354, millimeter-wave transceiver 356, memory 358, and various additional processors 360. Interconnect / bus modules 326, 350, and 364 may include arrays of reconfigurable logic gates and / or implement bus architectures (e.g., CoreConnect, AMBA, etc.). Communication can be provided by advanced interconnects such as high-performance on-chip networks (NoC).

[0068] The first and / or second SOCs 302, 304 may further include input / output modules (not described) for communicating with external resources (such as clock 306 and voltage regulator 308). External resources (e.g., clock 306, voltage regulator 308) may be shared by two or more internal SOC processors / cores.

[0069] In addition to the example computing system 300 discussed above, various embodiments can also be implemented in a wide variety of computing systems, which may include a single processor, multiple processors, multi-core processors, or any combination thereof.

[0070] Figure 4 This is a conceptual diagram illustrating various aspects of a method 400 for presenting facial expressions in a virtual meeting, according to various embodiments. (Refer to...) Figure 1-4The computing device 402 may be configured to participate in the virtual meeting 430 via a communication network or communication system (the aspects of which are discussed above with reference to communication system 100). In some embodiments, the virtual meeting 430 may include one or more avatars 432, 434 of various users, each of whom may participate via a computing device (such as computing device 402).

[0071] The computing device 402 may include one or more image sensors (such as a camera 404) and one or more sound sensors (such as a microphone 406). The camera 404 may be directed to capture images, such as an image of a user's face 410. The microphone 406 may capture one or more expressive sounds 412 of the user.

[0072] Computing device 402 may be configured to detect a user's facial expressions based on information received from sensors (such as camera 404 and / or microphone 406) of computing device 402. Computing device 402 may be further configured to generate a representation of the user's facial expressions 420. In some embodiments, the representation of the user's facial expressions 420 may include one or more images that may be presented on or incorporated into a user avatar 432 in a virtual meeting 430. In some embodiments, the representation of the user's facial expressions 420 may include one or more instructions, processed by main memory or a device executing the virtual meeting 430, for presenting the user's facial expressions on an avatar in the virtual meeting 430. The representation of the user's facial expressions 420 may be presented on the avatar within the virtual meeting 430 in a manner perceptible to other participants in the virtual meeting 430 (e.g., associated with avatar 434).

[0073] The computing device 402 may also be configured to generate a representation of expressive sound 422. In various embodiments, the representation of expressive sound 422 may be an audio file, a digital bitstream, or another suitable representation. In some embodiments, the representation of expressive sound 422 may be presented in conjunction with a representation of a user's facial expressions 420 in a virtual meeting 430.

[0074] As further described below, computing device 402 may be configured to determine whether a detected user facial expression is approved or not displayed in the virtual meeting 430. In some embodiments, computing device 402 may be configured to display the detected user facial expression for user approval. For example, computing device 402 may be configured to generate a prompt 412, which may be displayed on an avatar, for example, on a display device (such as a screen). The prompt may include one or more user interface elements that enable the user to provide input approving or disapproving / rejecting the display of the facial expression on the avatar in the virtual meeting 430. Computing device 402 may store indications of whether the user facial expression is approved or not displayed on the avatar in response to receiving input approving or disapproving the display of the user facial expression. In some embodiments, if a detected facial expression is preset to be approved or not displayed on the avatar, the computing device may update the preset in response to user input that differs from the preset. In this way, computing device 402 can learn and / or adjust the list of permitted or acceptable expressions and / or the list of unacceptable expressions through user interaction.

[0075] Furthermore, computing device 402 can learn to automatically determine whether to display virtual facial expressions without prompting the user to respond. In some embodiments, the detected expression may be one that the user has previously approved or rejected on the avatar 432 in the virtual meeting 430. In such embodiments, computing device 402 may bypass or not present prompt 412 and may proceed to sending a representation of the approved user facial expression 420 for display on the avatar 432 in the virtual meeting 430. In some embodiments, computing device 402 may determine that the detected facial expression was previously unapproved or previously rejected by the user. In such embodiments, computing device 402 may generate a representation of the approved or already approved facial expression for display on the avatar 432 in the virtual meeting 430.

[0076] Figure 5 This is a flowchart illustrating a method 500 for presenting facial expressions in a virtual meeting according to various embodiments. (Refer to...) Figure 1-5 Method 500 can be implemented by a processor (e.g., 202, 204, 310-318, 352, 360) of a computing device (e.g., 120a-120e, 200, 404).

[0077] In block 502, the processor may detect a user's facial expressions based on information received from sensors of the computing device. In some embodiments, the processor may detect a user's facial expressions based on information received from a camera (e.g., camera 404) of the computing device. In some embodiments, the processor may detect a user's facial expressions based on information received from a microphone (e.g., microphone 406) of the computing device. In some embodiments, the processor may detect a user's expressive voice and may use that expressive voice to detect the user's facial expressions.

[0078] In decision box 504, the processor may determine whether the detected user facial expression is approved for presentation on the avatar in the virtual meeting. In some embodiments, the processor may determine whether the detected user facial expression was previously approved for presentation on the avatar in the virtual meeting. In some embodiments, the processor may determine whether the detected user facial expression is included in a list of pre-approved or pre-disapproved facial expressions (such as a default or preset list). In some embodiments, the processor may determine whether the detected user facial expression is approved for presentation on the avatar in the virtual meeting based on the user's expressive voice. In some embodiments, the expressive voice may be related to or associated with the facial expression.

[0079] In response to determining that a detected user facial expression is not approved for display on an avatar in a virtual meeting (i.e., decision box 504 = "No"), the processor may generate an avatar in box 506 displaying an approved facial expression that differs from the detected user facial expression. In some embodiments, the processor may generate an avatar displaying a recently approved expression on the user's meeting avatar. In some embodiments, the processor may generate an avatar displaying the most recently approved user facial expression. In some embodiments, the processor may generate an avatar displaying the most recently approved user facial expression that is consistent with or closest to the detected user facial expression. In some embodiments, the processor may maintain the facial expression displayed on the avatar before detecting the user facial expression; in other words, it continues to display the currently displayed avatar.

[0080] In response to determining that the detected user facial expression is approved to be presented in the avatar in the virtual meeting (i.e., decision box 504 = "Yes"), the processor can generate an avatar that displays facial expressions consistent with the detected user facial expressions.

[0081] After performing the operation of box 506 or 508, the processor may, in optional box 510, store in memory an instruction regarding whether the avatar displaying the user's facial expressions has been approved or not, in response to receiving input on the user interface of the computing device instructing the user's facial expressions to be approved or not displayed in the virtual meeting.

[0082] In box 512, the processor can present the avatar and a representation of the generated user facial expressions in the virtual meeting. For example, the processor can send a representation of user facial expression 420 to be presented on user avatar 432 in virtual meeting 430.

[0083] In option 514, the processor may display a representation of the user's expressive voice in a virtual meeting in conjunction with the presentation of the generated avatar (i.e., a representation of the generated avatar and the user's facial expressions). For example, the processor may display a representation of expressive voice 422 in a virtual meeting (e.g., 430) in conjunction with the presentation of the generated avatar.

[0084] The processor can periodically or continuously execute method 500 by detecting the user's next facial expression again in box 502 and performing operations as described throughout boxes 504-512 of the virtual meeting.

[0085] Figure 6A and 6B Operations 600a and 600b, according to various embodiments, that can be performed as part of a method 500 for presenting facial expressions in a virtual meeting are explained. See also... Figures 1-6B Operations 600a and 600b can be implemented by the processors (e.g., 202, 204, 310-318, 352, 360) of computing devices (e.g., 120a-120e, 200, 404).

[0086] Reference Figure 6A In execution box 502 ( Figure 5 Following the operation, the processor can display an avatar of a facial expression consistent with the detected user facial expression on a user interface configured to receive user approval or rejection in box 602. For example, the processor can generate a prompt (e.g., 412) configured to receive input of the avatar facial expression presented for approval or rejection of a user interface element.

[0087] In decision box 602, the processor can determine whether input has been received in response to the user interface approving or rejecting the presented avatar facial expression.

[0088] In response to receiving input of an avatar facial expression that indicates rejection (i.e., decision box 604 = "reject"), the processor may perform the operation of box 506 of method 500 as described.

[0089] In response to receiving an input of an approved facial expression (i.e., decision box 604 = "approved"), the processor may perform the operation of box 508 of method 500 as described.

[0090] If there is no responsive input to the user interface during a predetermined time period (TH) (i.e., decision box 604 = "No response during the TH time period"), the processor may perform a default action in box 606. In some embodiments, the processor may determine that the presented avatar facial expression is approved for use in the virtual meeting and perform the action in box 508, as described, if there is no responsive input to the user interface during a threshold time period. In some embodiments, the processor may determine that the presented avatar facial expression is not approved for use in the virtual meeting and perform the action in box 506, as described, if there is no responsive input to the user interface during a threshold time period.

[0091] Reference Figure 6B After the operation of block 502 of method 500 is performed, in some embodiments, the processor may update the preset list based on user input that approves or rejects / disapproves facial expressions presented on the user's persona in the virtual meeting.

[0092] For example, in box 610, the processor can determine whether the presented avatar facial expression is stored in a preset list as approved or not approved.

[0093] In box 612, the processor can display an avatar of a facial expression consistent with the detected user facial expression on a user interface configured to accept user approval or rejection. For example, the processor can display a prompt 412 on the display.

[0094] In box 614, the processor may update the preset list of approved or unapproved avatars in response to receiving input that indicates whether a user's facial expression has been approved or not approved for presentation in the virtual meeting, which is different from the preset list.

[0095] The various embodiments illustrated and described are provided merely as examples illustrating the various features of the claims. However, the features shown and described with respect to any given embodiment are not necessarily limited to the associated embodiment and may be used in conjunction with or combined with other embodiments shown and described. Furthermore, the claims are not intended to be limited to any single example embodiment. For example, one or more of methods 500, 600a, and 600b and their operations may replace or be combined with one or more operations of methods 500, 600a, and 600b.

[0096] The following paragraphs describe various implementation examples. While some of the following implementation examples are described as example methods, further example implementations may include: example methods discussed in the following paragraphs implemented by a computing device including a processor configured with processor-executable instructions to perform the operations of the methods discussed in the following implementation examples; example methods discussed in the following paragraphs implemented by a computing device including means for performing the functions of the methods discussed in the following implementation examples; and example methods discussed in the following paragraphs may be implemented having a non-transient processor-readable storage medium having processor-executable instructions stored thereon, these processor-executable instructions being configured to cause the processor of the computing device to perform the operations of the methods discussed in the following implementation examples.

[0097] Example 1: A method for presenting facial expressions on an avatar in a virtual meeting, executed by a processor of a computing device, comprising: detecting a user's facial expressions based on information received from sensors of the computing device; determining whether the detected user facial expressions are approved for presentation on an avatar in the virtual meeting; generating an avatar displaying facial expressions consistent with the detected user facial expressions in response to determining that the detected user facial expressions are approved for presentation on an avatar in the virtual meeting; and presenting the generated avatar in the virtual meeting.

[0098] Example 2: The method of Example 1 further includes: continuing to present the currently presented avatar in response to determining that the detected user facial expression is not approved for presentation on the avatar in the virtual meeting.

[0099] Example 3: The method of Example 1 further includes: in response to determining that the detected user facial expression is not approved for presentation on the avatar in the virtual meeting, generating an avatar that displays a facial expression approved for presentation on the avatar in the virtual meeting but different from the detected user facial expression.

[0100] Example 4: The method as described in any of Examples 1-3, wherein generating an avatar that displays facial expressions approved for presentation in the virtual meeting but different from the detected user facial expressions in response to determining that the detected user facial expressions are not approved for presentation in the avatar in the virtual meeting includes: generating an avatar that displays the recently approved facial expressions.

[0101] Example 5: The method as described in any of Examples 1-4, wherein detecting a user's facial expression based on information received from a sensor of a computing device includes: detecting a user's facial expression based on information received from an image sensor of a computing device.

[0102] Example 6: The method as described in any of Examples 1-5, wherein determining whether the detected user facial expression is approved to be presented on an avatar in the virtual meeting includes: determining whether the user facial expression was previously approved to be presented on an avatar in the virtual meeting; and presenting the generated avatar in the virtual meeting includes: in response to determining that the user facial expression was previously approved to be presented on an avatar in the virtual meeting, presenting the generated avatar displaying the previously approved facial expression in the virtual meeting.

[0103] Example 7: The method as described in any of Examples 1-6, wherein determining whether the detected user facial expression is approved for presentation in the virtual meeting includes: displaying an avatar exhibiting a facial expression consistent with the detected user facial expression on a user interface configured to receive user approval or rejection; and determining that the detected user facial expression is approved in response to receiving input on the user interface of the computing device indicating that the user facial expression is approved for presentation in the virtual meeting.

[0104] Example 8: The method of Example 7 further includes: determining that the detected user facial expression is approved to be displayed on the avatar in the virtual meeting if there is no responsive input to the user interface during a threshold time period.

[0105] Example 9: The method of any of Examples 1-8 further includes: in response to receiving on the user interface of the computing device an instruction that the detected user facial expression is approved or not approved to be presented in the virtual meeting on an avatar, storing in memory an instruction that an avatar displaying a facial expression consistent with the detected user facial expression is approved or not approved to be presented in the virtual meeting.

[0106] Example 10. The method as described in any of Examples 1-9, wherein determining whether a detected user facial expression is approved for presentation on an avatar in a virtual meeting includes: determining whether the detected user facial expression is stored in a preset list as approved or not approved; displaying an avatar displaying a facial expression consistent with the detected user facial expression on a user interface configured to receive user approval or rejection; and updating the preset list in response to receiving input that differs from the preset list indicating whether the user facial expression is approved or not approved for presentation on an avatar in a virtual meeting.

[0107] Example 11: The method as described in any of Examples 1-10, wherein determining whether the detected user facial expressions are approved for presentation on the avatar in the virtual meeting includes: determining whether the detected user facial expressions are approved for presentation on the avatar in the virtual meeting based on the user's expressive voice.

[0108] Example 12: The method as described in any of Examples 1-11, wherein presenting the generated avatar in the virtual meeting includes: displaying a representation of the user's expressive voice in the virtual meeting in conjunction with presenting the generated avatar in the virtual meeting.

[0109] The foregoing method descriptions and process flowcharts are provided as illustrative examples only and are not intended to require or imply that the operations of the various embodiments must be performed in the given order. As those skilled in the art will appreciate, the order of operations in the foregoing embodiments can be performed in any order. Terms such as “afterward,” “following,” and “next” are not intended to limit the order of operations; these terms are used to guide the reader through the description of the method. Furthermore, any reference to a singular claim element (e.g., references using the articles “a,” “some,” or “the”) should not be construed as limiting that element to the singular.

[0110] The various illustrative logic blocks, modules, components, circuits, and algorithmic operations described in conjunction with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this hardware-software interchangeability, the various illustrative components, blocks, modules, circuits, and operations are described above in a generalized manner in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in different ways for each specific application, but such implementation decisions should not be construed as causing a departure from the scope of the claims.

[0111] The hardware used to implement the various illustrative logics, logic blocks, modules, and circuits described in conjunction with the embodiments disclosed herein may be implemented or executed by a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof, designed to perform the functions described herein. The general-purpose processor may be a microprocessor, but in alternatives, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of receiver intelligent objects, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors cooperating with a DSP core, or any other such configuration. Alternatively, some operations or methods may be performed by a circuit system dedicated to a given function.

[0112] In one or more embodiments, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, such functionality may be stored as one or more instructions or code on a non-transient computer-readable storage medium or a non-transient processor-readable storage medium. The operation of the methods or algorithms disclosed herein may be implemented in a processor-executable software module or processor-executable instructions, which may reside on a non-transient computer-readable or processor-readable storage medium. A non-transient computer-readable or processor-readable storage medium may be any storage medium accessible to a computer or processor. By way of example and not limitation, such a non-transient computer-readable or processor-readable storage medium may include RAM, ROM, EEPROM, flash memory, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage smart objects, or any other medium that can be used to store desired program code in the form of instructions or data structures and is accessible to a computer. As used herein, disk and disc include compact discs (CDs), laser discs, optical discs, digital multi-purpose discs (DVDs), floppy disks, and Blu-ray discs. Disks typically reproduce data magnetically, while discs reproduce data optically using lasers. Combinations of these are also included within the scope of non-transient computer-readable and processor-readable media. Furthermore, the operation of a method or algorithm may reside as a piece of code and / or instructions, or any combination or set of codes and / or instructions, on a non-transient processor-readable and / or computer-readable storage medium that can be incorporated into a computer program product.

[0113] The prior description of the disclosed embodiments is intended to enable any person skilled in the art to make or use these claims. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments without departing from the scope of the claims. Thus, this disclosure is not intended to be limited to the embodiments shown herein, but should be accorded the broadest scope consistent with the appended claims and the principles and novel features disclosed herein.

Claims

1. A method for displaying facial expressions on an avatar in a virtual meeting, executed by a processor of a computing device, comprising: Detecting user facial expressions based on information received from sensors of the computing device; Determine whether the detected user facial expressions were previously approved to be displayed on the avatar in the virtual meeting; In response to determining that the detected user facial expression was not previously approved for presentation in the virtual meeting, an avatar is generated that displays a facial expression that was approved for presentation in the virtual meeting but is different from the detected user facial expression; In response to determining that the detected user facial expression was previously approved to be presented in the avatar in the virtual meeting, an avatar displaying facial expressions consistent with the detected user facial expression is generated; as well as The generated avatar is presented in the virtual meeting.

2. The method as described in claim 1, wherein, In response to determining that the detected user facial expression was not previously approved for presentation in the virtual meeting, generating an avatar that displays a facial expression that is approved for presentation in the virtual meeting but is different from the detected user facial expression includes continuing to present the currently presented avatar.

3. The method as described in claim 1, wherein, In response to determining that the detected user facial expressions were not previously approved for presentation in the virtual meeting, generating an avatar that displays facial expressions approved for presentation in the virtual meeting but different from the detected user facial expressions includes generating an avatar that displays recently approved facial expressions.

4. The method of claim 1, wherein, Detecting a user's facial expression based on information received from the sensors of the computing device includes: detecting the user's facial expression based on information received from the image sensors of the computing device.

5. The method of claim 1, further comprising determining whether the detected user facial expressions are approved for display on the avatar in the virtual meeting, including: An avatar displaying a facial expression consistent with the detected user facial expression is shown on a user interface configured to accept the user's approval or rejection. as well as In response to receiving input on the user interface of the computing device indicating that the user's facial expression is approved for presentation in the virtual meeting, it is determined that the detected user facial expression is approved.

6. The method of claim 5, further comprising: If there is no responsive input to the user interface within a threshold time period, it is determined that the detected user facial expression is approved to be displayed on the avatar in the virtual meeting.

7. The method of claim 1, further comprising: In response to receiving an input on the user interface of the computing device indicating whether a detected user facial expression is approved or not approved for presentation in an avatar in a virtual meeting, the device stores in memory an indication that an avatar displaying a facial expression consistent with the detected user facial expression is approved or not approved for presentation in the virtual meeting.

8. The method of claim 1, wherein, Determining whether detected user facial expressions were previously approved for display on an avatar in a virtual meeting includes: Determine whether the detected user facial expressions are stored in a preset list as approved or not; On a user interface configured to receive the user's approval or rejection, an avatar displaying a facial expression consistent with the detected user's facial expression is shown; and The preset list is updated in response to receiving input that indicates, but is different from the preset list, that the user's facial expressions are approved or not approved for presentation on the avatar in the virtual meeting.

9. The method of claim 1, further comprising determining whether the detected user facial expression is approved for display on an avatar in a virtual meeting, including: The system determines whether detected user facial expressions are approved for display on the avatar in the virtual meeting based on the user's expressive voice.

10. The method of claim 1, wherein, Presenting the generated avatar in the virtual meeting includes: displaying the user's expressive voice in the virtual meeting in conjunction with presenting the generated avatar in the virtual meeting.

11. A computing device, comprising: Sensors configured to detect facial expressions of the user of the computing device; as well as A processor coupled to the sensor and configured with processor-executable instructions for the following operations: Detecting user facial expressions based on information received from the sensor; Determine whether the detected user facial expressions were previously approved to be displayed on the avatar in the virtual meeting; In response to determining that the detected user facial expression was not previously approved for presentation in the virtual meeting, an avatar is generated that displays a facial expression that was approved for presentation in the virtual meeting but is different from the detected user facial expression; In response to determining that the detected user facial expression was previously approved to be presented in the avatar in the virtual meeting, an avatar displaying facial expressions consistent with the detected user facial expression is generated; as well as The generated avatar is presented in the virtual meeting.

12. The computing device of claim 11, wherein, The processor is further configured with processor-executable instructions for: generating an avatar that displays facial expressions approved for presentation in the virtual meeting but different from the detected user facial expressions, in response to determining that the detected user facial expressions were not previously approved for presentation in the avatar, by continuing to present the currently presented avatar.

13. The computing device of claim 11, wherein, The processor is further configured with processor-executable instructions for generating avatars that display recently approved facial expressions.

14. The computing device of claim 11, wherein, The sensor is an image sensor.

15. The computing device of claim 11, wherein, The processor is further configured with processor-executable instructions for the following operations: An avatar displaying a facial expression consistent with the detected user facial expression is shown on a user interface configured to accept the user's approval or rejection. as well as In response to receiving input on the user interface of the computing device indicating that the user's facial expression is approved for presentation in the virtual meeting, it is determined that the detected user facial expression is approved.

16. The computing device of claim 15, wherein, The processor is further configured with processor-executable instructions for the following operations: If there is no responsive input to the user interface within a threshold time period, it is determined that the detected user facial expression is approved to be displayed on the avatar in the virtual meeting.

17. The computing device of claim 11, wherein, The processor is further configured with processor-executable instructions for the following operations: In response to receiving an input on the user interface of the computing device indicating whether a detected user facial expression is approved or not approved for presentation in an avatar in a virtual meeting, the device stores in memory an indication that an avatar displaying a facial expression consistent with the detected user facial expression is approved or not approved for presentation in the virtual meeting.

18. The computing device of claim 11, wherein, The processor is further configured with processor-executable instructions for the following operations: Determine whether the detected user facial expressions are stored in a preset list as approved or not; An avatar displaying a facial expression consistent with the detected user facial expression is shown on a user interface configured to accept the user's approval or rejection. as well as The preset list is updated in response to receiving input that indicates, but is different from the preset list, that the user's facial expressions are approved or not approved for presentation on the avatar in the virtual meeting.

19. The computing device of claim 11, further comprising a microphone coupled to the processor, wherein, The processor is further configured with processor-executable instructions for determining whether detected user facial expressions are approved to be presented on the avatar in the virtual meeting, based on the user's expressive voice detected by the microphone.

20. The computing device of claim 11, wherein, The processor is further configured with processor-executable instructions for displaying a representation of the user's expressive voice in the virtual meeting in conjunction with the generated avatar presented in the virtual meeting.

21. A computing device, comprising: A device for detecting a user's facial expressions based on information received from sensors of the computing device; A device for determining whether a detected user's facial expression was previously approved to be presented on an avatar in a virtual meeting; Device for generating an avatar that displays facial expressions that are approved to be presented in the virtual meeting but are different from the detected user facial expressions, in response to determining that the detected user facial expressions have not been previously approved to be presented in the avatar in the virtual meeting; Device for generating an avatar that displays facial expressions consistent with the detected user facial expressions in response to determining that the detected user facial expressions were previously approved to be presented in the avatar in the virtual meeting; as well as Devices for presenting the generated avatars in the virtual meeting.

22. The computing device of claim 21, wherein, The means for generating an avatar that displays facial expressions approved for presentation in the virtual meeting but different from the detected user facial expressions, in response to determining that the detected user facial expressions were not previously approved for presentation in the virtual meeting, includes means for continuing to present the currently presented avatar.

23. The computing device of claim 21, wherein, The means for generating an avatar that displays facial expressions approved for presentation in the virtual meeting but different from the detected user facial expressions, in response to determining that the detected user facial expressions have not been previously approved for presentation in the virtual meeting, includes: means for generating an avatar that displays recently approved facial expressions.

24. The computing device of claim 21, wherein, The apparatus for detecting a user's facial expression based on information received from a sensor of the computing device includes: means for detecting the user's facial expression based on information received from an image sensor of the computing device.

25. The computing device of claim 21, further comprising means for determining whether the detected user facial expression is approved to be displayed on an avatar in a virtual meeting, the means comprising: Device for displaying an avatar of a facial expression consistent with the detected facial expression of the user on a user interface configured to receive the user's approval or rejection; as well as A means for determining that a detected user facial expression is approved in response to receiving input on the user interface of the computing device indicating that the user's facial expression is approved to be presented in the virtual meeting.

26. The computing device of claim 25, further comprising: A means for determining, in the absence of responsive input to the user interface within a threshold time period, that a detected user facial expression is approved to be displayed on the avatar in the virtual meeting.

27. The computing device of claim 21, further comprising: A means for storing in memory, in response to receiving on the user interface of the computing device an input indicating whether a detected user facial expression is approved or not approved for presentation in an avatar in a virtual meeting, an instruction that an avatar displaying a facial expression consistent with the detected user facial expression is approved or not approved for presentation in the virtual meeting.

28. The computing device of claim 21, wherein, The device for determining whether a detected user facial expression was previously approved to be displayed on an avatar in a virtual meeting includes: Device for determining whether a detected user facial expression is stored in a preset list as approved or unapproved; A means for displaying an avatar of a facial expression consistent with the detected facial expression of the user on a user interface configured to receive the user's approval or rejection; and A means for updating the preset list in response to receiving an input that differs from the preset list, indicating whether the user's facial expressions are approved or not displayed on the avatar in the virtual meeting.

29. The computing device of claim 21, further comprising means for determining whether the detected user facial expression is approved to be displayed on an avatar in a virtual meeting, the means comprising: A device for determining, based on the user's expressive voice, whether a detected user facial expression is approved to be presented on an avatar in the virtual meeting.

30. The computing device of claim 21, wherein, The means for presenting the generated avatar in the virtual meeting includes: means for displaying a representation of the user's expressive voice in the virtual meeting in conjunction with presenting the generated avatar in the virtual meeting.

31. A non-transient processor-readable medium having processor-executable instructions stored thereon, the processor-executable instructions being configured to cause a processor of a computing device in a computing device to perform operations, the operations including: Detecting the user's facial expressions based on information received from the sensors of the computing device; Determine whether the detected user facial expressions were previously approved to be displayed on the avatar in the virtual meeting; In response to determining that the detected user facial expression was not previously approved for presentation in the virtual meeting, an avatar is generated that displays a facial expression that was approved for presentation in the virtual meeting but is different from the detected user facial expression; In response to determining that the detected user facial expression was previously approved to be presented in the avatar in the virtual meeting, an avatar displaying facial expressions consistent with the detected user facial expression is generated; as well as The generated avatar is presented in the virtual meeting.

Images