Presenting facial expressions in virtual meetings

Abstract

An embodiment system and method for presenting facial expressions in a virtual meeting may include detecting a facial expression of a user based on information received from a sensor of a computing device; determining whether the detected user's facial expression is approved for presentation on an avatar in the virtual meeting; in response to a determination that the detected user's facial expression is approved for presentation on an avatar in the virtual meeting, generating an avatar representing a facial expression matching the detected user's facial expression; in response to a determination that the detected user's facial expression is not approved for presentation on an avatar in the virtual meeting, generating an avatar representing the facial expression approved for presentation; and presenting the generated avatar in the virtual meeting.

Description

Technical Field

[0001] Related Applications This application claims the benefit of priority of U.S. Patent Application No. 17 / 320,627, filed on May 14, 2021, entitled "Presenting A Facial Expression In A Virtual Meeting", the entire content of which is incorporated herein by reference.

Background Art

[0002] Communication networks have enabled the development of applications and services for online meetings and gatherings. Some systems provide a virtual environment that presents a visual representation of attendees known as "avatars" that can range from simplified images or images like cartoons to photo-realistic images. Some of these systems include VR devices, such as virtual reality (VR) headsets or other VR equipment, that record the user's movements and voice. Such systems can generate facial expressions on the user's avatar based on the user's movements and utterances.

[0003] However, the expressions, words, and actions of virtual meeting attendees may not be relevant to the meeting. The user may react to any number of things that occur in the user's real-world environment, such as distractions from children, pets, or others, external noise, phone calls, and other disruptions. Also, the system may detect and display expressions that the user does not want to show to others in an online meeting, such as anger, discomfort, frustration, etc. Further, the system may inaccurately capture and display expressions, resulting in a mismatch between what the user is trying to convey and the expression that is displayed, which can be awkward, confusing, or rude. Current systems do not provide a mechanism for the user to review or approve the facial expressions to be displayed on the user's avatar.

Summary of the Invention

Means for Solving the Problems

[0004] Various embodiments include a method for presenting facial expressions in a virtual meeting and a computing device configured to perform the method. Various embodiments may include the steps of: detecting a user's facial expression based on information received from sensors of the computing device; determining whether the detected user's facial expression is approved for presentation on an avatar in a virtual meeting; generating an avatar representing a facial expression consistent with the detected user's facial expression in response to the determination that the detected user's facial expression is approved for presentation on an avatar in a virtual meeting; generating an avatar representing a facial expression that is approved for presentation on an avatar in a virtual meeting but is different from the detected user's facial expression in response to the determination that the detected user's facial expression is not approved for presentation on an avatar in a virtual meeting; and presenting the generated avatar in a virtual meeting. Some embodiments may further include the step of continuing to present the currently presented avatar in response to the determination that the detected user's facial expression is not approved for presentation on an avatar in a virtual meeting. In some embodiments, in response to a decision that the detected user's facial expression is not approved for display on an avatar in a virtual meeting, the step may further include generating an avatar that represents a facial expression that is approved for display on an avatar in a virtual meeting, but is different from the detected user's facial expression.

[0005] In some embodiments, in response to a decision that the detected user's facial expression is not approved for display on an avatar in a virtual meeting, the step of generating an avatar that represents a facial expression approved for display on an avatar in a virtual meeting, but different from the detected user's facial expression, may include the step of generating an avatar that represents a recent facial expression approved for display. In some embodiments, the step of detecting a user's facial expression based on information received from a computing device's sensor may include the step of detecting a user's facial expression based on information received from an image sensor of a computing device.

[0006] In some embodiments, the step of determining whether a detected user facial expression is approved for presentation on an avatar in a virtual meeting may include the step of determining whether the user's facial expression has been previously approved for presentation on an avatar in a virtual meeting, and the step of presenting the generated avatar in a virtual meeting may include, in response to the determination that the user's facial expression has been previously approved for presentation on an avatar in a virtual meeting, the step of presenting the generated avatar in a virtual meeting that represents the previously approved facial expression. In some embodiments, the step of determining whether a detected user facial expression is approved for presentation on an avatar in a virtual meeting may include the step of rendering an avatar representing a facial expression consistent with the detected user facial expression on a user interface configured to receive approval or rejection from the user, and the step of determining that the detected user facial expression is approved in response to the reception on the computing device's user interface of an input indicating that the user's facial expression is approved for presentation in a virtual meeting.

[0007] Some embodiments may include the step of determining that if there is no response input to the user interface within a threshold time period, the detected user's facial expression is approved for presentation on an avatar in a virtual meeting. Some embodiments may include the step of storing in memory an instruction that an avatar representing a facial expression consistent with the detected user's facial expression is approved or not approved for presentation in a virtual meeting, in response to the reception on the computing device's user interface of input indicating whether the detected user's facial expression is approved or not approved for presentation on an avatar in a virtual meeting.

[0008] In some embodiments, the step of determining whether a detected user facial expression is approved for display on an avatar in a virtual meeting may include the steps of determining whether the detected user facial expression is stored in a preset list as approved or not approved; rendering an avatar representing a facial expression consistent with the detected user facial expression on a user interface configured to receive approval or rejection from the user; and updating the preset list in response to receiving an input different from the preset list indicating whether the user's facial expression is approved or not approved for display on an avatar in a virtual meeting.

[0009] In some embodiments, the step of determining whether a detected user's facial expression is approved for display on an avatar in a virtual meeting may include the step of determining whether a detected user's facial expression is approved for display on an avatar in a virtual meeting based on the user's expressive voice. In some embodiments, the step of displaying a generated avatar in a virtual meeting may include, in conjunction with displaying a generated avatar in a virtual meeting, the step of rendering a representation of the user's expressive voice in a virtual meeting.

[0010] Further embodiments may include a computing device comprising memory and a processor coupled to the memory, wherein the processor is comprised of processor-executable instructions for performing any of the operations described above. Further embodiments may include a processor-readable storage medium storing processor-executable instructions configured to cause a controller of the computing device to perform any of the operations described above. Further embodiments may include a computing device comprising means for performing any of the functions described above.

[0011] The accompanying drawings incorporated herein and constituting part of this specification illustrate exemplary embodiments and, together with the general description given above and the detailed description given below, help to illustrate the features of some embodiments. [Brief explanation of the drawing]

[0012] [Figure 1] This is a system block diagram showing an exemplary communication system suitable for implementing any of the various embodiments. [Figure 2] This is an exemplary block diagram of the components of a computing device suitable for implementing any of the various embodiments. [Figure 3] This is a block diagram of components showing an exemplary computing system architecture suitable for implementing any of the various embodiments. [Figure 4] This is a conceptual diagram illustrating various embodiments of methods for presenting facial expressions in a virtual meeting. [Figure 5] This is a process flow diagram illustrating various methods for presenting facial expressions in a virtual meeting. [Figure 6A] This figure shows actions that can be performed as part of a method for presenting facial expressions in a virtual meeting, according to various embodiments. [Figure 6B]This figure shows actions that can be performed as part of a method for presenting facial expressions in a virtual meeting, according to various embodiments. [Modes for carrying out the invention]

[0013] Various embodiments will be described in detail with reference to the attached drawings. Wherever possible, the same reference numerals will be used throughout the drawings to refer to the same or similar parts. References made to specific examples and implementations are for illustrative purposes only and are not intended to limit the various embodiments or claims.

[0014] Various embodiments provide methods for displaying facial expressions that participants deem appropriate on their avatars in a virtual meeting, which may be implemented within a device such as a mobile computing device. Various embodiments enable a computing device to learn facial expressions approved and disapproved by the user for display on the user's avatar in a virtual meeting. Various embodiments eliminate the need for additional bulky and expensive peripherals such as virtual reality headsets and similar devices. Various embodiments improve the operation of computing devices and virtual meeting systems by enabling automatic filtering of facial expressions rendered on participant avatars in order to improve the conduct of virtual meetings.

[0015] The terms “components,” “modules,” and “systems” include, but are not limited to, computer-related entities such as hardware, firmware, hardware-software combinations, software, or running software that are configured to perform a particular operation or function. For example, a component may be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. As an example, both an application running on a computing device and the computing device itself may be referred to as components. One or more components may reside within a process and / or an execution thread, and components may be localized on one processor or core and / or distributed across two or more processors or cores. In addition, these components may be executed from various non-temporary computer-readable media storing various instructions and / or data structures. Components may communicate by local and / or remote processes, function or procedure calls, electronic signals, data packets, memory reads / writes, and other known computer, processor, and / or process-related communication methods.

[0016] The term “computing device” is used herein to refer to any or all of the following similar electronic devices, including cellular phones, smartphones, portable computing devices, personal or mobile multimedia players, laptop computers, tablet computers, smartbooks, ultrabooks, palmtop computers, email receivers, multimedia internet-enabled cellular phones, router devices, medical devices and equipment, biosensors / devices, wearable devices including smartwatches, smart clothing, smart glasses, smart wristbands, and smart jewelry (e.g., smart rings, smart bracelets, etc.), entertainment devices (e.g., game controllers, music and video players, satellite radios, etc.), smart meters / sensors, Internet of Things (IoT) devices including industrial manufacturing equipment and large and small machinery and appliances for home or business use, computing devices in autonomous and semi-autonomous vehicles, mobile devices fixed to or embedded in various mobile platforms, global positioning system devices, and memory and programmable processors.

[0017] The term “system on a chip” (SOC) is used herein to refer to a single integrated circuit (IC) chip that includes multiple resources and / or processors integrated on a single substrate. A single SOC may include circuitry for digital, analog, mixed-signal, and radio frequency functions. A single SOC may also include any number of general-purpose and / or dedicated processors (such as digital signal processors, modem processors, and video processors), memory blocks (e.g., ROM, RAM, and flash), and resources (e.g., timers, voltage regulators, and oscillators). A SOC may also include software for controlling the integrated resources and processors, as well as for controlling peripheral devices.

[0018] The term “system in package” (SIP) may be used herein to refer to a single module or package that includes multiple resources, computing units, cores, and / or processors on two or more IC chips, substrates, or SOCs. For example, a SIP may include a single substrate on which multiple IC chips or semiconductor dies are stacked in a vertical configuration. Similarly, a SIP may include one or more multi-chip modules (MCMs) on which multiple ICs or semiconductor dies are packaged on a unifying substrate. A SIP may also include multiple independent SOCs coupled to each other via high-speed communication circuits and packaged in very close proximity, such as on a single motherboard or within a single wireless device. The proximity of the SOCs facilitates high-speed communication, as well as the sharing of memory and resources.

[0019] As used herein, the terms “Network,” “System,” “Wireless Network,” “Cellular Network,” and “Wireless Communication Network” may interchangeably refer to a portion or all of the wireless network of a carrier associated with a wireless device and / or the subscription of a wireless device. The techniques described herein may be used with respect to various wireless communication networks, including Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), FDMA, Orthogonal FDMA (OFDMA), Single Carrier FDMA (SC-FDMA), and other networks. In general, any number of wireless networks may be deployed in a given geographical area. Each wireless network may support at least one radio access technology, which may operate on one or more frequencies or frequency ranges. For example, a CDMA network may implement Universal Terrestrial Radio Access (UTRA) (including the Wideband Code Division Multiple Access (WCDMA®) standard), CDMA2000 (including the IS-2000, IS-95, and / or IS-856 standards), etc. In another example, a TDMA network may implement the Global System for Mobile Communications (GSM), GSM Evolutionary High-Speed ​​Data Rate (EDGE). In yet another example, an OFDMA network may implement Advanced UTRA (E-UTRA) (including the LTE standard), IEEE 802.11 (WiFi), IEEE 802.16 (WiMAX), IEEE 802.20, Flash OFDM®, etc. Wireless networks using the LTE standard may be mentioned, and therefore the terms “Advanced Universal Terrestrial Radio Access,” “E-UTRAN,” and “eNode B” may also be used interchangeably to refer to wireless networks in this specification. However, such mentions are merely examples and are not intended to exclude wireless networks using other communication standards.For example, various third-generation (3G) systems, fourth-generation (4G) systems, and fifth-generation (5G) systems are discussed herein, but these systems are mentioned merely as examples and may be replaced in various instances by future-generation systems (e.g., systems beyond the sixth generation (6G)).

[0020] Some online meeting and conferencing systems provide a virtual environment that presents visual "avatars" of attendees instead of just name labels or live images. Such avatars can range from simplified images or cartoon-like images to photo-realistic images. Some systems that support virtual meetings include VR devices that record a user's movements and voice and then generate facial expressions on the user's avatar based on the user's movements and utterances.

[0021] As described above, the expressions, words, and actions of virtual meeting attendees may be irrelevant to the meeting under various circumstances. For example, a user may react to any number of things occurring in the user's real-world environment, such as children, pets, or distractions from others, external noise, phone calls, and other interruptions. Also, sometimes virtual meeting participants may react negatively or prematurely to something said or presented in the meeting, and it is best for these participants to avoid showing inappropriate expressions that the user does not want their avatar to display to others in the online meeting. For example, meeting participants may not want instantaneous expressions such as anger, displeasure, frustration, etc. to be presented on their meeting avatar. Additionally, virtual meeting software may misinterpret or inaccurately detect a participant's expression, which can cause a mismatch between what the participant is trying to convey in the meeting and the expression represented on the participant's avatar, which can be awkward, confusing, or rude to other meeting participants.

[0022] Various embodiments include methods for presenting facial expressions on a participant's avatar in a virtual meeting, which include the step of filtering out or avoiding rendering facial expressions that the participant does not wish to display on the participant's avatar, and computing devices configured to perform such methods. In various embodiments, the computing device may detect a user's facial expression or emotion based on information received from one or more sensors of the computing device (e.g., a camera). The computing device may determine whether the detected user's facial expression is approved for presentation on the participant's avatar in a virtual meeting. In response to a decision that the detected user's facial expression is approved for presentation on the avatar in a virtual meeting, the computing device may generate an avatar representing a facial expression consistent with the detected user's facial expression. In response to a decision that the detected user's facial expression is not approved for presentation on the avatar in a virtual meeting, the computing device may avoid rendering a facial expression consistent with the detected user's facial expression on the participant's avatar, for example, by generating an avatar representing a facial expression approved for presentation and / or by not changing the expression currently displayed on the participant's avatar. In some embodiments, in response to a determination that a detected user facial expression is not approved for presentation on an avatar in a virtual meeting, the computing device may generate an avatar representing the most recent or immediate expression approved for presentation. In some embodiments, the computing device may generate an avatar representing an expression consistent with the most recent or immediate expression approved for presentation. The computing device may present the avatar with the generated representation of the user's facial expression in a virtual meeting. In various embodiments, the facial expressions presented on the avatar may include expressions that definitively convey meaning or emotion (e.g., joy, sadness, surprise, anger, contempt, disgust, anxiety, etc.), expressive actions (e.g., laughter, sigh, wink, smile, gasp, etc.), indistinct expressions, a "calm" face, or a "blank" expression.

[0023] In some embodiments, the computing device may detect a user's facial expression based on an image and / or information received from an image sensor of the computing device. In some embodiments, the computing device may process an image of the user's face (which may be a single image or a video) to detect the facial expression. In some embodiments, the computing device may include a multi-mode sensor or a group of sensors including an image sensor for capturing an image or video of the user's face. In various embodiments, the system may not use or require additional devices, such as a VR system.

[0024] In some embodiments, the computing device may also record and process speech and / or other meaningful sounds and determine words, phrases, and sounds associated with or corresponding to certain facial expressions. In some embodiments, the computing device may capture the user's speech and other expressive sounds (such as sighs, laughter, etc.) with a microphone or another suitable audio sensor. In some embodiments, the computing device may detect the user's facial expression based on, for example, the user's expressive sounds received via a microphone of the computing device. The term "expressive sounds" is used herein to collectively refer to words and phrases (i.e., language) as well as meaningful sounds. Examples of meaningful sounds include laughter, sighs, humming, coughing, hesitant voices (such as "um", "ah", "er", "uh", etc.), stuttering, or other sounds that convey meaning to another person. In some embodiments, the computing device may correlate or associate virtual facial expressions with the user's expressive sounds. In some embodiments, the computing device may learn words, phrases, and / or sounds associated with or correlated to various expressions of the user's face.

[0025] In some embodiments, the computing device may automatically determine whether a generated virtual facial expression should be rendered on the user's avatar presented in a virtual meeting. In some embodiments, the system compares the generated virtual facial expression to a database, list, or other suitable data structure for permitted or acceptable expressions (e.g., smile, laugh, enthusiasm, etc.) and / or a database of unacceptable expressions (e.g., anger, frown, disgust, etc.), and if the generated virtual facial expression is on the permitted or acceptable expression list, the computing device may render that expression on the user's avatar. The list of permitted or acceptable expressions and / or unacceptable expressions may be pre-configured within the computing device, such as a default list, a preset list provided by the manufacturer, or a user-selected or generated list set during the setup procedure (e.g., for software related to virtual meetings).

[0026] In some embodiments, the computing device may learn and / or adjust a database, list, or other preferred data structure of facial expressions that are permitted or acceptable through user interaction, and / or a list of facial expressions that are not permitted or acceptable. For example, the computing device may determine that a detected facial expression is neither approved nor rejected for presentation (e.g., in a preset list, or in a database or list of previously approved or rejected facial expressions). In response to the determination that a detected user facial expression is neither approved nor rejected for presentation, in other words, the indecision regarding the acceptance of the detected facial expression, the computing device may present an avatar on the computing device's user interface that represents a facial expression consistent with the detected user facial expression within a user interface configured to receive an approval or rejection user input. In some embodiments, the computing device may render an avatar on the computing device's user interface that represents a facial expression consistent with the detected user facial expression using a solicitation or prompt that requires the user to indicate whether the detected user facial expression is approved or rejected for presentation. In response to receiving input on the computing device's user interface indicating that a user's facial expression is approved for presentation in a virtual meeting, the computing device may determine that the detected user's facial expression is approved. In some embodiments, the computing device may store instructions indicating whether an avatar representing a user's facial expression is approved or not, such as a listing of facial expressions for an avatar in a database or list of approved or rejected facial expressions.

[0027] In some embodiments, a computing device may display virtual facial expressions on a user interface to allow the user to input whether to approve or disapprove the displayed virtual facial expressions, and may give the user a short time period (e.g., 3-5 seconds) to respond to the user interface by selecting to approve or disapprove. This opportunity for consideration may allow the user to filter or screen facial expressions before they are rendered on the user's avatar in a virtual meeting. To allow the user to approve or disapprove displayed virtual facial expressions without requiring input each time a facial expression is displayed on the user interface, the system may take a default action if the user does not respond within a short time period, such as before a threshold time period expires. The default decision of approving or disapproving virtual facial expressions presented on the user interface may be user-defined (for example, the user may enter settings in a configuration interface and choose whether ignoring the user interface prompt should be interpreted as approval or disapproval).

[0028] In some embodiments, a computing device may determine, based on the user's expressive voice, whether a generated virtual facial expression should be rendered on the user's avatar. For example, the user's expressive voice may be associated with an unacceptable expression, such as a mocking and contemptuous facial expression, or a laughing and surprised or disbelieving facial expression. In some embodiments, a computing device may determine, based on the user's expressive voice, whether a detected facial expression of the user is approved for presentation on the user's avatar. For example, the computing device may determine that the user's expressive voice is associated with an unacceptable expression. In response, instead of an unacceptable facial expression, the computing device may generate an avatar representing a facial expression that is approved (or previously approved) for presentation.

[0029] In some embodiments, the computing device may be configured to learn user facial expressions that the user wishes to display and user facial expressions that the user does not wish to display (i.e., acceptable and unacceptable facial expressions) over time (e.g., by a learning algorithm). The computing device may also be configured to learn user facial expressions that are acceptable or suitable for use when the image of the user's face is ambiguous. The computing device may also be configured to learn expressive sounds associated with acceptable and unacceptable facial expressions. In some embodiments, the computing device may be configured to determine (e.g., based on a database, list, etc.) whether a detected user facial expression has been previously approved for presentation on an avatar in a virtual meeting, and in response to the determination that the detected user facial expression has been previously approved for presentation on an avatar in a virtual meeting, the computing device may automatically generate an avatar representing a facial expression consistent with the detected user facial expression. As described above, the computing device may check and / or update preset or default facial expressions in response to the receipt of inputs that differ from the preset or default facial expressions. Thus, a computing device can learn acceptable and unacceptable user facial expressions to update preset facial expressions, or update a database of acceptable and unacceptable facial expressions to add new facial expressions.

[0030] Thus, computing devices can learn how to automatically decide whether to display virtual facial expressions without prompting the user for a response. Over time, such as in response to several consistent user decisions, computing devices can be configured to reduce the number and frequency of prompts they display for approval or disapproval. Ultimately, computing devices can stop prompting the user about expressions that are always acceptable for display or never acceptable for display, and can immediately implement acceptable expressions without bothering the user with prompts to approve or disapprove of a particular virtual avatar's expressions.

[0031] Various embodiments improve the operation of computing devices and virtual conferencing systems by filtering or preventing the generation of unacceptable virtual avatar facial expressions in virtual conferencing. Various embodiments also improve the operation of computing devices and virtual conferencing systems by learning various facial expressions that are approved or unapproved to be presented in virtual conferencing, thus operating in an increasingly seamless manner that requires fewer and fewer user prompts over time. Various embodiments also improve their operation by improving the speed and efficiency of the operation of computing devices and virtual conferencing systems. Various embodiments further improve their operation by improving the speed and efficiency of the operability and user interactivity of computing devices and virtual conferencing systems.

[0032] Figure 1 is a system block diagram showing an exemplary communication system 100. The communication system 100 could be any other suitable network, such as a 5G New Radio (NR) network or a Long-Term Evolution (LTE) network. While Figure 1 shows a 5G network, later generations of networks may include the same or similar elements. Therefore, references to 5G networks and 5G network elements in the following description are illustrative and not intended to be limiting.

[0033] The communication system 100 may include a heterogeneous network architecture, including a core network 140 and various wireless devices (shown as wireless devices 120a to 120e in Figure 1). The communication system 100 may also include several base stations (shown as BS110a, BS110b, BS110c, and BS110d) and other network entities. A base station is an entity that communicates with wireless devices and may also be called a node B, an LTE-evolved node B (e-node B or eNB), an access point (AP), a radio head, a transmit / receive point (TRP), a new radio base station (NR BS), a 5G node B (NB), a next-generation node B (g-node B or gNB), etc. Each base station may provide communication coverage to a specific geographic area. In 3GPP®, the term “cell” may refer to a base station’s coverage area, a base station subsystem serving that coverage area, or a combination thereof, depending on the context in which the term is used. The core network 140 may be any type of core network, such as an LTE core network (e.g., an Advanced Packet Core (EPC) network), a 5G core network, etc.

[0034] Base stations 110a-110d may provide communication coverage to macrocells, picocells, femtocells, other types of cells, or combinations thereof. Macrocells may cover relatively large geographical areas (e.g., a radius of several kilometers) and may enable unrestricted access by wireless devices subscribed to the service. Picocells may cover relatively small geographical areas and may enable unrestricted access by wireless devices subscribed to the service. Femtocells may cover relatively small geographical areas (e.g., a home) and may enable limited access by wireless devices associated with the femtocell (e.g., wireless devices within a limited subscriber group (CSG)). Base stations for macrocells are sometimes called macro BS. Base stations for picocells are sometimes called pico BS. Base stations for femtocells are sometimes called femto BS or home BS. In the example shown in Figure 1, base station 110a may be a macro BS for macrocell 102a, base station 110b may be a pico BS for picocell 102b, and base station 110c may be a femto BS for femtocell 102c. Base stations 110a to 110d may support one or more (e.g., three) cells. The terms “eNB”, “base station”, “NR BS”, “gNB”, “TRP”, “AP”, “Node B”, “5G NB”, and “cell” may be used interchangeably in this specification.

[0035] In some examples, cells may not be stationary, and the geographical area of ​​a cell may move according to the location of the mobile base station. In some examples, base stations 110a-110d may be interconnected with each other in the communication system 100, as well as with one or more other base stations or network nodes (not shown), through various types of backhaul interfaces, such as direct physical connections, virtual networks, or a combination thereof, using any suitable transport network.

[0036] Base stations 110a to 110d can communicate with the core network 140 over wired or wireless communication link 126. Wireless devices 120a to 120e can communicate with base stations 110a to 110d over wireless communication link 122.

[0037] The wired communication link 126 may use a variety of wired networks (e.g., Ethernet, TV cable, telephony, fiber optics, and other forms of physical network connections) that may use one or more wired communication protocols, such as Ethernet, Point-to-Point Protocol, High-Level Data Link Control (HDLC), Advanced Data Communication Control Protocol (ADCCP), and Transmit Control Protocol / Internet Protocol (TCP / IP).

[0038] The communication system 100 may also include a relay station (such as relay BS110d). A relay station is an entity that can receive data transmissions from upstream stations (e.g., base stations or wireless devices) and send data transmissions to downstream stations (e.g., wireless devices or base stations). A relay station may also be a wireless device that can relay transmissions for other wireless devices. In the example shown in Figure 1, relay station 110d may communicate with macro base station 110a and wireless device 120d to facilitate communication between base station 110a and wireless device 120d. A relay station may also be called a relay base station, relay station, repeater, etc.

[0039] The communication system 100 may be a heterogeneous network including different types of base stations, such as macro base stations, pico base stations, femto base stations, and relay base stations. These different types of base stations may have different transmission power levels, different coverage areas, and different effects on interference in the communication system 100. For example, macro base stations may have high transmission power levels (e.g., 5 to 40 watts), while pico base stations, femto base stations, and relay base stations may have lower transmission power levels (e.g., 0.1 to 2 watts).

[0040] The network controller 130 may be coupled to a set of base stations and may provide coordination and control for these base stations. The network controller 130 may communicate with base stations via backhaul. Base stations may also communicate with each other, for example, directly or indirectly via wireless or wireline backhaul.

[0041] Wireless devices 120a, 120b, and 120c may be distributed throughout the communication system 100, and each wireless device may be fixed or mobile. Wireless devices may also be referred to as access terminals, terminals, mobile stations, subscriber units, stations, user equipment (UEs), etc.

[0042] Macro base station 110a can communicate with core network 140 over wired or wireless communication link 126. Wireless devices 120a, 120b, and 120c can communicate with base stations 110a to 110d over wireless communication link 122.

[0043] Wireless communication links 122 and 124 may include multiple carrier signals, frequencies, or frequency bands, each of which may include multiple logical channels. Wireless communication links 122 and 124 may utilize one or more radio access technologies (RATs). Examples of RATs that may be used in wireless communication links include 3GPP® LTE, 3G, 4G, 5G (NR, etc.), GSM, Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA®), Worldwide Interoperability for Microwave Access (WiMAX), Time Division Multiple Access (TDMA), and other mobile telephony communication technologies, including cellular RATs. Further examples of RATs that may be used in one or more of the various wireless communication links within communication system 100 include medium-range protocols such as Wi-Fi, LTE-U, LTE-Direct, LAA, and MuLTEfire, as well as relatively short-range RATs such as ZigBee, Bluetooth, and Bluetooth Low Energy (LE).

[0044] Some wireless networks (e.g., LTE) utilize orthogonal frequency division multiplexing (OFDM) on the downlink and single-carrier frequency division multiplexing (SC-FDM) on the uplink. OFDM and SC-FDM divide the system bandwidth into multiple (K) orthogonal subcarriers, commonly called tones or bins. Each subcarrier can be modulated with data. Generally, the modulation symbol is transmitted using OFDM in the frequency domain and SC-FDM in the time domain. The spacing between adjacent subcarriers may be fixed, and the total number of subcarriers (K) may depend on the system bandwidth. For example, the subcarrier spacing may be 15 kHz, and the minimum resource allocation (called a "resource block") may be 12 subcarriers (or 180 kHz). As a result, the nominal Fast File Transfer (FFT) sizes can be equal to 128, 256, 512, 1024, or 2048 for system bandwidths of 1.25, 2.5, 5, 10, or 20 megahertz (MHz), respectively. The system bandwidth can also be divided into subbands. For example, a subband can cover 1.08 MHz (i.e., 6 resource blocks), and there may be 1, 2, 4, 8, or 16 subbands for system bandwidths of 1.25, 2.5, 5, 10, or 20 MHz, respectively.

[0045] While some implementation descriptions may use terminology and examples related to LTE technology, some implementations may be applicable to other wireless communication systems, such as New Radio (NR) or 5G networks. NR may utilize OFDM with cyclic prefixes (CP) on the uplink (UL) and downlink (DL) and may include support for half-duplex operation using time-division duplex (TDD). A single component carrier bandwidth of 100 MHz may be supported. An NR resource block may span 12 subcarriers with a subcarrier bandwidth of 75 kHz over a duration of 0.1 milliseconds (ms). Each radio frame may consist of 50 subframes, each with a length of 10 ms. Consequently, each subframe may have a length of 0.2 ms. Each subframe may indicate the link direction for data transmission (i.e., DL or UL), and the link direction per subframe may be dynamically switched. Each subframe may contain DL / UL data and DL / UL control data. Beamforming may be supported, and the beam direction may be dynamically configured. Multiple-input multiple-output (MIMO) transmission with precoding may also be supported. MIMO configurations in DL can support up to eight transmitting antennas, along with multi-layer DL transmission of up to eight streams and up to two streams per wireless device. Multi-layer transmission with up to two streams per wireless device may also be supported.

[0046] Aggregation of multiple cells can be supported with up to eight serving cells. Alternatively, NR may support air interfaces other than OFDM-based air interfaces.

[0047] Some wireless devices may be considered machine-type communications (MTC) or advanced or enhanced machine-type communications (eMTC) wireless devices. MTC and eMTC wireless devices include, for example, robots, drones, remote devices, sensors, meters, monitors, and location tags that can communicate with base stations, other devices (e.g., remote devices), or any other entities. Wireless computing platforms may provide, for example, connectivity to or for a network (e.g., the Internet or a wide area network such as a cellular network) via wired or wireless communication links. Some wireless devices may be considered Internet of Things (IoT) devices or may be implemented as NB-IoT (Narrowband Internet of Things) devices. Wireless devices 120a-120e may be contained within a housing that accommodates the components of wireless devices 120a-120e, such as processor components, memory components, similar components, or combinations thereof.

[0048] In general, any number of communication systems and wireless networks can be deployed in a given geographical area. Each communication system and wireless network may support a specific radio access technology (RAT) and may operate on one or more frequencies. RATs are sometimes called radio technologies or air interfaces. Frequencies are sometimes called carriers or frequency channels. Each frequency may support a single RAT in a given geographical area to avoid interference between communication systems with different RATs. In some cases, 4G / LTE and / or 5G / NR RAT networks may be deployed. For example, a 5G non-standalone (NSA) network may utilize both 4G / LTE RATs on the 4G / LTE RAN side of the 5G NSA network and 5G / NR RATs on the 5G / NR RAN side of the 5G NSA network. Both the 4G / LTE RAN and the 5G / NR RAN may connect to each other and to the 4G / LTE core network (e.g., an Advanced Packet Core (EPC) network) within the 5G NSA network. Other exemplary network configurations may include a 5G standalone (SA) network where the 5G / NR RAN connects to the 5G core network.

[0049] In some implementations, two or more wireless devices (for example, indicated as wireless device 120a and wireless device 120e) may communicate directly using one or more sidelink channels (for example, without using base stations 110a-110d as a medium for communication with each other). For example, wireless devices 120a-120e may communicate using peer-to-peer (P2P) communication, device-to-device (D2D) communication, vehicle-to-everything (V2X) protocols (which may include vehicle-to-vehicle (V2V) protocols, vehicle-to-infrastructure (V2I) protocols, or similar protocols), mesh networks, or similar networks, or a combination thereof. In this case, wireless devices 120a-120e may perform scheduling operations, resource selection operations, and other operations described elsewhere in this specification as being performed by base stations 110a-110d.

[0050] Figure 2 is a block diagram of the components of an exemplary computing device 200 suitable for implementing any of the various embodiments. Referring to Figures 1 and 2, the computing device 200 may include a first SOC processor 202 (e.g., an SOC-CPU) coupled to a second system-on-chip (SOC) 204 (e.g., a 5G-enabled SOC). The first SOC 202 and the second SOC 204 may be coupled to internal memory 206, 216, a display 212, and a speaker 214. In addition, the computing device 200 may include an antenna 218 for transmitting and receiving electromagnetic radiation, which may be connected to a wireless data link and / or wireless transceiver 208, coupled to one or more processors in the first SOC 202 and / or the second SOC 204. One or more processors may be configured to determine the signal intensity level of the signal received by the antenna 218. The computing device 200 may also include a menu selection button or rocker switch 220 for receiving user input. In addition, a software virtual button for receiving user input may be displayed on the display 212.

[0051] The computing device 200 may also include an audio coding / decoding (codec) circuit 210 that digitizes sound received from a microphone into data packets suitable for wireless transmission and decodes the received sound data packets to generate an analog signal provided to a speaker for sound generation. Furthermore, one or more of the processors in the first SOC 202 and the second SOC 204, the wireless transceiver 208, and the codec 210 may include digital signal processor (DSP) circuits (not shown individually). The computing device 200 may also include one or more optical sensors 222, such as a camera. The optical sensors 222 may be coupled to one or more processors in the first SOC 202 and / or the second SOC 204 to control and receive information from the optical sensors 222 (e.g., images, video, etc.).

[0052] The processors of the computing device 200 (e.g., SOCs 202, 204) may be any programmable microprocessor, microcomputer, or one or more multiprocessor chips that can be configured by software instructions (applications) to perform a variety of functions, including the functions of the various embodiments described below. Some wireless devices may have multiple processors, such as one processor in SOC 204 dedicated to wireless communication functions and one processor in SOC 202 dedicated to running other applications. Generally, software applications containing processor-executable instructions may be stored in non-temporary processor-readable storage media, such as memories 206, 216, before the processor-executable instructions are accessed and loaded into the processor. Processors 202, 204 may include internal memory sufficient to store application software instructions. Mobile devices 102a-102e may also include optical sensors, such as cameras (not shown).

[0053] Figure 3 is a component block diagram showing an exemplary computing system 300 architecture suitable for implementing any of the various embodiments. Referring to Figures 1 to 3, the various embodiments can be implemented on several single-processor and multi-processor computer systems, including a system-on-chip (SOC) or a system-in-package (SIP). Computing system 300 may include two SOCs 302, 304, a clock 306, and a voltage regulator 308. In some embodiments, the first SOC 302 operates as a central processing unit (CPU) for a wireless device, executing instructions of a software application program by performing arithmetic, logic, control, and input / output (I / O) operations specified by the instructions. In some embodiments, the second SOC 304 may operate as a dedicated processing unit. For example, the second SOC 304 may operate as a dedicated 5G processing unit, responsible for managing high-capacity, high-speed (e.g., 5 Gbps) and / or very high frequency (e.g., 28 GHz mmWave spectrum) communications.

[0054] The first SOC 302 may include a digital signal processor (DSP) 310, a modem processor 312, a graphics processor 314, an application processor 316, one or more coprocessors 318 (such as a vector coprocessor) connected to one or more of the processors, memory 320, custom circuitry 322, system components and resources 324, an interconnect / bus module 326, one or more temperature sensors 330, a thermal management unit 332, and a thermal power envelope (TPE) component 334. The second SOC 304 may include a 5G modem processor 352, a power management unit 354, an interconnect / bus module 364, multiple mmWave transceivers 356, memory 358, and various additional processors 360, such as an application processor and a packet processor.

[0055] Each processor 310, 312, 314, 316, 318, 352, 360 may include one or more cores, and each processor / core may perform operations independent of other processors / cores. For example, the first SOC302 may include processors running a first type of operating system (e.g., FreeBSD, LINUX, OS X, etc.) and processors running a second type of operating system (e.g., MICROSOFT WINDOWS® 10). In addition, any or all of processors 310, 312, 314, 316, 318, 352, 360 may be included as part of a processor cluster architecture (e.g., a synchronous processor cluster architecture, an asynchronous or heterogeneous processor cluster architecture, etc.).

[0056] The first SOC302 and the second SOC304 may include various system components, resources, and custom circuitry for managing sensor data, analog-to-digital conversion, wireless data transmission, and for performing other specialized operations such as decoding data packets and processing encoded audio and video signals for rendering in a web browser. For example, the system components and resources 324 of the first SOC302 may include a power amplifier, voltage regulator, oscillator, phase-locked loop, peripheral bridge, data controller, memory controller, system controller, access port, timer, and other similar components used to support the processor and software client running on the wireless device. The system components and resources 324, and / or custom circuitry 322, may also include circuitry for interfaced with peripheral devices such as a camera, electronic display, wireless communication device, and external memory chip.

[0057] The first SOC 302 and the second SOC 304 may communicate via the interconnect / bus module 350. Various processors 310, 312, 314, 316, and 318 may be interconnected via the interconnect / bus module 326 to one or more memory elements 320, system components and resources 324, and custom circuitry 322, as well as a thermal management unit 332. Similarly, processor 352 may be interconnected via the interconnect / bus module 364 to a power management unit 354, a mmWave transceiver 356, memory 358, and various additional processors 360. The interconnect / bus modules 326, 350, and 364 may include arrays of reconfigurable logic gates and / or implement a bus architecture (e.g., CoreConnect, AMBA, etc.). Communication may be performed by an advanced interconnect unit such as a high-performance network on chip (NoC).

[0058] The first SOC 302 and / or the second SOC 304 may further include input / output modules (not shown) for communicating with resources outside the SOC, such as a clock 306 and a voltage regulator 308. The resources outside the SOC (e.g., clock 306, voltage regulator 308) may be shared by two or more of the internal SOC processors / cores.

[0059] Various embodiments may be implemented in a wide variety of computing systems in addition to the exemplary computing system 300 described above, and the computing system may include a single processor, multiple processors, a multi-core processor, or any combination thereof.

[0060] Figure 4 is a conceptual diagram showing various embodiments of the method 400 for presenting facial expressions in a virtual conference. Referring to Figures 1 to 4, the computing device 402 may be configured to participate in the virtual conference 430 via a communication network or communication system (the embodiments thereof are discussed above with respect to the communication system 100). In some embodiments, the virtual conference 430 may include one or more avatars 432, 434 of various users, each of which may participate via a computing device such as the computing device 402.

[0061] The computing device 402 may include one or more image sensors, such as a camera 404, and one or more sound sensors, such as a microphone 406. The camera 404 may be instructed to capture an image, such as an image of the user's face 410. The microphone 406 may capture one or more expressive voices 412 of the user.

[0062] The computing device 402 may be configured to detect the user's facial expressions based on information received from the computing device 402's sensors, such as a camera 404 and / or a microphone 406. The computing device 402 may be further configured to generate a representation of the user's facial expression 420. In some embodiments, the representation of the user's facial expression 420 may include one or more images that can be presented on or incorporated into the user's avatar 432 in the virtual conference 430. In some embodiments, the representation of the user's facial expression 420 may include one or more instructions that a device hosting or running the virtual conference 430 can process to present the user's facial expression on the avatar in the virtual conference 430. The representation of the user's facial expression 420 may be presented on an avatar in the virtual conference 430 in a manner perceptible to other participants (e.g., associated with an avatar 434) in the virtual conference 430.

[0063] The computing device 402 may also be configured to generate a representation of the expressive voice 422. In various embodiments, the representation of the expressive voice 422 may be an audio file, a digital bitstream, or another preferred representation. In some embodiments, the representation of the expressive voice 422 may be presented in conjunction with a representation of the user's facial expressions 420 in the virtual conference 430.

[0064] As further described below, the computing device 402 may be configured to determine whether a detected user facial expression is approved or not to be presented in the virtual meeting 430. In some embodiments, the computing device 402 may be configured to present the detected user facial expression for user approval. For example, the computing device 402 may be configured to generate a prompt 412 that the computing device 402 can display on an avatar, for example, on a display device such as a screen. The prompt may include one or more user interface elements that enable the user to provide input approving or not approving / rejecting the facial expression to be presented on the avatar in the virtual meeting 430. In response to receiving input approving or not approving the user's facial expression for presentation, the computing device 402 may store instructions that the user's facial expression is approved or not to be rendered on the avatar. In some embodiments, if a detected facial expression is preset as approved or not to be presented on the avatar, the computing device may update the preset in response to user input different from the preset. Thus, the computing device 402 can learn and / or adjust a list of permitted or acceptable facial expressions and / or unacceptable facial expressions through user interaction.

[0065] Furthermore, the computing device 402 may learn to automatically decide whether to display a virtual facial expression without prompting the user for a response. In some embodiments, the detected expression may be one that the user has previously approved or rejected for presentation on the avatar 432 in the virtual meeting 430. In such embodiments, the computing device 402 may bypass or not present the prompt 412 and proceed to send a representation of the user's facial expression 420 that has been approved for presentation on the avatar 432 in the virtual meeting 430. In some embodiments, the computing device 402 may determine that the detected facial expression has not been previously approved or has been previously rejected by the user. In such embodiments, the computing device 402 may generate a representation of the facial expression that is approved or has been approved for presentation on the avatar 432 in the virtual meeting 430.

[0066] Figure 5 is a process flow diagram showing method 500 for presenting facial expressions in a virtual meeting according to various embodiments. Referring to Figures 1 to 5, method 500 can be implemented by the processors (e.g., 202, 204, 310 to 318, 352, 360) of a computing device (e.g., 120a to 120e, 200, 404).

[0067] In block 502, the processor may detect the user's facial expressions based on information received from sensors of the computing device. In some embodiments, the processor may detect the user's facial expressions based on information received from a camera of the computing device (e.g., camera 404). In some embodiments, the processor may detect the user's facial expressions based on information received from a microphone of the computing device (e.g., microphone 406). In some embodiments, the processor may detect the user's expressive voice and use the expressive voice when detecting the user's facial expressions.

[0068] In decision block 504, the processor may determine whether the detected user's facial expression is approved for presentation on the avatar in the virtual meeting. In some embodiments, the processor may determine whether the detected user's facial expression has been previously approved for presentation on the avatar in the virtual meeting. In some embodiments, the processor may determine whether the detected user's facial expression is included in a list of previously approved or previously unapproved facial expressions, such as a default list or a preset list. In some embodiments, the processor may determine whether the detected user's facial expression is approved for presentation on the avatar in the virtual meeting based on the user's expressive voice. In some embodiments, the expressive voice may be correlated with or associated with the facial expression.

[0069] In response to a decision that the detected user's facial expression is not approved for presentation on the avatar in the virtual meeting (i.e., decision block 504 = "No"), the processor may, in block 506, generate an avatar representing a facial expression that is approved for presentation but is different from the detected user's facial expression. In some embodiments, the processor may generate an avatar representing a recent facial expression approved for presentation on the user's meeting avatar. In some embodiments, the processor may generate an avatar representing the most recently approved user's facial expression. In some embodiments, the processor may generate an avatar representing the most recently approved user's facial expression that is consistent with or closest to the detected user's facial expression. In some embodiments, the processor may maintain the facial expression rendered on the avatar prior to detecting the user's facial expression; in other words, it may continue to present the currently presented avatar.

[0070] In response to a decision that the detected user's facial expression should be approved for display on the avatar in the virtual meeting (i.e., decision block 504 = "Yes"), the processor may generate an avatar that represents a facial expression consistent with the detected user's facial expression.

[0071] Following the execution of the operations in block 506 or 508, the processor may, in any block 510, store in memory an instruction that an avatar representing the user's facial expression is approved or not approved, in response to the reception on the user interface of the computing device of an input indicating whether the user's facial expression is approved or not approved for presentation in a virtual meeting.

[0072] In block 512, the processor may present an avatar in the virtual meeting that has a generated representation of the user's facial expression. For example, the processor may send a representation of the user's facial expression 420 to render on the user's avatar 432 in the virtual meeting 430.

[0073] In any block 514, the processor may, in conjunction with presenting the generated avatar in the virtual conference (i.e., presenting the generated avatar with a representation of the user's facial expressions), render a representation of the user's expressive voice in the virtual conference. For example, the processor may, in conjunction with presenting the generated avatar, render a representation of the expressive voice 422 in the virtual conference (e.g., 430).

[0074] In this case as well, the processor may periodically or continuously execute method 500 by detecting the user's next facial expression in block 502 and performing the operations in blocks 504-512, as explained through the virtual conference.

[0075] Figures 6A and 6B show operations 600a and 600b that can be performed as part of method 500 for presenting facial expressions in a virtual meeting, according to various embodiments. Referring to Figures 1 to 6B, operations 600a and 600b can be implemented by the processors (e.g., 202, 204, 310 to 318, 352, 360) of a computing device (e.g., 120a to 120e, 200, 404).

[0076] Referring to Figure 6A, following the execution of the operation in block 502 (Figure 5), the processor may, in block 602, render an avatar representing a facial expression consistent with the detected user's facial expression onto a user interface configured to receive approval or rejection from the user. For example, the processor may generate a prompt (e.g., 412) configured to receive input to a user interface element indicating whether to approve or reject the facial expression of the presented avatar.

[0077] In decision block 602, the processor may, in response to the user interface, determine whether an input has been received to approve or reject the facial expression of the presented avatar.

[0078] In response to receiving input rejecting the presented avatar's facial expression (i.e., decision block 604 = "reject"), the processor may perform the operation of block 506 of method 500 as described.

[0079] In response to receiving an input approving a facial expression (i.e., decision block 604 = "approved"), the processor may perform the operation of block 508 of method 500 as described.

[0080] If there is no response input to the user interface within the threshold (TH) time period (i.e., decision block 604 = "no response within TH time period"), the processor may perform a default action in block 606. In some embodiments, if there is no response input to the user interface within the threshold time period, the processor may determine that the presented avatar's facial expression is approved for use in the virtual conference and perform the action of block 508 as described. In some embodiments, if there is no response input to the user interface within the threshold time period, the processor may determine that the presented avatar's facial expression is not approved for use in the virtual conference and perform the action of block 506 as described.

[0081] Referring to Figure 6B, following the execution of block 502 of method 500, in some embodiments, the processor may update the preset list based on the user's input, based on the user's input approving or rejecting / not approving facial expressions to be presented on the user's avatar in the virtual meeting.

[0082] For example, in block 610, the processor may determine whether the presented avatar's facial expression is stored in the preset list as either approved or not approved.

[0083] In block 612, the processor may render an avatar representing a facial expression consistent with the detected user's facial expression onto a user interface configured to receive user approval or rejection. For example, the processor may present a prompt 412 on the display.

[0084] In block 614, the processor may update a preset list of approved or unapproved avatar expressions in response to receiving input different from the preset list, indicating whether the user's facial expression is approved or unapproved for presentation in a virtual meeting.

[0085] The various embodiments illustrated and described are provided merely as examples to illustrate various features of the claims. However, features illustrated and described with respect to any given embodiment are not necessarily limited to the embodiment in question and may be used together with or in combination with other embodiments illustrated and described. Furthermore, the claims are not limited by any single exemplary embodiment. For example, one or more of the methods and operations 500, 600a, and 600b may be replaced by or in combination with one or more operations of methods 500, 600a, and 600b.

[0086] Examples of implementations are described in the following paragraphs. Some of the following examples of implementations are described in terms of exemplary methods, but further exemplary implementations may include exemplary methods described in the following paragraphs, which are implemented by a computing device including a processor configured with processor-executable instructions to perform the operations of the methods of the following implementations; exemplary methods described in the following paragraphs, which are implemented by a computing device including means for performing the functions of the methods of the following implementations; and exemplary methods described in the following paragraphs, which can be implemented as a non-temporary processor-readable storage medium storing processor-executable instructions configured to cause the processor of the computing device to perform the operations of the methods of the following implementations.

[0087] Example 1. A method performed by the processor of a computing device for presenting facial expressions on an avatar in a virtual meeting, comprising: detecting a user's facial expression based on information received from sensors of the computing device; determining whether the detected user's facial expression is approved for presentation on an avatar in a virtual meeting; generating an avatar representing a facial expression consistent with the detected user's facial expression in response to the determination that the detected user's facial expression is approved for presentation on an avatar in a virtual meeting; and presenting the generated avatar in a virtual meeting.

[0088] Example 2. The method of Example 1, further comprising the step of continuing to display the currently displayed avatar in response to a decision that the detected user's facial expression is not approved for display on the avatar in a virtual meeting.

[0089] Example 3. The method of Example 1, further comprising the step of generating an avatar that represents a facial expression that is approved for display on an avatar in a virtual meeting, but is different from the facial expression of the detected user, in response to a decision that the detected user's facial expression is not approved for display on an avatar in a virtual meeting.

[0090] Example 4. Any method of Examples 1 to 3, in response to a decision that the detected user's facial expression is not approved for display on an avatar in a virtual meeting, the step of generating an avatar representing a facial expression that is approved for display on an avatar in a virtual meeting but is different from the detected user's facial expression, includes the step of generating an avatar representing a recent facial expression that has been approved for display.

[0091] Example 5. Any method of Examples 1 to 4, wherein the step of detecting a user's facial expression based on information received from a sensor of a computing device includes the step of detecting a user's facial expression based on information received from an image sensor of a computing device.

[0092] Example 6. Any method of Examples 1 to 5, wherein the step of determining whether a detected user's facial expression is approved for display on an avatar in a virtual meeting includes the step of determining whether the user's facial expression has been previously approved for display on an avatar in a virtual meeting, and the step of presenting a generated avatar in a virtual meeting includes, in response to the determination that the user's facial expression has been previously approved for display on an avatar in a virtual meeting, the presenting of a generated avatar in a virtual meeting that represents the previously approved facial expression.

[0093] Example 7. Any method of Examples 1 to 6, wherein the step of determining whether a detected user facial expression is approved for presentation on an avatar in a virtual meeting includes the steps of rendering an avatar representing a facial expression consistent with the detected user facial expression on a user interface configured to receive approval or rejection from the user, and determining that the detected user facial expression is approved in response to the reception on the computing device's user interface of input indicating that the user's facial expression is approved for presentation in a virtual meeting.

[0094] Example 8. The method of Example 7, further comprising the step of determining that if no response input is received to the user interface within a threshold time period, the detected user's facial expression is approved for display on the avatar in the virtual meeting.

[0095] Example 9. Any method of Examples 1 to 8, further comprising the step of storing in memory an instruction that an avatar representing a facial expression consistent with the detected user's facial expression is approved or not approved for presentation on an avatar in a virtual meeting, in response to the reception on the user interface of a computing device of input indicating that the detected user's facial expression is approved or not approved for presentation on an avatar in a virtual meeting.

[0096] Example 10. Any method of Examples 1 to 9, comprising the steps of determining whether a detected user's facial expression is approved for presentation on an avatar in a virtual meeting, determining that the detected user's facial expression is stored in a preset list as approved or not approved, rendering an avatar representing a facial expression consistent with the detected user's facial expression on a user interface configured to receive approval or rejection from the user, and updating the preset list in response to receiving an input different from the preset list indicating whether the user's facial expression is approved or not approved for presentation on an avatar in a virtual meeting.

[0097] Example 11. Any method of Examples 1 to 10, wherein the step of determining whether a detected user's facial expression is approved for display on an avatar in a virtual meeting includes the step of determining whether a detected user's facial expression is approved for display on an avatar in a virtual meeting based on the user's expressive voice.

[0098] Example 12. Any method of Examples 1 to 11, wherein the step of presenting the generated avatar in a virtual meeting includes, in conjunction with the step of presenting the generated avatar in a virtual meeting, the step of rendering a representation of the user's expressive voice in a virtual meeting.

[0099] The above description of the method and process flow diagram are provided only as illustrative examples and do not require or imply that the operations of the various embodiments must be performed in the order presented. As will be understood by those skilled in the art, the order of operations in the above embodiments may be performed in any order. Words such as “then,” “next,” and “then” do not limit the order of operations and are used to guide the reader throughout the description of the method. Furthermore, any reference to a claim element in the singular form using, for example, the articles “a,” “an,” or “the” should not be interpreted as limiting the element to the singular form.

[0100] The various exemplary logic blocks, modules, components, circuits, and algorithmic operations described in relation to the embodiments disclosed herein may be implemented as electronic hardware, computer software, or a combination of both. To clearly demonstrate this hardware and software compatibility, various exemplary components, blocks, modules, circuits, and operations have generally been described above in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. A person skilled in the art may implement the described functionality in various ways for each specific application, but such a determination of embodiment should not be construed as causing a departure from the claims.

[0101] The hardware used to implement the various exemplary logics, logic blocks, modules, and circuits described in relation to the embodiments disclosed herein may be implemented or run using general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of receiver smart objects, for example, a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors working with a DSP core, or any other such configuration. Alternatively, some operations or methods may be performed by circuits specific to a given function.

[0102] In one or more embodiments, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or codes on a non-temporary computer-readable storage medium or a non-temporary processor-readable storage medium. The operation of the methods or algorithms disclosed herein may be embodied in processor-executable software modules or processor-executable instructions that may reside on a non-temporary computer-readable or processor-readable storage medium. A non-temporary computer-readable or processor-readable storage medium may be any storage medium accessible by a computer or processor. Such non-temporary computer-readable or processor-readable storage medium may include, but are not limited to, RAM, ROM, EEPROM, FLASH memory, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage smart objects, or any other medium that may be used to store desired program code in the form of instructions or data structures and may be accessed by a computer. The terms "disk" and "disc" as used herein include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray® discs, where a disk typically reproduces data magnetically, and a disc reproduces data optically using a laser. Combinations of these also fall within the scope of non-temporary computer-readable media and non-temporary processor-readable media. Furthermore, the operation of a method or algorithm may exist as one or any combination or set of code and / or instructions on non-temporary processor-readable storage media and / or non-temporary computer-readable storage media, which may be incorporated within computer program products.

[0103] The foregoing description of the embodiments disclosed is provided to enable any person skilled in the art to construct or use the claims. Various modifications to these embodiments will be readily apparent to a person skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the claims. Accordingly, this disclosure is not intended to be limited to the embodiments shown herein, but should be given the broadest scope that corresponds to the following claims and the principles and novel features disclosed herein. [Explanation of Symbols]

[0104] 100 Communication Systems 102a Macrocell, Mobile Device 102b picocell, mobile device 102c femtocell, mobile device 102d Mobile Device 102e Mobile Device 110a base station (BS), macro base station 110b Base Station (BS) 110c Base Station (BS) 110d base station (BS), relay BS 120a~120e Wireless Devices 122 Wireless communication links 124 Wireless communication links 126 Wired or wireless communication links 130 Network Controllers 140 Core Network 200 computing devices 202 First SOC processor, SOC, processor 204 Second System-on-a-Chip (SOC), Processor 206 internal memory, memory 208 Wire Restaurant Seaba 210 Audio coding / decoding (codec) circuit 212 displays 214 speakers 216 internal memory, memory 218 Antenna 220 Menu selection button or rocker switch 222 Light Sensor 300 Computing Systems 302 SOC, the first SOC 304 SOC, the second SOC 306 Clock 308 Voltage Regulator 310 Digital Signal Processor (DSP), Processor, 312 Modem Processor, Processor 314 Graphics processors, processors 316 Application processors, processors 318 coprocessors, processors 320 memory, memory elements 322 Custom Circuits 324 System Components and Resources 326 Interconnection / Bus Modules 330 Temperature Sensor 332 Thermal Management Unit 334 Thermoelectric Envelope (TPE) Components 350 Interconnection / Bus Modules 352 5G modem processor, processor 354 Power Management Unit 356 mmWave Transceiver 358 memory 360 Processor 364 Interconnection / Bus Modules 400 ways 402 Computing Devices 404 Camera 406 Microphone 410 User Faces 412 Expressive voice, prompt 420 user facial expressions 422 Expressive voice 430 Virtual Meeting 432 Avatars 434 Avatars 500 ways 600a operation 600b operation

Claims

1. A method performed by the processor of a computing device to display facial expressions on an avatar in a virtual meeting, The steps include detecting the user's facial expression based on information received from the sensors of the computing device, The steps include determining whether the detected user's facial expression has been previously approved for display on an avatar in a virtual meeting, In response to the determination that the detected user's facial expression was not previously approved for display on an avatar in the virtual meeting, the steps include generating an avatar that represents a facial expression that is approved for display on an avatar in the virtual meeting, but is different from the detected user's facial expression, In response to the determination that the detected user's facial expression has been previously approved for display on an avatar in the virtual meeting, the steps include generating an avatar that represents a facial expression consistent with the detected user's facial expression, The steps include presenting the generated avatar in the virtual meeting and Methods that include...

2. The method according to claim 1, wherein, in response to a determination that the detected user's facial expression has not been previously approved for display on an avatar in a virtual meeting, the step of generating an avatar that represents a facial expression that is approved for display on an avatar in the virtual meeting, but is different from the detected user's facial expression, includes the step of continuing to display the currently displayed avatar.

3. The method according to claim 1, wherein, in response to a determination that the detected user's facial expression has not been previously approved for presentation on an avatar in a virtual meeting, the step of generating an avatar representing a facial expression that is approved for presentation on an avatar in the virtual meeting, but is different from the detected user's facial expression, includes the step of generating an avatar representing a recent facial expression that has been approved for presentation.

4. The method according to claim 1, wherein the step of detecting a user's facial expression based on information received from a sensor of the computing device includes the step of detecting a user's facial expression based on information received from an image sensor of the computing device.

5. A step of determining whether the detected user's facial expression is approved for display on an avatar in a virtual meeting, The steps include rendering an avatar representing a facial expression consistent with the detected user's facial expression onto a user interface configured to receive approval or rejection from the user, The steps include: determining that the detected user's facial expression is approved in response to the receipt on the user interface of the computing device of input indicating that the user's facial expression is approved for presentation in the virtual meeting; The method according to claim 1, further comprising the step of including

6. If no response input is received to the user interface within a threshold time period, the step of determining that the detected user's facial expression is approved for display on the avatar in the virtual meeting. The method according to claim 5, further comprising:

7. Step 1: In response to receiving input on the user interface of the computing device indicating whether the detected user's facial expression is approved or not to be presented on an avatar in a virtual meeting, store in memory an instruction that an avatar representing a facial expression consistent with the detected user's facial expression is approved or not to be presented in the virtual meeting. The method according to claim 1, further comprising:

8. The step of determining whether the detected user's facial expression has been previously approved for display on an avatar in a virtual meeting is: The steps include determining whether the detected user's facial expression is stored in the preset list as approved or not approved, The steps include rendering an avatar representing a facial expression consistent with the detected user's facial expression onto a user interface configured to receive approval or rejection from the user, The steps include updating the preset list in response to receiving an input different from the preset list that indicates whether the user's facial expression is approved or not to be displayed on the avatar in the virtual meeting, and The method according to claim 1, including the method described in claim 1.

9. The method according to claim 1, further comprising the step of determining whether the detected user's facial expression is approved for display on an avatar in a virtual meeting, the step of determining whether the detected user's facial expression is approved for display on an avatar in the virtual meeting based on the user's expressive voice.

10. The method according to claim 1, wherein the step of presenting the generated avatar in the virtual conference includes, in conjunction with presenting the generated avatar in the virtual conference, the step of rendering a representation of the user's expressive voice in the virtual conference.

11. A computing device, A means for detecting the user's facial expression based on information received from the sensor of the computing device, Means for determining whether the detected user's facial expression has been previously approved for display on an avatar in a virtual meeting, In response to the determination that the detected user's facial expression was not previously approved for display on an avatar in the virtual meeting, means for generating an avatar that represents a facial expression that is approved for display on an avatar in the virtual meeting, but is different from the detected user's facial expression, In response to the determination that the detected user's facial expression has been previously approved for display on an avatar in the virtual conference, means for generating an avatar that represents a facial expression consistent with the detected user's facial expression, means for presenting the generated avatar in the virtual conference Computing devices, including [this].

12. The computing device according to claim 11, which, in response to a determination that the detected user's facial expression has not been previously approved for display on an avatar in a virtual meeting, continues to display the currently displayed avatar, which includes generating an avatar that represents a facial expression that is approved for display on an avatar in the virtual meeting, but is different from the detected user's facial expression.

13. The computing device according to claim 11, wherein, in response to a determination that the detected user's facial expression has not been previously approved for presentation on an avatar in a virtual meeting, the means for generating an avatar representing a facial expression that is approved for presentation on an avatar in the virtual meeting, but is different from the detected user's facial expression, the means for generating an avatar representing a recent facial expression that has been approved for presentation.

14. The computing device according to claim 11, further comprising means for performing the method described in any one of claims 1 to 10.

15. A non-temporary processor-readable storage medium storing processor-executable instructions, wherein the instructions cause a processor of a computing device to execute the method according to any one of claims 1 to 10.

Images