System and method for recording and playing animated emojis - Patents.com

Abstract

Systems and methods are provided for managing animated emoji playback. In particular, a server may monitor user interactions, including users interacting with objects of other users. The server may detect an action performed by a first one of users on an object of a second one of the users, and determine whether the detected action is a trigger action defined by the second user that triggers playback of an animated emoji of the second user. In response to determining that the detected action is a trigger action, the server may display the playback of the animated emoji of the second user to the first user.

Description

[Background technology]

[0001] [CROSS REFERENCE TO RELATED APPLICATIONS] This application claims priority to U.S. Application No. 17 / 507,507, filed on October 22, 2021 (titled "System and Method for Recording and Playing Animated Emojis"), the entire contents of which are incorporated herein by reference in their entirety. [Background technology] Emojis are widely used in various applications to provide users with a means to complement input text with image elements, thereby providing emotional or tonal context. However, the emojis provided by applications are usually limited to a set of predefined emojis and require manual input from the user to send to another user. Therefore, there is a need to develop a personalized emoji recording and playback function to improve the user experience.

[0002] It is with respect to these and other general considerations that the aspects disclosed herein are described, and while relatively specific problems are discussed, it should be understood that these examples are not to be construed as being limited to solving the specific problems identified in the Background or other parts of this disclosure. Summary of the Invention [Means for solving the problem]

[0003] In accordance with at least one example of the present disclosure, a method for managing animated emoji playback is described that includes monitoring user interactions including users interacting with objects of other users, detecting an action performed by a first one of the users, determining whether the detected action is a trigger action defined by the second user that triggers playback of animated emoji of the second user, and, in response to determining that the detected action is a trigger action, displaying a playback of the animated emoji of the second user to the first user.

[0004] In accordance with at least one example of the present disclosure, a computing device for managing animated emoji playback is described that includes a processor and a memory having stored thereon a number of instructions that, when executed by the processor, cause the computing device to: monitor user interactions, including a user interacting with an object of another user; detect an action performed by a first one of the users; determine whether the detected action is a trigger action defined by the second user that triggers playback of an animated emoji of the second user; and, in response to determining that the detected action is a trigger action, display a playback of the animated emoji of the second user to the first user.

[0005] In accordance with at least one example of the present disclosure, a non-transitory computer-readable medium having stored thereon instructions for managing animated emoji playback is described. In an example, the instructions, when executed by one or more processors of a computing device, cause the computing device to: monitor user interactions, including a user interacting with an object of another user; detect an action performed by a first one of the users; determine whether the detected action is a trigger action defined by the second user that triggers playback of an animated emoji of the second user; and, in response to determining that the detected action is a trigger action, display a playback of the animated emoji of the second user to the first user.

[0006] Any of the one or more aspects may be combined with any other aspect of the one or more aspects. Any of the one or more aspects described herein.

[0007] This Summary is provided to introduce in a simplified form a selection of concepts that are further described below in the Detailed Description. This Summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Additional aspects, features, and / or advantages of the examples will be set forth in part in the description that follows, and in part will be obvious from the description, or may be learned by practice of the disclosure. [Brief description of the drawings]

[0008] Non-limiting and non-exhaustive examples are described with reference to the following drawings:

[0009] [Figure 1] FIG. 2 illustrates an exemplary animated emoji management system according to an example of the present disclosure.

[0010] [Diagram 2] FIG. 2 illustrates details of a method for generating one or more animated emojis of a user according to an example of the present disclosure. [Diagram 3] FIG. 2 illustrates details of a method for generating one or more animated emojis of a user according to an example of the present disclosure.

[0011] [Figure 4] FIG. 2 illustrates details of a method for managing animated emoji playback according to an example of the present disclosure.

[0012] [Diagram 5] FIG. 1 is a block diagram illustrating the physical components (e.g., hardware) of a computing device that can be used to implement aspects of the present disclosure.

[0013] [Figure 6A] FIG. 1 illustrates a first example of a computing device that can be used to implement aspects of the present disclosure.

[0014] [Figure 6B] FIG. 2 illustrates a second example of a computing device that can be used to implement aspects of the present disclosure.

[0015] [Figure 7] FIG. 1 illustrates at least one aspect of an architecture of a system for processing data in accordance with an example of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] In the following detailed description, reference is made to the accompanying drawings, which form a part hereof, and in which specific aspects or examples are shown by way of illustration. These aspects may be combined, other aspects may be utilized, and structural changes may be made without departing from the disclosure. These aspects may be embodied as methods, systems, or devices. As such, these aspects may take the form of a hardware implementation, an entirely software implementation, or an implementation combining software and hardware aspects. Therefore, the following detailed description should not be taken as limiting, and the scope of the disclosure is limited by the appended claims and their equivalents.

[0017] According to an example of the present disclosure, the animated emoji management system allows users to create personalized animated emojis with customized facial expressions for their avatars in real time and / or offline. The animated emojis may be used to interact with other users. To that end, some animated emojis may be associated with some actions. For example, a user's profile picture may show different animated emojis based on actions triggered by another user. For example, when a host user uploads content (posts, video clips, etc.), other users may interact with the content by pressing a "Like" button on the content (in the host user feed). When a user "likes" the content, the user sees an automatic change in the host's profile picture to the host's particular animated emoji associated with the "Like" action. In such an example, after the animated emoji is played, the host's profile picture reverts to a still image. It should be understood that the still image may be a still image associated with a particular animated emoji or may be the host's original profile picture.

[0018] In other aspects, a user may use their avatar's personalized animated emoji as a predefined animation when sending messages to or replying to other users. Additionally, animated emoji may be used to respond to messages received from other users. For example, if a user receives a smiley face emoji from another user, that user's particular animated emoji may be automatically sent to the other user.

[0019] In another example, the host user may start recording a live video with avatar effects. Once the avatar effects are applied to the live video, the host's user avatar may follow the host in live mode. For example, the user avatar's facial expressions are driven by the actual host user's face. During the live video, other users may interact with the host user by giving gifts or pressing a "Like" button on the live video. When a gift is given or the "Like" button is pressed by another user, the user's avatar automatically follows the predefined animation of the host's animated emoji that is associated with the triggered action on the live stream video. In other words, the host user's user avatar may respond (e.g., by laughing) to the trigger action (e.g., receiving a gift) no matter what facial expression the host user actually makes. Alternatively, the host user may manually select an animated emoji and play a facial expression of the user avatar that reflects the facial expression of the selected animated emoji. In other words, the animated emoji management system allows users to record and play back personalized animated emojis and interact with other users in real time.

[0020] FIG. 1 illustrates an animated emoji management system 100 for recording and playing personalized animated emojis according to an example of the present disclosure. To this end, a user 110 may generate a user avatar and an animated emoji using a user avatar. For example, the user 110 may generate a user avatar by selecting one or more features from an avatar feature library using an avatar feature selection tool. To this end, the animated emoji management system 100 includes a computing device 120 associated with the user 110 and an animated emoji management server 150 communicatively coupled to the computing device 120 via a network 170. The network 170 may include any type of computing network, including but not limited to a wired or wireless local area network (LAN), a wired or wireless wide area network (WAN), and / or the Internet. The computing device 120 is illustrated in FIG. 1 as, for example, a desktop computer, but may be any portable or non-portable computing device. For example, the computing device 120 may be a smartphone, a laptop, a desktop computer, a server, a wearable electronic device, a smart home appliance, and the like.

[0021] The user 110 may utilize the computing device 120 to generate a user avatar and one or more animated emojis. To this end, the computing device 120 may include a communication interface 126, a processor 122, and a computer-readable storage 124. In an example, the communication interface 126 may be coupled to a network 170 to receive a user avatar database 128, an animated emoji database 130, and / or an avatar feature library 164. The user 110 may select one or more features from the avatar feature library 144 or 164 to generate a user avatar. It should be appreciated that in some aspects, the user 110 may generate multiple avatars associated with the user. To this end, the computing device 120 is configured to include an avatar feature selection tool to provide to the user 110. The avatar feature selection tool includes one or more avatar features stored in the avatar feature library (e.g., 144 and / or 164). For example, the computing device 120 may communicate with the animated emoji management server 150 to access an avatar feature library (e.g., 164) stored in the animated emoji management server 150 via the network 170. Additionally or alternatively, the avatar feature library may be downloaded onto the computing device 120 and stored in the avatar feature library 144. In such an example, the computing device 120 may access the avatar feature library (e.g., 144). By way of example, the one or more avatar features may include any facial features or shapes (e.g., head, hair, eyes, eyebrows, nose, mouth, ears, and wrinkles) and any accessories that may be placed on the facial features (e.g., glasses, hats, earrings, and nose rings). It should be understood that some avatar features may be pre-selected as defaults. Once a user avatar is created, the user avatar is stored in the user avatar database 128 of the computing device 120.Additionally, the computing device 120 may be configured to transmit the user avatar to the animated emoji management server 150 for storage in the user avatar database 158 .

[0022] Additionally, the user 110 may utilize the computing device 120 to generate one or more animated emojis using the user avatar. To this end, the computing device 120 may include one or more applications provided by the computing device 120. In an exemplary embodiment, the one or more applications comprise an animated emoji generator 132. The animated emoji generator 132 further comprises an animated emoji recorder 134, an animated emoji definition unit 136, and an animated emoji updater 138.

[0023] The animated emoji generator 132 is configured to generate one or more animated emojis. In some aspects, the animated emoji generator 132 is configured to receive a request from a user to generate an animated emoji. If the user 110 has more than one user avatar, the request may include an indication of a particular user avatar to be used to generate the animated emoji associated with the user 110.

[0024] The animated emoji recorder 134 records an animation of the user avatar with a customized facial expression of the user. For example, the animated emoji may be an animation of the user avatar with a smiling facial expression of the user. Additionally, in some aspects, the animated emoji recorder 134 may be configured to generate the animated emoji based on predefined facial expressions or gestures. For example, the predefined facial expressions or gestures may include smiling, kissing, crying, or frowning facial expressions, and may include predefined effects such as hearts, stars, tears, confetti, etc.

[0025] The animated emoji definition unit 136 is configured to associate the animated emoji with an action that triggers the playing of the animated emoji. For example, the trigger action may include receiving a "like" on content or receiving a gift during a live video. As an example, an animated emoji with a smiling or laughing facial expression may be associated with the action of receiving a "like" on content. In another example, an animated emoji with a smiling facial expression with a floating heart may be associated with the action of receiving a gift during a live video.

[0026] The animated emoji updater 138 is configured to store the animated emoji with the associated action. For example, the animated emoji may be stored in the animated emoji database 130 of the computing device 120. Additionally, the animated emoji updater 138 is further configured to transmit the animated emoji to the animated emoji management server 150 for storage in the animated emoji database 160.

[0027] Additionally, the animated emoji management server 150 is configured to monitor user interactions with objects uploaded to one or more databases associated with the animated emoji management server 150 and manage the user's animated emojis based on the user's interactions. To this end, the animated emoji management server 150 comprises a communication interface 156, a processor 152, and a computer-readable storage 154. In an example, the communication interface 156 may be coupled to a network 170 and may communicate with one or more computing devices associated with the user to receive and / or transmit user avatars, animated emojis, and / or avatar features. The user avatars are stored in a user avatar database 158. The animated emojis are stored in an animated emoji database 160. The avatar features are stored in an avatar feature library 164.

[0028] As described above, the animated emoji management server 150 is configured to manage animated emojis of users. To this end, one or more applications may be provided by the animated emoji management server 150. In an exemplary embodiment, the one or more applications comprise an animated emoji management unit 162. The animated emoji management unit 162 further comprises an animated emoji update unit 164, a trigger action determination unit 166, and an animated emoji display unit 168.

[0029] In particular, the animated emoji updater 165 is configured to update and store one or more animated emojis created or modified by a user in the animated emoji database 160. The animated emoji updater 165 may be configured to transmit one or more new or updated animated emojis created by a user to a computing device associated with the user.

[0030] The trigger action determiner 166 is configured to monitor user interaction with the object. For example, the trigger action determiner 166 is configured to determine whether a user interacts with another user's (e.g., the host user's) object by "liking," commenting, or otherwise reacting to the object. The object may include any content (e.g., photos, videos, live stream videos) uploaded to the animated emoji management server 150 and / or other servers associated with the animated emoji management system 100 communicatively coupled to the animated emoji management server 150. The object may be stored in the object database 167 of the emoji management server 150. Alternatively, in some examples, the object may be stored in a remote database communicatively coupled to the animated emoji management server 150. If a user interaction is determined, the trigger action determiner 166 is further configured to determine whether the detected action triggers the playing of an animated emoji of the host user. To this end, the trigger action determiner 166 is configured to determine whether the detected action is associated with any one of the animated emojis defined by the host user. If so, the trigger action determiner 166 triggers the playing of the animated emoji.

[0031] The animated emoji display portion 168 is configured to display the animated emoji of the host user to another user (eg, 114) who triggers an action associated with the animated emoji.

[0032] It should be appreciated that in exemplary aspects, the associated animated emoji may be automatically displayed to the user who triggered the detected action without receiving any host input. For example, if the visiting user "likes" a video clip posted by the host user, the animated emoji display unit 168 may automatically display the host user's animated emoji associated with the "like" action to the visiting user. In other words, an autoplay of the animated emoji is displayed to the visiting user who triggered the action associated with the animated emoji. In another example, if the visiting user gives a gift to the host user while watching a live video of the host user, the animated emoji display unit 168 may automatically display the host user's animated emoji associated with such action to the visiting user. It should be appreciated that in some aspects, the animated emoji display unit 168 may display input options to the host user so that the host user can manually send the animated emoji to the guest user. For example, the input options may be embodied as images or icons that appear on a display screen of the host user's computing device. The image or icon may be a shortcut to send a particular animated emoji of the host user associated with receiving the gift, and in response, when the host user selects or touches the image or icon, the animated emoji display unit 168 sends the animated emoji of the host user to the guest user.

[0033] With reference to Figures 2 and 3, a simplified method for generating one or more animated emojis of a user is provided according to an example of the present disclosure. The overall sequence of steps of method 200 is shown in Figures 1 and 2. In general, method 200 begins at 202 and ends at 234. Method 200 may include more or fewer steps, or the sequence of steps may be configured differently than those shown in Figures 2 and 3. Method 200 may be implemented as a set of computer-executable instructions executed by a computer system, and may be encoded or stored on a computer-readable medium. In an exemplary aspect, method 200 is performed by a computing device (e.g., 120) associated with a user (e.g., 110). However, it should be understood that aspects of method 200 may be performed by one or more processing devices, such as computing devices or servers (e.g., 120, 150). Furthermore, method 200 may be performed by gates or circuits associated with a processor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a system on a chip (SOC), a neural processing unit, or other hardware device. In the following, method 200 should be described with reference to the systems, components, modules, software, data structures, user interfaces, etc. described in connection with FIG.

[0034] Method 200 begins at 202 and flow may proceed to process 204. At 204, computing device 120 determines whether a user avatar exists associated with user 110. As will be further described below, if computing device 120 determines in operation 206 that a user avatar exists, method 200 skips further operations and proceeds directly to operation 218 of Figure 3, as indicated by alphanumeric character A in Figures 2 and 3. However, if computing device 120 determines in operation 206 that a user avatar does not exist, method 200 proceeds to operation 208.

[0035] In operation 208, the computing device 120 receives a request to create a user avatar from the user 110. If the computing device 120 determines in operation 210 that a request has not been received, the method 200 returns to operation 208 to continue waiting for a user request. However, if the computing device 120 determines that a request has already been received, the method 200 proceeds to operation 212.

[0036] In operation 212, the computing device 120 provides the user 110 with an avatar feature selection tool including one or more avatar features stored in the avatar feature library 144. It should be understood that the avatar feature library 144 may be received from an avatar feature library 164 stored in the animated emoji management server 150. For example, the one or more avatar features may include any facial features or shapes (e.g., head, hair, eyes, eyebrows, nose, mouth, ears, and wrinkles) and any accessories that can be placed over the facial features (e.g., glasses, hats, earrings, and nose rings). It should be understood that some avatar features may be pre-selected as defaults.

[0037] In operation 214, the computing device 120 receives the user avatar having the one or more features selected by the user. Then, in operation 216, the computing device 120 stores the user avatar in the user avatar database 128 of the computing device 120. Additionally, the computing device 120 transmits the user avatar to the animated emoji management server 150 for storage in the user avatar database 158. Thereafter, the method 200 proceeds to operation 218 in FIG. 3, as indicated by alphanumeric character A in FIGS. 2 and 3, as will be described further below.

[0038] In operation 218, the computing device 120 receives a request from a user to generate an animated emoji. If the computing device 120 determines in operation 220 that a request has not been received, the method 200 returns to operation 218 to continue waiting for the user request. However, if the computing device 120 determines in operation 220 that a request has already been received, the method 200 proceeds to operation 222.

[0039] In operation 222, computing device 120 obtains, retrieves, or otherwise receives a user avatar to be used to generate an animated emoji associated with user 110. As described above, the user avatar may be stored in user avatar database 128 on computing device 120 and / or user avatar database 158 on animated emoji management server 150.

[0040] Then, in operation 224, computing device 120 generates an animated emoji of the user avatar. To this end, computing device 120 records an animation of the user avatar with the user's customized facial expression. It should be appreciated that user 110 may generate animated emoji and play personalized animated emoji for interaction with other users (e.g., 114) in real time (e.g., in user feeds, in real time stream gifting, in chat rooms, or in response to user interactions).

[0041] Additionally, in some aspects, computing device 120 may generate animated emojis based on predefined facial expressions or gestures, as shown in operation 228. For example, the predefined facial expressions or gestures may include facial expressions such as smiling, kissing, crying, or frowning, and may also include predefined effects such as hearts, stars, tears, confetti, etc.

[0042] In operation 230, computing device 120 associates the animated emoji with an action that can trigger playing of the animated emoji. For example, the trigger action may include receiving a "like" on content or receiving a gift during a live video. As an example, an animated emoji with a smiling or laughing facial expression may be associated with the action of receiving a "like" on content. In another example, an animated emoji with a smiling facial expression with a floating heart may be associated with the action of receiving a gift during a live video.

[0043] At operation 232, the computing device 120 stores the animated emoji with the associated action. For example, the animated emoji may be stored in the animated emoji database 130 of the computing device 120. Additionally, the computing device 120 transmits the animated emoji to the animated emoji management server 150 for storage in the animated emoji database 160. The method may end at 234.

[0044] It should be understood that in some examples, the generation of the animated emoji may be performed by the animated emoji management server 150 having similar details as described above. It should also be understood that parts of the animated emoji generation may be performed by the computing device 120 and the animated emoji management server 150 having similar details as described above.

[0045] Referring now to FIG. 4, a detailed method for managing animated emoji playback according to an example of the present disclosure is provided. A general sequence of steps of method 400 is shown in FIG. 4. In general, method 400 begins at 402 and ends at 416. Method 400 may include more or fewer steps, or the order of steps may be configured differently than that shown in FIG. 4. Method 400 may be implemented as a set of computer-executable instructions executed by a computer system, and may be encoded or stored on a computer-readable medium. In an exemplary embodiment, method 400 is performed by a server (e.g., animated emoji management server 150). However, it should be understood that aspects of method 400 may be performed by one or more processing devices, such as computing devices or servers (e.g., 120, 150). Additionally, method 400 may be performed by gates or circuits associated with a processor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a system on a chip (SOC), a neural processing unit, or other hardware device. The method 400 is described below with reference to the systems, components, modules, software, data structures, user interfaces, etc. described in connection with FIGS. 1-3.

[0046] Method 400 may begin at 402 and flow may proceed to process 404. In operation 404, animated emoji management server 150 monitors user interactions with the object. For example, users may interact with the object by "liking," commenting, or otherwise reacting to other users' objects. The object includes any content (e.g., photos, videos, live stream videos) uploaded to animated emoji management server 150 and / or other servers associated with animated emoji management system 100 communicatively coupled to animated emoji management server 150. The object is stored in an object database of emoji management server 150. For example, the object database (e.g., 166) may be stored in emoji management server 150. Alternatively, in some examples, the object database may be communicatively coupled to animated emoji management server 150.

[0047] In operation 406, the animated emoji management server 150 detects an action taken by a user on an object on the host. For example, an action performed by a user may include "liking," commenting, or otherwise reacting to another user's content.

[0048] The animated emoji management server 150 then determines whether the detected motion triggers the playing of an animated emoji of the host, in operation 408. To this end, the animated emoji management server 150 determines 410 whether the detected motion is associated with any one of the animated emojis defined by the host.

[0049] If the animated emoji management server 150 determines in operation 412 that playback has not been triggered, the method 400 returns to operation 404 to continue monitoring user interaction with the object. However, if the animated emoji management server 150 determines that playback of the host's animated emoji is triggered by the detected motion, the method 400 proceeds to operation 414.

[0050] In operation 414, the animated emoji management server 150 automatically displays the animated emoji of the host associated with the detected action. It should be appreciated that in an exemplary aspect, the associated animated emoji may be automatically displayed to the user who triggered the detected action without receiving any host input. For example, if the visiting user "likes" a video clip posted by the host user, the animated emoji management server 150 may automatically display the animated emoji of the host user associated with the "like" action to the visiting user. In other words, the autoplay of the animated emoji is displayed to the visiting user who triggered the action associated with the animated emoji.

[0051] In another example, the host user may start recording a live video with avatar effects. When the avatar effects are applied to the live video, the host's user avatar may follow the host in live mode. For example, the user avatar's facial expressions are driven by the actual host user's face. During the live video, other users may interact with the host user by sending gifts or pressing a "Like" button on the live video. When a gift is sent or the "Like" button is pressed by a guest user, the user's avatar automatically follows a predefined animation of the host user's animated emoji that is associated with the triggered action on the live stream video. In other words, the host user's user avatar may respond (e.g., by laughing) to the trigger action (e.g., receiving a gift) no matter what facial expression the host user actually makes.

[0052] It should be appreciated that in some embodiments, the animated emoji management server 150 may display input options to the host user so that the host user can manually send an animated emoji to the guest user. For example, the input options may be embodied as an image or icon that appears on a display screen of the host user's computing device. The image or icon may be a shortcut to send the host user's particular animated emoji associated with receiving the gift. In response, if the host user selects or touches the image or icon, the host user's animated emoji is sent to the guest user. Once the playback of the animated emoji is displayed to the user who triggered the action, the method may end at 416.

[0053] Although method 400 is described as being performed by animated emoji management server 150, it should be understood that one or more operations of method 400 may be performed by any computing device, such as computing device 120.

[0054] 5 is a block diagram illustrating the physical components (e.g., hardware) of a computing device 500 that can perform the personalized animated emoji recording and playback operations described above and that can be used to implement aspects of the present disclosure. The computing device components described below may be suitable for the computing devices described above. For example, computing device 500 corresponds to computing device 120 of FIG. 1. In a basic configuration, computing device 500 may include at least one processing unit 502 and system memory 504. Depending on the configuration and type of computing device, system memory 504 may include, but is not limited to, volatile storage (e.g., random access memory), non-volatile storage (e.g., read-only memory), flash memory, or any combination of such memory.

[0055] The system memory 504 may include an operating system 505 and one or more program modules 506 suitable for carrying out various aspects disclosed herein. For example, the operating system 505 may be suitable for controlling the operation of the computing device 500. Additionally, aspects of the present disclosure may be carried out in conjunction with a graphics library, other operating systems, or any other application program, but are not limited to any particular application or system. This basic configuration is illustrated in FIG. 5 by those components within dashed line 508. The computing device 500 may have additional features or functionality. For example, the computing device 500 may further include additional data storage devices (removable and / or non-removable), such as magnetic disks, optical disks, or tape. Such additional storage is illustrated in FIG. 5 by removable storage device 509 and non-removable storage device 510.

[0056] As mentioned above, a number of program modules and data files may be stored in the system memory 504. The program modules 506, when executed on at least one processing unit 502, may perform processes including, but not limited to, one or more aspects as described herein. The application 507 includes an animated emoji generator 520, an animated emoji recorder 522, an animated emoji definition unit 524, an animated emoji updater 526, a trigger action determiner 528, and an animated emoji displayer 530, as described in more detail in FIG. 1. Other program modules that may be used in accordance with aspects of the present disclosure may include email and contact applications, word processing applications, spreadsheet applications, database applications, slide presentation applications, drawing programs, or computer-aided application programs, etc., and / or one or more components supported by the system described herein.

[0057] Additionally, aspects of the present disclosure may be implemented in electrical circuits including discrete electronic components, packaged or integrated electronic chips including logic gates, circuits utilizing a microprocessor, or a single chip including electronic components or a microprocessor. For example, aspects of the present disclosure may be implemented via a system-on-chip (SOC) in which each or multiple components shown in FIG. 5 may be integrated onto a single integrated circuit. Such an SOC device may include one or more processing units, graphics units, communication units, system virtualization units, and various application functions, all of which are integrated (or "baked") onto a chip substrate as a single integrated circuit. When operating via a SOC, the functionality described herein regarding the ability of a client to switch protocols may operate via application specific logic integrated with other components of the computing device 500 on a single integrated circuit (chip). Aspects of the present disclosure may also be implemented using other technologies, including but not limited to mechanical, optical, fluidic, and quantum technologies, capable of performing logical operations such as AND, OR, and NOT. Additionally, aspects of the present disclosure may be implemented within a general purpose computer or within any other circuit or system.

[0058] The computing device 500 may also include one or more input devices 512, such as a keyboard, mouse, pen, acoustic or voice input device, touch or swipe input device, etc. An output device 514A, such as a display, speakers, printer, etc. may also be included. An output 514B corresponding to a virtual display may also be included. The above devices are examples and other devices may be used. The computing device 500 may include one or more communication connections 516 that enable communication with other computing devices 550. Examples of suitable communication connections 516 include, but are not limited to, radio frequency (RF) transmitter, receiver, and / or transceiver circuitry, a universal serial bus (USB), a parallel port, and / or a serial port.

[0059] The term computer readable medium as used herein may include computer storage media. Computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, or program modules. System memory 504, removable storage device 509, and non-removable storage device 510 are all examples of computer storage media (e.g., memory storage). Computer storage media may include RAM, ROM, Electrically Erasable Read Only Memory (EEPROM), Flash memory or other memory technology, CD-ROM, Digital Versatile Disk (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other article used to store information and accessible by computing device 500. Any such computer storage media may be part of computing device 500. Computer storage media does not include carrier waves or other propagated or modulated data signals.

[0060] Communication media may be embodied by computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and includes any information delivery media. The term "modulated data signal" may refer to a signal that has one or more characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media may include wired media, such as a wired network or direct-wired connection, and wireless media, such as acoustic, radio frequency (RF), infrared and other wireless media.

[0061] 6A and 6B show a computing device or mobile computing device 600 suitable for performing various aspects disclosed herein, such as a mobile phone, a smartphone, a wearable computer (such as a smart watch), a tablet computer, a laptop computer, a smart home appliance, etc. that can be used to implement aspects of the present disclosure, which can perform the operations of recording and playing personalized animated emojis as described above. Referring to FIG. 6A, one embodiment of a mobile computing device 600 for implementing these aspects is shown. In a basic configuration, the mobile computing device 600 is a handheld computer with both input and output elements. The mobile computing device 600 typically includes a display 605 and one or more input buttons 609 / 610 that allow a user to input information into the mobile computing device 600. The display 605 of the mobile computing device 600 can also function as an input device (e.g., a touch screen display). An optional secondary input element 615 (if included) further allows for another user input. The secondary input element 615 may be a rotary switch, a button, or any other type of manual input element. In alternative embodiments, the mobile computing device 600 may incorporate more or fewer input elements. For example, in some embodiments, the display 605 may not be a touch screen. In yet another alternative embodiment, the mobile computing device 600 is a mobile telephone system, such as a cellular telephone. The mobile computing device 600 may further include an optional keypad 635. The optional keypad 635 may be a physical keypad or a "soft" keypad generated on a touch screen display. In various embodiments, the output elements include the display 605 for displaying a graphical user interface (GUI), a visual indicator 631 (e.g., a light emitting diode), and / or an audio transducer 625 (e.g., a speaker).In some embodiments, the mobile computing device 600 incorporates a vibration transducer for providing haptic feedback to the user. In yet other embodiments, the mobile computing device 600 incorporates input and / or output ports 630, such as an audio input (e.g., a microphone jack), an audio output (e.g., a headphone jack), and a video output (e.g., an HDMI port) to receive and transmit signals from external sources.

[0062] 6B is a block diagram illustrating an architecture of one aspect of a computing device, server, or mobile computing device. That is, the mobile computing device 600 can implement some aspects by incorporating a system (e.g., architecture) 602. The system 602 may be implemented as a "smartphone" capable of running one or more applications (e.g., browser, email, calendar, contact manager, messaging client, games, and media client / player). In some aspects, the system 602 is integrated as a computing device, such as an integrated personal digital assistant (PDA), wireless phone, etc.

[0063] One or more application programs 666 may be loaded into memory 662 and execute on or in conjunction with operating system 664. Examples of application programs include a telephone dialing program, an email program, a personal information manager (PIM) program, a word processing program, a spreadsheet program, an Internet browser program, a messaging program, and / or one or more components supported by the system described herein. System 602 also includes a non-volatile storage area 668 in memory 662. Non-volatile storage area 668 may be used to store persistent information that should not be lost when power is removed from system 602. Applications 666 may use and store information in non-volatile storage area 668, such as emails or other messages used by the email application, in non-volatile storage area 1168. A synchronization application (not shown) also resides on system 602 and is programmed to interact with a corresponding synchronization application resident on the host computer to maintain synchronization of information stored in non-volatile storage area 668 with corresponding information stored on the host computer. It should be understood that other applications (e.g., animated emoji generator 520, animated emoji recorder 522, animated emoji definer 524, animated emoji updater 526, trigger action determiner 528, and animated emoji displayer 530, etc.) may be loaded into memory 662 and executed on the mobile computing device 600 described herein.

[0064] The system 602 includes a power supply 670, which may be implemented as one or more batteries. The power supply 670 may also include an external power source, such as an AC adapter or a powered station that replenishes or recharges the batteries.

[0065] The system 602 may further include a wireless interface layer 672 that performs functions to transmit and receive radio frequency communications. The wireless interface layer 672 facilitates wireless connectivity between the system 602 and the "outside world" via a communications carrier or service provider. Transmissions to and from the wireless interface layer 672 are performed under the control of the operating system 664. In other words, communications received by the wireless interface layer 672 may be distributed to application programs 666 via the operating system 664, and vice versa.

[0066] The visual indicator 620 may be used to provide visual notifications. And / or the audio interface 674 may be used to generate audible notifications via the audio transducer 625. In the illustrated setup, the visual indicator 620 is a light emitting diode (LED) and the audio transducer 625 is a speaker. These devices may be directly coupled to the power source 670 so that when activated, they remain on for a duration specified by the notification mechanism even if the processor 660 / 661 and other components are powered off to conserve battery power. The LED may be programmed to remain illuminated indefinitely until the user performs an action that indicates the powered on state of the device. The audio interface 674 is used to provide audible signals to the user and receive audible signals from the user. For example, the audio interface 674, in addition to being coupled to the audio transducer 625, may be coupled to a microphone to receive audible input, for example, to facilitate a telephone conversation. According to aspects of the present disclosure, the microphone may function as an audio sensor to facilitate control of notifications, as described below. The system 602 may further include a video interface 676 that enables operation of an on-board camera to record still images, video streams, and the like.

[0067] A mobile computing device 600 implementing system 602 may have additional features or functionality. For example, mobile computing device 600 may further include additional data storage devices (removable and / or non-removable) such as magnetic disks, optical disks, or tape. Such additional storage is illustrated in FIG. 6B by non-volatile storage area 668.

[0068] As described above, data / information generated or captured by the mobile computing device 600 and stored via the system 602 may be stored locally on the mobile computing device 600. Alternatively, the data may be stored in any number of storage media accessible to the device via the wireless interface layer 672 or via a wired connection between the mobile computing device 600 and another computing device associated with the mobile computing device 600 (e.g., a server computer in a distributed computing network, such as the Internet). Such data / information may be accessed via the mobile computing device 600, via the wireless interface layer 672, or via a distributed computing network. Similarly, such data / information may be readily transmitted between computing devices for storage and use in accordance with known data / information transmission and storage means, including electronic mail and collaborative data / information sharing systems.

[0069] 7 illustrates one aspect of a system architecture for processing data received at a computing system from a remote source, such as a personal computer 704, a tablet computing device 706, or a mobile computing device 708, as described above. Content displayed at the server device 702 may be stored on different communication channels or other storage types. For example, the computing devices 704, 706, 708 may represent the computing devices 120 of FIG. 1. The server device 702 may represent the animated emoji management server 150 of FIG. 1.

[0070] In some aspects, one or more of the animated emoji updater 724, the trigger action determiner 726, and the animated emoji displayer 728 may be employed by the server device 702. The server device 702 may provide data to and from client computing devices, such as a personal computer 704, a tablet computing device 706, and / or a mobile computing device 708 (e.g., a smartphone) via the network 712. As an example, the computing system described above may be implemented within a personal computer 704, a tablet computing device 706, and / or a mobile computing device 708 (e.g., a smartphone). In addition to receiving graphical data that can be used for pre-processing in the graphically originating system or post-processing in the receiving computing system, any of these aspects of the computing device may retrieve content from the storage 716. The content store may include a user avatar database 718, an animated emoji database 720, an avatar feature library 722, and an object database 725.

[0071] FIG. 7 illustrates an example of a mobile computing device 708 that may perform one or more aspects disclosed herein. Additionally, aspects and functions described herein may operate on a distributed system (e.g., a cloud-based computing system) where application functions, memory, data storage and retrieval, and various processing functions may operate remotely from one another over a distributed computing network (e.g., the Internet or an intranet). Various types of user interfaces and information may be displayed via on-board computing device displays or via remote display units associated with one or more computing devices. For example, various types of user interfaces and information may be displayed and interacted with on a wall that projects the various types of user interfaces and information. Interactions with multiple computing systems that may be utilized to implement aspects of the present invention include keystroke input, touch screen input, voice or other audio input, gesture input, and the like. In the case of gesture input, the associated computing device includes detection (e.g., camera) capabilities to capture and interpret user gestures to control functions of the computing device.

[0072] Terms such as "at least one," "one or more," "or," and "and / or" are open-ended expressions that are conjunctive and disjunctive in operation. For example, each of the expressions "at least one of A, B, and C," "at least one of A, B, or C," "one or more of A, B, and C," "one or more of A, B, or C," "A, B, and / or C," and "A, B, or C" means A only, B only, C only, A and B, A and C, B and C, or A, B and C.

[0073] The term "an" or "an" entity means one or more of that entity. Thus, the terms "one," "one or more," and "at least one" may be used interchangeably herein. It should also be noted that the terms "comprising," "including," and "having" may be used interchangeably.

[0074] As used herein, the term "automatic" and variations thereof refer to any process or operation, usually continuous or semi-continuous, that is performed without significant manual input as the process or operation is performed. However, the performance of a process or operation can be performed automatically with significant or non-critical manual input, provided that the input is accepted prior to the performance of the process or operation. A manual input is considered a significant input if it affects how the process or operation is performed. A manual input to consent to the performance of a process or operation is not considered "critical."

[0075] Any of the steps, functions, and operations discussed herein may be performed continuously and automatically.

[0076] The exemplary systems and methods of the present disclosure have been described in conjunction with a computing device. However, in order to avoid unnecessarily obscuring the present disclosure, the foregoing description has omitted some known structures and devices. This omission should not be construed as limiting. Specific details are described to provide an understanding of the present disclosure. However, it should be understood that the present disclosure may be implemented in a variety of ways in addition to the specific details set forth herein.

[0077] Furthermore, although in the exemplary embodiments illustrated herein, various components of the system are shown to be co-located, some components of the system may be located remotely in a distal portion of a distributed network, such as a LAN and / or the Internet, or may be located in a dedicated system. Thus, it should be understood that the components of the system may be coupled to one or more devices, e.g., a server, a communication device, or may be co-located on a particular node of a distributed network, such as an analog and / or digital telecommunications network, a packet-switched network, or a circuit-switched network. As will be appreciated from the foregoing description, for reasons of computing efficiency, the components of the system may be located anywhere within the distributed network of components without affecting the operation of the system.

[0078] It should further be understood that the various links connecting the elements may be wired or wireless links, or any combination thereof, or any other known or future developed elements capable of providing data to and / or communicating data from the connected elements. These wired or wireless links may also be secure links or capable of communicating encrypted information. For example, the transmission medium used as the link may be any suitable carrier of electrical signals, including coaxial cable, copper wire, and optical fiber. For example, it may take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.

[0079] Although the flowcharts have been discussed and illustrated with reference to particular sequences of events, it should be understood that modifications, additions, and omissions may be made to the sequences without substantially affecting the operation of the disclosed configurations and aspects.

[0080] Several variations and modifications of the disclosure may be used, and it is possible to provide some features of the disclosure and not other features.

[0081] In yet another setting, the systems and methods of the present disclosure can be implemented in combination with special purpose computers, programmed microprocessors or microcontrollers and peripheral integrated circuit elements, ASICs or other integrated circuits, digital signal processors, hardwired electronic or logic circuits (e.g., discrete element circuits), programmable logic devices or gate arrays (e.g., PLDs, PLAs, FPGAs, PALs), special purpose computers, any similar devices, and the like. Overall, any device or means capable of implementing the methods shown herein can be used to implement various aspects of the present disclosure. Exemplary hardware that can be used in the present disclosure includes computers, handheld devices, phones (e.g., cellular phones, Internet-enabled phones, digital phones, analog phones, hybrid phones, etc.), and other hardware known in the art. Some of these devices include a processor (e.g., single or multiple microprocessors), memory, non-volatile storage, input devices, and output devices. Alternative software implementations may also be constructed to implement the methods described herein, including, but not limited to, distributed processing or component / object distributed processing, parallel processing, or virtual machine processing.

[0082] In yet another setting, the disclosed methods can be readily implemented in combination with software using object or object-oriented software development environments that provide portable source code that can be used on a variety of computer or workstation platforms. Alternatively, the disclosed systems may be implemented partially or fully in hardware using standard logic circuits or VLSI designs. Whether software or hardware is used to implement a system according to the present disclosure depends on the speed and / or efficiency requirements of the system, the particular functionality, and the particular software or hardware system or microprocessor or microcomputer system being used.

[0083] In yet another configuration, the disclosed methods may be implemented in part in software that may be stored on a storage medium and executed on a programmed general purpose computer, special purpose computer, microprocessor, etc. in cooperation with a controller and memory. In these instances, the disclosed systems and methods may be implemented as programs, e.g., applets, JAVA, or CGI scripts, installed on a personal computer, as resources resident on a server or computer workstation, as routines installed in dedicated measurement systems, system components, etc. The systems and / or methods may also be realized by physically incorporating them into software and / or hardware systems.

[0084] This disclosure is not limited to the standards and protocols described. Other similar disclosures. Furthermore, the standards and protocols described herein, and other similar standards and protocols not described herein, are periodically replaced by faster or more efficient equivalents having substantially the same functionality. Such replacement standards and protocols having the same functionality are considered equivalents included in this disclosure.

[0085] In various configurations and aspects, the present disclosure includes the components, methods, processes, systems, and / or devices as illustrated and described herein, including various combinations, subcombinations, and subsets thereof. Those skilled in the art, upon understanding the present disclosure, will understand how to make and use the systems and methods of the present disclosure. The present disclosure, in various configurations and aspects, includes providing devices and processes in the absence of items that may have been used in previous devices or processes, for example, to improve performance, provide ease of use, and / or reduce implementation costs, and includes providing devices and processes in the absence of items not illustrated and / or described herein or in its various configurations or aspects.

[0086] The present disclosure relates to systems and methods for rendering motion audio-visual images on a display, at least according to examples provided in the following sections.

[0087] (Al) In one aspect, some examples include a method for rendering motion audio-visual images on a display, the method including monitoring user interactions, detecting an action performed by a first one of users on an object of a second one of the users, determining whether the detected action is a trigger action defined by the second user that triggers playback of an animated emoji of the second user, and, in response to determining that the detected action is a trigger action, displaying playback of the animated emoji of the second user to the first user.

[0088] (A6) In some examples of A1-A5, displaying a playback of the animated emoji of the second user to the first user includes automatically displaying a playback of the animated emoji of the second user to the first user without input from the second user.

[0089] (A7) In some examples of A1-A6, automatically displaying a playback of the animated emoji of the second user includes automatically updating a profile picture of the second user from a user profile picture to the animated emoji, displaying a playback of the animated emoji, and changing the profile picture back to the user profile picture after playback of the animated emoji.

[0090] (A8) In some examples of A1-A7, automatically displaying a playback of the animated emoji of the second user includes automatically updating a user avatar of the second user during live video streaming to display a playback of the animated emoji of the user avatar regardless of a current facial expression of the second user.

[0091] In another aspect, some examples include a computing device for managing animated emoji playback, the computing device comprising one or more processors and a memory coupled to the one or more processors, the memory storing one or more instructions that, when executed by the one or more processors, cause the computing device to perform any of the methods described herein (e.g., A1-A8 above).

[0092] In yet another aspect, some examples include a non-transitory computer-readable medium having stored thereon instructions for managing animated emoji playback that, when executed by one or more processors of a computing device, cause the computing device to perform any of the methods described herein (e.g., A1-A8 above).

[0093] For example, aspects of the disclosure are described above with reference to block diagrams and / or operational diagrams of methods, systems, and computer program products according to aspects of the disclosure. The functions / acts noted in the blocks may occur in an order different from the order shown in any flowchart. For example, two blocks shown in succession may be executed substantially simultaneously or the blocks may sometimes be executed in the reverse order, depending on the functions / acts involved.

[0094] The description and illustration of one or more aspects provided in this application are not intended to restrict or limit the scope of the claimed disclosure in any manner. The aspects, examples, and details described herein are deemed sufficient to convey ownership and to enable others to make and use the best mode of the claimed disclosure. The claimed disclosure should not be construed as being limited to the aspects, examples, or details described herein. Various features (structural features and method features), whether illustrated or described in combination or individually, are intended to be selectively included or omitted to form an embodiment having a particular set of features. The description and illustrations provided in this application may enable those skilled in the art to envision changes, modifications, and alternative embodiments that fall within the spirit of the broader aspects of the general inventive concept embodied in this application without departing from the broader scope of the claimed disclosure.

Claims

1. 1. A method for managing animated emoji playback, comprising: monitoring user interactions, including users interacting with other users' objects; Detecting an action performed by a first one of the users; determining whether the detected action is a trigger action defined by a second user that triggers playing an animated emoji of the second user; responsive to determining that the detected motion is the trigger motion, displaying a playback of the animated emoji of the second user to the first user; generating one or more animated emojis by recording an animation of a user avatar having a facial expression of the second user; and associating the one or more animated emojis with one or more trigger actions defined by the second user; Determining whether the detected motion is a trigger motion includes determining whether the detected motion is associated with an animated emoji defined by the second user. method.

2. The objects include photos, videos, and / or live stream videos. The method of claim 1.

3. The actions include commenting on and / or reacting to actions performed by the first one of the users on an object of a second one of the users. The method of claim 1.

4. Displaying the playback of the animated emoji of the second user to the first user includes automatically displaying the playback of the animated emoji of the second user to the first user without input from the second user. The method of claim 1.

5. Automatically displaying a playback of the animated emoji of the second user includes: automatically updating a profile picture of the second user from a user profile picture to the animated emoji; displaying a playback of the animated emoji; changing the profile picture back to the user profile picture after playing the animated emoji; The method of claim 4 , comprising:

6. Automatically displaying the reproduction of the animated emoji of the second user includes: and automatically updating the user avatar during live video streaming to display a playback of the animated emoji of the user avatar of the second user regardless of a current facial expression of the second user. The method according to claim 4.

7. 1. A computing device for managing animated emoji playback, comprising: A processor; and a memory storing a plurality of instructions, which, when executed by the processor, cause the computing device to: monitoring user interactions, including users interacting with other users' objects; Detecting an action performed by a first one of the users; determining whether the detected action is a trigger action defined by a second user that triggers playing an animated emoji of the second user; responsive to determining that the detected motion is a trigger motion, displaying a playback of the animated emoji of the second user to the first user; generating one or more animated emojis by recording an animation of a user avatar having a facial expression of the second user; associating the one or more animated emojis with one or more trigger actions defined by the second user; Determining whether the detected motion is a trigger motion includes determining whether the detected motion is associated with an animated emoji defined by the second user. Computing device.

8. The object includes a photo, a video, and / or a live stream video, and the action includes commenting and / or reacting to an action performed by the first one of the users on an object of a second one of the users.

8. The computing device of claim 7.

9. Displaying the playback of the animated emoji of the second user to the first user includes automatically displaying the playback of the animated emoji of the second user to the first user without input from the second user.

8. The computing device of claim 7.

10. Automatically displaying the reproduction of the animated emoji of the second user includes: automatically updating a profile picture of the second user from a user profile picture to the animated emoji; displaying a playback of the animated emoji; changing the profile picture back to the user profile picture after playing the animated emoji; 10. The computing device of claim 9, comprising:

11. Automatically displaying the playback of the animated emoji of the second user includes automatically updating the user avatar during live video streaming to display a playback of the animated emoji of a user avatar of the second user regardless of a current facial expression of the second user.

10. The computing device of claim 9.

12. 1. A non-transitory computer-readable medium storing instructions for managing animated emoji playback, comprising: The instructions, when executed by one or more processors of a computing device, cause the computing device to: monitoring user interactions, including users interacting with other users' objects; Detecting an action performed by a first one of the users; determining whether the detected action is a trigger action defined by a second user that triggers playing an animated emoji of the second user; responsive to determining that the detected motion is a trigger motion, displaying a playback of the animated emoji of the second user to the first user; generating one or more animated emojis by recording an animation of a user avatar having customized facial expressions of the second user; associating the one or more animated emojis with one or more trigger actions defined by the second user; Determining whether the detected motion is a trigger motion includes determining whether the detected motion is associated with an animated emoji defined by the second user. Non-transitory computer-readable medium.

13. Displaying the playback of the animated emoji of the second user to the first user includes automatically displaying the playback of the animated emoji of the second user to the first user without input from the second user.

13. The non-transitory computer readable medium of claim 12.

14. The objects include photos, videos, and / or live stream videos.

13. The non-transitory computer readable medium of claim 12.

15. When executed by one or more processors of a computing device, the computing device is monitoring user interactions, including users interacting with other users' objects; Detecting an action performed by a first one of the users; determining whether the detected action is a trigger action defined by a second user that triggers playing an animated emoji of the second user; responsive to determining that the detected motion is a trigger motion, displaying a playback of the animated emoji of the second user to the first user; generating one or more animated emojis by recording an animation of a user avatar having customized facial expressions of the second user; associating the one or more animated emojis with one or more trigger actions defined by the second user; Determining whether the detected motion is a trigger motion includes determining whether the detected motion is associated with an animated emoji defined by the second user. Computer program.

Images