Computing system, computing method, server computing device, and program for rendering an avatar

Abstract

A computing system comprising a processor is provided, the processor being configured to execute an augmented reality avatar program to receive, via a computer network, second user avatar data object and second user pose information, receive an image from a live image feed obtained by a camera of the computing device, identify a virtual plane corresponding to a physical plane appearing in the image, calculate a position and pose of a second user avatar with respect to the virtual plane based on the second user pose information, and display the second user avatar of the second user superimposed on the image by rendering the second user avatar data object. The second user avatar is anchored to the virtual plane according to the calculated position and pose of the second user avatar. The virtual plane is world-locked to the physical plane in the image within a real-world 3D environment.

Description

Technical Field

[0001] Cross - Reference to Related Applications This application claims priority to U.S. Patent No. 17 / 806,701, filed on June 13, 2022 (invention title: Computing System and Computing Method for Rendering an Avatar), the disclosure of which is hereby incorporated by reference in its entirety.

Background Art

[0002] As online platforms for human interaction, such as social media services, messaging services, and online game services, have proliferated, people are increasingly using avatars to represent themselves in the online world. An avatar, broadly speaking, is an image that represents a user in a virtual world and often takes the form of a human, a cartoon character, or any object. Some avatars are two - dimensional while others are three - dimensional. Some avatars are set to move or change expressions in accordance with the movements and expressions of the user captured by a camera. However, even with such animated avatars, the interaction with other people's avatars in a video chat, for example, can be said to lack a sense of unity among users. Therefore, there is an opportunity to improve the user experience when using avatars in computer - based interactions with others.

Summary of the Invention

Means for Solving the Problems

[0003] In view of the above, a computing system is provided that includes a processor and a memory of a first user's computing device. The processor is configured to execute an extended reality avatar program using a portion of the memory to receive, via a computer network, a second user avatar data object and second user pose information of a second user. The processor is further configured to receive an image from a live image feed obtained by a camera of the computing device, identify a virtual surface corresponding to a physical surface appearing in the image, calculate a position and pose of the second user avatar with respect to the virtual surface based on the second user pose information, and render the second user avatar data object to display the second user avatar of the second user superimposed on the image. The second user avatar is anchored to the virtual surface according to the calculated position and pose of the second user avatar. The virtual surface is world-locked to the physical surface in the image within a three-dimensional environment of the real world.

[0004] This summary is provided to introduce, in a simplified form, a selection of concepts that are further described in detail in the following embodiments for implementing the invention. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to implementations that solve any or all of the disadvantages described in any part of this disclosure.

Brief Description of the Drawings

[0005]

Figure 1

[0006]

Figure 2

[0007]

Figure 3

[0008]

Figure 4

[0009]

Figure 5

[0010]

Figure 6

[0011]

Figure 7

[0012]

Figure 8

[0013]

Figure 9

DETAILED DESCRIPTION OF THE INVENTION

[0014] Referring to FIGS. 1 and 2, computing system 10 includes a first computing device 12A for a first user, a second computing device 12B for a second user, and a third computing device 12C for a third user. The number of users and their computing devices is not particularly limited, and it will be understood that the number of computing devices in computing system 10 may be three or more, or only two. The first computing device 12A of the first user includes a processor 28, a memory 30, a front camera 24A configured to acquire a first user image 25A of the first user, a rear camera 26A configured to acquire a live image feed and an image 27A, and a display 22A configured to display the image 27A. Since other computing devices in computing system 10 also include hardware similar to that of the first computing device 12A, it will be understood that the second computing device 12B also has a processor, a memory, a front camera 24B, a rear camera 26B, and a display 22B, and the third computing device 12C also includes a processor, a memory, a front camera 24C, a rear camera 26C, and a display 22C.

[0015] The processor 28 is configured to receive, via the computer network 136, the second user avatar data object 106b and the second user pose information 120 of the second user by executing the augmented reality program 14 using a part of the memory 30. The augmented reality program 14 receives the image 27A from the live image feed acquired by the rear camera 26A, identifies the virtual plane 34 corresponding to the physical plane appearing in the image 27A, calculates the position and pose of the second user avatar 132b relative to the virtual plane 34 based on the second user pose information 120, and renders the second user avatar data object to display the second user avatar 132b of the second user superimposed on the image 27A. The second user avatar 132b is anchored to the virtual plane 34 according to the calculated position and pose of the second user avatar 132b. The virtual plane 34 is world-locked to the physical plane of the image 27A in the three-dimensional environment of the real world. The second user pose information 120 and the image 27A may be the initial second user pose information and the initial image of the session of the augmented reality program 14, respectively.

[0016] The processor 28 also executes the pose tracking module 16. The pose tracking module 16 receives the first user image 25A acquired by the front camera 24A, identifies the tracking features in the first user image 25A of the first user, generates the first user pose information 108a of the first user based on the identified tracking features, and transmits the first user pose information 108a via the computer network 136 to the second computing device 12B of the second user and also to the three-dimensional engine 20 of the first computing device 12A for rendering and displaying the first user avatar 132a.

[0017] The image 27A from the live image feed acquired by the rear camera 26A is received by the augmented reality module 18, which identifies the virtual surface 34 corresponding to the physical surface appearing in the image 27A by identifying the surface features 36 of the virtual surface 34. Optionally, the augmented reality module 18 may perform simultaneous location and mapping (SLAM) in the physical environment and create a map of the physical environment including a mesh of the reconstructed surface including the virtual surface 34. The identified virtual surface 34 may be included in the SLAM data 32 including the map generated by the augmented reality module 18.

[0018] To render the avatars 132a, 132b, 132c so as to be overlaid on the image 27A, the three-dimensional engine 20 receives the SLAM data 32 of the first computing device 12A, the pose information 108a-c of all users in the image 27A (the first user pose information 108a of the first user, the second user pose information 108b of the second user, the third user pose information 108c of the third user), and the avatar data objects 106a-c of all users in the image 27A (the first user avatar data object 106a of the first user, the second user avatar data object 106b of the second user, the third user avatar data object 106c of the third user), and may calculate the positions and poses of some or all of the users' avatars 132a-c (the first user avatar 132a, the second user avatar 132b, the third user avatar 132c) with respect to the identified virtual surface 34. Then, the three-dimensional engine 20 displays the rendered avatars 132a-c of the users overlaid on the image 27A on the display 22A of the first computing device 12A by rendering some or all of the users' avatars 132a-c. The three-dimensional engine 20 may be, for example, UNITY3D, UNREAL ENGINE, or EffectSDK.

[0019] Next, execute the augmented reality program 14. In the loop, receive the updated second user pose information 126 of the second user via the computer network 136, receive the updated image 27A from the live image feed acquired by the rear camera 26A of the first computing device 12A, calculate the position and pose of the second user avatar 132b with respect to the virtual plane 34 based on the updated second user pose information 126, and display the second user avatar 132b of the second user superimposed on the updated image 27A by rendering the second user avatar data object 106b. The second user avatar 132b remains anchored to the virtual plane 34 locked to the physical plane within the image 27A in the three-dimensional real-world environment, and poses according to the updated second user pose information 126 each time it passes through the loop.

[0020] To implement a similar loop feedback process for the other avatars 132a, 132c, the first user avatar 132a poses according to the first user pose information 108a that is updated each time it passes through the loop while the first user avatar 132a remains anchored to the virtual plane 34, and the third user avatar 132c poses according to the third user pose information 108c that is updated each time it passes through the loop while the third user avatar 132c remains anchored to the virtual plane 34.

[0021] The second user pose information 120 and the updated second user pose information 126 may each include second face tracking data 122, 128. The processor 28 may be configured to display a second user avatar 132b of the second user posing with a face corresponding to the expression of the second user by rendering a second user avatar data object 132b of the second user based on the second face tracking data 122, 128 by executing the three-dimensional engine 20. The face of the second user avatar 132b may be rendered such that its appearance changes as the second face tracking data 128 continuously passes through the loop. By also implementing a similar loop feedback process for the third user avatar 132c, the third user avatar 132c may be rendered such that its appearance changes as the face tracking data of the third user pose information 108c continuously passes through the loop.

[0022] A similar loop feedback process may also be implemented for the first user avatar 132a. The first user pose information 108a may include first face tracking data 138. The processor 28 may be configured to display a first user avatar 132a of the first user posing with a face corresponding to the expression of the first user by rendering a first user avatar data object 106a of the first user based on the first face tracking data 138 of the first user pose information 108a by executing the three-dimensional engine 20. The face of the first user avatar 132a may be rendered such that its appearance changes as the first face tracking data 138 continuously passes through the loop.

[0023] The second user pose information 120 and the updated second user pose information 126 may each include second skeleton tracking data 124, 130. By executing the three-dimensional engine 20, the processor 28 is configured to render the body of the second user avatar 132b so that its appearance changes as the second skeleton tracking data 124, 130 change while continuously passing through a loop. By implementing a similar loop feedback process for the third user avatar 132c as well, the body of the third user avatar 132c may be rendered so that its appearance changes as the skeleton tracking data of the third user pose information 108c changes while continuously passing through a loop.

[0024] A similar loop feedback process may also be implemented for the first user avatar 132a. The first user pose information 108a may include first skeleton tracking data 140. By executing the three-dimensional engine 20, the processor 28 is configured to render the body of the first user avatar 132a so that its appearance changes as the first skeleton tracking data 140 changes while continuously passing through a loop.

[0025] In response to generating the first user pose information 108a, the processor 28 is configured to display the first user avatar 132a of the first user superimposed on the updated image 27A by rendering the first user avatar data object 106a of the first user, so that the first user avatar 132a poses according to the first user pose information 108a.

[0026] In this example, the pose tracking module 16 is shown to receive the front camera image 25A from the front camera 24A. However, alternatively or additionally, the pose tracking module 16 may be configured to receive the rear camera image 27A from the rear camera 26A. Thereby, the processor 28, by executing the pose tracking module 16, identifies tracking features in the image 27A and the updated image 27A acquired by the rear camera 26A and the first user image 25A of the first user, generates the first user pose information 108a of the first user based on the identified tracking features, and is configured to transmit the first user pose information 108a to the second computing device 12B of the second user via the computer network 136. In this example, the front camera 24A and the rear camera 26A are operated to acquire the first user image 25A and the updated image 27A in real time, respectively, with a time stamp that synchronizes the identified tracking features in the image 27A and the updated image 27A with the identified tracking features in the first user image 25A of the first user.

[0027] In one scenario, a user participates in a live augmented reality session 134 hosted by a server computing system 100 of a social media platform, where the user has their respective associated accounts. In the example of FIG. 1, a first computing device 12A of a first user, a second computing device 12B of a second user, and a third computing device 12C of a third user each execute an instance of an augmented reality avatar program 15 configured to communicate with the server computing system 100 of the social media platform, and also execute instances of an augmented reality program 14, a pose tracking module 16, an augmented reality module 18, and a three-dimensional engine 20. First user pose information 108a of the first user, a first user avatar data object 106a of the first user, second user pose information 120, updated second user pose information 126, and a second user avatar data object 106b are exchanged between the augmented reality avatar program 15 executed on the first computing device 12A and the augmented reality avatar program 15 executed on the second computing device 12B via the server computing system 100. It will be appreciated that the pose information of each user is continuously sampled within the control loop for the live augmented reality session 134. Thus, to clarify the first time the second user pose information 120 is acquired, this second user pose information 120 may be referred to as initial second user pose information.

[0028] As shown in FIG. 1 and more particularly shown in FIG. 2, the avatar data object and pose information may be packaged by the server computing device 100 into a data container or a configuration file, and the server computing device 100 transmits the data container to each computing device of the computing system 10. The avatar data object may be a configuration file, which is a description file that defines a list of properties and attributes of the avatar, and is transmitted to the server computing device 100 by the computing devices 12A - C. For example, the configuration file may define hair color, eye color, clothing, accessories, virtual skeleton structure, body skin shape, expression control points, etc.

[0029] In the example of FIG. 1, the server computing device 100 transmits a first avatar data container 118a including a second user avatar data object 106b and a third user avatar data object 106c to the first computing device 12A, and transmits a first pose data container 116a including a second user pose information 108b and a third user pose information 108c to the first computing device 12A. The second computing device 12B transmits a second configuration file 104b including the second user avatar data object 106b to the server computing device 100, and the third computing device 12C transmits a third configuration file 104c including the third user avatar data object 106c to the server computing device 100.

[0030] Figure 2 is a schematic diagram of a server computing device 100 that exchanges pose information and avatar data objects with the three computing devices 12A - C of Figure 1. The server computing device 100 includes a processor 114, a non - volatile storage device 102 operatively coupled to the processor 114, and an avatar hosting program 110 stored in the non - volatile storage device 102, which may be loaded into a volatile memory 112 and executed by the processor 114 of the server computing device 100.

[0031] In an initialization process, when avatars 132a - c are first rendered on the displays 22A - C of the computing devices 12A - C, the avatar hosting program 110 is configured to receive, by hosting an extended reality session 134, second user pose information 108b and a second configuration file 104b including a second user avatar data object 106b from a second computing device 12B of a second user. The server computing device also receives an initial first user pose information 108a and a first configuration file 104a including a first user avatar data object 106a from a first computing device 12A of a first user, and an initial third user pose information 108c and a third configuration file 104c including a third user avatar data object 106c from a third computing device 12C of a third user. Then, the avatar hosting program 110 stores the first configuration file 104a, the second configuration file 104b, and the third configuration file 104c in an avatar configuration database 104, and stores the initial first user pose information 108a, the second user pose information 108b, and the initial third user pose information 108c in an avatar pose information database 108.

[0032] In response to receiving a first request from the first computing device 12A, the avatar hosting program 110 transmits a second user avatar data object 106b and second user pose information 108b to the first computing device 12A. The first request may be the first configuration file 104a and the first user pose information 108a of the first user. In this example, the avatars of three users are rendered on the display 22A of the first computing device 12A, and the avatar hosting program 110 transmits to the first computing device 12A a first avatar data container 118a including the second user avatar data object 106b and the third user avatar data object 106c, and a first pose data container 116a including the second user pose information 108b and the third user pose information 108c. Thereby, by rendering the second user avatar data object 106b and the third user avatar data object 106c on the first computing device 12A, the first computing device 12A is caused to display the second user avatar and the third user avatar on the display 22A.

[0033] In response to receiving a second request from the second computing device 12B, the avatar hosting program 110 transmits the first user avatar data object 106a and the initial first user pose information 108a to the second computing device 12B. The second request may be the second configuration file 104b and the second user pose information 108b of the second user. In this example, the avatars of three users are rendered on the display 22B of the second computing device 12B, and the avatar hosting program 110 transmits to the second computing device 12B a second avatar data container 118b including the first user avatar data object 106a and the third user avatar data object 106c, and a second pose data container 116b including the first user pose information 108a and the third user pose information 108c. Thereby, by rendering the first user avatar data object 106a and the third user avatar data object 106c on the second computing device 12B, the second computing device 12B is caused to display the first user avatar and the third user avatar on the display 22B.

[0034] In response to receiving a third request from the third computing device 12C, the avatar hosting program 110 transmits a third user avatar data object 106c and initial third user pose information 108a to the third computing device 12C. The third request may be a third configuration file 104c and third user pose information 108c of the third user. In this example, the avatars of the three users are rendered on the display 22C of the third computing device 12C, and the avatar hosting program 110 transmits to the third computing device 12C a third avatar data container 118c including the first user avatar data object 106a and the second user avatar data object 106b, and a third pose data container 116c including the first user pose information 108a and the second user pose information 108b. Thereby, by rendering the first user avatar data object 106a and the second user avatar data object 106b on the third computing device 12C, the second user avatar and the third user avatar are caused to be displayed on the display 22C.

[0035] A runtime process in which an avatar is continuously re-rendered using pose information that is continuously updated in a loop by a display of a computing device is executed following an initialization process. In this loop, the processor 114 of the server computing device 100 is configured to receive updated second user pose information 108b of a second user from the second computing device 12B. In response to receiving the updated second user pose information 108b, the processor 114 transmits the updated second user pose information 108b to the first computing device 12A and the third computing device 12C, causing the first computing device 12A to render the second user avatar data object 106b using the updated second user pose information 108b and display the second user avatar on the display 22A, and causing the third computing device 12C to render the second user avatar data object 106b using the updated second user pose information 108b and display the second user avatar on the display 22C. Then, updated first user pose information 108a of the first user is received from the first computing device 12A, and updated third user pose information 108c of the third user is received from the third computing device 12C. In response to receiving the updated first user pose information 108a and the updated third user pose information 108c, the updated first user pose information 108a and the updated third user pose information 108c are transmitted to the second computing device 12B, causing the second computing device 12B to render the first user avatar data object 106a using the updated first user pose information 108a, render the third user avatar data object 106c using the updated third user pose information 108c, and display the first user avatar and the third user avatar on the display 22B.

[0036] By executing similar runtime processes on the first computing device 12A, the updated second user pose information 108b is used to render the second user avatar data object 106b, the updated third user pose information 108c is used to render the third user avatar data object 106c, and it will be understood that the second user avatar and the third user avatar may be displayed on the display 22A. Similarly, by executing similar runtime processes on the third computing device 12C, the updated second user pose information 108b is used to render the second user avatar data object 106b, the updated first user pose information 108a is used to render the first user avatar data object 106a, and the first user avatar and the second user avatar may be displayed on the display 22C.

[0037] The initial first user pose information 108a and the updated first user pose information 108a may include first face tracking data, the second user pose information 108b and the updated second user pose information 108b may include second face tracking data, and the third user pose information 108c may include third face tracking data. For example, the second computing device 12B may be made to display the first user avatar of the first user posing with a face corresponding to the expression of the first user by rendering the first user avatar data object 106a of the first user based on the first face tracking data. The first computing device 12A may be made to display the second user avatar of the second user posing with a face corresponding to the expression of the second user on the display 22A by rendering the second user avatar data object 106b of the second user based on the second face tracking data.

[0038] Here, with reference to FIGS. 3 and 4, an exemplary use case demonstrating aspects of the present disclosure is presented. As schematically shown in FIG. 3, a first user 202 stands within a dining room 212 in a real-world three-dimensional environment 200 and may hold a first computing device 12A, which may take the form of a smartphone in this example. However, the computing device is not particularly limited to a smartphone and may take any form, such as a dedicated smartphone and tablet device, a mixed reality head-mounted device (HMD), etc., that enables a user to view an augmented reality image overlaid on the real-world three-dimensional environment 200.

[0039] The first computing device 12A creates a map of the dining room 212 that includes a mesh of the reconstructed virtual surface and a virtual surface 234 corresponding to the physical surface 210, which is the table surface in this exemplary use case, by performing SLAM mapping within the physical environment 200 of the dining room 212. The virtual surface 234 may be planar or curved (as shown in the figure) and is typically identified by tracking visual features in the rear camera image using an optical flow camera or a depth camera. Object recognition technology may be used to identify the virtual surface. Instead of the virtual surface, appropriate anchor points for anchoring each avatar may be programmatically identified within the rear camera image. The first computing device 12A receives the second user pose information of the second user from the second computing device and the third user pose information of the third user from the third computing device.

[0040] The first computing device 12A calculates the positions and poses of the first user avatar 132a, the second user avatar 132b, and the third user avatar 132c with respect to the identified virtual surface world-locked to the physical surface 210 in the real-world three-dimensional environment 200 of the dining room 212. Then, the first computing device 12A renders the first user avatar 132a, the second user avatar 132b, and the third user avatar 132c and displays the rendered avatars 132a - c superimposed on the image 27A on the display 22A of the first computing device 12A. And the first user 202 looking at the display 22A of the first computing device 12A sees the complete avatars 132a - c rendered at an appropriate screen position and in an appropriate pose on the display 22A, which may give the first user 202 the illusion that the avatars 132a - c are standing in the real-world three-dimensional environment 200 of the dining room 212 as shown in FIG. 3. Presenting other user avatars in the same room as the user's avatar may enhance the sense of togetherness for the interaction.

[0041] Referring to FIG. 5, the exemplary use cases shown in FIGS. 3 and 4 are used to demonstrate changes in the appearance of avatars 132a - c on the display 22A of the first computing device 12A during the runtime process such that one or more portions of the avatars 132a - c are associated with movement. The first computing device 12A is configured to render the body of the first user avatar 132a based on the initial first user pose information 120a, the body of the second user avatar 132b based on the second user pose information 120b, and the body and face of the third user avatar 132c based on the initial third user pose information 120c during the initialization process. During the runtime process, as indicated by the dashed lines, the first computing device 12A is configured to render the body of the first user avatar 132a based on the updated first user pose information 126a (the first user turns their neck), the body of the second user avatar 132b based on the updated second user pose information 126b (the second user raises a thumb), and the body and face of the third user avatar 132c based on the updated third user pose information 126c (the third user raises a hand and frowns).

[0042] Accordingly, during the runtime process, as the faces and bodies of the first user avatar 132a, the second user avatar 132b, and the third user avatar 132c change as the updated first user pose information 126a, the updated second user pose information 126b, and the updated third user pose information 126c continuously pass through the loop, they are rendered such that their appearance changes. For this reason, the rendering of avatars 132a - c realizes a reality enhanced by the realistic rendering of the faces and bodies of avatars 132a - c, enabling users using their respective computing devices to have a more intimate community experience in their online interactions during the augmented reality session. In the online interaction, in addition to voice and text, they can communicate with each other using expressions and body language, and can enhance the physical proximity and sense of solidarity among the participants in the augmented reality session.

[0043] FIG. 6 shows a flowchart of a method 300 for an initialization process when an avatar is first rendered on the display of a computing device. With reference to the software and hardware components shown in FIGS. 1 - 5 as described above, the following description of method 300 is provided. It will be understood that method 300 may be executed in other contexts using other suitable hardware and software components.

[0044] In step 302, in the first display device, a first configuration file having a first user avatar data object is transmitted to the server computing device. In step 304, in the second display device, a second configuration file having a second user avatar data object is transmitted to the server computing device. In step 306, in the server computing device, the first configuration file including the first user avatar data object and the second configuration file including the second user avatar data object are received and stored. In step 308, in the server computing device, the avatar data object to be transmitted to the first display device is selected and transmitted, and the avatar data object to be transmitted to the second display device is selected and transmitted. In step 310, in the first display device, the selected avatar data object is received from the server computing device. In step 314, in the first display device, using the selected avatar data object, the positions and poses of the selected avatar data object including the first user avatar data object and the second user avatar data object with respect to the virtual plane are calculated. In step 318, the selected avatar data object including the first user avatar data object and the second user avatar data object is rendered on the display based on the calculated positions and poses.

[0045] In step 312, in the second display device, the selected avatar data object is received from the server computing device. In step 316, in the second display device, using the selected avatar data object, the positions and poses of the selected avatar data object including the first user avatar data object and the second user avatar data object with respect to the virtual surface are calculated. In step 320, the selected avatar data object including the first user avatar data object and the second user avatar data object is rendered on the display based on the calculated positions and poses.

[0046] FIG. 7 shows a flowchart of a method 400 for a runtime process in which an avatar is continuously re-rendered using pose information that is continuously updated in a loop by a display of a computing device. As described above, the following description of method 400 is provided with reference to the software and hardware components shown in FIGS. 1-5. It will be understood that method 400 may be executed in other contexts using other suitable hardware and software components.

[0047] In step 402, in the first display device, a first user image of the first user is acquired using the camera. In step 404, in the first display device, tracking features of the first user image are identified. In step 406, in the first display device, first user pose information is generated based on the identified tracking features. In step 408, in the first display device, the first user pose information is transmitted to the server computing device.

[0048] In step 410, in the second display device, a second user image of the second user is acquired using the camera. In step 412, in the second display device, tracking features of the second user image are identified. In step 414, in the second display device, second user pose information is generated based on the identified tracking features. In step 416, in the second display device, the second user pose information is transmitted to the server computing device.

[0049] In step 418, the first user pose information and the second user pose information are respectively received as updated pose information of the first user and the second user and stored in the server computing device. In step 420, in the server computing device, the updated pose information is transmitted to the first display device and the second display device respectively.

[0050] In the first display device, steps 422, 426 and 428 are executed. In step 422, updated second user pose information is received. In step 426, the positions and poses of the first user avatar and the second user avatar are calculated based on the first user pose information and the second user pose information. In step 428, the first user avatar and the second user avatar are rendered on the display based on the calculated positions and poses.

[0051] In the second display device, steps 424, 430 and 432 are executed. In step 424, updated first user pose information is received. In step 430, the positions and poses of the first user avatar and the second user avatar are calculated based on the first user pose information and the second user pose information. In step 432, the first user avatar and the second user avatar are rendered on the display based on the calculated positions and poses.

[0052] FIG. 8 shows a flowchart of method 500. With reference to the software and hardware components shown in FIGS. 1-5 as described above, the following description of method 500 is provided. It will be understood that method 500 may be executed in other contexts using other suitable hardware and software components.

[0053] In step 502, a second user avatar data object and second user pose information of a second user are received via a computer network. In step 504, an image is received from a live image feed acquired by a camera of the first computing device. In step 506, a virtual surface corresponding to a physical surface appearing in the image is identified.

[0054] In step 508, the position and pose of the second user avatar with respect to the virtual surface are calculated based on the second user pose information. In step 510, by rendering the second user avatar data object, the second user avatar of the second user superimposed on the image is displayed. The second user avatar is anchored to the virtual surface according to the calculated position and pose of the second user avatar, and the virtual surface is world-locked to the physical surface of the image in a three-dimensional environment of the real world.

[0055] The following steps 512, 514, 516, and 518 are executed in a loop. In step 512, updated second user pose information of a second user is received via a computer network. In step 514, an updated image is received from the live image feed acquired by the camera of the first computing device. In step 516, the position and pose of the second user avatar with respect to the virtual plane are calculated based on the updated second user pose information. In step 518, by rendering the second user avatar data object, the second user avatar of the second user superimposed on the updated image is displayed, and the second user avatar remains anchored to the virtual plane that is world-locked to the physical plane in the image within the three-dimensional environment of the real world, and poses according to the updated second user pose information each time it passes through the loop.

[0056] It will be understood that methods 300, 400, and 500 are provided by way of example and are not intended to be limiting. Thus, methods 300, 400, and 500 may include additional and / or alternative steps as compared to those shown in FIGS. 6-8. Further, it should be understood that methods 300, 400, and 500 may be executed in any suitable order.

[0057] The above system and method enable multiple users using respective computing devices to view each other's avatars on their devices during an extended reality session, such that their positions and appearances change in real time in response to changes in the users' positions and appearances. Since the rendering of the avatars' faces and bodies is enhanced, users participating in the extended reality session can have a more intimate communal experience in their online interactions with each other. This increases the physical sense of proximity and solidarity, which is further enhanced when users communicate with each other via their respective computing devices using expressions and body language in addition to voice and text.

[0058] In some embodiments, the methods and processes described herein may be associated with a computing system of one or more computing devices. Specifically, such methods and processes may be implemented as a computer application program or service, an application programming interface (API), a library, and / or other computer program products.

[0059] FIG. 9 schematically shows a non-limiting embodiment of a computing system 600 capable of implementing one or more of the methods and processes described above. The computing system 600 is shown in a simplified form. The computing system 600 may include the first computing device 12A, the second computing device 12B, the third computing device 12C, the fourth computing device 12D, and the server computing device 100 respectively shown in FIGS. 1-5 described above. The computing system 600 may take the form of one or more personal computers, server computers, tablet computers, home entertainment computers, network computing, gaming devices, mobile computing devices, mobile communication devices (e.g., smartphones) and / or other computing devices, and wearable computing devices, such as smart watches and head-mounted augmented reality devices.

[0060] The computing system 600 includes a logical processor 602, a volatile memory 604, and a non-volatile storage device 606. The computing system 600 may optionally include a display subsystem 608, an input subsystem 610, a communication subsystem 612, and / or other components not shown in FIG. 9.

[0061] The logical processor 602 includes one or more physical devices configured to execute instructions. For example, the logical processor may be configured to execute instructions that are part of one or more applications, programs, routines, libraries, objects, components, data structures, or other logical structures. Such instructions may be implemented to perform tasks, realize data types, transform the state of one or more components, achieve technical effects, or achieve desired results.

[0062] The logical processor may include one or more physical processors (hardware) configured to execute software instructions. Additionally or alternatively, the logical processor may include one or more hardware logic circuits or firmware devices configured to execute logic or firmware instructions implemented by hardware. The processor of the logical processor 602 may be single-core or multi-core, and the instructions executed thereon may be configured for sequential, parallel, and / or distributed processing. The individual components of the logical processor may optionally be located remotely and / or distributed among two or more separate devices configured for coordinated processing. Aspects of the logical processor may be virtualized and executed by a computing device connected to a remotely accessible network configured in a cloud computing setting. In such cases, it should be understood that these virtualized aspects are executed on different physical logical processors of various different machines.

[0063] The non-volatile memory device 606 includes one or more physical devices configured to hold instructions executable by the logical processor for implementing the methods and processes described herein. When such method and process operations are implemented, the state of the non-volatile memory device 606 may be transformed, for example, to hold different data.

[0064] The non-volatile memory device 606 may include removable and / or built-in physical devices. The non-volatile memory device 606 may include optical memory (e.g., CD, DVD, HD-DVD, Blu-Ray disk, etc.), semiconductor memory (e.g., ROM, EPROM, EEPROM, flash memory, etc.), and / or magnetic memory (e.g., hard disk drive, floppy disk drive, tape drive, MRAM, etc.), or other mass storage device technologies. The non-volatile memory device 606 may include non-volatile, dynamic, static, read / write, read-only, sequential access, location-addressable, file-addressable, and / or content-addressable devices. It will be understood that the non-volatile memory device 606 is set to hold instructions even when power to the non-volatile memory device 606 is cut off.

[0065] The volatile memory 604 may include a physical device with random access memory. The volatile memory 604 is typically utilized by the logic processor 602 to temporarily store information during the processing of software instructions. It can be seen that when power to the volatile memory 604 is cut off, the volatile memory 604 typically no longer stores instructions.

[0066] Aspects of the logic processor 602, the volatile memory 604, and the non-volatile memory device 606 may be integrated together into one or more hardware logic components. Such hardware logic components may include, for example, field programmable gate arrays (FPGAs), application specific integrated circuits (PASIC / ASICs), application specific standard products (PSSP / ASSPs), system-on-chips (SOCs), and complex programmable logic devices (CPLDs), etc.

[0067] The terms "module", "program", and "engine" are often used to describe aspects of a computing system 600 that are typically implemented in software to perform a specific function that includes conversion processing in which a processor specially configures a program to execute a function using a portion of volatile memory. Thus, a module, program, or engine may be instantiated via a logic processor 602 that executes instructions held by a non-volatile storage device 606, using a portion of volatile memory 604. It will be understood that different modules, programs, and / or engines may be instantiated from the same application, service, code block, object, library, routine, API, function, etc. Similarly, the same module, program, and / or engine may be instantiated by different applications, services, code blocks, objects, routines, APIs, functions, etc. The terms "module", "program", and "engine" may include an individual or group such as an executable file, data file, library, driver, script, database record, etc.

[0068] The display subsystem 608, if included, may be used to present a visual representation of data held by the non-volatile storage device 606. This visual representation may take the form of a graphical user interface (GUI). Since the methods and processes described herein change data held by the non-volatile storage device and transform the state of the non-volatile storage device, the state of the display subsystem 608 may similarly be transformed to visually represent changes in the underlying data. The display subsystem 608 may include one or more display devices that utilize substantially any type of technology. Such display devices may be combined with the logic processor 602, volatile memory 604, and / or non-volatile storage device 606 within a shared housing, or such display devices may be peripheral display devices.

[0069] The input subsystem 610, if included, may comprise and / or interact with one or more user input devices, such as a keyboard, mouse, touch screen, or game controller. In some embodiments, the input subsystem may comprise and / or interact with a selected natural user input (NUI) component. These components may be integral or peripheral, and the transmission and / or processing of input operations may be processed on-board or off-board. Exemplary NUI components may include a microphone for speech and / or voice recognition, an infrared camera for machine vision and / or gesture recognition, a color camera, a stereo camera, and / or a depth camera, a head tracker, an eye tracker, an accelerometer and / or gyroscope for motion detection and / or intent recognition, an electric field sensing component for evaluating brain activity, and / or any other suitable sensor.

[0070] The communication subsystem 612, if included, may be configured to communicatively couple the various computing devices described herein to each other or to other devices. The communication subsystem 612 may include wired and / or wireless communication devices compatible with one or more different communication protocols. By way of non-limiting example, the communication subsystem may be configured to communicate via a radio telephone network or a wired or wireless local or wide area network, such as via Wi-Fi connection through HDMI (registered trademark). In some embodiments, the communication subsystem may enable the computing system 600 to send and receive messages to and from other devices via a network such as the Internet.

[0071] The following paragraphs provide additional support for the claims of this application. One aspect provides a computing system comprising a processor of a first user's computing device and a memory, wherein the processor uses a portion of the memory to execute an augmented reality (AR) avatar program to receive, via a computer network, a second user avatar data object and second user pose information of a second user, receive an image from a live image feed acquired by a camera of the computing device, identify a virtual surface corresponding to a physical surface appearing in the image, calculate a position and pose of the second user avatar with respect to the virtual surface based on the second user pose information, and display the second user avatar of the second user overlaid on the image by rendering the second user avatar data object. The second user avatar is anchored to the virtual surface and poses according to the calculated position and pose of the second user avatar, and the virtual surface is world-locked to the physical surface of the image in a three-dimensional environment of the real world. In this aspect, additionally or alternatively, the second user pose information is initial second user pose information of a session of the AR avatar program, and the image is an initial image of the session of the AR avatar program. In this aspect, additionally or alternatively, by executing the AR avatar program, the loop receives, via the computer network, updated second user pose information of the second user, receives an updated image from the live image feed acquired by the camera of the computing device, calculates a position and pose of the second user avatar with respect to the virtual surface based on the updated second user pose information, and renders the second user avatar data object to display the second user avatar that remains anchored to the virtual surface that remains world-locked to the physical surface in the image within the three-dimensional environment of the real world and overlaid on the updated image, and poses according to the updated pose information each time the loop is passed.In this aspect, additionally or alternatively, the second user pose information and the updated second user pose information include second face tracking data, and the processor is configured to display the second user avatar of the second user posing with a face corresponding to the expression of the second user by rendering the second user avatar data object of the second user based on the second face tracking data, and the face of the second user avatar is rendered such that its appearance changes as the second face tracking data continuously passes through a loop. In this aspect, additionally or alternatively, the second user pose information includes second face tracking data, and the processor is configured to display the second user avatar of the second user posing with a face corresponding to the expression of the second user by rendering the second user avatar data object of the second user based on the second face tracking data. In this aspect, additionally or alternatively, the second user pose information includes second skeleton tracking data, and the processor is configured to render the body of the second user avatar such that its appearance changes as the second skeleton tracking data changes. In this aspect, additionally or alternatively, the camera is a rear camera, the computing device is a first computing device, the computing system further includes a front camera configured to acquire a first user image of the first user, the processor is configured to identify tracking features in the first user image of the first user and generate first user pose information of the first user based on the identified tracking features, and the first user pose information is configured to be transmitted to a second computing device of the second user via the computer network.In this aspect, additionally or alternatively, the first user pose information includes first face tracking data, and the processor is configured to display the first user avatar of the first user who poses with a face corresponding to the expression of the first user by rendering the first user avatar data object of the first user based on the first face tracking data. In this aspect, additionally or alternatively, the processor is further configured to display the first user avatar of the first user superimposed on the image that poses according to the first user pose information by rendering the first user avatar data object of the first user. In this aspect, additionally or alternatively, the front camera and the rear camera are operated to respectively obtain the first user image and the image in real time with a timestamp that synchronizes the identified tracking features in the image and the identified tracking features in the first user image of the first user. In this aspect, additionally or alternatively, the processor identifies tracking features in the image obtained by the rear camera and the first user image of the first user, generates first user pose information of the first user based on the identified tracking features, and is configured to transmit the first user pose information to the second computing device of the second user via the computer network.In this aspect, additionally or alternatively, the computing device is a first computing device, the first user and the second user each have associated accounts on a social media platform, the first computing device and the second computing device each execute an instance of the AR avatar program configured to communicate with a server computing system of the social media platform, and the first user pose information of the first user, the first user avatar data object of the first user, the second user pose information, and the second user avatar data object are exchanged between the AR avatar program executed on the first computing device and the AR avatar program executed on the second computing device via the server computing system.

[0072] Another aspect provides a method for a client program to be executed on a computing device operated by a first user, the method comprising receiving, via a computer network, second user avatar data object and second user pose information of a second user, receiving an image from a live image feed acquired by a camera of the computing device, identifying a virtual surface corresponding to a physical surface appearing in the image, calculating a position and a pose of a second user avatar with respect to the virtual surface based on the second user pose information, and displaying the second user avatar of the second user superimposed on the image by rendering the second user avatar data object, wherein the second user avatar is anchored to the virtual surface according to the calculated position and pose of the second user avatar, and the virtual surface is world-locked to the physical surface of the image in a three-dimensional environment of the real world. In this aspect, additionally or alternatively, the method further comprises, in a loop, receiving updated second user pose information of the second user via the computer network, receiving an updated image from the live image feed acquired by the camera of the computing device, calculating a position and a pose of the second user avatar with respect to the virtual surface based on the updated second user pose information, and displaying the second user avatar of the second user superimposed on the updated image by rendering the second user avatar data object, wherein the second user avatar remains anchored to the virtual surface that remains world-locked to the physical surface in the image within the three-dimensional environment of the real world, and poses according to the updated second user pose information each time passing through the loop.In this aspect, additionally or alternatively, the second user pose information includes second face tracking data, and the method further includes displaying the second user avatar of the second user posing with a face corresponding to the expression of the second user by rendering the second user avatar data object of the second user based on the second face tracking data. In this aspect, additionally or alternatively, the computing device is a first computing device, and the method further includes obtaining a first user image of the first user, identifying tracking features in the first user image of the first user, generating first user pose information of the first user based on the identified tracking features, and transmitting the first user pose information to a second computing device of the second user via the computer network. In this aspect, additionally or alternatively, the first user pose information includes first face tracking data, and the method further includes displaying the first user avatar of the first user posing with a face corresponding to the expression of the first user by rendering the first user avatar data object of the first user based on the first face tracking data.

[0073] Another aspect includes a processor, a non-volatile memory device operatively coupled to the processor, and an avatar hosting program stored in the non-volatile memory device and executed by the processor of the server computing device. The avatar hosting program hosts an avatar session to receive and store a second user avatar data object and second user pose information from a second computing device, and in response to receiving a first request for a second user avatar from a first computing device, transmits the second user avatar data object and the second user pose information to the first computing device to cause the first computing device to render the second user avatar data object and display the second user avatar. The avatar hosting program also receives and stores a first user avatar data object and initial first user pose information from the first computing device, and in response to receiving a second request for a first user avatar from the second computing device, transmits the first user avatar data object and the initial first user pose information to the second computing device to cause the second computing device to render the first user avatar data object and display the first user avatar. The loop includes receiving updated second user pose information of the second user from the second computing device and transmitting the updated second user pose information to the first computing device in response to receiving the updated second user pose information, causing the first computing device to render the second user avatar data object using the updated second user pose information and display the second user avatar, receiving updated first user pose information of the first user from the first computing device, and transmitting the updated first user pose information to the second computing device in response to receiving the updated first user pose information,The second computing device is set to render the first user avatar data object using the updated first user pose information and display the first user avatar. In this aspect, additionally or alternatively, the initial first user pose information and the updated first user pose information include first face tracking data, and the second computing device renders the first user avatar data object of the first user based on the first face tracking data, so as to display the first user avatar of the first user posing with a face corresponding to the expression of the first user. The second user pose information and the updated second user pose information include second face tracking data, and the first computing device renders the second user avatar data object of the second user based on the second face tracking data, so as to display the second user avatar of the second user posing with a face corresponding to the expression of the second user. In this aspect, additionally or alternatively, the initial first user pose information and the updated first user pose information include first skeleton tracking data, and the second computing device renders the first user avatar data object of the first user based on the first skeleton tracking data, so as to display the first user avatar of the first user posing with a body whose appearance changes as the first skeleton tracking data changes while continuously passing through the loop. The second user pose information and the updated second user pose information include second skeleton tracking data, and the first computing device renders the second user avatar data object of the second user based on the second skeleton tracking data of the second user, so as to display the second user avatar of the second user posing with a body whose appearance changes as the second skeleton tracking data changes while continuously passing through the loop.

[0074] The configurations and / or approaches described herein are exemplary in nature and, since numerous variations are possible, it will be understood that these specific embodiments or examples are not to be considered in a limiting sense. The particular routines or methods described herein may represent one or more of any number of processing strategies. Accordingly, the various acts illustrated and / or described may be performed in the order illustrated and / or described, in other orders, in parallel, or may be omitted. Similarly, the order of the processes described above may be changed.

[0075] The subject matter of the present disclosure includes novel and non-obvious combinations and sub-combinations of the various processes, systems, configurations, and other features, functions, acts, and / or properties disclosed herein, as well as any and all equivalents thereof.

[0076] To the extent that the terms "comprise," "comprising," "include," "including," and variations thereof are used herein, such terms are intended to be inclusive in a manner similar to the term "including" as an open transition word without excluding any additional or other elements.

[0077] The configurations and / or approaches described herein are exemplary in nature and, since numerous variations are possible, it will be understood that these specific embodiments or examples are not to be considered in a limiting sense. The particular routines or methods described herein may represent one or more of any number of processing strategies. Accordingly, the various acts illustrated and / or described may be performed in the order illustrated and / or described, in other orders, in parallel, or may be omitted. Similarly, the order of the processes described above may be changed.

[0078] The subject matter of the present disclosure includes novel and non-obvious combinations and sub-combinations of the various processes, systems, configurations, and other features, functions, acts, and / or properties disclosed herein, as well as any and all equivalents thereof.

Claims

1. A computing system comprising a processor and a memory of a computing device of a first user, wherein the processor executes an augmented reality (AR) avatar program using a part of the memory to receive, via a computer network, a second user avatar data object and second user pose information of a second user, receive an image from a live image feed acquired by a camera of the computing device, identify a virtual plane corresponding to a physical plane appearing in the image, calculate a position and a pose of a second user avatar with respect to the virtual plane based on the second user pose information, and is configured to display the second user avatar of the second user superimposed on the image by rendering the second user avatar data object, wherein the second user avatar is anchored to the virtual plane and poses according to the calculated position and pose of the second user avatar, and the virtual plane is world-locked to the physical plane in the image in a three-dimensional environment of the real world. A computing system.

2. The second user pose information is initial second user pose information of a session of the AR avatar program, and the image is an initial image of the session of the AR avatar program. The computing system according to claim 1.

3. By executing the AR avatar program, within a loop, receive, via the computer network, updated second user pose information of the second user, receive an updated image from the live image feed acquired by the camera of the computing device, calculate a position and a pose of the second user avatar with respect to the virtual plane based on the updated second user pose information, and while displaying a second user avatar superimposed on the updated image and posing according to updated pose information each time passing through the loop by rendering the second user avatar data object, the second user avatar remains anchored to the virtual plane, and the virtual plane remains world-locked to the physical plane in the image within the three-dimensional environment of the real world. The computing system according to claim 1.

4. The second user pose information and the updated second user pose information include second face tracking data, and the processor is configured to display the second user avatar of the second user posing with a face corresponding to the expression of the second user by rendering the second user avatar data object of the second user based on the second face tracking data, wherein the face of the second user avatar is rendered to change its appearance as the second face tracking data continuously passes through a loop. The computing system according to claim 3.

5. The second user pose information includes second face tracking data, and the processor is configured to display the second user avatar of the second user posing with a face corresponding to the expression of the second user by rendering the second user avatar data object of the second user based on the second face tracking data. The computing system according to claim 1.

6. The second user pose information includes second skeleton tracking data, and the processor is configured to render the body of the second user avatar to change its appearance as the second skeleton tracking data changes. The computing system according to claim 1.

7. The camera is a rear camera, the computing device is a first computing device, and the computing system further comprises a front camera configured to acquire a first user image of the first user, and the processor is configured to: identify tracking features in the first user image of the first user; generate first user pose information of the first user based on the identified tracking features; and transmit the first user pose information to a second computing device of the second user via the computer network. The computing system according to claim 1.

8. The first user pose information includes first face tracking data, ​ The processor is configured to display the first user avatar of the first user, which poses with a face corresponding to the expression of the first user, by rendering the first user avatar data object of the first user based on the first face tracking data. The computing system according to claim 7.

9. The processor is further set to display the first user avatar of the first user superimposed on the image, which poses according to the first user pose information, by rendering the first user avatar data object of the first user. The computing system according to claim 7.

10. Operate the front camera and the rear camera to respectively obtain the first user image and the image in real time with time stamps that synchronize the identified tracking features in the image and the identified tracking features in the first user image of the first user. The computing system according to claim 9.

11. The processor identifies tracking features in the image obtained by the rear camera and the first user image of the first user, generates first user pose information of the first user based on the identified tracking features, and is set to transmit the first user pose information to the second computing device of the second user via the computer network. The computing system according to claim 7.

12. The computing device is a first computing device, the first user and the second user each have associated accounts on a social media platform, the first computing device and the second computing device each execute an instance of the AR avatar program configured to communicate with the server computing system of the social media platform. The first user pose information of the first user, the first user avatar data object of the first user, the second user pose information, and the second user avatar data object are exchanged between the AR avatar program executed on the first computing device and the AR avatar program executed on the second computing device via the server computing system. The computing system according to claim 1.

13. A method for a client program executed on a computing device operated by a first user, comprising: Receiving, via a computer network, a second user avatar data object and second user pose information of a second user; Receiving an image from a live image feed acquired by a camera of the computing device; Identifying a virtual plane corresponding to a physical plane appearing in the image; Calculating the position and pose of the second user avatar with respect to the virtual plane based on the second user pose information; Displaying the second user avatar of the second user superimposed on the image by rendering the second user avatar data object, wherein the second user avatar is anchored to the virtual plane according to the calculated position and pose of the second user avatar, and the virtual plane is world-locked to the physical plane in the image in a real-world three-dimensional environment. Method.

14. Within a loop, Receiving, via the computer network, updated second user pose information of the second user; Receiving an updated image from the live image feed acquired by the camera of the computing device; Calculating the position and pose of the second user avatar with respect to the virtual plane based on the updated second user pose information; Further comprising displaying the second user avatar of the second user superimposed on the updated image by rendering the second user avatar data object. The second user avatar poses according to the updated second user pose information each time it passes through the loop, while the second user avatar remains anchored to the virtual plane, and the virtual plane remains world-locked to the physical plane in the image in the three-dimensional environment of the real world. The method according to claim 13.

15. The second user pose information includes second face tracking data. The method further includes displaying the second user avatar of the second user posing with a face corresponding to the expression of the second user by rendering the second user avatar data object of the second user based on the second face tracking data. The method according to claim 13.

16. The computing device is a first computing device. The method includes: acquiring a first user image of the first user; identifying tracking features in the first user image of the first user; generating first user pose information of the first user based on the identified tracking features; further including the step of transmitting the first user pose information to a second computing device of the second user via the computer network. The method according to claim 13.

17. The first user pose information includes first face tracking data. The method further includes displaying the first user avatar of the first user posing with a face corresponding to the expression of the first user by rendering the first user avatar data object of the first user based on the first face tracking data. The method according to claim 16.

18. a processor; a non-volatile memory device operatively coupled to the processor; an avatar hosting program stored in the non-volatile memory device and executed by the processor of the server computing device, wherein the avatar hosting program hosts an avatar session, receives and stores a second user avatar data object and second user pose information of a second user from a second computing device. In response to receiving a first request for a second user avatar from a first computing device, by transmitting the second user avatar data object and the second user pose information to the first computing device, causing the first computing device to render the second user avatar data object and display the second user avatar, Receiving and storing, from the first computing device, a first user avatar data object of a first user and initial first user pose information, In response to receiving a second request for a first user avatar from the second computing device, by transmitting the first user avatar data object and the initial first user pose information to the second computing device, configured to cause the second computing device to render the first user avatar data object and display the first user avatar, Within the loop, Receiving, from the second computing device, updated second user pose information of the second user, In response to receiving the updated second user pose information, by transmitting the updated second user pose information to the first computing device, causing the first computing device to render the second user avatar data object using the updated second user pose information and display the second user avatar, Receiving, from the first computing device, updated first user pose information of the first user, In response to receiving the updated first user pose information, by transmitting the updated first user pose information to the second computing device, causing the second computing device to render the first user avatar data object using the updated first user pose information and display the first user avatar, A server computing device.

19. The initial first user pose information and the updated first user pose information include first face tracking data. Cause the second computing device to display the first user avatar of the first user posing with a face corresponding to the expression of the first user by rendering the first user avatar data object of the first user based on the first face tracking data. The second user pose information and the updated second user pose information include second face tracking data. Cause the first computing device to display the second user avatar of the second user posing with a face corresponding to the expression of the second user by rendering the second user avatar data object of the second user based on the second face tracking data. The server computing device according to claim 18.

20. The initial first user pose information and the updated first user pose information include first skeleton tracking data. Cause the second computing device to display the first user avatar of the first user posing with a body whose appearance changes as the first skeleton tracking data changes while continuously passing through the loop by rendering the first user avatar data object of the first user based on the first skeleton tracking data. The second user pose information and the updated second user pose information include second skeleton tracking data. Cause the first computing device to display the second user avatar of the second user posing with a body whose appearance changes as the second skeleton tracking data changes while continuously passing through the loop by rendering the second user avatar data object of the second user based on the second skeleton tracking data of the second user. The server computing device according to claim 18.

Images