MAPPING AND ADJUSTING USER-ON-AVATAR ACTIONS

Abstract

Methods, devices, systems, and manufacturing articles for user-to-avatar mapping and adaptation are disclosed. An exemplary device includes a processor circuit arrangement for instantiating or executing machine-readable instructions for: determining a dissonance between a first orientation of a user in a real-world environment and a second orientation of an avatar in a virtual environment, where the avatar corresponds to the user; determining an avatar adaptation value based on the dissonance; and applying the avatar adaptation value to the avatar model to modify the second orientation.

Description

AREA OF REVELATION

[0001] This revelation generally concerns the presentation of avatars and, in particular, the mapping and adaptation of user-to-avatar actions. STATE OF THE ART

[0002] In virtual reality environments, such as the Metaverse, users create or select avatars. Actions that users perform in the real world are mapped into the virtual environment and mimicked by their respective avatars. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is an illustration of a user in an environment with an avatar in a misaligned position. Fig. Figure 2 is an illustration of the user in the environment of Fig. 1 with the avatar in a newly aligned position according to the teachings of this revelation. Fig. Figure 3 is a block diagram of an exemplary system, including an exemplary avatar customization application according to the teachings of this revelation. Fig. 4 is a block diagram of the avatar customization application of Fig. 3. Fig. Figure 5 illustrates an exemplary correlation of exemplary coordinate systems, achieved through the avatar adaptation application of Fig. 3 can be used, with a user face in a first position. Fig. Figure 6 illustrates an exemplary correlation of the coordinate systems obtained through the avatar adaptation application of Fig. 3 can be used, with the user face in a second position. Fig. 7, Fig. 8 to Fig. 9 are flowcharts that represent exemplary machine-readable instructions and / or exemplary operations that can be performed by an exemplary processor circuit arrangement to implement the avatar customization application of Fig. 3 to implement. Fig. Figure 10 is a block diagram of an exemplary processing platform, including a processor circuit arrangement, structured to illustrate the exemplary machine-readable instructions and / or the exemplary operations of Fig. 7, Fig. 8 to Fig. 9 to run the avatar customization application of Fig. 3 to implement. Fig. Figure 11 is a block diagram of an exemplary implementation of the processor circuit arrangement of Fig. 10. Fig. Figure 12 is a block diagram of another exemplary implementation of the processor circuit arrangement of Fig. 10. Fig. Figure 13 is a block diagram of an exemplary software distribution platform (e.g., one or more servers) for distributing software (e.g., software that follows the exemplary machine-readable instructions of Fig. 7, Fig. 8 to Fig. 9 corresponds) to client devices associated with end users and / or consumers (e.g., for licensing, sale, and / or use), retailers (e.g., for sale, resale, licensing, and / or sublicensing), and / or original equipment manufacturers (OEMs) (e.g., for inclusion in products intended for distribution to retailers and / or other end users, such as direct customers).

[0003] Generally, the same reference symbols are used throughout the drawing(s) and the accompanying written description to refer to the same or similar parts. The figures are not to scale. DETAILED DESCRIPTION

[0004] In virtual environments, such as the Metaverse, users can represent themselves as avatars. A user can use head-mounted displays (HMDs) as portals to the virtual environment. HMDs feature devices for virtual reality (VR), augmented reality (AR), and / or mixed reality (MR), including headsets, goggles, glasses, etc. With HMDs, the position of the user's eyes relative to cameras and other sensors in the HMD is predictable. Thus, the user's avatar appears naturally oriented in the virtual environment.

[0005] In addition to HMDs, other electronic devices can be used as portals for virtual environments. For example, a personal computer (PC) can be used as a metaverse portal, either as an alternative or in addition to HMDs. In some PC setups, such as laptop computers with external displays and / or multi-display desktop computers, the location of the camera and / or other sensors may be offset from the display of the virtual environment. For example, Fig. 1 An exemplary laptop computer 102 with an exemplary camera 104 positioned adjacent to an exemplary external display 106. An exemplary virtual environment 108 is presented on the external display 106. An exemplary avatar 110 of an exemplary user 112 is presented in the virtual environment 108. The position or orientation of the avatar 110 is determined based on data collected by the camera 104.

[0006] In the example of Fig. The user's gaze (114) is directed at the external display (106). However, the camera (104) is positioned to the right of the external display (106). Therefore, the avatar (110) appears to be looking sideways, which is an incorrect orientation. This is because the camera (104) on the laptop computer (102) is not aligned with the user (112) relative to the content of interest (i.e., the virtual environment (108)). Similar problems exist in other scenarios, such as a desktop computer setup where an external camera can be placed on top of a monitor or positioned to the side, etc.

[0007] Furthermore, avatars in virtual environments can look at each other and even make virtual eye contact, giving the user the impression that other users' avatars are making eye contact. Virtual eye contact is inconsistent or prevented if an avatar is misaligned.

[0008] In some virtual environments, such as a Metaverse session, if a user creates an unintended input (e.g., scratching their nose or getting up to close a window, etc.) while immersed in the virtual environment, the user's avatar reproduces the unintended input or interaction. In some examples, the avatar's reproduction of the unintended action appears as a jerky movement or other incongruent action. The avatar's unintended movement can cause confusion for other participants in the virtual environment, as well as potential frustration for the user. There are many types of unintended input, including physical input, such as when the user moves their body, for example, micro-gestures, sweeping movements, or briefly stepping away from a PC presenting the virtual environment.Unintentional inputs also include, for example, auditory inputs such as sneezing, yawning, coughing, etc. Unintentional inputs also include emotional inputs, such as when a user provides an emotional output that differs from an emotion intended for the avatar. Unintentional inputs also include, for example, activity in the user's real-world environment, such as another person entering a camera's field of view, external noises, etc. Unintentional inputs also include, for example, when a user operates multiple devices (e.g., dual monitors and / or cameras), some of which are not connected to the virtual environment. The user might look away from the primary monitor, where the camera is installed, to a secondary screen. For example, a user might look at a smartwatch. The avatar's reproduction of this movement may be unintentional.In some examples, unintentional input may be a combination of different types of unintentional input.

[0009] Currently, there are no significant warnings to allow for a temporary pause in sensor tracking (e.g., "It looks like you're about to sneeze, would you like to pause and / or mute your avatar control?" or "It looks like you're about to walk away. Would you like to pause your avatar control?"), and there are also no reminders that a user is still in a virtual collaboration environment, even though the user might be multitasking and forgetting they are controlling their avatar. Thus, in current systems, all user activity is translated into an avatar action.

[0010] The examples revealed here improve the mapping of users to their avatars, including facial expressions and body movements. These examples also feature avatar alignment correction when the camera is offset from the user. For example, [example example] shows Fig. 2 the user 112 in the vicinity of Fig. 1, where the camera 104 is offset from the external display 106, which presents the virtual environment 108. The user 112 is focused on the virtual environment 108, and the avatar 110, despite the offset position of the camera 104, which collects the user orientation data, is oriented according to the user 112's orientation to the virtual environment 108. Thus, the avatar 110 is reoriented to be consistent with the user's actual view of the virtual environment.

[0011] The examples revealed here correct the avatar's orientation within the application (the virtual environment) while taking into account multiple tasks the user might perform on or near Computer 102. This includes the ability to detect user activity and then correct or otherwise adjust the avatar's orientation within virtual content to be consistent with the user's activity in the real world. In some examples, user activity is determined by analyzing user gaze tracking, objects in the environment, and / or available digital content (e.g., content presented on one or more electronic devices). In some examples, the avatar reorientation is not limited to the face and eyes but can be applied to the entire avatar model.

[0012] The examples disclosed here also demonstrate dynamic management of intended and unintended user and environment inputs for avatar reproduction of outputs in virtual environments, such as the Metaverse, virtual meetings, games, etc. These examples further enhance the user experience and avatar control in the Metaverse and other virtual environments by validating and allowing intended inputs and eliminating unintended inputs through the use of available user presence information via input and output data (e.g., data collected via a camera, microphone, keyboard, mouse, additional and / or alternative sensors, etc.). These examples also utilize predefined behavioral models (e.g., logged and categorized types of movements, including, for example, that an urgent stand-up instance is unintentional, a sneeze is unintentional, etc.).The examples revealed here establish a baseline or normal default behavior for comparison with unintended inputs. These examples allow the user to block unintended inputs. In some examples, users actively block unintended inputs at the time of the event, preventing them from being translated into the avatar. In some examples, users pre-configure events to be blocked before they occur.

[0013] The examples revealed here identify, categorize, and translate user actions for visual and auditory reproduction by a corresponding avatar with high-resolution control to correct misalignment and block unintended actions.

[0014] Unless specifically stated otherwise, descriptors such as "first," "second," "third," etc., are used here without implying any significance in terms of priority, physical order, arrangement in a list, and / or sequence, or otherwise indicating anything. They are used merely as labels and / or arbitrary names to distinguish elements to facilitate understanding of the disclosed examples. In some examples, the descriptor "first" may be used to refer to an element in the detailed description, while the same element in a claim is referred to by a different descriptor, such as "second" or "third." In these cases, it is understood that these descriptors are used solely to specifically identify those elements that might otherwise, for example, share the same name.

[0015] As used here, "approximately" and "about" modify their subjects / values ​​to account for the potential presence of variations that occur in real-world applications. For example, the terms "approximately" and "about" can modify dimensions that, due to manufacturing tolerances and / or other imperfections in the real world, are not exact as understood by the average person. For example, "approximately" and "about" can indicate that such dimensions may vary within a tolerance range of + / - 10%, unless otherwise specified in the subsequent description. As used here, "essentially real-time" refers to an occurrence in a virtually instantaneous manner, acknowledging that there may be real-world delays for computing time, transmission, etc. Unless otherwise specified, "essentially real-time" thus refers to real-time + / - 1 second.

[0016] When used in this document, the term “in communication”, including variations thereof, includes direct communication and / or indirect communication through one or more intermediaries and does not require direct physical (e.g., wired) communication and / or constant communication, but instead additionally features selective communication at periodic intervals, scheduled intervals, aperiodic intervals and / or at one-off events.

[0017] As used here, the term "processor circuit arrangement" is defined as comprising (i) one or more specialized electrical circuit(s) structured to perform (a) specific operation(s) and comprising one or more semiconductor-based logic devices (e.g., electrical hardware implemented by one or more transistors), and / or (ii) one or more general-purpose semiconductor-based electrical circuit(s) programmed with instructions to perform specific operations and comprising one or more semiconductor-based logic devices (e.g., electrical hardware implemented by one or more transistors).Examples of processor circuit arrangements include programmable microprocessors, field-programmable gate arrays (FPGAs) capable of instantiating instructions, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), XPUs or microcontrollers, and integrated circuits such as application-specific integrated circuits (ASICs). For example, an XPU can be implemented by a heterogeneous computing system comprising several types of processor circuit arrangements (e.g., one or more FPGAs, one or more CPUs, one or more GPUs, one or more DSPs, etc., and / or a combination thereof) and an application programming interface (API) that can assign a computational task to any of the several types of processor circuit arrangements best suited to perform the computational task(s).

[0018] As used here, the terms "consistent," "agree," and "correlate" can indicate a degree of compatibility or similarity. In some examples, the degree of similarity is within a threshold of similarity, such as ten percent. In some examples, the degree of similarity is higher, such as within a threshold of one percent. In some examples, the degree of similarity is very high, including, for example, exactly the same. In some examples, the terms are qualitative and indicate agreement or consonance between two or more elements, such as elements of a presentation or virtual environment. Opposite terms, such as "inconsistent" or "dissonance," are used here to indicate a degree of dissimilarity. Degrees of dissimilarity do not meet the similarity threshold.

[0019] Fig. Figure 3 is a block diagram of an exemplary System 300 for user-to-avatar mapping and customization. System 300 includes an exemplary computing device 302. The computing device 302 is in Fig. 3 labelled “Calculating Device 1”. The exemplary system 300 also includes one or more other calculating devices 304 (“Calculating Device 2”), 306 (“Calculating Device N”). The system 300 can include any number of calculating devices. The other calculating device 304, 306 can include some or all of the devices described in Fig. The computing device 302 exhibits the features shown in the three illustrations. The computing device 302 can be any device used to access and / or display a virtual environment and / or otherwise enable input and / or output of user activity. Examples of computing devices include HMDs, laptop PCs, desktop PCs, smartwatches, smartphones, tablets, external displays, smart mirrors, smart TVs, game consoles, and / or combinations of devices, etc.

[0020] The computing device 302 includes an exemplary sensor array 308. The sensor array 308 includes an exemplary camera 310 and an exemplary microphone 312. The computing device 302 also includes an exemplary program identification circuit arrangement 314, an exemplary avatar customization application 316, an exemplary graphics subsystem 318, an exemplary display 320, and an exemplary database 322. The exemplary display 320 includes at least one exemplary program list 324 and exemplary logged actions 326. The system 300 also includes an exemplary cloud service 328 and an exemplary host application 330.

[0021] The sensor array 308 collects data relating to the user and the real-world environment. For example, the example camera 310 collects visual data that can be used to identify or determine the user's position, orientation, gaze direction, face tracking, and so on. The visual data collected by camera 310 can also be used to identify objects in the real world, including, for example, an object being held or otherwise near the user that might be of interest. In some examples, camera 310 is an RGB camera. Additionally or alternatively, the sensor array 308 can include an infrared camera, a time-of-flight camera, multiple cameras, and so forth.

[0022] The microphone 312 collects audio data. Audio also has a directional component. Thus, the audio data can also be used to identify or determine the user's position and orientation. In some examples, the microphone 312 has directional microphones. In some examples, the sensor array 308 has an array of microphones.

[0023] In other examples, the sensor array 308 includes more or fewer devices for collecting data relating to a user and / or a real-world environment. For example, the sensor array 308 may include one or more gyroscopes, accelerometers, lidar sensors, electric field sensors, capacitive sensors, structured light sensors, vertical cavity surface-emitting lasers, and / or other types of positioning sensors. One or more of the sensors in the sensor array 308 may be integrated into one or more computers and / or other electronic devices and / or could be external peripherals for these computers and / or other electronic devices.

[0024] The program identification circuit arrangement 314 identifies and tracks which applications or programs are used on the computing device 302. In some examples, the program identification circuit arrangement 314 identifies the screen buffer of content. The program identification circuit arrangement 314 stores the identified program list 324 in the database 322.

[0025] The avatar customization application 316 manages and updates an avatar model for presenting an avatar in the virtual environment based on data collected via the sensor array 308 and the program identification circuit arrangement 314. The operation of the avatar customization application 316 is revealed in more detail below. In some examples, the avatar customization application 316 communicates with the host application 330 of the cloud service 328. In such examples, one or more of the functionality and capabilities of the avatar customization application 316 may be distributed between the compute device 302 and the host application 330. In some examples, the host application 330 coordinates the avatar customization application across the compute devices 302, 304, and 306. For example, different compute devices 302, 304, and 306 may have different types of sensors that collect different data for use in the avatar model.In some examples, the host application 330 is located on a non-cloud-based server. In some examples, the host application 330 is edge-based. In some examples, the host application 330 resides locally on one or more of the compute devices 302, 304, or 306. In some examples, the host application 330 and the avatar customization application 316 are integrated.

[0026] The graphics subsystem 318 creates the avatar images based on the avatar model of the avatar customization application 316. In some examples, the avatar model resides in the cloud service 328. In such examples, the computing device 302 and the graphics subsystem 318 receive rendered frames from the cloud service 328. In some examples, the avatar is displayed on the display 320.

[0027] Fig. Figure 4 is a block diagram of the avatar customization application 316 by Fig. 3. The avatar adaptation application 316 includes an exemplary user alignment circuit arrangement 402, an exemplary object detection circuit arrangement 404, an exemplary user activity classification circuit arrangement 406, an exemplary alignment offset circuit arrangement 408, an exemplary avatar mapping circuit arrangement 410 and an exemplary avatar modeling circuit arrangement 412.

[0028] The user orientation circuit arrangement 402 analyzes data from the sensor array 308 and the program identification circuit arrangement 314 to determine a user's position and orientation. The user orientation circuit arrangement 402 analyzes the visual data and tracks one or more parts of the user's body, including, for example, one or more eyes, a face, a torso, arms, one or more fingers, etc. In some examples, the user orientation circuit arrangement 402 tracks the user's voice by analyzing the audio data. The user orientation circuit arrangement 402 also analyzes the program list 324 to identify content displayed on the display 320. The user orientation circuit arrangement 402 uses the visual, audio, and / or program list data to identify a user's position and what the user is focused on.

[0029] In some examples, the 402 user orientation circuit determines the location and orientation of relevant displays and peripherals of the system or the user's environment based on visual data and / or known specifications. For example, in an integrated laptop camera, the camera's position is fixed, and the screen angle can be determined. However, an external display can be located at various angles relative to the laptop camera. Several approaches can be used to accurately estimate the location of an external display. For example, the 402 user orientation circuit can analyze radio signals, such as millimeter wave or ultra-wideband in a 5G specification, which can be used to obtain precise positional information. Another exemplary approach involves the use of eye reflections from displays. For example, a camera (e.g., a laptop camera) might be positioned at an angle relative to the laptop camera.The camera 310, which faces the user, detects displays in the reflection of the user's eye. The user orientation circuit arrangement 402 can determine the user's position based on the visual data of the reflection. Another exemplary approach uses wide-field-of-view cameras to capture more visual data. In another example with two or more displays, the user orientation circuit arrangement 402 determines, based on the program identification circuit arrangement 314 determining that the program is displayed on that display (and has visibility to other programs running on the electronic device) and / or that there is user input into that display and / or program, whether a user is interacting with a window or program on any of the displays.The user orientation circuit arrangement 402 estimates the user's gaze and / or facial orientation with respect to the display presented by the program. In some examples, the user orientation circuit arrangement 402 maps the gaze and / or facial orientation to specific coordinates on the display 320, as disclosed here.

[0030] In some examples, the input from camera 310 can be used to calibrate the user's gaze on the screen or the display content. For example, the user focuses on a point moving around the display 320. The user alignment circuit arrangement 402 tracks the gaze relative to camera 310 and display 320. The calibration can be used to map the position of camera 310 as long as camera 310 and display 320 do not move relative to each other.

[0031] The user's orientation with respect to the content of interest can be determined by a set of world coordinates (X). W , Y W , Z W ). The user's orientation relative to the position of camera 310 can be represented by a set of camera image coordinates (X C , Y C , Z C ) are represented. In some examples, capturing or determining the user orientation involves transforming from the world coordinate system to the camera image coordinate system.

[0032] The object detection circuit arrangement 404 determines the presence of an object based on visual data. The detected object can be used in some examples to evaluate user activity. For instance, some real-world objects are objects with which the user intends to interact and are therefore relevant for mapping user action to the avatar. In some examples, the object detection circuit arrangement 404 can also detect the location and orientation of various electronic devices, displays, and / or peripherals, which can be used in determining user orientation.

[0033] The User Activity Classification Circuit Arrangement 406 determines the user's action based on audio, visual, program tracking, and / or orientation. For example, the User Activity Classification Circuit Arrangement 406 can determine whether the user is looking at the Display 320, whether the user is looking at another device, whether the user is looking at a device not coupled to the virtual environment, whether the user has been enjoying themselves, whether the user has yawned, whether the user's head is facing toward or away from the Display 320, whether the user is holding an object, whether the user is looking at an object, whether the user is speaking, whether the user is busy, whether the user is making a rapid movement, whether the user is temporarily looking away from the Display 320, and it can determine other actions or combinations of actions.

[0034] In some examples, the user activity classification circuit arrangement 406 determines, classifies, or categorizes the action as intentional or unintentional. In some examples, the user activity classification circuit arrangement 406 has accessed the logged actions 326 in the database 322 and evaluates whether an action is logged as intentional or unintentional. For example, a sneeze may be logged as unintentional. A thumbs-up gesture or a hand wave may be logged as intentional. Turning the head toward the display 320, which shows the virtual environment, may be logged as intentional. A whisper may be logged as unintentional. A sigh may be logged as unintentional. In some examples, the user specifies or defines certain actions as intentional or unintentional.In some examples, the 406 user activity classification circuit learns, based on a history of user activity, whether an action is intentional or unintentional. Artificial intelligence, including, for example, machine learning technologies, can be implemented through the 406 user activity classification circuit to learn intentional or unintentional classifications of actions.

[0035] In some examples, the User Activity Classification Circuit Arrangement 406 accesses sources of context data to determine whether an action is intentional or unintentional. Sources of context data include, for example, current user dialogue, previous conversations, background audio, a level of background noise, other audio data, the framework of the virtual environment, a time of day, connectivity of a device to the virtual environment, the position of an object, other visual data, the presence or operating mode of a program on a device, and other sources from which a real-world or virtual environment context can be evaluated by the User Activity Classification Circuit Arrangement 406.

[0036] In some examples, the User Activity Classification Circuit 406 determines whether a user's audio activity is consistent with the context. For instance, the User Activity Classification Circuit 406 analyzes the audio data and evaluates whether the audio is consistent with the context of the virtual environment. If the User Activity Classification Circuit 406 determines that the audio is consistent with the context, the action is classified as intentional. If the User Activity Classification Circuit 406 determines that the audio is not consistent with the context, the action is classified as unintentional. If the User Activity Classification Circuit 406 determines that the user action is a sneeze, a sigh, a cough, a change of topic in a conversation, etc.In some examples, User Activity Classification Circuit 406 classifies the action as unintentional. If User Activity Classification Circuit 406 determines that the user action is conversationally consistent with the topic of a current or previous conversation, it determines that the user action is intentional in some examples. In some examples, certain words can trigger User Activity Classification Circuit 406 to classify an action as intentional or unintentional. For example, an action in which a user says the name of another avatar in the virtual environment may be classified as intentional, while an action in which the user says a name that is unrelated to anything in the virtual environment may be classified as unintentional.

[0037] In some examples, audio activity may be outside the user's control. For instance, background noise or sounds in the real-world environment could be introduced as audio data that the user cannot control. The User Activity Classification Circuit Arrangement 406 analyzes the audio data to determine whether the audio source is the user or an external source, thus determining whether the audio is consistent with the context of the virtual environment, as revealed above. Unintended audio data is categorized as an unintended action and is not reproduced or otherwise manipulated in a virtual environment. This determination of unintended action based on audio data can also be used by the User Activity Classification Circuit Arrangement 406 to classify response behavior as unintended.For example, a loud bang in the background of the real-world environment can increase the likelihood of an unintentional physical input or behavior by the user (such as a sudden movement or head turn). The User Activity Classification Circuit Arrangement 406 can classify the physical action as unintentional based on the associated audio data being categorized as such.

[0038] In some examples, the User Activity Classification Circuit Arrangement 406 determines whether a user has directed their attention to a device that is not coupled to the virtual environment. Devices are coupled to the virtual environment if the device can be used to input or output data to or from the virtual environment(s). For example, a device that presents a display of the virtual environment to the user is coupled to the virtual environment. A device that measures movement from the user that can be translated to an avatar is a device coupled to the virtual environment. A device that does not collect data for use in the virtual environment and does not present any aspects of the virtual environment is a device that is not coupled to the virtual environment.If the User Activity Classification Circuit Arrangement 406 determines that the user's attention remained on a device connected to the virtual environment or was directed toward another device connected to the virtual environment, the action is classified as intentional. If the User Activity Classification Circuit Arrangement 406 determines that the user's attention was directed toward a device not connected to the virtual environment, the action is classified as unintentional. For example, if the User Activity Classification Circuit Arrangement 406 determines that a user briefly looked at a clock, and the clock is not connected to the virtual environment, the User Activity Classification Circuit Arrangement 406 may classify the action as unintentional.If the user activity classification circuit arrangement 406 determines that the user action is a movement of a head orientation from a first display to a second display, and both displays are connected to the virtual environment, the user activity classification circuit arrangement 406 can classify the action as intentional.

[0039] In some examples, when the action is a movement, User Activity Classification Circuit 406 determines whether the action is consistent with the context of the virtual environment. If User Activity Classification Circuit 406 determines that the movement is consistent with the context, the action is classified as intentional. If User Activity Classification Circuit 406 determines that the movement is not consistent with the context, the action is classified as unintentional. For example, if User Activity Classification Circuit 406 determines that the action is a quick turn of the head away from the display of the virtual environment, User Activity Classification Circuit 406 may classify the action as unintentional.If the user activity classification circuit arrangement 406 determines that the action is a head movement associated with a sneeze, the user activity classification circuit arrangement 406 may classify the action as unintentional. If the user activity classification circuit arrangement 406 determines that the movement is a position or orientation of the user's head or face that is offset or misaligned relative to the display of the virtual environment, the user activity classification circuit arrangement 406 may classify the action as unintentional. If the user activity classification circuit arrangement 406 determines that the action is a clap and the context of the virtual environment is a game, the user activity classification circuit arrangement 406 may classify the action as intentional.If the user activity classification circuit arrangement 406 determines that the action is bending down into the field of view of camera 310 and the virtual environment context is a game, the user activity classification circuit arrangement 406 can classify the action as unintentional. If the user activity classification circuit arrangement 406 determines that the action is getting up to close a window and there is no window in the virtual environment, the user activity classification circuit arrangement 406 can classify the action as unintentional.

[0040] In some examples, motion activity may be outside the user's control. For instance, movements of people or objects in the background of the real-world environment could be introduced as motion data that the user cannot control. The User Activity Classification Circuit Arrangement 406 analyzes the motion data to determine whether the source of the motion data is the user or an external source, in order to determine if the motion is consistent with the context of the virtual environment, as revealed above. Unintended motion data is categorized as an unintended action and is not reproduced or otherwise manipulated in a virtual environment. This determination of unintended action based on motion data can also be used by the User Activity Classification Circuit Arrangement 406 to classify response behavior as unintended.For example, another person or animal moving in the background of the real-world environment can increase the likelihood of unintentional audio input by the user (e.g., speaking to the person or animal). The User Activity Classification Circuit Arrangement 406 can classify the audio action as unintentional based on the associated motion data being categorized as such.

[0041] In some examples, the action is an emotion. User Activity Classification Circuit 406 can identify an emotion based on, for example, a facial expression, an auditory tone, etc. If User Activity Classification Circuit 406 determines that the emotion is consistent with the context, the action is classified as intentional. If User Activity Classification Circuit 406 determines that the emotion is inconsistent with the context, the action is classified as unintentional. If the emotion is anger, but the virtual environment is a business meeting, User Activity Classification Circuit 406 can classify the action as unintentional. In some examples, the user can predefine a desired emotional state or a target emotional state.The User Activity Classification Circuit Arrangement 406 compares the user's current emotion with the desired emotional state or emotional target state. If the user emotion matches the emotional target state, the User Activity Classification Circuit Arrangement 406 can classify the action as intended. If the user emotion does not match the emotional target state, the User Activity Classification Circuit Arrangement 406 can classify the action as unintended.

[0042] When the activity classification circuit arrangement 406 determines that an action is intended, the action is integrated into the avatar model for translation or reproduction by the avatar. For actions involving head or face orientation, the orientation offset circuit arrangement 408 determines whether an orientation offset, inconsistency, or dissonance exists between the user's head position in the real world and the avatar's position in the virtual world. Dissonance is a perceptible and / or unnatural difference between the user's state (user position or orientation) and an avatar reproduction (avatar position or orientation). For example, the user may speak to other members in a virtual environment (e.g.,(during a meeting presented in the metaverse), and he might be interrupted or distracted and turn his attention to a second screen located 25 degrees to the left of the main screen and the camera. If the user continues speaking while looking at the second screen, the face of the avatar representing the user in the virtual environment would be oddly rotated by 25 degrees. The examples revealed here account for the dissonance, and the avatar maintains its focus (e.g., the orientation or position of its head, face, and eyes) on the elements within the virtual environment.

[0043] In some examples, the dissonance is caused by an offset between camera 310 and display 320 (or another display that is the focus of the user's attention). The offset is due to camera 310 being positioned at an angle relative to display 320. The alignment offset circuit arrangement 408 makes two estimates of the user's alignment, one to camera 310 and one to the content of interest in the activity (e.g., display 320). As noted above, the alignment estimates can be provided as sets of coordinates. As revealed above, the user's alignment with respect to the content of interest can be provided by the set of real coordinates (X). W , Y W , Z W ) can be represented, and the user's orientation in relation to the position of camera 310 can be represented by a set of camera image coordinates (X C , Y C , Z C) can be represented. The orientation of the avatar in the virtual environment can be represented by a set of target coordinates (X). T , Y T , Z T ) are represented.

[0044] The alignment offset circuit arrangement 408 determines whether the orientation of the avatar (X) T , Y T , Z T ) with the user's orientation towards the content of interest (X W , Y W , Z W ) matches or is consistent with it. If the avatar's orientation (X) T , Y T , Z T ) not in line with the user's orientation regarding the content of interest (X W , Y W , Z W ) whether it matches or is inconsistent with it, the alignment offset circuit arrangement 408 determines a difference between the user's alignment with respect to the content of interest (X W , Y W , Z W) and the user's orientation in relation to the camera image coordinates (X C , Y C , Z C ).

[0045] The avatar model circuit arrangement 412 updates the avatar model based on a difference between the user's orientation relative to the content of interest (X). W , Y W , Z W ) and the user's orientation in relation to the camera image coordinates (X C , Y C , Z CThe avatar mapping circuit arrangement 410 encodes or maps the user action onto the avatar and adjusts the avatar's position or orientation based on the avatar adjustment. In some examples, a local system receives input from devices, such as gamepads, and maps the input to a Human Interface Device input stream (HID input stream). In some examples, the HID data is streamed to the host application 330. Thus, in some examples, avatar input is corrected for orientation before being placed in the HID stream. Therefore, in some examples, a compressed data set is transmitted to the server and / or the host application 330 instead of large sets of frames.

[0046] Fig. Figure 5 is an illustration of an exemplary correlation of exemplary coordinate systems, generated by the avatar fitting application 316 of Fig. 3 can be used, with a user face in a first position. As in Fig. As shown in Figure 5, a user 500 is oriented towards the content of interest (the virtual environment) displayed on a first electronic device 502. The user's orientation directly in front of the content of interest results in the world coordinates (0 W , 0 W , 0 W ) are.

[0047] A camera 504 on a second electronic device 506 is offset from the user 500. In this example, the camera image 508 shows the user facing right. This results in the camera image coordinates (X) being C , Y C , Z CThe positional offset between camera 504 and the first display 502, which shows the content of interest, was taken into account during avatar modeling by the avatar model circuit arrangement 412. Thus, the avatar mapping circuit arrangement 410 renders an avatar 512 that is aligned with the same position as the user in the real world. Therefore, the target coordinates (0 T , 0 T , 0 T ), agree with or are consistent with those of the world coordinates.

[0048] Fig. Figure 6 illustrates the correlation of the coordinate systems achieved by the avatar fitting application 316. Fig. 3 can be used, with the user's face in a second position while the user moves 500. As in Fig. As shown in Figure 6, the user has moved 500 and is oriented at an angle relative to the content of interest (the virtual environment) displayed on the first electronic device 502. The user's orientation, which is turned away from the content of interest, results in the world coordinates (X) W , Y W , Z W ) are.

[0049] In this example, user 500 is facing camera 504. Therefore, camera image 508 shows the user directly aligned with camera 504. This results in the camera image coordinates (0 C , 0 C , 0 C). Since the positional offset between the camera 504 and the first display 502, which shows the content of interest, was taken into account in the avatar modeling by the avatar model circuit arrangement 412, the avatar mapping circuit arrangement 410 renders an avatar 512 that is oriented in the same position as the user in the real world (and thus facing sideways). Thus, the target coordinates (X) T , Y T , Z T ), agree with or are consistent with those of the world coordinates.

[0050] In some examples, the Avatar Customization Application 316 considers an estimate of whether a user is interacting with a real-world object, rather than looking at the content of interest in the virtual environment. When a user is interacting with a real-world object, the Avatar Customization Application 316 does not evaluate the consistency or dissonance of the user and avatar orientation.

[0051] Furthermore, in some examples, the avatar model circuit arrangement 412 adjusts the direction of audio data so that the playback of audio (e.g., surround sound audio) correlates with or is consistent with the corrected avatar orientation.

[0052] In some examples, the Avatar Model Circuit Arrangement 412 uses three-dimensional avatar models. And in some examples, the Avatar Model Circuit Arrangement 412 uses algorithms to augment and fill in non-visible parts of the user that are to be mapped onto the avatar model (e.g., second eye or other side of the face, etc.).

[0053] In some examples, the avatar adaptation application 316 performs the mapping and adaptation of user-to-avatar actions for multiple avatars for multiple users within a camera's field of view. The multiple mapping and adaptation applications can be performed simultaneously and / or independently.

[0054] In some examples, the System 300 can have two or more cameras. To align the avatar, a camera with a fixed position relative to the content of interest would have higher priority. For example, a smart display with an integrated webcam would track the user's gaze on the content of the smart display's screen. In some examples, panning a virtual camera on a display is used to map a panning movement of an avatar.

[0055] In some examples, multiple microphone inputs (e.g., from multiple audio devices) or head-related transfer function (HRTF) data from low-energy audio (LE audio) can be further used or optimized to realign the avatar orientation.

[0056] If the activity classification circuit arrangement 406 determines that an action is unintended, the action is blocked. A blocked action is not mapped to the avatar. Instead, the avatar model circuit arrangement determines whether the avatar retains a previous position (for example, temporarily freezes) or whether the blocked input is replaced with an alternative input.

[0057] If the unintended input is a misaligned head position, the replacement input is determined based on the avatar customization mechanisms revealed above. If the unintended action is something other than head alignment, an alternative input is provided. For example, the alternative input can be user-defined, where the user chooses what the alternative input will be. For instance, if the user's unintended action is an angry emotion, the user can select (or preselect) that the replacement input for the avatar is a happy emotion. In another example, the user can choose that their avatar paces restlessly back and forth during the alternative input until the unintended action is complete.

[0058] In some examples, the alternative input can be character-based. For instance, the avatar model circuit arrangement 412 can select an alternative input for the avatar mapping circuit arrangement 410 that corresponds to the avatar's character and is to be mapped to a user's avatar during an unintended action. For example, a dragon avatar might breathe fire during an unintended action. In some examples, the alternative input is selected and / or modified so that the avatar appears to naturally revert to a user-to-avatar mapping when the unintended action is complete.

[0059] Because the Avatar Customization Application 316 blocks or prevents unintended actions from being processed and mapped to an avatar, users of avatars in virtual environments do not experience unintended actions or interruptions.

[0060] Fig. Figure 4 is a block diagram of the Avatar Customization Application 316 for performing a user-to-avatar mapping and customization. The Avatar Customization Application 316 from Fig. 4 can be instantiated by a processor circuit arrangement, such as a central processing unit, which executes instructions (e.g., creating an instance, realizing for any duration, realizing, implementing, etc.). Additionally or alternatively, the avatar customization application 316 can be Fig. 4. are instantiated by an ASIC or FPGA structured to perform operations according to the instructions (e.g., creating an instance, realizing for a specified duration, realizing, implementing, etc.). It is understood that some or all of the circuit arrangements of Fig. 4. Thus, they can be instantiated at the same time or at different times. For example, part or all of the circuit arrangement can be instantiated in one or more threads that are executed concurrently on hardware and / or sequentially on hardware. Furthermore, in some examples, part or all of the circuit arrangement can be Fig. 4. are implemented by a microprocessor circuit arrangement that executes instructions for implementing one or more virtual machines and / or one or more containers.

[0061] In some examples, the program identification circuit arrangement 314 is instantiated by a processor circuit arrangement that executes program identification instructions and / or is designed to perform operations such as those described by the flowcharts of Fig. 7, Fig. 8 to Fig. 9, to perform. In some examples, the user alignment circuit arrangement 402 is instantiated by a processor circuit arrangement that executes user alignment instructions and / or is designed to perform operations such as those shown in the flowcharts of Fig. 7, Fig. 8 to Fig. 9, to be carried out. In some examples, the object detection circuit arrangement 404 is instantiated by a processor circuit arrangement that executes object detection instructions and / or is designed to perform operations such as those shown in the flowcharts of Fig. 7, Fig. 8 to Fig. 9, to be carried out. In some examples, the user activity classification circuit arrangement 406 is instantiated by a processor circuit arrangement that executes user activity classification instructions and / or is designed to perform operations such as those shown in the flowcharts of Fig. 7, Fig. 8 to Fig. 9, to perform. In some examples, the alignment offset circuit arrangement 408 is instantiated by a processor circuit arrangement that executes alignment offset instructions and / or is designed to perform operations such as those shown in the flowcharts of Fig. 7, Fig. 8 to Fig. 9, to be carried out. In some examples, the avatar mapping circuit arrangement 410 is instantiated by a processor circuit arrangement that executes avatar mapping instructions and / or is designed to perform operations such as those shown in the flowcharts of Fig. 7, Fig. 8 to Fig. 9, to be carried out. In some examples, the avatar model circuit arrangement 402 is instantiated by a processor circuit arrangement that executes avatar model instructions and / or is designed to perform operations such as those described in the flowcharts of Fig. 7, Fig. 8 to Fig. 9 shown, to be carried out.

[0062] In some examples, the setup includes a means of mapping user-to-avatar actions. This mapping means can be implemented, for example, by the Avatar Customization Application 316. In some examples, the Avatar Customization Application 316 can be implemented by a processor circuit arrangement, such as the exemplary processor circuit arrangement 1012 from Fig. 10. For example, the avatar customization application 316 can be instantiated by the exemplary microprocessor 1100 from Fig. 11 are instantiated, the machine-executable instructions, such as those defined at least by blocks 710 to 732 of Fig. 7 and the blocks of Fig. 8 and Fig. 9 implemented, executes. In some examples, the Avatar Customization Application 316 can be instantiated by a hardware logic circuit arrangement implemented by an ASIC, an XPU, or the FPGA circuit arrangement 1200 from Fig. 12 can be implemented, which is structured to perform operations corresponding to the machine-readable instructions. Additionally or alternatively, the Avatar Adaptation Application 316 can be instantiated by any other combination of hardware, software, and / or firmware. For example, the Avatar Adaptation Application 316 can be implemented by at least one or more hardware circuits (e.g., a processor circuit arrangement, a discrete and / or integrated analog and / or digital circuit arrangement, an FPGA, an ASIC, an XPU, a comparator, an operational amplifier (op-amp), a logic circuit, etc.) that are structured to execute some or all of the machine-readable instructions and / or to perform some or all of the operations corresponding to the machine-readable instructions without executing any software or firmware, although other structures are also suitable.

[0063] While one exemplary way of implementing the avatar customization application 316 of Fig. 3 in Fig. As illustrated in section 4, one or more of the elements, processes and / or devices that are in Fig. 4, are illustrated, may be combined, divided, rearranged, omitted, removed, and / or implemented in any other way. Furthermore, the exemplary program identification circuit arrangement 314, the exemplary user alignment circuit arrangement 402, the exemplary object detection circuit arrangement 404, the exemplary user activity classification circuit arrangement 406, the exemplary alignment offset circuit arrangement 408, the exemplary avatar mapping circuit arrangement 410, the exemplary avatar modeling circuit arrangement 412, and / or more generally, the exemplary avatar customization application 316 of Fig. 3 and Fig. 4. can be implemented by hardware alone or by hardware in combination with software and / or firmware. Thus, for example, each of the exemplary user alignment circuit arrangement 402, the exemplary object detection circuit arrangement 404, the exemplary user activity classification circuit arrangement 406, the exemplary alignment offset circuit arrangement 408, the exemplary avatar mapping circuit arrangement 410, the exemplary avatar modeling circuit arrangement 412, and / or more generally, the exemplary avatar adaptation application 316 could be implemented by a processor circuit arrangement, analog circuit(s), digital circuit(s), logic circuit(s), programmable processor(s), programmable microcontroller(s), graphics processing unit(s) (GPU(s)), digital signal processor(s) (DSP(s)), application-specific integrated circuit(s) (ASIC(s)),Programmable logic devices (PLDs) and / or field-programmable logic devices (FPLDs), such as field-programmable gate arrays (FPGAs), can be implemented. Furthermore, the exemplary avatar customization application 316 from [reference missing] can be used. Fig. 4. Furthermore, one or more elements, processes and / or devices in addition to or instead of those in Fig. 4 illustrated elements, processes and devices and / or may include more than one or all of the illustrated elements, processes and devices.

[0064] Flowcharts that represent exemplary machine-readable instructions that can be executed to configure a processor circuit arrangement to run the Avatar Customization Application 316 by Fig. 4 to implement are in Fig. 7, Fig. 8 to Fig. 9 shown. The machine-readable instructions can be one or more executable programs or sections of an executable program for execution by a processor circuit arrangement, such as the processor circuit arrangement 1012 shown in the exemplary processor platform 1000, which is described below in conjunction with Fig. 10 is discussed, and / or the exemplary processor circuit arrangement, which is further discussed below in connection with Fig. 11 and / or 12. The program may be executed in software stored on one or more non-transient, computer-readable storage media, such as a compact disc (CD), a floppy disk, a hard disk drive (HDD), a solid-state drive (SSD), a digital versatile disc (DVD), a Blu-ray disc, volatile memory (e.g., random-access memory (RAM) of any type, etc.), or non-volatile memory (e.g., electrically erasable programmable read-only memory (EEPROM), flash memory, an HDD, an SSD, etc.), associated with a processor circuit arrangement located in one or more hardware devices; alternatively, the entire program and / or parts thereof may be executed by one or more hardware devices other than the processor circuit arrangement and / or be executed in firmware or dedicated hardware.The machine-readable instructions can be distributed across multiple hardware devices and / or executed by two or more hardware devices (e.g., a server and a client hardware device). For example, the client hardware device can be implemented as a gateway to an endpoint client hardware device (e.g., a hardware device associated with a user) or an intermediate client hardware device (e.g., a wireless access network (RAN)) that can support communication between a server and an endpoint client hardware device. Similarly, the non-transitory machine-readable storage media can comprise one or more media located in one or more hardware devices. Furthermore, the exemplary program refers to the one in... Fig. 7, Fig. 8 to Fig. As described in the illustrated flowchart 9, many other methods can alternatively be used to implement the exemplary avatar customization application 316. For example, the execution order of the blocks can be changed, and / or some of the described blocks can be modified, removed, or combined. Additionally or alternatively, any or all of the blocks can be implemented by one or more hardware circuits (e.g., a processor circuit arrangement, a discrete and / or integrated analog and / or digital circuit arrangement, an FPGA, an ASIC, a comparator, an operational amplifier (op-amp), a logic circuit, etc.) structured to perform the appropriate operation without executing any software or firmware. The processor circuit arrangement can be located at different network locations and / or locally on one or more hardware devices (e.g., a single-core processor (e.g., a 3-core processor)).a single-core central processing unit (CPU)), a multi-core processor (e.g., a multi-core CPU, an XPU, etc.) in a single machine, multiple processors distributed across multiple servers in a server rack, multiple processors distributed across one or more server rack(s), a CPU and / or an FPGA located in the same package (e.g., in the same package of an integrated circuit (IC package) or distributed across two or more separate packages, etc.).

[0065] The machine-readable instructions described here can be stored in one or more formats, including compressed, encrypted, fragmented, compiled, executable, and packed. Machine-readable instructions, as described here, can be stored as data or a data structure (e.g., as sections of instructions, code, representations of code, etc.) that can be used to create, manufacture, and / or generate machine-executable instructions. For example, the machine-readable instructions can be fragmented and stored on one or more storage devices and / or computing devices (e.g., servers) located in the same or different locations within a network or collection of networks (e.g., in the cloud, at edge devices, etc.).The machine-readable instructions may require one or more of the following operations: installation, modification, adaptation, updating, combining, augmenting, configuring, decryption, decompression, unpacking, distribution, reassignment, compilation, etc., to make them directly readable, interpretable, and / or executable by a computing device and / or another machine. For example, the machine-readable instructions may be stored in multiple parts, each individually compressed, encrypted, and / or stored on separate computing devices. When these parts are decrypted, decompressed, and combined, they form a set of machine-executable instructions that implement one or more operations which together can constitute a program, such as the one described here.

[0066] In another example, the machine-readable instructions might be stored in a state where they can be read by a processor circuit arrangement, but require the addition of a library (e.g., a dynamically linked library (DLL)), a software development kit (SDK), an application programming interface (API), etc., to execute the machine-executable instructions on a specific computing device or other device. In yet another example, the machine-readable instructions might need to be configured (e.g., settings saved, data entered, network addresses captured, etc.) before the machine-readable instructions and / or the corresponding program(s) can be executed in whole or in part.Thus, machine-readable media, as used here, can contain machine-readable instructions and / or (a) program(s), regardless of the specific format or state of the machine-readable instructions and / or program(s), when stored or otherwise in a dormant or transmission state.

[0067] The machine-readable instructions described herein can be represented by any past, present, or future instruction language, scripting language, programming language, etc. For example, the machine-readable instructions can be represented using any of the following languages: C, C++, Java, C#, Perl, Python, JavaScript, HyperText Markup Language (HTML), Structured Query Language (SQL), Swift, etc.

[0068] As mentioned above, the exemplary operations of Fig. 7, Fig. 8 to Fig. 9. are implemented using executable instructions (e.g., computer- and / or machine-readable instructions) stored on one or more non-transitory computer- and / or machine-readable media, such as optical storage devices, magnetic storage devices, a hard disk drive (HDD), flash memory, read-only memory (ROM), a CD, a DVD, a cache, RAM of any type, a register, and / or any other storage device or disk on which information is stored for any duration (e.g., for extended periods, permanently, for short periods, for temporary buffering, and / or for intermediate storage of information).As used herein, the terms non-transitory computer-readable medium, non-transitory computer-readable storage medium, non-transitory machine-readable medium, and non-transitory machine-readable storage medium are expressly defined to include any type of computer-readable storage device and / or storage disk, but exclude signal propagation and transmission media. The terms "computer-readable storage device" and "machine-readable storage device" are defined herein to include any physical (mechanical and / or electrical) structure for storing information, but exclude signal propagation and transmission media.Examples of computer-readable storage devices and / or machine-readable storage devices include random-access memory (RAM) of any type, read-only memory (ROM) of any type, solid-state memory, flash memory, optical disks, magnetic disks, disk drives, and / or redundant array of independent disks (RAID) systems. As used herein, the term "device" refers to a physical structure, such as mechanical and / or electrical equipment, hardware, and / or a circuit arrangement, that can or cannot be configured by computer-readable instructions, machine-readable instructions, etc.

[0069] "Including" and "comprising" (and all forms and tenses thereof) are used here as open terms. It is understood that whenever a claim uses any form of "including" or "comprising" (e.g., encompasses, exhibits, encompassing, including, exhibiting, etc.) as a preamble or within any kind of claim statement, additional elements, terms, etc., may be present without falling outside the scope of protection of the claim or statement in question. When used in this document, the expression "at least" is used as the transitional term in, for example, a preamble of a claim, it is open in the same way as the terms "comprising" and "including" are open.The term "and / or," when used in a form such as A, B and / or C, refers to any combination or subset of A, B, C, such as (1) A alone, (2) B alone, (3) C alone, (4) A with B, (5) A with C, (6) B with C, or (7) A with B and with C. When used in this document in the context of describing structures, components, elements, objects, and / or things, the phrase "at least one of A and B" refers to implementations that include any one of (1) at least one A, (2) at least one B, or (3) at least one A and at least one B. Similarly, when used in this document in the context of describing structures, components, elements, objects and / or things, the phrase “at least one of A or B” shall refer to implementations that include any one of (1) at least one A, (2) at least one B, or (3) at least one A and at least one B.When used in this document in the context of describing the execution or performance of processes, instructions, actions, activities, and / or steps, the phrase "at least one of A and B" shall refer to implementations that include any one of (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. Similarly, when used in this document in the context of describing the execution or performance of processes, instructions, actions, activities, and / or steps, the phrase "at least one of A or B" shall refer to implementations that include any one of (1) at least one A, (2) at least one B, and (3) at least one A and at least one B.

[0070] When used in this document, singular references (e.g., "a," "a," "first," "second," etc.) do not preclude plurality. The term "a" object, when used in this document, refers to one or more of that object. The terms "a" (or "a"), "a or more," and "at least one" are used interchangeably here. Furthermore, although listed individually, multiple means, elements, or procedural actions can be implemented by, for example, the same entity or object. Additionally, although individual features may be included in different examples or claims, they may potentially be combined, and inclusion in different examples or claims does not imply that a combination of features is not feasible and / or advantageous.

[0071] Fig. Figure 7 is a flowchart representing exemplary machine-readable instructions and / or exemplary operations 700 that can be executed and / or instantiated by a processor circuit arrangement to map real-world user actions to an avatar in a virtual environment. The machine-readable instructions and / or operations 700 of Fig. Figure 7 shows the acquisition of visual data (block 702) by the sensor array 308, including the camera. The sensor array 308, including the microphone 312, acquires audio data (block 704). In addition, the program identification circuit arrangement 314 acquires program user data indicating which programs are used on devices operated by the user (block 706).

[0072] Operations 700 also include the user activity classification circuit arrangement 406, which determines whether a user action is intended or unintended (block 710). Further details regarding processes for determining whether a user action is intended or unintended are given above and in conjunction with Fig. 8 disclosed. If and / or when the user activity classification circuit arrangement 406 determines that an action is intended (block 710: INTENTIONED), the alignment offset circuit arrangement 408 evaluates for an alignment offset or dissonance between a user's position or orientation in the real world and the position or orientation of a corresponding avatar in the virtual environment (block 712). The alignment offset circuit arrangement 408 determines whether there is an offset between a user's position or orientation in the real world and the position or orientation of a corresponding avatar in the virtual environment (block 714).For example, the alignment offset circuit arrangement 408 can identify that there is an offset between the position or orientation of a user in the real world and the position or orientation of a corresponding avatar in the virtual environment if the difference between the position or orientation of a user in the real world and the position or orientation of a corresponding avatar in the virtual environment is greater than or otherwise reaches a threshold difference.Similarly, the alignment offset circuit arrangement 408 can identify that there is no offset between the position or orientation of a user in the real world and the position or orientation of a corresponding avatar in the virtual environment if the difference between the position or orientation of a user in the real world and the position or orientation of a corresponding avatar in the virtual environment is less than a threshold difference or does not otherwise reach it.

[0073] If and / or when the orientation offset circuit arrangement 408 determines that there is an offset between the position or orientation of a user in the real world and the position or orientation of a corresponding avatar in the virtual environment (Block 408: YES), the avatar model circuit arrangement 412 determines an avatar adjustment (Block 716). Further details regarding processes for determining an avatar adjustment are given above and in conjunction with Fig. Figure 9 discloses that the avatar adaptation specifies the values ​​or calculations for how to modify the avatar's position or orientation to align with, or otherwise correspond to, the user's position or orientation in the real world. Once the avatar adaptation is determined, the avatar model circuit arrangement 412 applies the adaptation to the avatar model (Block 718). The avatar mapping circuit arrangement 410 encodes, or translates, the user's action (i.e., the action specified in Block 708) with the avatar adaptation onto the avatar (Block 720). Process 700 indicates that the user activity classification circuit arrangement 406 determines to the avatar adaptation application 316 whether to map an additional user-to-avatar action (Block 732).

[0074] If and / or when the orientation offset circuit arrangement 408 determines that there is no offset between the position or orientation of a user in the real world and the position or orientation of a corresponding avatar in the virtual environment (block 408: NO), the avatar mapping circuit arrangement 410 encodes or translates the user's action (i.e., the action determined in block 708) with the avatar adaptation to the avatar (block 720). Process 700 indicates that the user activity classification circuit arrangement 406 of the avatar adaptation application 316 determines whether to map an additional user-to-avatar action (block 732).

[0075] If and / or when the user activity classification circuit arrangement 406 determines that an action is unintentional (Block 710: UNINTENTIONAL), the avatar model circuit arrangement 412 blocks the input (Block 722). In other words, the action determined at Block 708 is not processed for mapping to the avatar. The avatar model circuit arrangement 412 determines whether to replace the input or maintain the avatar's position (Block 724). For example, the avatar's actions may be replaced by an alternative action that does not map the unintentional action, or in some examples, the avatar may be maintained (e.g., Pause). If and / or when the avatar model circuit arrangement 412 determines that the avatar's position should be maintained (Block 724: MAINTAIN), the avatar mapping circuit arrangement 410 maintains the avatar in the same state (Block 726).Process 700 indicates that the user activity classification circuit arrangement 406 of the avatar customization application 316 determines whether to map an additional user-on-avatar action (block 732).

[0076] If and / or when the avatar model circuit arrangement 412 determines that the avatar position should be replaced (Block 724: REPLACE), the user activity classification circuit arrangement 406 determines whether the unintended action is a head / face alignment (Block 728). If and / or when the user activity classification circuit arrangement 406 determines that the unintended action is a head / face alignment (Block 728: YES), the exemplary process 700 continues with the avatar model circuit arrangement determining the avatar adjustment (Block 716).

[0077] If and / or when the user activity classification circuit arrangement 406 determines that the unintended action is not a head / face orientation (block 728: NO), the avatar model circuit arrangement 412 receives a new input (block 730). For example, the sample avatar model circuit arrangement 412 might receive an alternative input, such as an avatar pacing restlessly, to map to the avatar for the duration of the unintended action. In some examples, the avatar customization application 316 requests user input and / or user preferences to receive a new input. In some examples, the avatar customization application 316 prompts the user for input through a visual user interface prompt, such as on a screen. Additionally, in some examples, a user might provide a vision-based response, such as by making a gesture.In some examples, the avatar customization application 316 prompts the user for input via an audio request. Additionally, in some examples, a user can provide an audio-based response, such as a voice command.

[0078] In some examples, the new input might be an override command from the user indicating that an action perceived as an unintended input was intentional. In other words, to obtain new input (block 730), the avatar customization application 316 might prompt the user to provide new input or other preference data. In this example, the user can override, correct, or adjust the algorithm if the user has exhibited (or preemptively knows they will exhibit) behavior that the system or algorithm might consider unintended, but they want the behavior directly generated in their active avatar. For example, a user might want a fake sneeze or a moment of jumping up from their chair in shock to be reproduced by their avatar. In some examples, the new input of this type is an override control.In some examples, the override control is accessible at all times. In some examples, the override control is accessible as soon as the system begins to detect unintended behavior. In some examples, the override control is available in real time. Process 700 proceeds by having the avatar mapping circuitry 410 encode or translate (block 720) the new input action (i.e., the input from block 730) to the avatar.

[0079] If and / or when the user activity classification circuit arrangement 406 of the avatar customization application 316 determines that an additional user-on-avatar action should be mapped (Block 732: YES), exemplary process 700 continues from Block 702. If and / or when the user activity classification circuit arrangement 406 of the avatar customization application 316 determines that no additional user-on-avatar action should be mapped (Block 732: YES), exemplary process 700 terminates.

[0080] Fig. 8 is a flowchart that shows exemplary machine-readable instructions and / or exemplary operations 710 (of Fig. 7) represents operations that can be executed and / or instantiated by a processor circuit arrangement to determine whether a user action is intended or unintended. Exemplary operations 710 indicate that the user activity classification circuit arrangement 406 determines the action (i.e., the one in block 708 of Fig. The user activity classification circuit (406) compares the identified action (Block 802) with logged actions. Block 804 determines whether the action is a logged intended action. If and / or when Block 806 determines that the action is a logged intended action (Block 804: YES), Block 806 identifies the action as intended.

[0081] If and / or when the User Activity Classification Circuit 406 determines that the action is not a logged intended action (Block 804: NO), the User Activity Classification Circuit 406 determines whether the action is a logged unintended action (Block 808). If and / or when the User Activity Classification Circuit 406 determines that the action is a logged unintended action (Block 808: YES), the User Activity Classification Circuit 406 identifies the action as unintended (Block 810). If and / or when the User Activity Classification Circuit 406 determines that the action is not a logged unintended action (Block 808: NO), the User Activity Classification Circuit 406 retrieves context data (Block 812).In some examples, the contextual data includes, for example, audio data, visual data, motion data, data relating to the device a user is focusing on, data relating to the virtual environment, emotion data, and other types of data to establish a context of the virtual environment.

[0082] The user activity classification circuit arrangement 406 determines whether the audio data in the action (i.e., the one in block 708 of Fig. 7 identified action) are consistent with the context (Block 814). If and / or when the User Activity Classification Circuit Arrangement 406 determines that the audio is not consistent with the context (Block 814: NO), the User Activity Classification Circuit Arrangement 406 identifies the action as unintentional (Block 810). If and / or when the User Activity Classification Circuit Arrangement 406 determines that the audio is consistent with the context (Block 814: YES), the User Activity Classification Circuit Arrangement 406 determines whether the action (i.e., the one identified in Block 708 of Fig. 7 identified action) consists of the user's attention being directed to a device that is not connected to the virtual environment (Block 816).

[0083] If and / or when the user activity classification circuit arrangement 406 determines that the user's attention was directed toward a device not connected to the virtual environment (block 816: YES), the user activity classification circuit arrangement 406 identifies the action as unintentional (block 810). If and / or when the user activity classification circuit arrangement 406 determines that the user's attention was not directed toward a device not connected to the virtual environment (block 816: NO), the user activity classification circuit arrangement 406 determines whether the action (i.e., the one in block 708 of Fig. 7 identified action) is a movement that is consistent with the context of the virtual environment (Block 818).

[0084] If and / or when the User Activity Classification Circuit 406 determines that the movement is inconsistent with the context (Block 818: NO), the User Activity Classification Circuit 406 identifies the action as unintentional (Block 810). If and / or when the User Activity Classification Circuit 406 determines that the movement is consistent with the context (Block 818: YES), the User Activity Classification Circuit 406 determines whether the action (i.e., the one in Block 708 of Fig. 7 identified action) is an emotion that is consistent with the context of the virtual environment (Block 820).

[0085] If and / or when the user activity classification circuit 406 determines that the emotion is inconsistent with the context (Block 820: NO), the user activity classification circuit 406 identifies the action as unintentional (Block 810). If and / or when the user activity classification circuit 406 determines that the emotion is consistent with the context (Block 820: YES), the user activity classification circuit 406 identifies the action as intentional (Block 806).

[0086] Once the User Activity Classification Circuit 406 identifies the action as intentional (block 806) or as unintentional (block 810), it determines that the action and its classification should be logged (block 822). For example, logging the action and its classification can save processing resources by allowing the User Activity Classification Circuit 406 to identify the type of action (intentional or unintentional) and skip blocks 812 through 820 in this example process 710.

[0087] If and / or when the User Activity Classification Circuit 406 determines that the action and its corresponding classification should be logged (Block 822: YES), the User Activity Classification Circuit 406 logs the action (Block 824). After logging the action (Block 824), or if and / or when the User Activity Classification Circuit 406 determines that the action and its corresponding classification should not be logged (Block 822: NO), the exemplary process proceeds from Block 710 to Fig. 7 to block 712 if the action is intentional, or to block 722 if the action is unintentional.

[0088] Fig. 9 is a flowchart that shows exemplary machine-readable instructions and / or exemplary operations 716 (of Fig. 7) represents operations that can be executed and / or instantiated by a processor circuit arrangement to determine whether an avatar adjustment is required to change the position or orientation of an avatar. Exemplary operations 716 indicate that the offset orientation circuit arrangement 408 accesses world coordinates (block 902). The world coordinates correspond to the user's position or orientation relative to the content of interest (e.g., the display of the virtual environment). The offset orientation circuit arrangement 408 also accesses the camera image coordinates (block 904). The camera image coordinates correspond to the user's position or orientation relative to the camera used to map the user's actions onto the avatar.

[0089] The offset alignment circuit arrangement 408 determines the difference between the world coordinates and the camera image coordinates (block 906). The avatar model circuit arrangement 412 sets the difference as the fit for the avatar model (block 908). The example process 716 then moves to block 718. Fig. 7 continued.

[0090] Fig. Figure 10 is a block diagram of an exemplary processor platform 1000, structured to show the machine-readable instructions and / or operations of Fig. 7, Fig. 8 to Fig. 9 to execute and / or instantiate the avatar customization application 316 of Fig. 3 and Fig. 4. The Processor Platform 1000 can be, for example, a server, a personal computer, a workstation, a self-learning machine (e.g., a neural network), a mobile device (e.g., a mobile phone, a smartphone, a tablet such as an iPad™), a personal digital assistant (PDA), an internet device, a DVD player, a CD player, a digital video recorder, a Blu-ray player, a game console, a personal video recorder, a set-top box, a headset (e.g., an augmented reality (AR) headset, a virtual reality (VR) headset, etc.), or any other wearable device or any other type of computing device.

[0091] The processor platform 1000 of the illustrated example features a processor circuit arrangement 1012. The processor circuit arrangement 1012 of the illustrated example is hardware. For example, the processor circuit arrangement 1012 can be implemented by one or more integrated circuits, logic circuits, FPGAs, microprocessors, GPUs, DSPs, and / or microcontrollers from any desired family or manufacturer. The processor circuit arrangement 1012 can be implemented by one or more semiconductor-based (e.g., silicon-based) devices.In this example, the processor circuit arrangement 1012 implements the program identification circuit arrangement 314, the avatar customization application 316, the user alignment circuit arrangement 402, the object detection circuit arrangement 404, the activity classification circuit arrangement 406, the alignment offset circuit arrangement 408, the avatar mapping circuit arrangement 410, and the avatar model circuit arrangement 412.

[0092] The processor circuit arrangement 1012 of the illustrated example has a local memory 1013 (e.g., a cache, registers, etc.). The processor circuit arrangement 1012 of the illustrated example communicates with a main memory, including a volatile memory 1014 and a non-volatile memory 1016, via a bus 1018. The volatile memory 1014 can be accessed by Synchronous Dynamic Random Access Memory (SDRAM), Dynamic Random Access Memory (DRAM), or RAMBUS. ®Dynamic Random Access Memory (RDRAM ® ) and / or any other type of RAM device. The non-volatile memory 1016 can be implemented by flash memory and / or any other desired type of storage device. Access to the main memory 1014, 1016 of the illustrated example is controlled by a memory controller 1017.

[0093] The processor platform 1000 of the illustrated example also features an interface circuit arrangement 1020. The interface circuit arrangement 1020 can be implemented by hardware according to any type of interface standard, such as an Ethernet interface, a Universal Serial Bus interface (USB interface), or Bluetooth. ®-interface, a near field communication interface (NFC interface), a peripheral component interconnect interface (PCI interface) and / or a peripheral component interconnect express interface (PCle interface) can be implemented.

[0094] In the illustrated example, one or more input devices 1022 are connected to the interface circuit arrangement 1020. The input device(s) 1022 enable a user to input data and / or commands into the processor circuit arrangement 1012. The input device(s) 1022 can be implemented by, for example, an audio sensor, a microphone, a camera (photo or video), a keyboard, a button, a mouse, a touchscreen, a trackpad, a trackball, an isopoint device, and / or a speech recognition system.

[0095] One or more output devices 1024 are also connected to the interface circuit arrangement 1020 of the illustrated example. The output device(s) 1024 can be implemented, for example, by display devices (e.g., a light-emitting diode (LED), an organic light-emitting diode (OLED), a liquid crystal display (LCD), a cathode ray tube display (CRT), an in-place switching display (IPS), a touchscreen, etc.), a key output device, a printer, and / or a loudspeaker. The interface circuit arrangement 1020 of the illustrated example thus typically includes a graphics driver card, a graphics driver chip, and / or a graphics processing unit (GPU) circuit arrangement.

[0096] The interface circuit arrangement 1020 of the illustrated example also includes a communication device, such as a transmitter, receiver, transceiver, modem, residential gateway, wireless access point, and / or network interface, to support the exchange of data with external machines (e.g., computing devices of any kind) through a network 1026. Communication can be effected through, for example, an Ethernet connection, a digital subscriber line (DSL) connection, a telephone line connection, a coaxial cable system, a satellite system, a wireless line-of-sight system, a mobile phone system, an optical link, etc.

[0097] The processor platform 1000 of the illustrated example also includes one or more mass storage devices 1028 for storing software and / or data. Examples of such mass storage devices 1028 include magnetic storage devices, optical storage devices, floppy disk drives, HDDs, CDs, Blu-ray disc drives, redundant array of independent disk (RAID) systems, solid-state storage devices such as flash memory devices and / or SSDs, and DVD drives.

[0098] The machine-executable instructions 1032, which are replaced by the machine-readable instructions of Fig. 7, Fig. 8 to Fig. 9 can be implemented, can be stored in the mass storage device 1028, in the volatile memory 1014, in the non-volatile memory 1016 and / or on a removable, non-transient, computer-readable storage medium, such as a CD or DVD.

[0099] Fig. Figure 11 is a block diagram of an exemplary implementation of the processor circuit arrangement 1012 by Fig. 10. In this example, the processor circuit arrangement 1012 is used. Fig. 10 is implemented by a microprocessor 1100. The microprocessor 1100 can, for example, be a general-purpose microprocessor (e.g., a general-purpose microprocessor circuit arrangement). The microprocessor 1100 executes some or all of the machine-readable instructions of the flowcharts of Fig. 7, Fig. 8 to Fig. 9 out to determine the circuit arrangement of Fig. 3 and Fig. 4. to effectively instantiate logic circuits in order to perform the operations corresponding to these machine-readable instructions. In some such examples, the circuit arrangement of Fig. 3 and / or 4 are instantiated by the hardware circuitry of the microprocessor 1100 in combination with the instructions. The microprocessor 1100 can be implemented, for example, by a multi-core hardware circuitry such as a CPU, DSP, GPU, XPU, etc. Although it can have any number of exemplary cores 1102 (e.g., 1 core), the microprocessor 1100 in this example is a multi-core semiconductor device with N cores. The cores 1102 of the microprocessor 1100 can operate independently or work together to execute machine-readable instructions. For example, machine code corresponding to a firmware program, an embedded software program, or a software program can be executed by one of the cores 1102, or it can be executed by several of the cores 1102 at the same time or at different times.In some examples, the machine code corresponding to the firmware program, embedded software program, or software program is divided into threads and executed in parallel by two or more of the 1102 cores. The software program may correspond to a section or all of the machine-readable instructions and / or operations defined by the flowcharts of [missing information]. Fig. 7, Fig. 8 to Fig. 9 are represented.

[0100] The 1102 cores can communicate via a first exemplary bus 1104. In some examples, the first bus 1104 can be implemented as a communication bus to enable communication associated with one or more of the 1102 cores. The first bus 1104 can be implemented, for example, by at least one Inter-Integrated Circuit bus (I2C bus), a Serial Peripheral Interface bus (SPI bus), a PCI bus, or a PCIe bus. Additionally or alternatively, the first bus 1104 can be implemented by any other type of computational or electrical bus. The 1102 cores can receive data, instructions, and / or signals from one or more external devices via the exemplary interface circuit arrangement 1106. The 1102 cores can output data, instructions, and / or signals to the one or more external devices via the interface circuit arrangement 1106.Although the cores 1102 of this example have an exemplary local memory 1120 (e.g., a Level 1 cache (L1 cache), which may be partitioned into an L1 data cache and an L1 instruction cache), the microprocessor 1100 also has an exemplary shared memory 1110 that can be shared by the cores (e.g., a Level 2 cache (L2 cache)) for high-speed access to data and / or instructions. The data and / or instructions can be transferred (e.g., shared) by writing to and / or reading from the shared memory 1110. The local memory 1120 of each of the cores 1102 and the shared memory 1110 can be part of a hierarchy of storage devices, including multiple levels of cache memory and main memory (e.g., main memory 1014, 1016 of ). Fig. 10). Higher storage levels in the hierarchy typically have lower access times and smaller storage capacities than lower storage levels. Changes at the various levels of the cache hierarchy are managed (e.g., coordinated) by a cache coherence policy.

[0101] Each Core 1102 can be referred to as a CPU, a DSP, a GPU, etc., or any other type of hardware circuit arrangement. Each Core 1102 has a Control Unit Circuit Arrangement 1114, an Arithmetic and Logic Circuit Arrangement (AL Circuit Arrangement) 1116 (sometimes called an ALU), a plurality of Registers 1118, Local Memory 1120, and a second exemplary bus 1122. Other structures may be present. For example, each Core 1102 may have a Vector Unit Circuit Arrangement, a Single Instruction Multiple Data Unit Circuit Arrangement (SIMD Unit Circuit Arrangement), a Load / Store Unit Circuit Arrangement (LSU Circuit Arrangement), a Branch / Jump Unit Circuit Arrangement, a Floating Point Unit Circuit Arrangement (FPU Circuit Arrangement), etc.The control unit circuit arrangement 1114 features semiconductor-based circuits structured to control (e.g., coordinate) data movement within the corresponding core 1102. The AL circuit arrangement 1116 features semiconductor-based circuits structured to perform one or more mathematical and / or logical operations on the data within the corresponding core 1102. In some examples, the AL circuit arrangement 1116 performs integer-based operations. In other examples, the AL circuit arrangement 1116 also performs floating-point operations. In still other examples, the AL circuit arrangement 1116 may include a first AL circuit arrangement that performs integer-based operations and a second AL circuit arrangement that performs floating-point operations. In some examples, the AL circuit arrangement 1116 may be referred to as an arithmetic logic unit (ALU).The 1118 registers are semiconductor-based structures for storing data and / or instructions, such as the results of one or more of the operations performed by the AL circuit arrangement 1116 of the corresponding 1102 core. For example, the 1118 registers can include a vector register, a SIMD register, a general-purpose register, a flag register, a segment register, a machine-specific register, an instruction pointer register, a control register, a debug register, a memory management register, a machine check register, etc. The 1118 registers can be arranged in a bank, as shown in... Fig. Figure 11 shows the following. Alternatively, the registers 1118 can be organized in any other arrangement, format, or structure, including distributed across the entire core 1102 to reduce access time. The second bus 1122 can be implemented by at least one I2C bus, SPI bus, PCI bus, or PCIe bus.

[0102] Each Core 1102 and / or, more generally, the Microprocessor 1100 may have additional and / or different structures compared to those shown and described above. For example, it may include one or more clock circuits, one or more power supplies, one or more power gates, one or more cache home agents (CHAs), one or more converged / common mesh stops (CMSs), one or more shifters (e.g., barrel shifters), and / or other circuit arrangements. The Microprocessor 1100 is a semiconductor device manufactured to include many interconnected transistors to implement the structures described above in one or more integrated circuits (ICs) contained in one or more packages. The processor circuit arrangement may include and / or interact with one or more accelerators.In some examples, accelerators are implemented through a logic circuit arrangement to perform certain tasks faster and / or more efficiently than is possible with a general-purpose processor. Examples of accelerators include ASICs and FPGAs, such as those discussed here. A GPU or other programmable device can also be an accelerator. Accelerators can be contained within the processor circuit arrangement, in the same chip package as the processor circuit arrangement, and / or in one or more packages separate from the processor circuit arrangement.

[0103] Fig. Figure 12 is a block diagram of another exemplary implementation of the processor circuit arrangement 1012 by Fig. 10. In this example, the processor circuit arrangement 1012 is implemented by the FPGA circuit arrangement 1200. For example, the FPGA circuit arrangement 1200 can be implemented by an FPGA. The FPGA circuit arrangement 1200 can, for example, be used to perform operations that would otherwise be impossible to perform by the exemplary microprocessor 1100. Fig. 11 could be performed, which executes the corresponding machine-readable instructions. However, once configured, the FPGA 1200 circuit assembly instantiates the machine-readable instructions in hardware and can therefore often perform the operations faster than they could be performed by a general-purpose microprocessor executing the corresponding software.

[0104] More precisely, in contrast to the microprocessor 1100 described above from Fig. 11 (which is a universal device that can be programmed to execute some or all of the machine-readable instructions provided by the flowcharts of Fig. 7, Fig. 8 to Fig. 9, whose intermediate connections and logic circuit arrangement are fixed after manufacture), contains the FPGA circuit arrangement 1200 of the example of Fig. 12 connections and a logic circuit arrangement that can be configured and / or connected in different ways after manufacture, for example to instantiate some or all of the machine-readable instructions defined by the flowcharts of Fig. 7, Fig. 8 to Fig. 9. In particular, the FPGA Circuit Arrangement 1200 can be viewed as an array of logic gates, interconnects, and switches. The switches can be programmed to change how the logic gates are interconnected by the interconnects, effectively forming one or more dedicated logic circuits (unless and until the FPGA Circuit Arrangement 1200 is reprogrammed). The configured logic circuits allow the logic gates to interact in different ways to perform various operations on data received by an input circuit arrangement. These operations can be represented by the entire flowcharts of Fig. 7, Fig. 8 to Fig. 9 represented software or a part thereof. The FPGA circuit arrangement 1200 can thus be structured to represent some or all of the machine-readable instructions of the flowcharts of Fig. 7, Fig. 8 to Fig. 9 effectively instantiate as dedicated logic circuits to perform the operations corresponding to these software instructions in a dedicated manner analogous to an ASIC. Therefore, the FPGA circuit arrangement 1200 can perform the operations that some or all of the machine-readable instructions of Fig. 7, Fig. 8 to Fig. 9, perform faster than the universal microprocessor can execute them.

[0105] In the example of Fig. Section 6 describes the FPGA 1200 circuit arrangement as structured to be programmed (and / or reprogrammed once or multiple times) by an end user using a hardware description language (HDL), such as Verilog. The FPGA 1200 circuit arrangement of Fig. Reference 12 includes an exemplary input / output (I / O) circuit arrangement 1202 for receiving and / or outputting data to / from an exemplary configuration circuit arrangement 1204 and / or external hardware 1206. The configuration circuit arrangement 1204 can, for example, be implemented by an interface circuit arrangement capable of receiving machine-readable instructions to configure the FPGA circuit arrangement 1200 or a section thereof. In some such examples, the configuration circuit arrangement 1204 can receive the machine-readable instructions from a user, a machine (e.g., a hardware circuit arrangement, such as a programmed or dedicated circuit arrangement, that can implement an artificial intelligence / machine learning (AI / ML) model to generate the instructions), etc.In some examples, the external hardware 1206 can be implemented by an external hardware circuit arrangement. For example, the external hardware 1206 can be implemented by the microprocessor 1100. Fig. 11 are implemented. The FPGA circuit arrangement 1200 also includes an array of exemplary logic gate circuit arrangements 1208, a plurality of exemplary configurable interconnects 1210, and an exemplary storage circuit arrangement 1212. The logic gate circuit arrangement 1208 and the configurable interconnects 1210 are configurable to instantiate one or more operations that implement at least some of the machine-readable instructions of Fig. 7, Fig. 8 to Fig. 9 and / or other desired operations may correspond to the following. Fig. The logic gate circuit assembly 1208 shown is manufactured in groups or blocks. Each block contains semiconductor-based electrical structures that can be configured into logic circuits. In some examples, the electrical structures include logic gates (e.g., AND gates, OR gates, NOR gates, etc.) that provide basic building blocks for logic circuits. Electrically controllable switches (e.g., transistors) are present within each of the logic gate circuit assemblies 1208 to allow configuration of the electrical structures and / or logic gates to form circuits for performing desired operations. The logic gate circuit assembly 1208 may also include other electrical structures, such as lookup tables (LUTs), registers (e.g., flip-flop memories or buffers), multiplexers, etc.

[0106] The configurable intermediate connections 1210 of the illustrated example are conductive paths, tracks, vias or the like, which may have electrically controllable switches (e.g. transistors) whose state can be changed by programming (e.g. using an HDL instruction language) to enable or disable one or more connections between one or more of the logic gate circuit arrangements 1208 in order to program desired logic circuits.

[0107] The storage circuit arrangement 1212 of the illustrated example is structured to store the result(s) of one or more of the operations performed by the corresponding logic gates. The storage circuit arrangement 1212 can be implemented using registers or the like. In the illustrated example, the storage circuit arrangement 1212 is distributed across the logic gate arrangement 1208 to facilitate access and increase execution speed.

[0108] The exemplary FPGA circuit arrangement 1200 from Fig. Figure 12 also includes an exemplary dedicated operational circuit arrangement 1214. In this example, the dedicated operational circuit arrangement 1214 includes a special-purpose circuit arrangement 1216 that can be called upon to implement frequently used functions, thus avoiding the need to program these functions locally. Examples of such a special-purpose circuit arrangement 1216 include a memory control circuit arrangement (e.g., a DRAM circuit arrangement), a PCIe control circuit arrangement, a clock circuit arrangement, a transceiver circuit arrangement, a memory, and a multiplier-accumulator circuit arrangement. Other types of special-purpose circuit arrangements may be present. In some examples, the FPGA circuit arrangement 1200 may also include an exemplary programmable general-purpose circuit arrangement 1218, such as an exemplary CPU 1220 and / or an exemplary DSP 1222.Another programmable universal circuit arrangement 1218 may be additionally or alternatively present, such as a GPU, an XPU, etc., which may be programmed to perform other operations.

[0109] Although Fig. 11 and Fig. 12 Two exemplary implementations of the processor circuit arrangement 1012 from Fig. As illustrated in Figure 10, many other approaches are considered. A modern FPGA circuit arrangement can, for example, as mentioned above, be an integrated CPU, such as one or more of the exemplary CPU 1220 from Fig. 12. Therefore, the processor circuit arrangement 1012 can be of Fig. 10 additionally by combining the exemplary microprocessor 1100 from Fig. 11 and the exemplary FPGA circuit arrangement 1200 of Fig. 6 can be implemented. In some such hybrid examples, a first section of machine-readable instructions, which is represented by the flowcharts of Fig. 7, Fig. 8 to Fig. 9 can be represented by one or more of the cores 1102 of Fig. 11 are executed, a second section of the machine-readable instructions, which are represented by the flowcharts of Fig. 7, Fig. 8 to Fig. 9 can be represented by the FPGA circuit arrangement 1200 of Fig. 12 are executed and / or a third section of the machine-readable instructions, which are determined by the flowcharts of Fig. 7, Fig. 8 to Fig. The circuit shown in Figure 9 can be implemented using an ASIC. It is understood that some or all of the circuit arrangements of Fig. 3 and Fig. 4. Thus, they can be instantiated at the same time or at different times. For example, part or all of the circuit arrangement can be instantiated in one or more threads that are executed simultaneously and / or sequentially. Furthermore, in some examples, part or all of the circuit arrangement can be instantiated by Fig. 3 and Fig. 4. Implemented within one or more virtual machine(s) and / or one or more containers running on the microprocessor.

[0110] In some examples, the processor circuit arrangement 1012 may differ from Fig. 10 are located in one or more housings. For example, the 1100 microprocessor from Fig. 11 and / or the FPGA circuit arrangement 1200 of Fig. 12 in one or more packages. In some examples, an XPU can be represented by the 1012 processor circuit arrangement of Fig. 10 can be implemented, which can be located in one or more packages. For example, the XPU can have a CPU in one package, a DSP in another package, a GPU in yet another package, and an FPGA in yet another package.

[0111] A block diagram illustrating an exemplary software distribution platform 1305 for distributing software, such as the exemplary machine-readable instructions 1032 of Fig. 10, illustrated by hardware devices owned and / or operated by third parties, is in Fig. Figure 13 illustrates this. The exemplary software distribution platform 1305 can be implemented by any computer server, data facility, cloud service, etc., capable of storing software and transferring it to other computing devices. The third parties can be customers of the entity that owns and / or operates the software distribution platform 1305. The entity that owns and / or operates the software distribution platform 1305 can be, for example, a developer, vendor, and / or licensor of software such as the exemplary machine-readable instructions 1032 of Fig. 10. Third parties may be consumers, users, retailers, OEMs, etc., who purchase and / or license the software for use and / or resale and / or sublicensing. In the illustrated example, the software distribution platform 1305 includes one or more servers and one or more storage device(s). The storage devices store the machine-readable instructions 1032, which correspond to the exemplary machine-readable instructions 700 of Fig. 7, Fig. 8 to Fig. 9 can correspond to, as described above. The one or more servers of the exemplary software distribution platform 1305 communicate with an exemplary network 1310, which can correspond to any one or more of the Internet and / or any of the exemplary networks described above. In some examples, the one or more servers respond to requests to transfer the software to a requesting party as part of a commercial transaction. Payment for the delivery, sale, and / or licensing of the software can be processed through the one or more servers of the software distribution platform and / or through a third-party payment agent. The servers enable purchasers and / or licensors to download the machine-readable instructions 1032 from the software distribution platform 1305. For example, the software, which corresponds to the exemplary machine-readable instructions 700 of . Fig. 7, Fig. 8 to Fig. 9 can be downloaded to the exemplary processor platform 1000, which is to execute the machine-readable instructions 1032 to implement the avatar customization application 316. In some examples, one or more servers of the software distribution platform 1305 periodically provide updates for the software (e.g., the exemplary machine-readable instructions 1032 of Fig. 10) to, transfer and / or enforce such measures to ensure that improvements, patches, updates, etc. are distributed and applied to the software on end-user devices.

[0112] From the foregoing, it is understood that exemplary systems, methods, devices, and articles have been disclosed that prevent the translation or reproduction of unintended actions of a user in a real-world environment onto an avatar in a virtual environment and also correct a misaligned avatar. Examples disclosed here enable a clearer and less confusing virtual environment. Thus, disclosed systems, methods, devices, and articles improve the efficiency of using a computing device by eliminating unnecessary input for avatar reproduction, thereby saving processing resources. Accordingly, disclosed systems, methods, devices, and articles are directed toward one or more improvements in the operation of a machine, such as a computer or other electronic and / or mechanical device.

[0113] The examples revealed here also enable better avatar representation for users accessing a virtual environment (e.g., metaverse applications) from a PC, phone, and / or other non-HMD device. These examples also correct avatar alignment and allow user-to-user eye contact (avatar to avatar).

[0114] Exemplary methods, devices, systems, and manufacturing articles for user-to-avatar mapping and customization are disclosed here. Example 1 includes a device for mapping user-to-avatar actions.The device includes at least one memory; machine-readable instructions and a processor circuit arrangement for instantiating or executing at least one of the machine-readable instructions for: determining whether the user action is intentional or unintentional; preventing a mapping of a user action to an avatar model based on a determination that the user action is unintentional; determining whether there is a dissonance between a first orientation of the user and a second orientation of the avatar, based on a determination that the user action is intentional; determining an avatar adaptation value based on (1) the dissonance or (2) based on a determination that the user action is an unintentional position of the user; and applying the avatar adaptation value to the avatar model.

[0115] Example 2 features the setup of Example 1, wherein the processor circuit arrangement is intended to determine whether the user action is intentional or unintentional by: evaluating the context of audio data; determining whether an audio input from a user device is consistent with the context; and categorizing the user action as intentional if the audio input is consistent with the context, and as unintentional if the audio input is inconsistent with the context.

[0116] Example 3 features the setup of Example 1 and / or 2, wherein the avatar is presented in a virtual environment and the processor circuitry is intended to determine whether the user action is intentional or unintentional by: evaluating the user's gaze direction; determining whether the gaze direction is towards a device coupled to the virtual environment; and categorizing the user action as intentional if the gaze is directed towards a device coupled to the virtual environment, and as unintentional if the gaze is directed towards a device not coupled to the virtual environment.

[0117] Example 4 features the setup of one of Examples 1 to 3, where the avatar is presented in a virtual environment and the processor circuitry is intended to determine whether the user action is intentional or unintentional by: evaluating the context of the virtual environment; determining whether the user action is consistent with the context; and categorizing the user action as intentional if the action is consistent with the context, and as unintentional if the action is inconsistent with the context.

[0118] Example 5 features the setup of one of Examples 1 to 4, where the avatar is presented in a virtual environment and the processor circuitry is to determine whether the user action is intentional or unintentional by: evaluating the context of the virtual environment; evaluating audio data; determining a mood of the virtual environment based on the context and audio data; evaluating a user emotion; determining whether the emotion is consistent with the mood; and categorizing the user action as intentional if the emotion is consistent with the mood, and as unintentional if the emotion is inconsistent with the mood.

[0119] Example 6 features the setup of one of Examples 1 to 5, where the processor circuit arrangement is intended to determine the dissonance by: identifying a first set of coordinates based on the first alignment; identifying a second set of coordinates based on the first alignment; and determining a difference between the first set of coordinates and the second set of coordinates.

[0120] Example 7 features the setup of Example 6, where the processor circuit arrangement is intended to determine the avatar adaptation value based on the difference.

[0121] Example 8 features the setup of Example 7, where the processor circuit arrangement is intended to define the second alignment to a third set of coordinates based on the avatar adaptation value.

[0122] Example 9 features the setup of one of Examples 1 to 8, where the processor circuit arrangement is intended to map the avatar model with the avatar adaptation value to a Human Interface Device stream.

[0123] Example 10 features the setup of one of Examples 1 to 9, where the processor circuit arrangement is intended to adapt audio data based on the avatar model with the avatar adaptation value.

[0124] Example 11 includes a non-transitory machine-readable storage medium that contains instructions to cause one or more processors to at least: categorize a user action as unintentional; prevent the mapping of the user action to an avatar model; identify a dissonance between an initial orientation of a user's head and a second orientation of an avatar's head; determine an avatar adjustment value based on the dissonance and apply the avatar adjustment value to the avatar model.

[0125] Example 12 features the storage medium of Example 11, wherein the avatar is presented in a virtual environment and the instructions cause one or more processors to categorize the user action as unintentional by: evaluating one or more of the context of audio data, the user's gaze direction, or the context of the virtual environment; determining one or more of (1) whether an audio input from a user device is consistent with the context of the audio data, (2) determining whether the gaze direction is toward a device coupled to a virtual environment, (3) determining whether the user action is consistent with the context of the virtual environment, or (4) determining a mood of the virtual environment based on the context of the virtual environment and the audio data, evaluating a user emotion, and determining whether the emotion is consistent with the mood.and categorizing the user action as unintentional if one or more of the following are present: (1) the audio input is inconsistent with the context of the audio data, (2) the gaze is directed at a device that is not coupled with the virtual environment, (3) the action is inconsistent with the context of the virtual environment, or (4) the emotion is inconsistent with the mood.

[0126] Example 13 features the storage medium of Example 11 and / or 12, wherein the avatar is presented in a virtual environment and the instructions cause one or more processors to categorize the user action as unintentional by: evaluating a context of audio data, a user's gaze direction, and a virtual environment context; determining (1) whether an audio input from a user device is consistent with the audio data context, (2) whether the gaze direction is toward a device coupled to the virtual environment, and (3) whether the user action is consistent with the virtual environment context;and categorizing the user action as unintentional if one or more of the following are present: (1) the audio input is inconsistent with the context of the audio data, (2) the gaze is directed at a device that is not coupled with the virtual environment, or (3) the action is inconsistent with the context of the virtual environment.

[0127] Example 14 features the storage medium of one of Examples 11 to 13, wherein the instructions cause the one or more processor(s) to identify the dissonance by: identifying a first set of coordinates based on the first alignment; identifying a second set of coordinates based on the first alignment; and determining a difference between the first set of coordinates and the second set of coordinates.

[0128] Example 15 features the storage medium of one of Examples 11 to 14, wherein the instructions cause one or more processor(s) to determine the avatar adaptation value based on the difference.

[0129] Example 16 features the storage medium of one of Examples 11 to 15, wherein the instructions cause the one or more processor(s) to set the second alignment to a third set of coordinates based on the avatar adaptation value.

[0130] Example 17 includes a device for mapping user-to-avatar actions, the device comprising: at least one memory; machine-readable instructions; and a processor circuit arrangement for instantiating or executing the machine-readable instructions to: determine a dissonance between a first orientation of a user's body in a real-world environment and a second orientation of an avatar's body in a virtual environment, where the avatar corresponds to the user; determine an avatar adaptation value based on the dissonance; and apply the avatar adaptation value to an avatar model to modify the second orientation.

[0131] Example 18 features the setup of Example 17, wherein the processor circuit arrangement is intended to determine the dissonance by: identifying a first set of coordinates based on the first alignment; identifying a second set of coordinates based on the first alignment; and determining a difference between the first set of coordinates and the second set of coordinates.

[0132] Example 19 features the setup of Example 18, wherein the processor circuit arrangement is designed to: determine the avatar adjustment value based on the difference and set the second alignment to a third set of coordinates based on the avatar adjustment value.

[0133] Example 20 features the setup of one of Examples 17 to 19, where the processor circuit arrangement is intended to adapt audio data in the virtual environment based on the avatar model with the avatar adaptation value.

[0134] The following claims are hereby incorporated into this detailed description by reference. Although certain exemplary systems, processes, devices, and articles are disclosed herein, the scope of protection of this patent is not limited thereto. Rather, this patent covers all systems, processes, devices, and articles that ordinarily fall within the scope of protection of the claims of this patent.

Claims

[1] A setup for mapping user-to-avatar actions, wherein the setup includes: at least one storage device; machine-readable instructions and a processor circuit arrangement for at least one of instantiating or executing the machine-readable instructions for: Determine whether a user action is intentional or unintentional; Preventing the mapping of user action onto an avatar model based on a determination that the user action is unintentional; Determine whether there is a dissonance between a user's initial orientation and an avatar's secondary orientation, based on a determination that the user action is intended; Determining an avatar adaptation value based on (1) the dissonance or (2) based on a determination that the user action is an unintended position of the user; and Applying the avatar customization value to the avatar model. [2] Device according to claim 1, wherein the processor circuit arrangement is to determine whether the user action is intentional or unintentional, by: Evaluating the context of audio data; Determine whether audio input from a user device is consistent with the context; and Categorize the user action as intended if the audio input is consistent with the context, and as unintentional if the audio input is inconsistent with the context. [3] Device according to claim 1, wherein the avatar is presented in a virtual environment and the processor circuit arrangement is to determine whether the user action is intentional or unintentional, by: Evaluating the user's gaze direction; Determine whether the viewing direction is towards a device that is coupled to the virtual environment; and Categorizing the user action as intentional when the view is directed towards a device that is coupled with the virtual environment, and as unintentional when the view is directed towards a device that is not coupled with the virtual environment. [4] Device according to claim 1, wherein the avatar is presented in a virtual environment and the processor circuit arrangement is to determine whether the user action is intentional or unintentional, by: Evaluating the context of the virtual environment; Determine whether the user action is consistent with the context; and Categorizing the user action as intended if the action is consistent with the context, and as unintentional if the action is inconsistent with the context. [5] Device according to claim 1, wherein the avatar is presented in a virtual environment and the processor circuit arrangement is to determine whether the user action is intentional or unintentional, by: Evaluating the context of the virtual environment; Evaluating audio data; Determining the mood of the virtual environment based on the context and audio data; Evaluating a user's emotion; Determine whether the emotion is consistent with the mood; and Categorizing the user action as intended if the emotion is consistent with the mood, and as unintentional if the emotion is inconsistent with the mood. [6] Device according to any one of claims 1 to 5, wherein the processor circuit arrangement is to determine the dissonance by: Identifying an initial set of coordinates based on the initial orientation; Identifying a second set of coordinates based on the first alignment and Determining a difference between the first set of coordinates and the second set of coordinates. [7] Device according to claim 6, wherein the processor circuit arrangement is to determine the avatar adaptation value based on the difference. [8] Device according to claim 7, wherein the processor circuit arrangement is to determine the second alignment to a third set of coordinates based on the avatar adaptation value. [9] Device according to claim 1, wherein the processor circuit arrangement is to map the avatar model with the avatar adaptation value onto a Human Interface Device stream. [10] Device according to claim 1, wherein the processor circuit arrangement is to adapt audio data based on the avatar model with the avatar adaptation value. [11] Machine-readable storage medium containing instructions to cause one or more processors to perform at least the following actions: Categorizing a user action as unintentional; Preventing user action from being mapped to an avatar model; Identifying a dissonance between a first orientation of the user's head and a second orientation of the avatar's head; Determining an avatar adaptation value based on dissonance and Applying the avatar customization value to the avatar model. [12] Storage medium according to claim 11, wherein the avatar is presented in a virtual environment and the instructions cause one or more processor(s) to categorize the user action as unintentional, by: Evaluating one or more of the context of audio data, the user's gaze direction, or the context of the virtual environment; Determine one or more of: (1) whether an audio input from a user device is consistent with the context of the audio data; (2) determine whether the gaze direction is towards a device coupled with a virtual environment; (3) determine whether the user action is consistent with the context of the virtual environment; or (4) determine a mood of the virtual environment based on the context of the virtual environment and the audio data, evaluate a user emotion, and determine whether the emotion is consistent with the mood. Categorizing the user action as unintentional if one or more of the following are present: (1) the audio input is inconsistent with the context of the audio data, (2) the gaze is directed towards a device that is not coupled with the virtual environment, (3) the action is inconsistent with the context of the virtual environment, or (4) the emotion is inconsistent with the mood. [13] Storage medium according to claim 11, wherein the avatar is presented in a virtual environment and the instructions cause one or more processor(s) to categorize the user action as unintentional, by: Evaluating a context of audio data, the user's gaze direction, and the context of the virtual environment; Determine (1) whether an audio input from a user device is consistent with the context of the audio data, (2) whether the gaze direction is towards a device that is coupled to the virtual environment, and (3) whether the user action is consistent with the context of the virtual environment; and Categorize the user action as unintentional if one or more of the following are present: (1) the audio input is inconsistent with the context of the audio data, (2) the gaze is directed at a device that is not coupled with the virtual environment, or (3) the action is inconsistent with the context of the virtual environment. [14] Storage medium according to any one of claims 11 to 13, wherein the instructions cause the one or more processor(s) to identify the dissonance by: Identifying an initial set of coordinates based on the initial orientation; Identifying a second set of coordinates based on the first alignment and Determining a difference between the first set of coordinates and the second set of coordinates. [15] Storage medium according to claim 14, wherein the instructions cause the one or more processor(s) to determine the avatar adaptation value based on the difference. [16] Storage medium according to claim 15, wherein the instructions cause the one or more processor(s) to set the second alignment to a third set of coordinates based on the avatar adaptation value. [17] Device for mapping user-to-avatar actions, wherein the device includes: at least one storage device; machine-readable instructions and a processor circuit arrangement for at least one of instantiating or executing the machine-readable instructions for: Determining a dissonance between a first orientation of a user's body in a real environment and a second orientation of an avatar's body in a virtual environment, where the avatar corresponds to the user; Determining an avatar adaptation value based on dissonance and Applying the avatar adjustment value to an avatar model to change the second alignment. [18] Device according to claim 17, wherein the processor circuit arrangement is to determine the dissonance by: Identifying an initial set of coordinates based on the initial orientation; Identifying a second set of coordinates based on the first alignment and Determining a difference between the first set of coordinates and the second set of coordinates. [19] Device according to claim 18, wherein the processor circuit arrangement is provided for: Determining the avatar customization value based on the difference and Setting the second alignment to a third set of coordinates based on the avatar customization value. [20] Device according to one of claims 17 to 19, wherein the processor circuit arrangement is to adapt audio data in the virtual environment based on the avatar model with the avatar adaptation value. [21] Device for mapping user-to-avatar actions, wherein the device includes: a means of storing instructions and a means of processing the instructions for: Determining a dissonance between a first orientation of a user's body in a real environment and a second orientation of an avatar's body in a virtual environment, where the avatar corresponds to the user; Determining an avatar adaptation value based on dissonance and Applying the avatar adjustment value to an avatar model to change the second alignment. [22] Device according to claim 21, wherein the means for processing is to determine the dissonance by: Identifying an initial set of coordinates based on the initial orientation; Identifying a second set of coordinates based on the first alignment and Determining a difference between the first set of coordinates and the second set of coordinates. [23] Device according to claim 22, wherein the means for processing is provided for: Determining the avatar customization value based on the difference and Setting the second alignment to a third set of coordinates based on the avatar customization value. [24] Device according to one of claims 21 to 23, wherein the means for processing audio data in the virtual environment is to adapt the avatar model with the avatar adaptation value.

Images