Volume control for audio and video conferencing applications

Abstract

Volume control for audio and video conferencing applications is disclosed. In various examples, when a local user initiates an instance of a video conferencing application, a user interface (UI) may be provided to the user that displays an icon corresponding to the user and several other icons corresponding to participants in the instance of the video conferencing application. As the users converse, the local user may discover that a particular participant is speaking louder than other remote users. The local user may then select the icon corresponding to the particular participant and move the icon away from the local user's icon in the UI. Based on moving the remote user's icon further away from the local user's icon, the system may reduce the output volume of the participant's audio data. Further, if the local user moves the participant icon closer to the local user's icon, the participant's volume may be increased.

Description

Background Art

[0001] Users of multi-party applications (such as online video conferencing, game streaming services, video streaming services, etc.) often find it difficult to focus their attention on one speaker or participant when multiple participants speak at the same time or when different participants speak at different volume levels. For example, two participants can speak at the same time, but one participant may speak very loudly compared to a quieter participant. In this case, it may be difficult for the user receiving the audio stream to hear and / or understand the quieter participant, especially if the two participants speak at the same time. Although the user can increase or decrease the output volume of all participants in the stream, the volume of each individual participant in the stream is determined by factors outside the user's control, such as how loud the participant speaks and / or where the other participants place their microphones relative to themselves. As a result, the receiving user must try to interpret crosstalk, ask the quieter participant to speak louder or the louder participant to speak quieter, and / or try to control the order in which the participants speak so that there is less crosstalk. However, these options may be uncomfortable for the user, or may be unavailable, such as in the case of a participant who is unable to access the microphone function or does not have other communication devices for conveying their experience of the volume level with other participants. Summary of the invention

[0002] Embodiments of the present disclosure relate to location-based volume control for audio and video conferencing. Systems and methods are disclosed for applying updated audio settings to application participants based on manipulation of participant icons. For example, and in contrast to conventional systems such as those described above, when a local user moves (e.g., toward or away from an icon associated with the local user) or otherwise manipulates (e.g., contracts to decrease volume, expands to increase volume) an icon (e.g., a thumbnail, tile, video feed, picture, avatar, graphic image, or other representation) associated with a remote participant, the systems and methods of the present disclosure automatically modify the volume output of audio data corresponding to a remote participant (e.g., another user within an instance of an application).

[0003] As a non-limiting example, when a local user initiates an instance of a video conferencing application, a user interface (UI) may be provided to the user that displays an icon (or other representation or indicator) corresponding to the user and one or more other icons corresponding to the participants in the instance of the video conferencing application. However, as the users converse, the local user may discover that a particular participant speaks very loudly compared to other participants. The local user may then select an icon corresponding to a particular participant and move the icon further away from the local user's icon in the UI. Based on moving the icon of a particular participant away from the local user's icon, the system may identify a subset of audio data corresponding to the particular participant (e.g., from within an audio data stream) and reduce the output volume of the audio data of the participant. Further or alternatively, if the local user moves the icon of the participant closer to the icon of the local user, the volume of the audio data of the participant may be increased. Advantageously, the local user may adjust the output volume corresponding to each remote user by positioning (or otherwise manipulating) the icon of each remote user relative to the local user's icon, thereby enabling the local user to more clearly understand and track the conversation within the video conferencing application. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] The following describes in detail a system and method for location-based volume control for audio and video conferencing with reference to the accompanying drawings, wherein:

[0005] Figure 1 is an example audio manipulation system suitable for use in implementing some embodiments of the present disclosure;

[0006] Figure 2A-2C An example user interface (UI) for manipulating user icons suitable for implementing some embodiments of the present disclosure is shown;

[0007] Figure 3 shows an example UI for manipulating user icons suitable for implementing some embodiments of the present disclosure;

[0008] Figure 4 is a flow chart illustrating a method for updating audio settings based on manipulation of an icon suitable for implementing some embodiments of the present disclosure;

[0009] Figure 5 is a block diagram of an example content streaming system suitable for implementing some embodiments of the present disclosure;

[0010] Figure 6 is a block diagram of an example computing device suitable for implementing some embodiments of the present disclosure; and

[0011] Figure 7 is a block diagram of an example data center suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0012] Systems and methods are disclosed relating to location-based volume control for audio and video conferencing. While primarily described herein with respect to video conferencing implementations, this is not intended to be limiting, and the systems and methods of the present disclosure may be implemented in any implementation for updating audio settings based on manipulation of an icon. For non-limiting examples, the systems and methods described herein may be implemented for video streaming applications, game streaming applications, virtual learning applications, social media content sharing applications, video sharing applications, and / or other types of applications that may adjust the volume for one or more participants.

[0013] In some embodiments, when a local user initiates an instance of a video conferencing application, a user interface (UI) may be provided to the user that displays an icon (or other representation or indicator) corresponding to the user and several other icons (or other representations or indicators) corresponding to the participants in the instance of the video conferencing application. Initially, a volume level may be set for each remote user, such as a default level set by the application or based on a learned or user-configured volume level corresponding to a particular participant. Additionally or alternatively, the local user may manually select or pre-specify a default volume level for a particular participant (e.g., by selecting a user icon and setting a volume level for the particular user before initiating the video conference). However, as the users converse, the local user may discover that a particular participant speaks very loudly or very quietly compared to other participants, and when the same general volume level is applied to all participants, the inconsistency causes hearing difficulties or discomfort. The local user may then select an icon corresponding to a particular participant and modify the volume output of audio data corresponding to (e.g., specific to) the particular participant by moving or otherwise manipulating the icon (e.g., reducing the icon size to reduce the volume, expanding the icon size to increase the volume). For example, a local user may move the icon of a louder participant away from the icon of the local user in the UI. Based on moving the icon of the louder participant away from the icon of the local user, the system may determine an updated volume level (e.g., reduce the volume) for the icon of the louder participant. Subsequently, the system may identify a subset of the audio data corresponding to the louder participant (e.g., from one or more audio data streams), and reduce the output volume of the audio data of the louder participant. In addition, if the local user moves the icon of the participant closer to the icon of the local user, the volume of the audio data output for the participant may be automatically increased. In one or more embodiments, a local user may position or reposition the icon or other representation of one or more participants to a (pre) designated area associated with a particular effect in the user interface. For example, a local user may reposition one or more participant icons to the area designated to amplify the volume level of the audio data corresponding to the participant with a representation positioned in an area. Similarly, a local user may reposition one or more participant icons to the area designated to reduce (or mute) the volume level of the audio data corresponding to the participant with a representation positioned in an area. Advantageously, a local user can adjust the output volume corresponding to each participant (including several participants simultaneously) by positioning each participant's icon relative to the local user's icon, thereby improving the local user's user experience by allowing dynamic adjustment of audio settings (e.g., volume, treble, bass, noise cancellation, etc.) on a per-participant basis.

[0014] In other embodiments, the user can adjust the volume level of the audio data corresponding to the participant by manipulating the size of the participant icon corresponding to the participant. For example, in an application on a touch screen mobile device (e.g., a smart phone, a tablet computer, etc.) - where the screen size may not be suitable for distance-based icon manipulation - the user can perform a pinch zoom in / out gesture on the participant icon, thereby increasing or decreasing the size of the participant icon, depending on whether the gesture is a pinch zoom in gesture (e.g., selecting an icon with two fingers and spreading the fingers outward) or a pinch zoom out gesture (e.g., selecting an icon with two fingers and moving the fingers toward each other). Therefore, the volume level can be associated with the relative size of the participant icon. For example, as the user increases the size of the participant icon, the volume level of the audio data associated with the participant can increase, and as the user reduces the size of the participant icon, the volume level of the audio data can decrease, or vice versa. Similarly, the icon size can be used to increase / decrease the volume level in a desktop application. However, in the case where a touch screen is not available, the user can select a corner or side of the participant's icon and manipulate the corner or side to adjust the size or size of the participant's icon.

[0015] In some embodiments, the configuration of the participant icons on the user's first device can be dynamically ported to the user's second device. For example, a user can configure one or more participant icons while participating in an instance of an application session on the user's personal computer (PC) and initiate a new instance of the application session on the user's mobile device (e.g., using common login information for both instances of the application session). When initiating a new instance of the application session on the user's mobile device, the system can access the configuration and / or volume level of one or more participant icons from the instance of the application session on the user's PC and adjust the one or more participant icons on the mobile device to match the volume level of each corresponding participant. The volume level can be used to recreate the configuration at the new user interface of the application session. For example, the volume level corresponding to each participant can be used to adjust the size of the participant icon on the mobile device.

[0016] In some embodiments, the volume level can be adjusted across applications based on the distance / position in the application. For example, two or more player avatars can be linked to each other in a virtual environment (such as in a video game) (e.g., via an API, a plug-in, etc.). The distance / position of one or more participant avatars relative to the user avatar in the virtual environment can be used to adjust the volume level of the participant avatar, including the spatial volume level (e.g., left, right, front, back, top, bottom, etc.). The output volume setting can be calibrated to mimic the real world environment. For example, based on the distance between the position of the participant avatar and the position orientation of the user avatar, the system can adjust the volume level corresponding to the participant avatar output to the user (e.g., via the user's stereo headset, stereo speakers, surround sound speakers, speaker arrays, etc.). As the distance and / or position change during the application session, the system can adjust the volume level accordingly. By adjusting the volume level based on the relative position of the user avatar and one or more participant avatars, the user can more quickly and easily locate one or more participant avatars in the virtual environment.

[0017] In some embodiments, each user participating in an application session can form icons with different configurations on their local devices. For example, a first user can move the second user's icon very close to the first user's icon on the first user's device, while a second user can move the first user's icon away from the second user's icon on the second user's device. As such, the second user's volume level will increase on the first user's device, while the first user's volume level will decrease on the second user's device. In some embodiments, real-time feedback can be provided to each user on how their icons are configured (e.g., moved away or closer) on other users' devices. For example, a user whose icon has been moved away from another user's icon (so that the user's volume level has been reduced (e.g., below a threshold)) can be provided with an indicator (such as a list of users who cannot or may not be able to hear the user) to notify the user that one or more other users can no longer hear the user. This information can help inform users how they are perceived in an application session (e.g., speaking too loudly / quietly or participating in crosstalk). In some embodiments, users can choose (e.g., opt in / out) whether they want to provide information about user icon movement / configuration to users participating in an application session.

[0018] In some embodiments, such as in a situation where the first user has reduced the volume level of the second user below a threshold and the first user can no longer hear the second user, the second user may be able to reengage the first user in a variety of ways. For example, the second user may provide non-verbal feedback to the first user in the form of a raised hand, a message, a pop-up indicator, and / or other means of non-verbal communication. Further, in some embodiments, the system may employ a natural language processor (NLP) that may determine when a user's name is spoken. Following the above example, based on determining that the second user has spoken the first user's name, the system may increase the volume level of the second user so that the first user can hear the second user, thereby enabling the second user to reengage the first user.

[0019] In further embodiments, the system can collect and record data from any number of application sessions of an application processed by one or more machine learning models, one or more deep neural networks (DNNs), and / or one or more other algorithm types locally or at a remote server to determine typical user behavior. For example, decay data, user lists, time of day, meeting topics, and / or other contextual data can be used to generate typical user behavior predictions for a given context. These predictions can be used to perform automatic decay operations for users. For example, if a user typically manipulates or otherwise makes the user icon associated with a whispering user closer to the user's icon during weekly budget meetings, but manipulates or otherwise makes the whispering user's icon away from the user's icon during monthly virtual happy hours, the system can learn this behavior and automatically adjust the soft voice user's icon position in future budget meetings and virtual happy hours.

[0020] Reference Figure 1 , Figure 1 is an example audio manipulation system 100 (referred to herein alternatively as "system 100") according to some embodiments of the present disclosure. It should be understood that this and other arrangements described herein are set forth by way of example only. Other arrangements and elements (e.g., machines, interfaces, functions, sequences, functional groupings, etc.) may be used in addition to or in place of those shown, and some elements may be omitted entirely. Further, many of the elements described herein are functional entities that may be implemented as discrete or distributed components or in conjunction with other components, and in any suitable combination and location. The various functions described herein as being performed by entities may be performed by hardware, firmware, and / or software. For example, the various functions may be performed by a processor executing instructions stored in a memory. In some embodiments, system 100 may include a processor that is configured to perform the audio manipulation system 100. Figure 5 Example content streaming system 500, Figure 6 The example computing device 600 of Figure 7Components, features, and / or functions similar to those of the example data center 700.

[0021] The audio manipulation system 100 may include client devices 102A and 102B (collectively referred to herein as “client devices 102”) and an application server 104, among others. Figure 1 102A and 102B are shown in the figure, but this is not intended to be limiting. In any example, there may be any number (e.g., one or more) of client devices 102. The components of the audio control system 100 can communicate via a network 106. The network can include a wide area network (WAN) (e.g., the Internet, a public switched telephone network (PSTN), etc.), a local area network (LAN) (e.g., Wi-Fi, ZigBee, Z-Wave, Bluetooth, Bluetooth low energy (BLE), Ethernet, ultra-wideband (UWB), etc.), a low power wide area network (LPWAN) (e.g., LoRaWAN, Sigfox, etc.), a global navigation satellite system (GNSS) network (e.g., a global positioning system (GPS)), and / or another network type. In any example, each of the components of the audio control system 100 can communicate with one or more of the other components via one or more of the one or more networks 106.

[0022] Client device 102 may include a smartphone, a laptop, a tablet computer, a desktop computer, a wearable device, a game console, a virtual / augmented / mixed reality system (e.g., a headset, a computer, a game console, one or more remote controls, one or more controllers, and / or other components), a streaming device (e.g., NVIDIA SHIELD), a smart home device that may include an intelligent personal assistant, another type of device capable of supporting video conferencing or other application types (e.g., playing games, video chat, customer service chat, etc.), and / or a combination thereof.

[0023] The client device 102A / 102B may include a client application 108A / 108B, a display 110A / 110B, a communication interface 112A / 112B, one or more input devices 114A / 114B, and / or local storage 116A / 116B. Figure 1 Only some components and / or features of the client device 102 are shown, but this is not intended to be limiting. For example, the client device 102 may include additional or alternative components, such as those described below with respect to Figure 6 and / or Figure 7 The components described.

[0024] The client applications 108A and 108B (collectively referred to herein as "client applications 108") may include a mobile application, a computer application, a console application, and / or another type of application. The client applications 108 may include instructions that, when executed by one or more processors, cause the one or more processors to, without limitation, receive input data representing user input to one or more input devices 114, transmit the input data to one or more application servers 104, retrieve application data from a memory or local storage 116, receive application data from one or more application servers 104 using a communication interface 112, and cause the application to be displayed on a display 110. For example, the client application 108 may operate as a facilitator for enabling streaming of a video conferencing session associated with the client application on the client device 102. The client application 108 may also include instructions that, when executed by the one or more processors, cause the one or more processors to send data to and receive data from one or more application servers 104. For example, client application 108A may transmit real-time feedback about icon manipulations performed on client application 108A to one or more application servers 104, and receive real-time feedback about icon manipulations performed on client application 108B from one or more application servers 104. In other examples, client application 108A may transmit session data (e.g., decay data, user lists, time of day, meeting topics, and / or other contextual data) from any number of application sessions to one or more application servers 104. In such examples, host application 118 may generate (e.g., via a machine learning model, a deep neural network (DNN), or another algorithm type) a typical user behavior prediction for a given context.

[0025] Displays 110A and 110B (collectively referred to herein as "displays 110") may include any type of display capable of displaying an application (e.g., a light emitting diode display (LED), an organic LED display (OLED), a liquid crystal display (LCD), an active matrix OLED display (AMOLED), a quantum dot display (QDD), a plasma display, an LED / LCD display, and / or another type of display). In some examples, display 110 may include more than one display (e.g., a dual monitor display for a computer). In some examples, the display is a touch screen display, such as a touch screen of a smart phone, a tablet computer, a laptop computer, etc., where the touch screen is at least one of the one or more input devices 114 of client device 102.

[0026] One or more input devices 114A and 114B (collectively referred to herein as "one or more input devices 114") may include any type of device capable of providing user input to an application. The one or more input devices may include a keyboard, a mouse, one or more microphones, a touch screen display, one or more controllers, one or more remote controls, a headset (e.g., a sensor of a virtual / augmented / mixed reality headset), and / or other types of input devices.

[0027] The communication interfaces 112A, 112B, 112C (collectively referred to herein as "communication interfaces 112") may include one or more components and features for communicating across one or more networks, such as one or more networks 106. The communication interfaces 112 may be configured to communicate via any number of the one or more networks 106 described herein. For example, to communicate between Figure 1 To communicate in the audio manipulation system 100 , the client device 102 may access the Internet using an Ethernet or Wi-Fi connection through a router to communicate with one or more application servers 104 and / or other client devices 102 .

[0028] Local storage devices 116A and 116B (collectively referred to herein as "local storage devices 116") may include any of a variety of computer-readable media. Computer-readable media may be any available media that can be accessed by client device 102. Computer-readable media may include volatile and non-volatile media, and removable and non-removable media. By way of example and not limitation, computer-readable media may include computer storage media and communication media. Local storage devices 116 may include additional or alternative components, such as those described herein with respect to Figure 6 According to some embodiments of the present disclosure, portions of the client application 108 may be stored in the local storage device 116. In some embodiments, application session data, such as decay data, user lists, time of day, meeting topics, and / or other contextual data, may be stored in the local storage device 116.

[0029] One or more application servers 104 may include a host application 118 and a communication interface 112C. The host application 118 may correspond to the client application 108. However, instead of performing audio manipulation operations locally on the client device 102 (e.g., modifying the volume output of audio data corresponding to a remote participant), the host application 118 may perform the audio manipulation operations described herein on the application server 104 and transmit the modified audio data to the client device 102. As described above, the communication interface 112C may include one or more components and features for communicating across one or more networks. Although Figure 1 Only some components and / or features of one or more application servers 104 are shown, but this is not intended to be limiting. For example, one or more application servers 104 may include additional or alternative components, such as the following: Figure 5 Example content streaming system 500, Figure 6 The example computing device 600 and / or Figure 7 Those described for example data center 700 .

[0030] In operation, when the client device 102 initiates an instance of a video conference application via the client application 108 as part of a video conference session, the client device 102 can each be provided with a graphical UI that displays an icon corresponding to each user participating in the instance of the video conference application. The icon may include, but is not limited to, a thumbnail, a sticker, a video feed (e.g., a window with live video, such as live video captured from a camera of a specific user), a static image picture, a virtual background, an avatar, a graphical image, or other representations. Initially, a volume level can be set for the user, such as a default level set by the client application 108, or a volume level based on a learned volume level or a user configuration corresponding to a specific user. For example, the client device 102A can be operated by a local user, and multiple client devices 102B can each be operated by multiple remote users. In such an example, when the user is talking, the local user can find that a specific remote user or participant speaks very loudly compared to other remote users. The client device 102A can then receive an icon selection corresponding to a specific remote user and manipulation of the icon corresponding to the specific remote user from the local user via the input device 114A. For example, the local user may manipulate or otherwise cause the particular remote user icon to move further away from the local user's icon, as presented visually (e.g., graphically) in the UI. Based on the particular remote user's icon being moved further away from the local user's icon (as presented visually in the UI), the client application 108A may identify at least a subset of the audio data received from the client device 102B associated with the particular remote user, and reduce the volume output by the client device 102A for at least the subset of the audio data.

[0031] As an example, and with reference to Figure 2A-2C , Figure 2A-2C An example UI for manipulating user icons suitable for implementing at least some embodiments of the present disclosure is shown. Figure 2AIncluded is an application UI 200A, presentation 202, user icon 204, and participant icons 206, 208, 210, and 212. In operation, a user associated with user icon 204 and users associated with each of participant icons 206, 208, 210, and 212 can participate in an application session. From the perspective of a local user associated with user icon 204, application UI 200A can correspond to an instance of an application—such as an online video conference, a game streaming service, a video streaming service, etc. Initially, a user associated with user icon 204 can set a volume level for each user associated with each of participant icons 206, 208, 210, and 212. Additionally or alternatively, a default level can be set for an application session by the application or based on a learned volume level corresponding to a particular participant.

[0032] In some embodiments, the output audio level of the participant icons 206, 208, 210, and 212 may be inversely related to the distance between the user icon 204 and the participant icons 206, 208, 210, and 212. Figure 2B , when the users are talking, the user associated with the user icon 204 may find that one or more users associated with the participant icons 206, 208, 210, and / or 212 are speaking very loudly / quietly compared to the other users. In order to more comfortably hear and understand the users participating in the application session, the user associated with the user icon 204 may then select the participant icon 208 and move the participant icon 208 closer to the user icon 204, thereby increasing the output volume of the user associated with the participant icon 208. Similarly, the user associated with the user icon 204 may select the participant icon 212 and move the participant icon 212 away from the user icon 204, thereby reducing the output volume of the user associated with the participant icon 212. However, depending on the embodiment and / or the settings of a particular user, changes in volume or other audio settings may have different relationships, such that moving closer may reduce the volume, while moving farther may increase the volume.

[0033] In some embodiments, the output audio level of the participant icons 206, 208, 210, and 212 may be directly related to the size of the participant icons 206, 208, 210, and 212. Figure 2C, when the users are talking, the user associated with the user icon 204 may find that one or more users associated with the participant icons 206, 208, 210, and / or 212 are speaking very loudly / quietly compared to other users. The user associated with the user icon 204 may then select the participant icon 206 and use a “pinch-to-zoom” gesture 212A (e.g., using two fingers to select the icon 206 and spreading the fingers outward) to zoom in on the participant icon 206 (or cause the participant icon 206 to be zoomed in), thereby increasing the size of the participant icon 206 and increasing the output volume of the user associated with the participant icon 206. Similarly, the user associated with the user icon 204 may select the participant icon 210 and use a “pinch-to-zoom” gesture 212B (e.g., using two fingers to select the icon 210 and closing the fingers toward each other) to zoom out (or cause the zooming out) of the participant icon 210, thereby decreasing the size of the participant icon 210 and the output volume of the user associated with the participant icon 210. In some embodiments, the output volume can be determined by comparing the size of the participant icons 206, 208, 210, and 212 to a default size (e.g., an initial size at the start of an application session). In other embodiments, the output volume can be determined by comparing the size of each of the participant icons 206, 208, 210, and / or 212 to each of the other participant icons 206, 208, 210, and / or 212. For example, the participant icon 206 can have an output volume of 100% when it is the largest, and the participant icons 208, 212 can have an output volume of 25% when the size is approximately 25% of the size of the participant icon 206. However, if the size of the participant icon 206 is reduced to match the size of the participant icons 208 and 212, each of the participant icons 206, 208, and 212 can have an output volume of 100% because each of the participant icons 206, 208, and 212 will now be the largest participant icon in the application UI 200C.

[0034] Reference Figure 3 , Figure 3 An example UI for manipulating user icons suitable for implementing some embodiments of the present disclosure is shown. Figure 3Included is an application UI 300, a user icon 304, and participant icons 306, 308, 310, 312, and 314. In operation, initially, each of the participant icons 306, 308, 310, 312, and 314 may be placed at an equal distance around the user icon 304. For example, each of the participant icons 306, 308, 310, 312, and 314 may be placed at one of the radial distance lines 302A, 302B, 302C, 302D, and 302E. When users are talking, the user associated with the user icon 304 may find that one or more users associated with the participant icons 306, 308, 310, 312, and 314 are speaking louder / softer than other users. As such, a user associated with user icon 304 may then select one or more of participant icons 306, 308, 310, 312, and 314 and move the one or more participant icons further away from user icon 304 or closer to user icon 304. Radial distance lines 302A, 302B, 302C, 302D, and 302E may correspond to volume output levels, where the volume output levels are inversely related to the distance from user icon 304 at the center of application UI 300. For example, participant icon 306, which is the participant icon closest to user icon 304, may be associated with the highest volume output compared to participant icons 308, 310, 312, and 314. As described below, participant icon 314 may be associated with the lowest volume output compared to participant icons 306, 308, 310, and 312. Although Figure 3 Radial distance lines 302A, 302B, 302C, 302D, and 302E are shown, but this is intended for illustrative purposes and is not intended to be limiting. The distance between the user icon 304 and each of the participant icons 306, 308, 310, 312, and 314 can be measured using any number of radial distance lines and / or by measuring the distance directly. In addition, embodiments of the application UI 300 are not limited to two dimensions and can be implemented in three dimensions (3D), thereby enabling a user associated with the user icon 304 to spherically position the participant icons 306, 308, 310, 312, and 314 relative to the user icon 304 in 3D.

[0035] See now Figure 4 , each block of the method 400 described herein includes a computing process that can be performed using any combination of hardware, firmware, and / or software. For example, various functions can be performed by a processor executing instructions stored in a memory. The method 400 can also be embodied as computer usable instructions stored on a computer storage medium. The method 400 can be provided by a stand-alone application, a service or a hosted service (standalone or in combination with another hosted service), or a plug-in to another product, to name a few. In addition, for example, with respect to Figure 1The method 400 is described with reference to a system of FIG. However, the method may additionally or alternatively be performed by any system or any combination of systems, including but not limited to the systems described herein.

[0036] Figure 4 4 is a flow chart illustrating a method 400 suitable for implementing some embodiments of the present disclosure for updating audio settings based on manipulation of an icon. At block B402, the method 400 includes initiating an instance of an application on a user's client device. For example, the user may initiate an instance of an application on the user's device by selecting an icon for opening the application, selecting a link for joining an application session, accessing a web page, and / or other means of initiating an instance of an application.

[0037] At block B404, method 400 includes causing a user icon corresponding to the user and one or more participant icons corresponding to one or more participants of an application session of the application to be displayed within a user interface of the instance of the application. For example, when a local user initiates an instance of a video conferencing application, a user interface (UI) may be provided to the user that displays an icon corresponding to the user and several other icons corresponding to participants in the instance of the video conferencing application.

[0038] At block B406, method 400 includes receiving input data representing one or more inputs for manipulating a participant icon in the one or more participant icons. For example, a local user may select an icon corresponding to a particular participant and move the icon away from the local user's icon (or move it) within the user interface.

[0039] At block B408, method 400 includes, in response to the one or more inputs, determining updated audio settings corresponding to the participant associated with the participant icon. For example, based on moving the remote user's icon away from the local user's icon (or causing it to move), the system can determine an updated volume level (e.g., reduce the volume) for the remote user's icon.

[0040] At block B410, method 400 identifies at least a portion of the audio data from the application session corresponding to the participant. For example, based on moving the icon of the remote user away from the icon of the local user, the system can identify a subset of the audio data corresponding to the remote user (e.g., from within the audio data stream).

[0041] At block B412, method 400 includes applying the updated audio settings to at least the portion of the audio data. For example, based on identifying the subset of audio data corresponding to the remote user, the system may decrease / increase the output volume of the remote user's audio data.

[0042] Sample content streaming system

[0043] See now Figure 5 , Figure 5 is an example system diagram of a content streaming system 500 according to some embodiments of the present disclosure. Figure 5 Includes one or more application servers 502 (which may include Figure 6 ), one or more client devices 504 (which may include components, features, and / or functionality similar to the example computing device 600 of Figure 6 ), and one or more networks 506 (which may be similar to one or more networks described herein). In some embodiments of the present disclosure, system 500 may be implemented. Application sessions may correspond to game streaming applications (e.g., NVIDIA GeFORCE NOW), remote desktop applications, simulation applications (e.g., autonomous or semi-autonomous vehicle simulation), computer-aided design (CAD) applications, virtual reality (VR) and / or augmented reality (AR) streaming applications, deep learning applications, and / or other application types.

[0044] In the system 500, for an application session, one or more client devices 504 may receive only input data in response to input to one or more input devices 526, transmit the input data to one or more application servers 502, receive encoded display data from one or more application servers 502, and display the display data on a display 524. In this way, more computationally intensive calculations and processing are offloaded to one or more application servers 502 (e.g., rendering (specifically ray or path tracing) is performed by one or more GPUs of one or more game servers 502 for graphical output of the application session). In other words, the application session is streamed from one or more application servers 502 to one or more client devices 504, thereby reducing the requirements of one or more client devices 504 for graphics processing and rendering.

[0045] For example, with respect to instantiation of an application session, the client device 504 may display a frame of the application session on a display 524 based on receiving display data from one or more application servers 502. The client device 504 may receive input to one of the one or more input devices and generate input data in response. The client device 504 may transmit the input data to the one or more application servers 502 over one or more networks 506 (e.g., the Internet) via a communication interface 520, and the one or more application servers 502 may receive the input data via a communication interface 518. The one or more CPUs may receive the input data, process the input data, and transmit the data to one or more GPUs, which cause the one or more GPUs to generate a rendering of the application session. For example, the input data may represent movement of a user's character, firing a weapon, reloading, passing a ball, turning a vehicle, etc. during a game session of a game application. The rendering component 512 may render the application session (e.g., representing the result of the input data), and the rendering capture component 514 may capture the rendering of the application session as display data (e.g., as image data capturing a rendered frame of the application session). The rendering of the application session may include ray or path tracing lighting and / or shadow effects calculated using one or more parallel processing units (such as GPUs), which may further employ one or more dedicated hardware accelerators or processing cores for executing ray or path tracing techniques of one or more application servers 502. In some embodiments, one or more application servers 502 may use one or more virtual machines (VMs) (e.g., including one or more virtual components, such as vGPUs, vCPUs, etc.) to support the application session. The encoder 516 may then encode the display data to generate encoded display data, and the encoded display data may be sent to the client device 504 over one or more networks 506 via the communication interface 518. The client device 504 may receive the encoded display data via the communication interface 520, and the decoder 522 may decode the encoded display data to generate the display data. The client device 504 may then display the display data via the display 524.

[0046] Example computing device

[0047] Figure 66 is a block diagram of an example computing device 600 suitable for implementing some embodiments of the present disclosure. The computing device 600 may include an interconnect system 602 that directly or indirectly couples the following devices: a memory 604, one or more central processing units (CPUs) 606, one or more graphics processing units (GPUs) 608, a communication interface 610, an input / output (I / O) port 612, an input / output component 614, a power supply 616, one or more presentation components 618 (e.g., a display), and one or more logic units 620. In at least one embodiment, the one or more computing devices 600 may include one or more virtual machines (VMs) and / or any components thereof, which may include virtual components (e.g., virtual hardware components). For non-limiting examples, the one or more GPUs 608 may include one or more vGPUs, the one or more CPUs 606 (which may include one or more vCPUs), and / or the one or more logic units 620 (which may include one or more virtual logic units). Thus, computing device 600 may include discrete components (eg, a complete GPU dedicated to computing device 600 ), virtual components (eg, a portion of a GPU dedicated to computing device 600 ), or a combination thereof.

[0048] although Figure 6 The various blocks of are shown as being connected via an interconnect system 602 having wires, but this is not intended to be limiting and is merely for clarity. For example, in some embodiments, a presentation component 618 such as a display device may be considered an I / O component 614 (e.g., if the display is a touch screen). As another example, CPU 606 and / or GPU 608 may include memory (e.g., memory 604 may represent a storage device in addition to the memory of GPU 608, CPU 606, and / or other components). In other words, Figure 6 The computing devices described herein are illustrative only. No distinction is made between categories such as "workstations," "servers," "laptops," "desktops," "tablets," "client devices," "mobile devices," "handheld devices," "game consoles," "electronic control units (ECUs)," "virtual reality systems," and / or other device or system types, as all are considered Figure 6 within the range of computing devices.

[0049] Interconnection system 602 can represent one or more links or buses, such as address bus, data bus, control bus or combination thereof.Interconnection system 602 can include one or more links or bus types, such as industry standard architecture (ISA) bus, extended industry standard architecture (EISA) bus, video electronics standard association (VESA) bus, peripheral component interconnect (PCI) bus, peripheral component interconnect fast (PCIe) bus, and / or another type of link or bus. In some embodiments, there is a direct connection between components. As an example, CPU 606 can be directly connected to memory 604. In addition, CPU 606 can be directly connected to GPU 608. In the case of direct or point-to-point connection between components, interconnection system 602 can include a PCIe link to perform linking. In these examples, the PCI bus does not have to be included in computing device 600.

[0050] Memory 604 may include any of a variety of computer-readable media. Computer-readable media may be any available media that can be accessed by computing device 600. Computer-readable media may include volatile and nonvolatile media and removable and non-removable media. By way of example and not limitation, computer-readable media may include computer storage media and communication media.

[0051] Computer storage media may include volatile and nonvolatile media and / or removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, and / or other data types. For example, memory 604 may store computer-readable instructions (e.g., representing programs and / or program elements, such as an operating system). Computer storage media may include, but are not limited to, RAM, ROM, EEPROM, flash memory or other storage technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by computing device 600. As used herein, computer storage media does not include the signals themselves.

[0052] Computer storage media may contain computer readable instructions, data structures, program modules, and / or other data types in a modulated data signal such as a carrier wave or other transmission mechanism, and include any information delivery media. The term "modulated data signal" may refer to a signal that has one or more of its characteristics set or changed in such a way that information is encoded into the signal. By way of example and not limitation, computer storage media may include wired media such as a wired network or a direct wired connection, and wireless media such as sound, RF, infrared, and other wireless media. Any combination of the above should also be included within the scope of computer readable media.

[0053] The CPU 606 may be configured to execute at least some computer-readable instructions to control one or more components of the computing device 600 to perform one or more of the methods and / or processes described herein. Each of the CPUs 606 may include one or more cores (e.g., one, two, four, eight, twenty-eight, seventy-two, etc.) capable of simultaneously processing a large number of software threads. The CPU 606 may include any type of processor, and may include different types of processors, depending on the type of computing device 600 implemented (e.g., a processor with fewer cores for mobile devices and a processor with more cores for servers). For example, depending on the type of computing device 600, the processor may be an Advanced RISC Machine (ARM) processor implemented using Reduced Instruction Set Computing (RISC) or an x86 processor implemented using Complex Instruction Set Computing (CISC). The computing device 600 may also include one or more CPUs 606 in addition to one or more microprocessors or supplementary coprocessors such as math coprocessors.

[0054] In addition to the one or more CPUs 606, the one or more GPUs 608 may be configured to execute at least some of the computer-readable instructions to control one or more components of the computing device 600 to perform one or more of the methods and / or processes described herein below. The one or more GPUs 608 may be an integrated GPU (e.g., with one or more CPUs 606 and / or the one or more GPUs 608 may be a discrete GPU. In one embodiment, the one or more GPUs 608 may be a coprocessor of the one or more CPUs 606. The computing device 600 may use the one or more GPUs 608 to render graphics (e.g., 3D graphics). For example, the GPU 608 may be used for general-purpose computing on a GPU (GPGPU). The GPU 608 may include hundreds or thousands of cores capable of processing hundreds or thousands of software threads simultaneously. The GPU 608 may generate pixel data for outputting an image in response to a rendering command (e.g., a rendering command received from the CPU 606 via a host interface). The GPU 608 may include graphics memory such as a display memory for storing pixel data. The display memory may be included as part of the memory 604. The GPU 608 may include two or more GPUs operating in parallel (e.g., via a link) or any other suitable data, such as GPGPU data. The link may directly connect the GPUs (e.g., using NVLINK), or may connect the GPUs through a switch (e.g., using NVSwitch). When combined together, each GPU 608 may generate different portions for output or different pixel data or GPGPU data for output (e.g., a first GPU for a first image and a second GPU for a second image). Each GPU may include its own memory, or may share memory with other GPUs.

[0055] In addition to or as an alternative to one or more CPUs 606 and / or one or more GPUs 608, one or more logic units 620 may be configured to execute at least some computer-readable instructions to control one or more components of the computing device 600 to perform one or more methods and / or processes described herein. In embodiments, one or more CPUs 606, one or more GPUs 608, and / or one or more logic units 620 may discretely or jointly perform any combination of methods, processes, and / or portions thereof. One or more logic units 620 may be a part of and / or integrated into one or more CPUs 606, and / or one or more GPUs 608 and / or the one or more logic units 620 may be discrete components or otherwise external to one or more CPUs 606 and / or one or more GPUs 608. In embodiments, one or more logic units 620 may be one or more coprocessors of one or more CPUs 606 and / or one or more GPUs 608.

[0056] Examples of one or more logic units 620 include and / or are configured to perform functions of one or more processing cores and / or components thereof, such as a tensor core (TC), a tensor processing unit (TPU), a pixel visual core (PVC), a visual processing unit (VPU), a graphics processing cluster (GPC), a texture processing cluster (TPC), a streaming multiprocessor (SM), a tree traversal unit (TTU), artificial intelligence accelerators (AIAs), a deep learning accelerator (DLA), an arithmetic logic unit (ALU), an application specific integrated circuit (ASIC), a floating point unit (FPU), an input / output (I / O) element, a peripheral component interconnect (PCI) or a peripheral component interconnect express (PCIe) element, etc.

[0057] The communication interface 610 may include one or more receivers, transmitters, and / or transceivers that enable the computing device 600 to communicate with other computing devices via an electronic communication network, including wired and / or wireless communications. The communication interface 610 may include components and functionality that enable communication over any of a number of different networks, such as a wireless network (e.g., Wi-Fi, Z-wave, Bluetooth, Bluetooth LE, ZigBee, etc.), a wired network (e.g., communicating over Ethernet or InfiniBand), a low power wide area network (e.g., LoRaWAN, SigFox, etc.), and / or the Internet.

[0058] The I / O ports 612 can enable the computing device 600 to be logically coupled to other devices including I / O components 614, presentation components 618, and / or other components, some of which can be built into (e.g., integrated into) the computing device 600. Illustrative I / O components 614 include microphones, mice, keyboards, joysticks, game pads, game controllers, satellite dishes, scanners, printers, wireless devices, and the like. The I / O components 614 can provide a natural user interface (NUI) that processes air gestures, voice, or other physiological inputs generated by a user. In some instances, the input can be transmitted to an appropriate network element for further processing. The NUI can implement any combination of voice recognition, stylus recognition, facial recognition, biometric recognition, gesture recognition on and adjacent to the screen, air gestures, head and eye tracking, and touch recognition associated with the display of the computing device 600 (as described in more detail below). The computing device 600 may include a depth camera such as a stereo camera system, an infrared camera system, an RGB camera system, touch screen technology, and combinations of these for gesture detection and recognition. Additionally, computing device 600 may include an accelerometer or gyroscope to enable motion detection (e.g., as part of an inertial measurement unit (IMU)). In some examples, the output of the accelerometer or gyroscope may be used by computing device 600 to render immersive augmented reality or virtual reality.

[0059] The power supply 616 may include a hardwired power supply, a battery power supply, or a combination thereof. The power supply 616 may provide power to the computing device 600 to enable the components of the computing device 600 to operate.

[0060] The presentation component 618 may include a display (e.g., a monitor, a touch screen, a television screen, a head-up display (HUD), other display types, or a combination thereof), a speaker, and / or other presentation components. The presentation component 618 may receive data from other components (e.g., GPU 608, CPU 606, etc.) and output the data (e.g., as an image, video, sound, etc.).

[0061] Sample Data Center

[0062] Figure 7 An example data center 700 that can be used in at least one embodiment of the present disclosure is shown. The data center 700 can include a data center infrastructure layer 710, a framework layer 720, a software layer 730, and / or an application layer 740.

[0063] like Figure 7As shown, the data center infrastructure layer 710 may include a resource coordinator 712, grouped computing resources 714, and node computing resources ("node CRs") 716(1)-716(N), where "N" represents any complete positive integer. In at least one embodiment, the node CRs 716(1)-716(N) may include, but are not limited to, any number of central processing units ("CPUs") or other processors (including accelerators, field programmable gate arrays (FPGAs), graphics processors, or graphics processing units (GPUs), etc.), memory devices (e.g., dynamic read-only memory), storage devices (e.g., solid-state drives or disk drives), network input / output ("NW I / O") devices, network switches, virtual machines ("VMs"), power modules and cooling modules, etc. In at least some embodiments, one or more of the node CRs 716(1)-716(N) may correspond to a server having one or more of the above computing resources. Additionally, in some embodiments, nodes CR 716 ( 1 )- 716 (N) may include one or more virtual components (such as vGPU, vCPU, etc.), and / or one or more nodes CR 716 ( 1 )- 716 (N) may correspond to a virtual machine (VM).

[0064] In at least one embodiment, the computing resources 714 of the grouping can include a separate grouping (not shown) of the node CR716 contained in one or more racks, or many racks (also not shown) contained in the data center of each geographical location. The separate grouping of the node CR716 in the computing resources 714 of the grouping can include the computing, network, memory or storage resources that can be configured or assigned to support the grouping of one or more workloads. In at least one embodiment, several node CR716 including CPU, GPU or other processors can be grouped in one or more racks to provide computing resources to support one or more workloads. One or more racks can also include any number of power modules, cooling modules and / or network switches, in any combination.

[0065] The resource coordinator 722 may configure or otherwise control one or more nodes CR 716(1)-716(N) and / or grouped computing resources 714. In at least one embodiment, the resource coordinator 722 may include a software design infrastructure ("SDI") management entity for the data center 700. The resource coordinator 722 may include hardware, software, or some combination thereof.

[0066] In at least one embodiment, Figure 7As shown, the framework layer 720 may include a job scheduler 732, a configuration manager 734, a resource manager 736, and a distributed file system 738. The framework layer 720 may include a framework that supports the software 732 of the software layer 730 and / or one or more applications 742 of the application layer 740. The software 732 or the application 742 may include web-based service software or applications, such as services or applications provided by Amazon Web Services, Google Cloud, and Microsoft Azure. In at least one embodiment, the framework layer 720 may be, but is not limited to, a free and open source software network application framework, such as Apache SparkTM (hereinafter referred to as "Spark") that can utilize the distributed file system 738 for large-scale data processing (e.g., "big data"). In at least one embodiment, the job scheduler 732 may include a Spark driver to facilitate scheduling of workloads supported by various layers of the data center 700. The configuration manager 734 may be able to configure different layers, such as the software layer 730 and the framework layer 720 including Spark and a distributed file system 738 for supporting large-scale data processing. The resource manager 736 can manage clustered or grouped computing resources mapped to or allocated to support the distributed file system 738 and the job scheduler 732. In at least one embodiment, the clustered or grouped computing resources may include grouped computing resources 714 on the data center infrastructure layer 710. The resource manager 736 can coordinate with the resource coordinator 712 to manage these mapped or allocated computing resources.

[0067] In at least one embodiment, software 732 included in software layer 730 may include software used by at least a portion of node CRs 716(1)-716(N), grouped computing resources 714, and / or a distributed file system 738 of framework layer 720. One or more types of software may include, but are not limited to, Internet web page search software, email virus scanning software, database software, and streaming video content software.

[0068] In at least one embodiment, one or more applications 742 included in the application layer 740 may include one or more types of applications used by at least a portion of the node CRs 716(1)-716(N), the grouped computing resources 714, and / or the distributed file system 738 of the framework layer 720. The one or more types of applications may include, but are not limited to, any number of genomics applications, cognitive computing, and machine learning applications, including training or inference software, machine learning framework software (e.g., PyTorch, TensorFlow, Caffe, etc.), and / or other machine learning applications used in conjunction with one or more embodiments.

[0069] In at least one embodiment, any of configuration manager 734, resource manager 736, and resource coordinator 712 can implement any number and type of self-modification actions based on any number and type of data acquired in any technically feasible manner. The self-modification actions can relieve a data center operator of data center 700 from making potentially bad configuration decisions and can avoid underutilized and / or poorly performing portions of the data center.

[0070] The data center 700 may include tools, services, software, or other resources to train one or more machine learning models or use one or more machine learning models to predict or infer information according to one or more embodiments described herein. For example, one or more machine learning models may be trained by calculating weight parameters according to a neural network architecture using the software and / or computing resources described above with respect to the data center 700. In at least one embodiment, by using weight parameters calculated by one or more training techniques (such as, but not limited to, the techniques described herein), the resources described above with respect to the data center 700 may be used to infer or predict information using a trained or developed machine learning model corresponding to one or more neural networks. Such as, but not limited to, what is described herein

[0071] In at least one embodiment, the data center 700 may use a CPU, an application-specific integrated circuit (ASIC), a GPU, an FPGA, and / or other hardware (or virtual computing resources corresponding thereto) to use the above resources to perform training and / or reasoning. In addition, one or more of the above software and / or hardware resources may be configured as a service to allow a user to train or perform information reasoning, such as image recognition, speech recognition, or other artificial intelligence services.

[0072] Example network environment

[0073] A network environment suitable for implementing embodiments of the present disclosure may include one or more client devices, servers, network attached storage (NAS), other backend devices, and / or other device types. The client devices, servers, and / or other device types (e.g., each device) may be configured to communicate with the client device, server, or other device types. Figure 6 The backend device 700 may be implemented on one or more instances of one or more computing devices 600. For example, each device may include similar components, features and / or functions of one or more computing devices 600. In addition, in the case of implementing a backend device (e.g., a server, NAS, etc.), the backend device may be included as part of the data center 700, and reference is made to the backend device 700 in this document. Figure 7 The examples are described in more detail.

[0074] The components of the network environment can communicate with each other via a network, which can be wired, wireless, or both. The network can include multiple networks or networks of networks. For example, the network can include one or more wide area networks (WANs), one or more local area networks (LANs), one or more public networks, such as the Internet and / or a public switched telephone network (PSTN), and / or one or more private networks. In the case where the network includes a wireless telecommunications network, components such as base stations, communication towers, and even access points (and other components) can provide wireless connections.

[0075] Compatible network environments may include one or more peer-to-peer network environments (in which case the server may not be included in the network environment) and one or more client-server network environments (in which case one or more servers may be included in the network environment). In a peer-to-peer network environment, the functionality described herein for the server may be implemented on any number of client devices.

[0076] In at least one embodiment, the network environment may include one or more cloud-based network environments, distributed computing environments, combinations thereof, and the like. A cloud-based network environment may include a framework layer, a job scheduler, a resource manager, and a distributed file system implemented on one or more servers, and the servers may include one or more core network servers and / or edge servers. The framework layer may include a framework for supporting software of the software layer and / or one or more applications of the application layer. The software or application may include web-based service software or applications, respectively. In an embodiment, one or more client devices may use web-based service software or applications (e.g., by accessing the service software and / or application via one or more application programming interfaces (APIs)). The framework layer may be, but is not limited to, a free and open source software Web application framework, such as a distributed file system that may be used for large-scale data processing (e.g., "big data").

[0077] A cloud-based network environment can provide cloud computing and / or cloud storage that performs any combination of the computing and / or data storage functions (or one or more parts thereof) described herein. Any of these different functions can be performed from a central or core server (e.g., in one or more data centers that can be distributed across states, regions, countries, the world, etc.). If the connection to a user (e.g., a client device) is relatively close to one or more edge servers, one or more core servers can specify at least a portion of the functions to one or more edge servers. A cloud-based network environment can be private (e.g., limited to a single organization), public (e.g., available to many organizations), and / or a combination thereof (e.g., a hybrid cloud environment).

[0078] The client device may include the Figure 6 The client device 600 may be embodied as a personal computer (PC), a laptop computer, a mobile device, a smart phone, a tablet computer, a smart watch, a wearable computer, a personal digital assistant (PDA), an MP3 player, a virtual reality headset, a global positioning system (GPS) or device, a video player, a camera, a surveillance device or system, a vehicle, a boat, a flying vessel, a virtual machine, a drone, a robot, a handheld communication device, a hospital device, a gaming device or system, an entertainment system, a vehicle computer system, an embedded system controller, a remote control, a device, a consumer electronic device, a workstation, an edge device, any combination of the depicted devices, or any other suitable device.

[0079] The present disclosure may be described in the general context of computer code or machine-usable instructions, including computer-executable instructions (such as program modules) executed by a computer or other machine (such as a personal data assistant or other handheld device). In general, program modules, including routines, programs, objects, components, data structures, etc., refer to code that performs a specific task or implements a specific abstract data type. The present disclosure may be practiced in a variety of system configurations, including handheld devices, consumer electronics, general-purpose computers, more specialized computing devices, etc. The present disclosure may also be practiced in a distributed computing environment, where tasks are performed by remote processing devices linked through a communication network.

[0080] As used herein, the statement of "and / or" about two or more elements should be interpreted as meaning only one element or a combination of more elements. For example, "element A, element B, and / or element C" may include only element A, only element B, only element C, element A and element B, element A and element C, element B and element C, or elements A, B, and C. In addition, "at least one of element A or element B" may include at least one of element A, at least one of element B, or at least one of element A and at least one of element B. Further, "at least one of element A and element B" may include at least one of element A, at least one of element B, or at least one of element A and at least one of element B.

[0081] The subject matter of the present disclosure is described in detail herein to meet statutory requirements. However, this description itself is not intended to limit the scope of the present disclosure. Rather, the inventors have anticipated that the claimed subject matter may also be embodied in other ways, in conjunction with other current or future technologies, to include different steps or combinations of steps similar to the steps described in this document. In addition, although the terms "step" and / or "frame" may be used herein to imply different elements of the method employed, these terms should not be interpreted as implying any particular order among or between the various steps disclosed herein, unless and except when the order of the various steps is explicitly described.

Claims

1. A method for volume control, comprising: causing display, within a user interface of an instance of an application executed using a client device of the user, a user representation of the user and one or more participant representations corresponding to one or more participants of an application session of the application, wherein a size of at least one of the one or more participant representations is determined based at least in part on one or more historical manipulations of one or more historical participant representations corresponding to at least one of the one or more participants during one or more historical application sessions of the application; determining, based at least in part on the size of the at least one participant representation, audio settings corresponding to the at least one participant associated with the at least one participant representation; and The audio settings are applied to at least a portion of audio data from the application session associated with the at least one participant.

2. The method according to claim 1, further comprising: receiving one or more inputs corresponding to manipulation of the at least one participant representation corresponding to repositioning the at least one participant representation relative to a position of the user representation depicted using the user interface; as well as Based at least in part on the one or more inputs, updated audio settings corresponding to the at least one participant are determined.

3. The method according to claim 2, wherein: The updated audio settings include adjusting a volume level corresponding to the at least one participant when the at least one participant representation is repositioned relative to the position of the user representation depicted using the user interface.

4. The method of claim 3, wherein adjusting the volume level comprises at least one of: increasing the volume level when the at least one participant representation is repositioned closer to the location of the user representation depicted using the user interface; or The volume level is decreased when the at least one participant representation is repositioned further away from the position of the user representation within the user interface.

5. The method according to claim 1, further comprising: receiving one or more inputs corresponding to manipulation of the at least one participant representation corresponding to adjusting at least one of a position or the size of the at least one participant representation; as well as Based at least in part on the one or more inputs, updated audio settings corresponding to the at least one participant are determined.

6. The method according to claim 5, wherein: The updated audio settings include adjusting a volume level corresponding to the at least one participant when at least one of a position or the size of the at least one participant representation depicted using the user interface is adjusted.

7. The method of claim 6, wherein adjusting the volume level comprises at least one of: increasing the volume level when the size of at least one participant representation increases relative to the size of the user representation depicted using the user interface; or The volume level is decreased when a size of the at least one participant representation decreases relative to the size of the user representation depicted using the user interface. 8 . The method of claim 1 , wherein the application corresponds to at least one of a conferencing application, a game streaming application, a content streaming application, or a telephony service application.

9. The method according to claim 1, further comprising: receiving data representing an indication that the at least one participant is in contact with the user; as well as The audio settings are adjusted to updated audio settings based at least in part on the at least one participant interacting with the user.

10. The method according to claim 1, further comprising: A position of the at least one participant representation is determined relative to a position of the user representation depicted using the user interface.

11. The method according to claim 1, further comprising: receiving data representing at least one of the participants manipulating a representation corresponding to the user within the application session; as well as Instructions to manipulate the representation corresponding to the user are caused to be displayed within the user interface.

12. A processor, comprising: One or more circuits for: causing display within a user interface of an application a representation of a user corresponding to a user and one or more participant representations corresponding to one or more participants of an application session, wherein a size of at least one of the one or more participant representations is determined based at least in part on one or more historical manipulations of one or more historical participant representations corresponding to at least one of the one or more participants during one or more historical application sessions of the application; generating updated audio data based at least in part on adjusting a volume level associated with at least a portion of received audio data corresponding to the at least one participant associated with the at least one participant representation based at least in part on the size of the at least one participant representation; as well as The updated audio data is caused to be output.

13. The processor of claim 12, wherein the one or more circuits are further configured to: receiving one or more inputs corresponding to repositioning the at least one participant representation relative to the user representation depicted using the user interface; and Second updated audio data is generated based at least in part on adjusting a second volume level associated with at least a second portion of the received audio data based at least in part on the one or more inputs.

14. The processor of claim 13, wherein adjusting the second volume level comprises: The second volume level corresponding to the at least one participant is adjusted when the at least one participant representation is repositioned relative to the position of the user representation depicted using the user interface.

15. The processor of claim 14, wherein adjusting the second volume level comprises at least one of: increasing the second volume level when the participant representation is repositioned closer to the user representation depicted using the user interface; or The second volume level is decreased when the participant representation is repositioned further away from the user representation depicted using the user interface.

16. The processor of claim 12, wherein the one or more circuits are further configured to: receiving one or more inputs corresponding to adjusting at least one of a position or the size of the at least one participant representation depicted using the user interface; and Second updated audio data is generated based at least in part on adjusting a second volume level associated with at least a second portion of the received audio data based at least in part on the one or more inputs.

17. The processor of claim 16, wherein adjusting the second volume level comprises: When at least one of the position or the size of the at least one participant representation depicted using the user interface is adjusted, the second volume level corresponding to the at least one participant is adjusted.

18. The processor of claim 17, wherein adjusting the second volume level comprises at least one of: increasing the second volume level when the size of the participant representation depicted using the user interface increases; or The second volume level is decreased when the size of the participant representation depicted using the user interface decreases.

19. The processor of claim 12, wherein the processor is included in at least one of the following: Systems for performing deep learning operations; Systems implemented using edge devices; A system for merging one or more virtual machines VM; A system implemented at least in part in a data center; or A system implemented at least in part using cloud computing resources.

20. A system for volume control, comprising: One or more processing units for Launching an instance of the application on the user's client device; causing display within a user interface of an instance of the application a user representation of the user and one or more participant representations corresponding to one or more participants of an application session of the application, wherein a size of at least one of the one or more participant representations is determined based at least in part on one or more historical manipulations of one or more historical participant representations corresponding to at least one of the one or more participants during one or more historical application sessions of the application; determining, based at least in part on the size, an audio setting corresponding to the at least one participant associated with the at least one participant representation; as well as The audio settings are applied to at least a portion of audio data from an instance of the application associated with the at least one participant.

21. The system of claim 20, wherein the one or more processing units are further configured to: receiving the one or more inputs manipulating the at least one participant representation to reposition the at least one participant representation relative to the user representation depicted using the user interface; and Based at least in part on the one or more inputs, updated audio settings corresponding to the at least one participant are determined.

22. The system of claim 21, wherein the updated audio settings include: An adjusted volume level corresponding to the at least one participant.

23. The system of claim 22, wherein the adjusted volume level comprises at least one of: an increased volume level when the at least one participant representation is repositioned closer to the user representation within the user interface; or A reduced volume level when the at least one participant representation is repositioned further away from the user representation within the user interface.

24. The system of claim 20, wherein the one or more processing units are further configured to: receiving the one or more inputs manipulating the at least one participant representation to adjust the size of the at least one participant representation; and Based at least in part on the one or more inputs, updated audio settings corresponding to the at least one participant are determined.

25. The system of claim 24, wherein the updated audio settings include an adjusted volume level corresponding to the at least one participant.

26. The system of claim 25, wherein the volume level that is adjusted comprises at least one of: increasing said volume level as said at least one participant representation increases in size within said user interface; or The volume level is reduced when the at least one participant representation is reduced in size within the user interface.

27. The system of claim 20, wherein the system is included in at least one of the following: Systems for performing deep learning operations; Systems implemented using edge devices; A system for merging one or more virtual machines VM; A system implemented at least in part in a data center; or A system implemented at least in part using cloud computing resources.

Images