Saliency-based digital environment adaptation

Abstract

In examples, a saliency metric can be determined for an instance of content and / or a location in a digital environment. A set of candidate content and / or locations can be ranked accordingly according to the associated saliency metric, and from the candidate set, content and / or associated locations can be determined for adapting the digital environment for a given user. For example, the digital environment can be adapted to present two-dimensional or three-dimensional assets to the user. As another example, game mechanics of the digital environment can be changed. In some examples, content from another digital environment can be identified and used to adapt the digital environment, thereby incorporating external content. Thus, the saliency metric associated with the location or content instance can be used to programmatically generate relative or absolute metrics for evaluating and adapting aspects of the digital environment.

Description

[Background technology]

[0001] background It is possible for a user to be presented with and / or to interact with content in a digital environment. However, different users find different content appealing. Thus, it can be difficult to adapt the digital environment to take into account user-specific differences and to enhance the associated user experience.

[0002] The embodiments are described with respect to these and other general considerations, and while relatively specific problems are discussed, it should be understood that the embodiments should not be limited to solving the specific problems identified in the context. Summary of the Invention

[0003] overview In examples, a saliency metric can be determined for an instance of content and / or a location in a digital environment. A set of candidate content and / or locations can be ranked accordingly according to the associated saliency metric, and from the candidate set, content and / or associated locations can be determined for adapting the digital environment for a given user. For example, the digital environment can be adapted to present two-dimensional or three-dimensional assets to the user. As another example, game mechanics of the digital environment can be changed. In some examples, content from another digital environment can be identified and used to adapt the digital environment, thereby incorporating external content. Thus, the saliency metric associated with the location or content instance can be used to programmatically generate relative or absolute metrics for evaluating and adapting aspects of the digital environment.

[0004] This Summary is provided to introduce some concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0005] BRIEF DESCRIPTION OF THE DRAWINGS Non-limiting and non-exhaustive examples are described with reference to the following figures. [Brief description of the drawings]

[0006] [Figure 1] 1 illustrates a schematic of an example system for saliency-based digital environment adaptation according to aspects described herein. [Diagram 2] 1 illustrates an overview of an example method for generating a content salience metric for adapting a digital environment, according to aspects described herein. [Diagram 3] 1 illustrates an overview of an example method for adapting a digital environment according to aspects described herein. [Figure 4] 1 illustrates a schematic of another example method for adapting a digital environment according to aspects described herein. [Diagram 5] FIG. 1 is a block diagram illustrating example physical components of a computing device capable of implementing aspects of the present disclosure. [Figure 6A] FIG. 1 is a simplified block diagram of a mobile computing device capable of implementing aspects of the present disclosure. [Figure 6B] FIG. 1 is a simplified block diagram of a mobile computing device capable of implementing aspects of the present disclosure. [Figure 7] FIG. 1 is a simplified block diagram of a distributed computing system in which aspects of the present disclosure may be implemented. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Detailed Description In the following detailed description, reference is made to the accompanying drawings which form a part hereof, and in which are shown, by way of illustration, specific embodiments or examples. These aspects can be combined, other aspects can be utilized, and structural changes can be made without departing from the present disclosure. The embodiments can be embodied as methods, systems, or devices. Accordingly, the embodiments can take the form of a hardware implementation, an entirely software implementation, or an implementation combining software and hardware aspects. Therefore, the following detailed description is not to be taken in a limiting sense, and the scope of the present disclosure is defined by the appended claims and their equivalents.

[0008] In an example, the content of a digital environment may be presented to a user. Exemplary digital environments include, but are not limited to, a video game environment, a virtual collaboration environment, an alternate reality (AR) environment, and / or a virtual reality (VR) environment. For example, the digital environment may be part of a metaverse. The content of the digital environment may include environmental assets (e.g., sounds, text overlays, two-dimensional (2D) or three-dimensional (3D) models or game items, scenery, weapons, non-player characters (NPCs), and player models), as well as environmental mechanics (e.g., those related to the behavior of a given asset, a mini-game, and / or a series of choices in a role-playing game). As another example, a non-fungible token may be an instance of content or be associated with an instance of content. However, creating a personalized experience for a user within a digital environment (e.g., based on a user profile and associated implicit or explicit preferences) may be difficult. In addition, adapting a digital environment based on content associated with another digital environment (e.g., to provide recommendations across different digital environments) can also be similarly difficult, for example, due to little or no insight into user preferences across multiple digital environments.

[0009] Accordingly, aspects of the present disclosure relate to digital environment adaptation based on salience. In examples, a digital environment can be adapted based on any of a variety of factors, including, but not limited to, content attributes (e.g., related to the usefulness and / or rarity of the content), environment attributes (e.g., related to spatial locations within the digital environment where the content may be presented), user profile attributes (e.g., related to a user's play style, number of user interactions with the content / similar types of content, attention habits based on the user's viewable area determined, for example, from an AR / VR headset, and / or explicit and / or implicit user-specific preferences), and / or population attributes (e.g., similar to user profile attributes and related to a larger population of users). Accordingly, a salience metric can be determined for content and / or locations of the digital environment based on such factors. Although exemplary factors are described herein, it will be appreciated that in other examples, any of a variety of additional and / or alternative factors can be used.

[0010] As an example, content from a content set can be determined based on an associated salience metric, and the content set is ranked according to the salience metric for each instance of the content. The highest ranked salience metric can be determined accordingly for presentation to the user. For example, 2D or 3D assets can be presented to the user, and / or environmental mechanics can be incorporated or modified (e.g., to present the user with a series of different story options and / or to have different levels of difficulty, prominence and / or length). As another example, a similar technique can be used to determine spatial locations (e.g., locations within a digital environment that can be overlaid on the user's physical environment) for presenting content to the user, and such spatial location sets can be ranked according to the associated salience metric. Thus, the experience provided by the digital environment can vary from user to user, thereby providing each user with a personalized experience.

[0011] Although exemplary adaptations have been described, it will be appreciated that in other examples, any of a variety of additional or alternative adaptations can be performed. For example, the representation of the digital environment can be adapted according to a saliency metric, such that content with a higher saliency metric is rendered at a higher quality level (e.g., using more detailed textures, more detailed 3D models, and / or video encoding at a higher bit rate) and content with a lower saliency metric is rendered at a lower quality level. As another example, content of the digital environment having a saliency metric above a predefined threshold can be cached in system memory, while content having a saliency metric below the predefined threshold can be loaded into system memory when needed to render the digital environment.

[0012] As another example, a determination may be made based on the salience metric that an NPC should be spawned in the digital environment, such as when relevant environmental and / or population factors indicate that the location is becoming more difficult. As a result, the NPC may assist the user, thereby reducing the likelihood of the user becoming frustrated. Similarly, a recommendation may be presented to request assistance from another user of the digital environment, and in response to accepting the recommendation, the other user may be identified and communication established between the users, thereby enabling the users to assist each other. In contrast, if the user factors indicate an improved aptitude for the game mechanics associated with the location, the salience metric of the NPC may be relatively decreased, and in such an example, the NPC may not be spawned, or a less available NPC may be spawned.

[0013] Further, while the salience metric is in the context of adapting a digital environment, it will be understood that there may be additional or alternative uses. As discussed above, the associated salience metric may be used to compare a first content item to a second content item, thereby establishing a relative "value" of the first and second content items. In other examples, such value need not be relative, but may instead be an estimated or predicted value, such as when the content has not yet been introduced to the digital environment. Thus, the salience metric may indicate the relevance, usefulness, estimated interest and / or perceived value of a given instance and / or location of content, in accordance with aspects described herein. As another example, the salience metric may be used to assign values ​​associated with spatial locations of a digital environment, such that corresponding content may be identified according to the assigned values ​​of the spatial locations.

[0014] It will be appreciated that the content used to adapt a digital environment need not be limited to the content initially associated with the digital environment. For example, content associated with a different digital environment can be determined and presented in the digital environment accordingly, according to aspects described herein. For example, such external content may be game items from a different game, or may include targeted content provided by a third party, among other examples. Thus, content from within the different digital environment itself may be presented to the user (e.g., if you like this mechanic or item, try a similar mechanic or item in a different digital environment) compared to instances where the digital environment itself is presented as a recommendation (e.g., a list of games that similar players also enjoyed).

[0015] Additionally, similar techniques can be used to adapt a feed, recording, stream, or other reproduction of a digital environment (e.g., in addition to the initial rendering presented to a user). For example, a digital environment can be adapted according to aspects described herein as it is rendered for a user, and the stream of the rendering (e.g., as it may be provided to another computing device via a streaming platform) can be further adapted with similar or different content. For example, the stream of the digital environment can be adapted to include external content (e.g., based on environmental factors associated with the stream, such as a portion of a story associated with the stream). In some examples, the digital environment can be adapted based at least in part on a set of attributes associated with a first user (e.g., for whom the digital environment is rendered), while the stream of the digital environment can be adapted based at least in part on a set of attributes associated with a second user or user group (e.g., for whom the stream of the digital environment is provided).

[0016] As discussed above, the digital environment may adapt based on any of a variety of factors, including, but not limited to, content attributes, environment attributes, user profile attributes, and / or population attributes. In some examples, data associated with such factors may be maintained locally on the user's device (e.g., an application associated with the digital environment may maintain user profile data locally on the computing device) and / or aggregated by the digital environment platform. For example, the digital environment platform may aggregate telemetry data associated with one or more digital environments and use the telemetry data to determine attributes associated with the user and / or associated with one or more user populations (e.g., associated with a geographic region, locale, or any other variety of demographics of one or more digital environments).

[0017] A content salience metric can be generated based on such factors using any of a variety of techniques, including, but not limited to, a statistical model (e.g., factors combined according to a set of weights) or a machine learning model. For example, a machine learning model can be trained based on user interactions with content of the digital environment (e.g., as may be determined based on telemetry data) that is annotated according to relevant factors. In some examples, the machine learning model can be tailored to a particular user or can be user-specific or population-specific, among other examples. As an example, the machine learning model can be trained to determine content or a location from a candidate set to maximize the likelihood that a user will engage with the determined content or location. Such techniques can be used to generate a relative salience metric (e.g., for comparing a first content item to a second content item) or an absolute salience metric, among other examples. In some examples, the salience metric can include a target content value or rating.

[0018] 1 illustrates an overview of an example system 100 for salience-based digital environment adaptation according to aspects described herein. As shown, the system 100 includes a digital environment platform 102, a digital environment service 104, a computing device 106, and a network 108. In an example, the digital environment platform 102, the digital environment service 104, and / or the computing device 106 communicate over the network 108, which may include a local area network, a wireless network, or the Internet, or any combination thereof, among other examples.

[0019] The digital environment platform 102 can aggregate telemetry data associated with one or more digital environments (e.g., as may be associated with the digital environment services 104). Although the digital environment platform 102 and the digital environment services 104 are shown separately in FIG. 1, it will be understood that in other examples, the digital environment services can implement aspects similar to those of the digital environment platform 102. Thus, aspects of the present disclosure can be applied in the context of a single digital environment (e.g., as may be provided by a digital environment service similar to the digital environment service 104) or multiple digital environments as well. For example, while the digital environment platform 102 is shown as including a salience metric engine 112 and an interaction data store 114, the digital environment services 104 can additionally or alternatively include further aspects, such as when the digital environment services 104 generate salience metrics independently of or in addition to the digital environment platform 102. Similarly, the digital environment platform 102 may incorporate similar aspects as the digital environment services 104, such that the digital environment platform 102 provides a digital environment platform in addition to performing telemetry data aggregation and data processing to generate salience metrics, as described herein.

[0020] The digital environment platform 102 is shown as including a request processor 110, a salience metric engine 112, an interaction data store 114, and a content data store 116. In an example, the request processor 110 processes various requests, such as may be received from the digital environment service 104 and the computing device 106. For example, the request processor 110 may process a request for a salience metric associated with content (e.g., as may be received from the salience processors 118 or 124 of the digital environment service 104 and the computing device 106, respectively). In some examples, the request may include an indication of the content to which the salience metric applies, an indication of the associated digital environment, and / or one or more demographics. As another example, the request may include at least a portion of a user profile that may be used to generate a salience metric accordingly. As another example, the request may be for content and / or a ranked set of content for use in adapting the digital environment.

[0021] The salience metric engine 112 can generate a salience metric and / or an indication of the location and / or content, which can be provided in response to the request. For example, the salience metric engine 112 can process telemetry data stored in the interaction data store 114 and / or content attributes associated with the candidate content (e.g., as may be received by the request processor 110, as may be obtained from the content data store 120 of the digital environment service 104, as may be obtained from the content data store 126 of the computing device 106, and / or as may be accessed from the content data store 116, among other examples). As discussed above, the salience metric engine 112 can use any of a variety of techniques to generate a salience metric and / or to determine the content and / or location from the candidate set based on the relevant factors, in accordance with aspects described herein. Additional examples of such aspects are discussed below with respect to the method 200 of FIG. 2.

[0022] As another example, the request processor 110 can aggregate telemetry data (e.g., from the digital environment service 104 and / or the computing device 106), which can be stored in the interaction data store 114. In some examples, the content data store 116 stores content associated with one or more digital environments and / or content that may not be associated with a digital environment, thereby adapting the digital environment according to external content (e.g., as compared to content associated with a given digital environment, as may be stored by the content data store 120 of the digital environment service 104).

[0023] The system 100 is further illustrated as including a digital environment service 104 that can be used to provide the digital environment. For example, the environment application 122 and the digital environment service 104 can act as a client and a server, respectively, in rendering the digital environment for display to a user of the computing device 106. In other examples, the environment application 122 can be operated locally such that the digital environment service 104 can distribute the environment application 122 to any of a variety of computing devices (e.g., to generate an offline digital environment). As another example, the digital environment service 104 can render at least a portion of the digital environment, which can be provided to the environment application 122 of the computing device 106 for display accordingly. Although the system 100 is illustrated as including a single computing device 106 having an associated environment application 122, it will be appreciated that in other examples, any number of computing devices and associated environment applications can be used.

[0024] The digital environment service 104 is illustrated as including a salience processor 118 and a content data store 120. In an example, the salience processor 118 may generate and / or obtain telemetry data (e.g., from the computing device 106), at least a portion of which may be provided for aggregation by the digital environment platform 102. In addition, the salience processor 118 may request salience metrics (e.g., as may be processed by the request processor 110, as described above) from the digital environment platform. For example, the salience processor 118 may request salience metrics for content (e.g., as may be associated with the digital environment) from the content data store 120 and / or request external content (e.g., as may be stored by the content data store 116) from the digital environment platform 102.

[0025] While generation of the saliency metrics is described above with respect to the saliency metric engine 112, it will be appreciated that in other examples, the digital environment service 104 can obtain machine learning models and / or weighting functions from the digital environment platform 102 and use them to generate the saliency metrics, such that at least some of the saliency metrics are generated at the digital environment service 104.

[0026] The computing device 106 may be any of a variety of computing devices, including, but not limited to, a mobile computing device, a tablet computing device, a laptop computing device, or a desktop computing device. In some examples, the computing device 106 may be an AR and / or VR headset or may be communicatively coupled to an AR and / or VR headset, among other examples. As shown, the computing device 106 includes an environment application 122, a salience processor 124, and a content data store 126. Aspects of the salience processor 124 and the content data store 126 may be similar to the salience processor 118 and the content data store 120, respectively, of the digital environment service 104, and thus will not necessarily be described again in detail below.

[0027] As described above, the environment application 122 can generate a digital environment for presentation to a user of the computing device 106. As another example, at least a portion of the digital environment can be rendered by the digital environment service 104. The salience processor 124 and / or the salience processor 118 can determine content for adapting the digital environment accordingly, according to aspects described herein. For example, the salience processor 124 can request salience metrics and / or content from the digital environment platform 102. In some examples, the request includes at least a portion of a user profile, as may be stored by the computing device 106. In other examples, such information may be stored by the digital environment service 104 and / or the digital environment platform 102. Thus, the environment application 122 can determine a spatial location for presenting content, determine content from a content set for presentation to the user, and / or adapt environment mechanics according to the determined content, among other examples.

[0028] Additionally or alternatively, in examples where the digital environment is streamed, recorded, or otherwise captured, the representation of the digital environment may be adapted according to the determined content, e.g., to include such content. In some examples, the content determined by the salience processor 124 may be overlaid or otherwise used to replace the content used to adapt the digital environment when rendered for display to the user.

[0029] Similar to the digital environment service 104, in other examples, it will be appreciated that the machine learning models and / or weighting functions for generating the salience metrics may be obtained from the digital environment platform 102, and at least some of the salience metrics and related content and / or location determinations may be made on the computing device 106.

[0030] Additionally, as discussed above, in other examples, any number of computing devices may be used. In such examples, the digital environment may be adapted for each user of the computing devices such that the same digital environment has different associated representations presented to each user. For example, a first user may view the digital environment as including a first content item, and a second user may view the digital environment as including a second content item. Similarly, different environmental mechanics may be adapted for different users, such as when a first user likes an environmental mechanic while a second user does not like the environmental mechanic. As a further example, a first instance of external content may be presented to a first user based on a first set of objects associated with the first user, and a second instance of external content may be presented to a second user based on a second set of objects associated with the second user.

[0031] 2 illustrates an overview of an example method 200 for generating a content saliency metric for adapting a digital environment according to aspects described herein. In an example, aspects of the method 200 are performed by a saliency metric engine, such as the saliency metric engine 112 discussed above with respect to FIG.

[0032] As shown, method 200 begins at operation 202, where a set of content attributes is obtained. For example, the set of attributes may include the relative usefulness and / or rarity of the content (e.g., as may be determined based on a content data store, such as content data store 116, content data store 120, and / or content data store 126 discussed above with respect to FIG. 1). Such attributes may be relative, for example, in relation to other content in the digital environment and / or content of one or more other digital environments. As another example, the set of attributes may include one or more attributes related to the content itself, such as type of gameplay mechanics, type of item, spatial location of the item within the digital environment, and / or associated difficulty level, among other examples.

[0033] Flow continues to operation 204 where a set of environmental attributes is obtained. In some examples, the set of environmental attributes is related to spatial locations within the digital environment where content may be presented, such as spatial locations proximate to a user (e.g., within a predefined distance and / or within a viewable area of ​​the user). As another example, the set of environmental attributes may include an indication regarding a user's progression within a storyline of the digital environment. Thus, it will be appreciated that the set of environmental attributes may be related to the digital environment itself (e.g., spatial locations and / or one or more associated environmental assets, such as items and / or sounds), as well as the user's perception of and / or user interaction with the digital environment.

[0034] Moving to operation 206, a set of user profile attributes is obtained. For example, the set of user profile attributes may relate to a user's play or interaction style, a number of user interactions with a given instance of the content and / or with similar types of content, a user's attentional habits (e.g., based on a user's viewable area determined from an AR / VR headset), and / or explicit and / or implicit user-specific preferences, among other examples.

[0035] At act 208, a population attribute set is obtained. In an example, the population attribute set includes similar attributes as obtained at act 206, but aggregated according to one or more population demographics. For example, the population attribute set can be determined based on telemetry data associated with one or more digital environments (e.g., as may be stored by an interaction data store, such as interaction data store 114). As an example, the population attribute can indicate a difficulty level and / or popularity associated with a game mechanic and / or spatial location where a user is located and / or to determine content.

[0036] Although exemplary attributes are described with respect to acts 202-208, it will be understood that in other examples, any of a variety of additional and / or alternative attributes can be used. As an example, in other examples, one or more of acts 202-208 can be omitted. Such attributes can be obtained based on telemetry data in an interaction data store (e.g., the interaction data store 114) and / or based on a user profile (e.g., as may be stored by a computing device, a digital environment service, and / or a digital environment platform), among other data sources, as part of a request for salience metrics or content for adapting the digital environment. In some examples, one or more attributes can be obtained based at least in part on other attributes. For example, it is possible to obtain user profile attributes based on environmental attributes (e.g., as when user profile attributes are associated with a spatial location of the digital environment) or population attributes based on user attributes.

[0037] Flow continues to operation 210, where a content saliency metric is generated based on the attributes obtained in operations 202-208. As described above, aspects of operation 210 may include generating the saliency metric using a machine learning model (e.g., trained based on user interactions with content in the digital environment and the associated attributes). As another example, operation 210 may include generating the saliency metric based on a set of weights associated with each attribute.

[0038] At operation 212, an indication of the generated saliency metric is provided. For example, the saliency metric may be provided in response to a request for the saliency metric (e.g., as may be received from a saliency processor, such as saliency processor 118 or 124 discussed above with respect to FIG. 1). As another example, the saliency metric may be provided and used to rank a content set, an example of which is described below with respect to method 300 of FIG.

[0039] Although method 200 is provided as an example of generating a saliency metric for content, it will be appreciated that a saliency metric can be generated for spatial locations as well based on location attributes (e.g., including rarity and / or usefulness of the location, proximity to other rare and / or desirable locations, as well as traffic, visibility, and / or ease of access to the location), environmental attributes, user profile attributes, and / or population attributes in accordance with aspects described herein. Method 200 ends at operation 212.

[0040] 3 illustrates an overview of an exemplary method 300 for adapting a digital environment according to aspects described herein. In an example, aspects of the method 300 are performed by a saliency metric engine or a saliency processor, such as the saliency metric engine 112 or the saliency processor 118 or 124, respectively, discussed above with respect to FIG. 1. For example, aspects of the method 300 may be performed by a digital environment service (e.g., the digital environment service 104) to adapt a digital environment for presentation to a user of a computing device, or similarly by a computing device (e.g., the computing device 106) in other examples. As another example, similar aspects may be performed by a digital environment platform to identify content for adapting the digital environment.

[0041] As shown, method 300 begins at operation 302, where a candidate content set is obtained. For example, at least a portion of the content set may be associated with the digital environment for which content is being determined. In another example, at least a portion of the content set may include external content (e.g., as may be associated with one or more other digital environments). The content set may be obtained from any of a variety of sources, such as the content data stores 116, 120, and / or 126 of the digital environment platform 102, the digital environment service 104, and the computing device 106, respectively, discussed above with respect to FIG.

[0042] Flow continues to operation 304 where a saliency metric is determined for the content set. For example, operation 304 may include generating a saliency metric for each instance of content in the content set obtained in operation 302. Aspects of operation 304 may be similar to those discussed above with respect to method 200 of FIG.

[0043] At operation 306, the content set is ranked based on the saliency metric generated at operation 304. As discussed above, the saliency metric may indicate relative saliency in some examples and absolute saliency in other examples. In some examples, operation 306 further includes filtering one or more instances of the content (e.g., to filter saliency metrics below a specified threshold or to generate a ranked content set having a specified number of items).

[0044] Moving to operation 308, content is determined from the ranked content set. In some examples, one or more highest ranked instances of content are selected, or as another example, content may be randomly selected from instances of content having a saliency metric above a predefined threshold. Thus, it will be appreciated that any of a variety of techniques may be used to determine content from the ranked content set.

[0045] Flow continues to operation 310, where the digital environment is adapted according to the content determined in operation 308. For example, operation 310 may include adapting the digital environment to include 2D or 3D assets or NPCs for presentation to the user. As another example, the environmental mechanics of the digital environment may be incorporated or modified, for example, to present a different set of RPG story choices to the user, and / or to have different difficulty levels, prominence, and / or length. Although an exemplary adaptation is described, it will be appreciated that the digital environment may be adapted according to any of a variety of alternative or additional techniques. In some examples, operation 310 may include providing an indication of the determined content, such as when the digital environment is rendered or a representation is otherwise created by a remote computing device (e.g., when aspects of method 300 are performed by a digital environment platform or digital environment service). Method 300 ends at operation 310.

[0046] 4 illustrates an overview of another exemplary method 400 for adapting a digital environment according to aspects described herein. In an example, aspects of the method 400 are performed by a saliency metric engine or a saliency processor, such as the saliency metric engine 112 or the saliency processor 118 or 124, respectively, discussed above with respect to FIG. 1. For example, aspects of the method 400 may be performed by a digital environment service (e.g., the digital environment service 104) to determine where to present content within the digital environment. As another example, such aspects may similarly be performed by a computing device (e.g., the computing device 106) or a digital environment platform (e.g., the digital environment platform 102).

[0047] Method 400 begins at operation 402, where a set of locations is determined. In an example, the set of locations may be determined based on a location of a user within a digital environment. For example, the set of locations may include surfaces proximate to the user (e.g., within a predefined radius or area or within the user's visible area). It will be appreciated that a spatial location may have a corresponding location within the user's physical environment, such as when the digital environment is overlaid on the physical environment via an AR headset.

[0048] Flow proceeds to operation 404 where a saliency metric is determined for the set of places. For example, operation 404 may include generating a saliency metric for each place in the set of places determined in operation 402. Aspects of operation 404 may be similar to those discussed above with respect to method 200 of FIG. 2, such that the saliency metric is generated based on one or more location attributes, environmental attributes, user profile attributes, and / or population attributes, in accordance with aspects described herein.

[0049] At operation 406, the set of places is ranked based on the saliency metric generated at operation 404. As described above, the saliency metric may indicate relative saliency in some examples and absolute saliency in other examples. In some examples, operation 406 further includes filtering one or more places (e.g., to filter saliency metrics below a specified threshold or to generate a ranked set of places having a specified number of items).

[0050] Moving to operation 408, a location is determined from the ranked set of locations. In some examples, the highest ranked location or locations are selected, or as another example, a location may be randomly selected from locations having a saliency metric above a predefined threshold. Thus, it will be appreciated that any of a variety of techniques may be used to determine a location from the ranked set of locations.

[0051] Flow continues to operation 410, where content is determined for adapting the digital environment (e.g., at the location determined at operation 408). For example, aspects of operation 410 may be similar to those discussed above with respect to operations 302-308 of method 300 of FIG. 3, and thus are not necessarily detailed again below. In an example, a content set from which the content is determined is generated based at least in part on the location determined at operation 408. Thus, different content sets may be generated for different locations within the digital environment.

[0052] At operation 412, the digital environment is adapted using the content determined at operation 410 according to the location determined at operation 408. For example, operation 412 may include adapting the digital environment to include 2D or 3D assets or NPCs for presentation to the user at the determined location. As another example, environmental mechanics may be incorporated or modified in association with the determined location. In some examples, the content determined at operation 410 is overlaid within the user's physical environment (e.g., as if the user were wearing an AR headset).

[0053] While an exemplary adaptation is described, it will be appreciated that the digital environment can be adapted according to any of a variety of alternative or additional techniques. In some examples, operation 412 may include providing an indication of the determined location and associated content, such as when the digital environment is rendered or a representation is otherwise created by a remote computing device (e.g., when aspects of method 400 are performed by a digital environment platform or digital environment service). Additionally, while method 400 is described as an example in which content is subsequently determined based on a location having a high salience metric, it will be appreciated that in other examples, a location can be subsequently determined based on content having a high salience metric. Method 400 ends at operation 412.

[0054] Figures 5-7 and the associated discussion provide a discussion of a variety of operating environments in which aspects of the present disclosure may be implemented. However, the devices and systems shown and discussed with respect to Figures 5-7 are for purposes of example and illustration, and are not intended to limit the vast number of computing device configurations that may be utilized to implement aspects of the present disclosure as described herein.

[0055] 5 is a block diagram illustrating the physical components (e.g., hardware) of a computing device 500 capable of implementing aspects of the present disclosure. The computing device components described below may be suitable for the computing devices described above, including the digital environment platform 102, the digital environment service 104, and / or the computing device 106 discussed above with respect to FIG. 1. In a basic configuration, the computing device 500 may include at least one processing unit 502 and a system memory 504. Depending on the configuration and type of computing device, the system memory 504 may include, but is not limited to, volatile storage (e.g., random access memory), non-volatile storage (e.g., read-only memory), flash memory, or any combination of such memory.

[0056] The system memory 504 may include an operating system 505 and one or more program modules 506 suitable for executing software applications 520, such as one or more components supported by the system described herein. By way of example, the system memory 504 may store an environmental application 524 and a saliency metric engine 526. The operating system 505 may be suitable for controlling the operation of the computing device 500, for example.

[0057] Moreover, embodiments of the present disclosure may be implemented in conjunction with graphics libraries, other operating systems, or any other application programs, and are not limited to any particular application or system. This basic configuration is illustrated in FIG. 5 by the components within dashed line 508. Computing device 500 may have additional features or functionality. For example, computing device 500 may also include additional data storage devices (removable and / or non-removable), such as, for example, magnetic disks, optical disks, or tape. Such additional storage is illustrated in FIG. 5 by removable storage 509 and non-removable storage 510.

[0058] As mentioned above, a number of program modules and data files can be stored in the system memory 504. While executing on the processing unit 502, the program modules 506 (e.g., applications 520) can perform processes including, but not limited to, aspects as described herein. Other program modules that can be used in accordance with aspects of the present disclosure can include email and contact applications, word processing applications, spreadsheet applications, database applications, slide presentation applications, drawing or computer-aided application programs, and the like.

[0059] Additionally, embodiments of the present disclosure may be implemented on electrical circuits including discrete electronic elements, packaged or integrated electronic chips including logic gates, microprocessor-based circuits, or single chips including electronic elements or microprocessors. For example, embodiments of the present disclosure may be implemented via a system-on-chip (SOC) where each or many of the components shown in FIG. 5 may be integrated onto a single integrated circuit. Such an SOC device may include one or more processing units, graphics units, communication units, system virtualization units, and various application functionalities, all integrated (or "burned") onto the chip substrate as a single integrated circuit. When operating via a SOC, the functionality described herein with respect to the client's ability to switch protocols may operate via application-specific logic integrated with other components of the computing device 500 on a single integrated circuit (chip). Also, embodiments of the present disclosure may be implemented using other technologies capable of performing logical operations such as, for example, AND, OR, and NOT, including, but not limited to, mechanical, optical, fluidic, and quantum technologies. Additionally, embodiments of the present disclosure may be implemented within a general-purpose computer or any other circuit or system.

[0060] The computing device 500 may also have one or more input devices 512, such as a keyboard, mouse, pen, sound or voice input device, touch or swipe input device, etc. It may also include output devices 514, such as a display, speakers, printer, etc. The aforementioned devices are examples and other devices may be used. The computing device 500 may include one or more communication connections 516 that enable communication with other computing devices 550. Examples of suitable communication connections 516 include, but are not limited to, radio frequency (RF) transmitter, receiver and / or transceiver circuitry, universal serial bus (USB), parallel and / or serial ports.

[0061] The term computer readable medium as used herein may include computer storage media. Computer storage media may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, or program modules. System memory 504, removable storage 509, and non-removable storage 510 are all examples of computer storage media (e.g., memory storage). Computer storage media may include RAM, ROM, Electrically Erasable Read Only Memory (EEPROM), Flash memory or other memory technology, CD-ROM, Digital Versatile Disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other article of manufacture that can be used to store information and that can be accessed by computing device 500. Any such computer storage media may be part of computing device 500. Computer storage media does not include carrier waves or other propagated or modulated data signals.

[0062] Communication media may be embodied by computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and includes any information delivery media. The term "modulated data signal" may describe a signal that has one or more characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media may include wired media, such as a wired network or direct-wired connection, and wireless media, such as acoustic, radio frequency (RF), infrared and other wireless media.

[0063] 6A and 6B illustrate a mobile computing device 600, such as a mobile phone, a smartphone, a wearable computer (such as a smart watch), a tablet computer, a laptop computer, and the like, on which embodiments of the present disclosure can be implemented. In some aspects, the client can be a mobile computing device. Referring to FIG. 6A, one aspect of a mobile computing device 600 for implementing those aspects is shown. In a basic configuration, the mobile computing device 600 is a handheld computer having both input and output elements. The mobile computing device 600 typically includes a display 605 and one or more input buttons 610 that allow a user to input information into the mobile computing device 600. The display 605 of the mobile computing device 600 can also function as an input device (e.g., a touch screen display).

[0064] If included, optional side input element 615 allows for further user input. Side input element 615 may be a rotary switch, a button, or any other type of manual input element. In alternative aspects, mobile computing device 600 may incorporate more or fewer input elements. For example, display 605 may not be a touch screen in some embodiments.

[0065] In yet another alternative embodiment, the mobile computing device 600 is a portable phone system, such as a cellular phone. The mobile computing device 600 may also include an optional keypad 635. The optional keypad 635 may be a physical keypad or a "soft" keypad generated on a touch screen display.

[0066] In various embodiments, the output elements include a display 605 for presenting a graphical user interface (GUI), a visual indicator 620 (e.g., a light emitting diode), and / or an audio transducer 625 (e.g., a speaker). In some aspects, the mobile computing device 600 incorporates a vibration transducer for providing haptic feedback to the user. In yet another aspect, the mobile computing device 600 incorporates input and / or output ports, such as an audio input (e.g., a microphone jack), an audio output (e.g., a headphone jack), and a video output (e.g., an HDMI port), for transmitting signals to or receiving signals from external devices.

[0067] 6B is a block diagram illustrating an architecture of one aspect of a mobile computing device. That is, the mobile computing device 600 can incorporate a system (e.g., architecture) 602 for implementing some aspects. In one embodiment, the system 602 is implemented as a "smartphone" capable of executing one or more applications (e.g., browser, email, calendaring, contact manager, messaging client, games and media client / player). In some aspects, the system 602 is integrated as a computing device such as an integrated personal digital assistant (PDA) and wireless phone.

[0068] One or more application programs 666 may be loaded into memory 662 and run on or in association with operating system 664. Examples of application programs include a phone dialer program, an email program, a personal information manager (PIM) program, a word processing program, a spreadsheet program, an Internet browser program, a messaging program, and the like. System 602 also includes a non-volatile storage area 668 in memory 662. Non-volatile storage area 668 may be used to store persistent information that should not be lost when system 602 is powered down. Application programs 666 may use and store information in non-volatile storage area 668, such as emails or other messages used by an email application, and the like. A synchronization application (not shown) also resides on system 602 and is programmed to interact with a corresponding synchronization application resident on the host computer to keep information stored in non-volatile storage area 668 synchronized with corresponding information stored on the host computer. Of course, other applications may also be loaded into the memory 662 and executed on the mobile computing device 600 described herein.

[0069] The system 602 includes a power source 670, which may be implemented as one or more batteries. The power source 670 may further include an external power source, such as an AC adapter or a powered docking cradle, that replenishes or charges the battery.

[0070] The system 602 may also include a radio interface layer 672 that performs the functions of transmitting and receiving radio frequency communications. The radio interface layer 672 facilitates wireless connectivity between the system 602 and the "outside world" via a communications carrier or service provider. Transmissions to and from the radio interface layer 672 are performed under the control of the operating system 664. In other words, communications received by the radio interface layer 672 can be disseminated to application programs 666 via the operating system 664, and vice versa.

[0071] The visual indicator 620 can be used to provide visual notifications and / or the audio interface 674 can be used to generate audible notifications via the audio transducer 625. In the embodiment shown, the visual indicator 620 is a light emitting diode (LED) and the audio transducer 625 is a speaker. These devices can be directly coupled to the power source 670 and when activated, they remain on for as long as dictated by the notification mechanism, even though the processor 660 and other components may shut down to conserve battery power. The LED can be programmed to remain on indefinitely until the user takes an action to indicate that they have powered on the device. The audio interface 674 is used to provide audible signals to and receive audible signals from the user. For example, in addition to coupling to the audio transducer 625, the audio interface 674 can also be coupled to a microphone to receive audible input, such as to facilitate a telephone conversation. According to an embodiment of the present disclosure, the microphone can also serve as an audio sensor to facilitate control of notifications, as described below. The system 602 may further include a video interface 676 that enables operation of the on-board camera 630 for recording still images, video streams, and the like.

[0072] A mobile computing device 600 implementing system 602 may have additional features or functionality. For example, mobile computing device 600 may also include additional data storage devices (removable and / or non-removable) such as magnetic disks, optical disks, or tape. Such additional storage is illustrated in FIG. 6B by non-volatile storage area 668.

[0073] Data / information generated or captured by the mobile computing device 600 and stored via the system 602 may be stored locally on the mobile computing device 600, as described above, or the data may be stored in any number of storage media accessible to the device via the wireless interface layer 672, or via a wired connection between the mobile computing device 600 and a separate computing device associated with the mobile computing device 600 (e.g., a server computer in a distributed computing network such as the Internet). Of course, such data / information may be accessed via the mobile computing device 600 via the wireless interface layer 672 or via a distributed computing network. Similarly, such data / information may be readily transferred between computing devices for storage and use in accordance with well-known data / information transfer and storage means, including electronic mail and collaborative data / information sharing systems.

[0074] 7 illustrates one aspect of an architecture of a system for processing data received at a computing system from a remote source, such as a personal computer 704, a tablet computing device 706, or a mobile computing device 708, as described above. Content displayed at the server device 702 may be stored in different communication channels or other storage types. For example, a directory service 722, a web portal 724, a mailbox service 726, an instant messaging store 728, or a social networking site 730 may be used to store various documents.

[0075] A client in communication with the server device 702 may employ an environmental application 720 and / or the server device 702 may employ a saliency metric engine 721. The server device 702 may provide data to and from client computing devices, such as a personal computer 704, a tablet computing device 706, and / or a mobile computing device 708 (e.g., a smartphone) over a network 715. As an example, the computer system described above may be embodied in a personal computer 704, a tablet computing device 706, and / or a mobile computing device 708 (e.g., a smartphone). Any of these embodiments of computing devices may obtain content from the store 716 in addition to receiving graphical data that may be used for either pre-processing in the graphics originating system or post-processing in the receiving computing system.

[0076] In addition, it will be appreciated that aspects and functionality described herein can operate across distributed systems (e.g., cloud-based computing systems), and application functionality, memory, data storage and retrieval, and various processing functions can operate remotely from one another over a distributed computing network, such as the Internet or an intranet. Various types of user interfaces and information can be displayed via on-board computing device displays or via remote display units associated with one or more computing devices. For example, various types of user interfaces and information can be displayed and interacted with on a wall surface onto which the various types of user interfaces and information are projected. Interactions with multiple computing systems on which embodiments of the present invention can be implemented include keystroke entry, touch screen entry, voice or other audio entry, gesture entry where the associated computing devices are equipped with detection (e.g., camera) functionality to capture and interpret user gestures to control functionality of the computing devices, and the like.

[0077] As can be appreciated from the foregoing disclosure, one aspect of the present technology relates to a system including at least one processor and a memory storing instructions that, when executed by the at least one processor, cause the system to perform a set of operations. The set of operations includes obtaining a set of factors associated with a digital environment, where the digital environment is an augmented reality digital environment; determining content from the candidate content set based on the set of factors and based on a salience metric associated with each instance of content in the candidate content set; and adapting the digital environment according to the determined content, where the adapting includes at least one of including the determined content in the digital environment for presentation to a user and modifying environmental mechanics of the digital environment based on the determined content. In one example, the candidate content set includes content associated with another digital environment or content not associated with the digital environment. In another example, the salience metric defines a relative salience of the determined content associated with a user compared to another instance of content in the candidate content set. In a further example, the set of operations includes determining a location from a set of candidate locations in the digital environment based on a saliency metric for each candidate location in the set of candidate locations, and including the determined content includes including the determined content in the determined location in the digital environment. In yet another example, the saliency metric associated with the instance of the candidate content is generated using a machine learning model, the machine learning model is trained using training data including user interactions with content of the digital environment, and the user interactions are annotated with one or more associated factors. In a further example, the set of factors includes content attributes associated with the determined content, environmental attributes associated with the digital environment, user profile attributes associated with the user, and population attributes associated with a population of the digital environment. In another example, the set of factors is obtained based on telemetry data associated with the digital environment.

[0078] In another aspect, the technology relates to a method for adapting a digital environment based on a saliency metric. The method includes generating a representation of the digital environment for presentation to a user, requesting content from a digital environment platform for adapting the digital environment for the user, receiving an indication of the content from the digital environment platform, and adapting the digital environment according to the received indication of the content, the adapting including at least one of including the content in the representation of the digital environment and modifying environmental mechanics of the digital environment based on the content. In one example, requesting the content to adapt the digital environment for the user includes providing an indication of a user profile associated with the user to the digital environment platform. In another example, the received content is associated with another digital environment. In a further example, the indication of the received content is a first instance of the content, and the method further includes generating a stream of the representation of the digital environment, and generating the stream includes determining a second instance of the content and adapting the stream to include the second instance of the content to adapt the stream. In yet another example, the stream is adapted to include the second instance of the content in place of the first instance of the content. In yet another example, the first instance of the content is determined based at least in part on a user and the second instance of the content is determined based at least in part on a population associated with the digital environment.

[0079] In a further aspect, the technology relates to another method for adapting a digital environment based on a saliency metric. The method includes: obtaining a set of factors associated with the digital environment, the digital environment being an augmented reality digital environment; determining content from the candidate content set based on the set of factors based on a saliency metric associated with each instance of content in the candidate content set; and adapting the digital environment according to the determined content, the adapting including at least one of including the determined content in the digital environment for presentation to the user and modifying environmental mechanics of the digital environment based on the determined content. In one example, the candidate content set includes content associated with another digital environment or content not associated with the digital environment. In another example, the saliency metric defines a relative saliency of the determined content associated with the user compared to another instance of content in the candidate content set. In a further example, the method includes determining a location from a candidate location set of the digital environment based on the saliency metric of each candidate location in the candidate location set, and including the determined content includes including the determined content in the determined location in the digital environment. In yet another example, the saliency metrics associated with the instances of candidate content are generated using a machine learning model, the machine learning model being trained using training data including user interactions with content of the digital environment, the user interactions being annotated with one or more relevant factors. In a further example, the set of factors includes content attributes associated with the determined content, environmental attributes associated with the digital environment, user profile attributes associated with the user, and population attributes associated with a population of the digital environment. In another example, the set of factors is obtained based on telemetry data associated with the digital environment.

[0080] Aspects of the present disclosure have been described above with reference to block diagrams and / or operational illustrations of, for example, methods, systems, and computer program products according to aspects of the present disclosure. The functions / acts noted in the blocks may occur out of the order noted in the flowcharts. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or in the reverse order, depending on the functionality / acts involved.

[0081] The description and illustration of one or more aspects provided in this application are not intended to limit or restrict the scope of the claimed disclosure in any way. The aspects, examples, and details provided in this application are believed to be sufficient to convey ownership and enable others to make and use the claimed aspects of the disclosure. The claimed disclosure should not be construed as being limited to the aspects, examples, or details provided in this application. Various features (both structural and methodological), whether shown and described in combination or shown and described separately, are intended to be selectively included or omitted to produce an embodiment having a set of specific features. Although the description and illustration of the application have been provided, those skilled in the art may envision variations, modifications, and alternatives that fall within the spirit of the broader aspects of the general inventive concept embodied in this application without departing from the broader scope of the claimed disclosure.

Claims

1. at least one processor; a memory storing instructions that, when executed by said at least one processor, cause the system to perform a series of operations; A system comprising: The series of operations is obtaining a set of factors associated with a digital environment, the digital environment being an augmented reality digital environment; determining content from the candidate content set based on a salience metric associated with each instance of content in the candidate content set based on the set of factors; determining a location from the set of candidate locations in the digital environment based on a saliency metric for each candidate location in the set; adapting the digital environment at the determined location according to the determined content; including the determined content in the digital environment for presentation to a user; and modifying environmental mechanics of the digital environment based on the determined content; and Including, the system.

2. The system described in claim 1, wherein the candidate content set includes content associated with another digital environment or content not associated with the digital environment.

3. The system described in claim 1, wherein the salience metric defines the relative salience of the determined content associated with the user compared to other instances of content in the candidate content set.

4. 10. The system of claim 1, wherein the salience metrics associated with instances of candidate content are generated using a machine learning model, the machine learning model being trained using training data including user interactions with content of the digital environment, the user interactions being annotated with one or more relevant factors.

5. The factor set: content attributes associated with the determined content; and environmental attributes associated with the digital environment; user profile attributes associated with the user; population attributes associated with the population of digital environments; and The system of claim 1 , comprising:

6. The system described in claim 1, wherein the set of factors is obtained based on telemetry data associated with the digital environment.

7. The system described in claim 1, wherein the salience metric for each candidate location is determined based on the user.

8. 1. A method for adapting a digital environment based on a saliency metric, comprising: generating a representation of the digital environment for presentation to a user; requesting content for adapting the digital environment for the user from a digital environment platform; receiving an indication of content from the digital environment platform; determining a location from the set of candidate locations in the digital environment based on a saliency metric for each candidate location in the set; adapting the digital environment at the determined location according to an indication of the received content; including the content in the representation of the digital environment; and modifying environmental mechanics of the digital environment based on the content; and A method comprising:

9. 10. The method of claim 8, wherein requesting the content to adapt the digital environment for the user includes providing the digital environment platform with an indication of a user profile associated with the user.

10. The method of claim 8, wherein the received content is associated with another digital environment.

11. the received indication of content is a first instance of content; The method further includes generating a stream of the representation of the digital environment, wherein generating the stream comprises: determining a second instance of content for adapting the stream; Adapting the stream to include the second instance of content; The method of claim 8, comprising:

12. The method of claim 11, wherein the stream is adapted to include the second instance of content instead of the first instance of content.

13. The method of claim 12, wherein the first instance of content is determined based at least in part on the user; The method of claim 11 , wherein the second instance of content is determined based at least in part on a population associated with the digital environment.

14. 1. A method for adapting a digital environment based on a saliency metric, comprising: obtaining a set of factors associated with a digital environment, the digital environment being an augmented reality digital environment; determining content from the candidate content set based on a salience metric associated with each instance of content in the candidate content set based on the set of factors; determining a location from the set of candidate locations in the digital environment based on a saliency metric for each candidate location in the set; adapting the digital environment at the determined location according to the determined content; including the determined content in the digital environment for presentation to a user; and modifying environmental mechanics of the digital environment based on the determined content; and A method comprising:

15. The method of claim 14 , wherein the set of candidate content includes content associated with another digital environment or content not associated with a digital environment.

16. The method of claim 14 , wherein the salience metric defines a relative salience of the determined content associated with the user compared to other instances of content in the candidate content set.

17. The method of claim 14, wherein the salience metrics associated with instances of candidate content are generated using a machine learning model, the machine learning model being trained using training data including user interactions with content in the digital environment, and the user interactions being annotated with one or more relevant factors.

18. The set of factors is: content attributes associated with the determined content; and environmental attributes associated with the digital environment; user profile attributes associated with the user; population attributes associated with the population of digital environments; and 15. The method of claim 14, comprising:

19. The method of claim 14 , wherein the set of factors is obtained based on telemetry data associated with the digital environment.

20. The method of claim 14, wherein the salience metric for each candidate location is determined based on the user.

Images