Saliency-based digital environment adaptation

Through the method based on significance measurement, the problem of insufficient user preference insight in the digital environment is solved, personalized digital environment adaptation is achieved, and the targetedness and satisfaction of the user experience is improved.

CN119013703APending Publication Date: 2024-11-22MICROSOFT TECHNOLOGY LICENSING LLC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202380032587.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-03-31
Filing Date
2023-02-16
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The prior art is difficult to provide different users with a personalized digital environment experience, especially when crossing multiple digital environments, making it difficult to understand user preferences, resulting in adaptation difficulties.

Method used

Through a method based on significance metrics, significance metrics are generated based on content attributes, environment attributes, user profile attributes and group attributes, which are used to sort and adapt content and locations in the digital environment, thereby providing users with a personalized experience.

Benefits of technology

It realizes personalized digital environment adaptation based on user characteristics, improves the pertinence and satisfaction of user experience, and can dynamically adjust content and environmental mechanisms to adapt to the preferences of different users.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119013703A_ABST
    Figure CN119013703A_ABST
Patent Text Reader

Abstract

In an example, a significance metric may be determined for a location of a content instance and / or a digital environment. Accordingly, a set of candidate content and / or locations may be ranked according to the associated significance metrics, such that content and / or associated locations for adapting the digital environment for a given user may be determined from the set of candidate content. For example, a digital environment may be adapted to present two-dimensional or three-dimensional assets to a user. As another example, a gaming mechanism of a digital environment may be altered. In an example, content from another digital environment may be identified and used to adapt the digital environment to incorporate external content. Thus, significance metrics associated with the location or instance of the content may be used to programmatically generate relative or absolute metrics, which are utilized to evaluate and adapt aspects of the digital environment.
Need to check novelty before this filing date? Find Prior Art

Description

Background Art

[0001] Users may be presented with and / or interact with content of a digital environment. However, different users may find different content more appealing. Therefore, adapting a digital environment to account for user-specific differences and improve the associated user experience may be difficult.

[0002] The embodiments have been described with respect to these and other general considerations.Furthermore, although relatively specific problems have been discussed, it should be understood that the embodiments should not be limited to solving the specific problems identified in the context. Summary of the invention

[0003] In an example, a saliency metric may be determined for a content instance and / or location of a digital environment. Thus, a set of candidate content and / or locations may be ranked according to the associated saliency metric, such that content and / or associated locations for adapting a digital environment for a given user may be determined from a set of candidate content. For example, a digital environment may be adapted to present a two-dimensional or three-dimensional asset to a user. As another example, the game mechanics of a digital environment may be altered. In an example, content from another digital environment may be identified and used to adapt the digital environment, thereby incorporating external content. Thus, a saliency metric associated with a location or instance of content may be used to programmatically generate a relative or absolute metric that is utilized to evaluate and adapt aspects of a digital environment.

[0004] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Non-limiting and non-exhaustive examples are described with reference to the following figures.

[0006] Figure 1 An overview diagram of an example system for saliency-based digital environment adaptation in accordance with various aspects described herein is shown.

[0007] Figure 2 An overview diagram of an example method for generating a content saliency metric for adapting a digital environment in accordance with various aspects described herein is shown.

[0008] Figure 3 An overview diagram of an example method for adapting a digital environment according to various aspects described herein is shown.

[0009] Figure 4 An overview diagram of another example method for adapting a digital environment according to aspects described herein is shown.

[0010] Figure 5 is a block diagram illustrating example physical components of a computing device with which aspects of the present disclosure may be implemented.

[0011] Fig. 6A and Figure 6B is a simplified block diagram of a mobile computing device with which aspects of the present disclosure may be implemented.

[0012] Figure 7 is a simplified block diagram of a distributed computing system in which aspects of the present disclosure may be implemented. DETAILED DESCRIPTION

[0013] In the following detailed description, reference is made to the accompanying drawings which form a part thereof, and specific embodiments or examples are shown by way of illustration in the accompanying drawings. These aspects may be combined, other aspects may be utilized, and structural changes may be made without departing from the present disclosure. The embodiments may be implemented as methods, systems, or devices. Therefore, the embodiments may take the form of hardware implementations, complete software implementations, or implementations combining software and hardware aspects. Therefore, the following detailed description should not be considered restrictive, and the scope of the present disclosure is defined by the appended claims and their equivalents.

[0014] In an example, the content of a digital environment can be presented to a user. Exemplary digital environments include, but are not limited to, video game environments, virtual collaborative environments, alternative reality (AR) environments, and / or virtual reality (VR) environments. For example, a digital environment can be part of a virtual reality. The content of a digital environment can include environmental assets (e.g., sounds, text overlays, two-dimensional (2D) or three-dimensional (3D) models or game items, scenery, weapons, non-player characters (NPCs), and player models) and environmental mechanisms (e.g., related to a set of choices for the behavior, mini-games, and / or role-playing games of a given asset). As another example, a non-replaceable token can be or can be otherwise associated with a content instance. However, it may be difficult to create a personalized experience for a user within a digital environment (e.g., based on a user profile and associated implicit or explicit preferences). In addition, adapting a digital environment based on content associated with another digital environment (e.g., to provide recommendations across different digital environments) may be similarly difficult, for example, due to little or no insight into user preferences across multiple digital environments.

[0015] Therefore, aspects of the present disclosure relate to digital environment adaptation based on saliency. In an example, a digital environment can be adapted based on any of a variety of factors, including, but not limited to: content attributes (e.g., related to the desirability and / or scarcity of the content), environment attributes (e.g., related to the spatial location within the digital environment where the content can be presented), user profile attributes (e.g., related to the user's play style, the number of user interactions with the content and / or similar types of content, attention habits, such as based on the user's field of view determined from an AR / VR helmet, and / or explicit and / or implicit user-specific preferences), and / or group attributes (e.g., user profile attributes of a larger group of users). Therefore, a saliency measure can be determined for the content and / or location of the digital environment based on such factors. Although example factors are described herein, it should be understood that any of a variety of additional and / or alternative factors can be used in other examples.

[0016] As an example, content from a set of content can be determined based on associated saliency metrics, where the set of content is ranked according to a saliency metric for each content instance. Thus, the highest ranked saliency metric can be determined for presentation to the user. For example, 2D or 3D assets can be presented to the user and / or environmental mechanisms can be incorporated or modified (e.g., to present different sets of story options to the user and / or with different difficulty levels, saliency, and / or lengths). As another example, similar techniques can be used to determine the spatial locations at which content is presented to the user (e.g., within a digital environment that can be overlaid on the user's physical environment), such that a set of spatial locations can be ranked according to an associated saliency metric. Thus, the experience provided by the digital environment can vary between users, thereby providing a personalized experience to each user.

[0017] Although example adaptations are described, it should be understood that any of a variety of additional or alternative adaptations may be performed in other examples. For example, the representation of the digital environment may be adapted based on the saliency metric such that content with a higher saliency metric is rendered at a higher quality level (e.g., using more detailed textures, more detailed 3D models, and / or encoding a higher bitrate video), while content with a lower saliency metric is rendered at a lower quality level. As another example, content of the digital environment with a saliency metric above a predetermined threshold may be cached in system memory, while content with a saliency metric below the predetermined threshold may be loaded into system memory when needed to render the digital environment.

[0018] As another example, it may be determined based on a salience metric that an NPC should be spawned within a digital environment, such as when associated environmental factors and / or group factors indicate a location with increased difficulty. As a result, the NPC may assist the user, thereby reducing the likelihood of user frustration. Similarly, a recommendation may be presented to request assistance from another user of the digital environment, and in response to accepting the recommendation, the other user may be identified and communication between the users may be established, enabling the users to help each other. In contrast, if the user factor indicates an increased capability of a game mechanic associated with the location, the NPC may have a relatively reduced salience metric such that the NPC may not be spawned, or an NPC with reduced utility may be spawned in such an example.

[0019] Furthermore, while there are saliency metrics in the context of adapting digital environments, it will be appreciated that there may be additional or alternative uses. As described above, the associated saliency metric may be used to compare a first content item with a second content item, thereby establishing a relative "value" of the first content item and the second content item. In other examples, such values ​​need not be relative, but may be estimated or predicted values, such as where the content has not yet been introduced into a digital environment. Thus, according to the various aspects described herein, a saliency metric may indicate the relevance, desirability, estimated interest, and / or perceived value of a given instance of content and / or location. As another example, a saliency metric may be used to assign a value associated with a spatial location of a digital environment, such that corresponding content may be identified based on the assigned value of the spatial location.

[0020] It should be understood that the content used to adapt the digital environment need not be limited to the content originally associated with the digital environment. For example, content associated with different digital environments can be determined according to the various aspects described herein and presented accordingly within the digital environment. For example, such external content can be game items from different games, or can include target content provided by a third party, among other examples. Therefore, compared to instances in which the digital environment itself is presented as a recommendation (e.g., a list of games that similar players also like), content from within different digital environments themselves can be presented to the user (e.g., if you like this mechanism or item, you can try similar mechanisms or items from different digital environments).

[0021] Additionally, similar techniques may be used to adapt a feed, recording, stream, or other reproduction of a digital environment (e.g., in addition to an initial rendering presented to a user). For example, when rendered for a user, the digital environment may be adapted according to aspects described herein, and the rendered stream (e.g., as may be provided by a streaming platform to another computing device) may be further adapted with similar or different content. For example, the stream of the digital environment may be adapted to include external content (e.g., based on environmental factors associated with the stream, such as a portion of a story associated with the stream). In an example, the digital environment may be adapted based at least in part on a set of attributes associated with a first user (e.g., for whom the digital environment is rendered), while the stream of the digital environment may be adapted based at least in part on a set of attributes associated with a second user or group of users (e.g., to whom the stream of the digital environment is provided).

[0022] As described above, the digital environment may be adjusted based on any of a variety of factors, including, but not limited to, content attributes, environment attributes, user profile attributes, and / or demographic attributes. In an example, data associated with such factors may be maintained locally on the user's device (e.g., an application associated with the digital environment may maintain user profile data local to the computing device) and / or may be aggregated by the digital environment platform. For example, the digital environment platform may aggregate telemetry data associated with one or more digital environments such that the telemetry data may be used to determine attributes associated with a user and / or one or more groups of users (e.g., associated with a geographic region, region, or any of a variety of other demographics of one or more digital environments).

[0023] Any of a variety of techniques may be used to generate a saliency measure for content based on such factors, including but not limited to a statistical model (e.g., in which factors are combined according to a set of weights) or a machine learning model. For example, a machine learning model may be trained based on a user's interaction with content of a digital environment annotated according to associated factors (e.g., as may be determined based on telemetry data). In some cases, the machine learning model may be tuned for a particular user, or may otherwise be user-specific or group-specific, among other examples. As an example, a machine learning model may be trained to determine content or a location from a set of candidates to maximize the likelihood that the user will engage with the determined content or location. Such techniques may be used to generate a relative saliency measure (e.g., for comparing a first content item to a second content item) or an absolute saliency measure, among others. In an example, a saliency measure may include a target content value or rating.

[0024] Figure 1An overview diagram of an example system 100 for saliency-based digital environment adaptation according to various aspects described herein is shown. As shown, the system 100 includes a digital environment platform 102, a digital environment service 104, a computing device 106, and a network 108. In an example, the digital environment platform 102, the digital environment service 104, and / or the computing device 106 communicate via the network 108, which may include a local area network, a wireless network, or the Internet, or any combination thereof, etc.

[0025] The digital environment platform 102 can aggregate telemetry data associated with one or more digital environments (e.g., can be associated with the digital environment service 104). Figure 1 104, but it should be understood that in other examples, the digital environment service can implement aspects similar to those of the digital environment platform 102. Therefore, aspects of the present disclosure can be similarly applied in the context of a single digital environment (e.g., which can be provided by a digital environment service similar to the digital environment service 104) or multiple digital environments. For example, although the digital environment platform 102 is shown as including the saliency metric engine 112 and the interaction data repository 114, the digital environment service 104 can additionally or alternatively include various aspects, such as when the digital environment service 104 generates saliency metrics independently of the digital environment platform 102 or in addition to the digital environment platform 102. Similarly, the digital environment platform 102 can incorporate aspects similar to the digital environment service 104, such that in addition to performing telemetry data aggregation and data processing to generate saliency metrics as described herein, the digital environment platform 102 also provides a digital environment platform.

[0026] The digital environment platform 102 is shown to include a request processor 110, a saliency metric engine 112, an interaction data repository 114, and a content data repository 116. In an example, the request processor 110 processes various requests, such as may be received from the digital environment service 104 and the computing device 106. For example, the request processor 110 may process a request for a saliency metric associated with content (e.g., as may be received from the saliency processors 118 or 124 of the digital environment service 104 and the computing device 106, respectively). In an example, the request may include a saliency metric, an indication of an associated digital environment, and / or an indication of one or more demographic content. As another example, the request may include at least a portion of a user profile, which may be used to generate a saliency metric accordingly. As another example, the request may be for content used in adapting the digital environment and / or a ranked set of content.

[0027] The salience metric engine 112 may generate a salience metric and / or an indication of a location and / or content that may be provided in response to such a request. For example, the salience metric engine 112 may process telemetry data stored in the interaction data repository 114 and / or content attributes associated with candidate content (e.g., as may be received by the request processor 110, obtained from the content data repository 120 of the digital environment service 104, obtained from the content data repository 126 of the computing device 106, and / or accessed from the content data repository 116, etc.). As described above, in accordance with the various aspects described herein, the salience metric engine 112 may use any of a variety of techniques to generate a salience metric and / or determine content and / or location from a set of candidates based on associated factors. The following description of Figure 2 Method 200 of illustrative embodiments discusses additional examples of such aspects.

[0028] As another example, the request processor 110 may aggregate telemetry data (e.g., from the digital environment service 104 and / or the computing device 106), which may be stored in the interaction data store 114. In some cases, the content data store 116 stores content associated with one or more digital environments and / or content that may not be related to the digital environment, thereby enabling the digital environment to be adapted based on external content (e.g., compared to content associated with a given digital environment as may be stored by the content data store 120 of the digital environment service 104).

[0029] The system 100 is further shown to include a digital environment service 104, which can be used to provide a digital environment. For example, when rendering a digital environment for display to a user of a computing device 106, an environment application 122 and a digital environment service 104 can operate as a client and a server, respectively. In other examples, the environment application 122 can operate locally (e.g., to generate an offline digital environment) so that the digital environment service 104 can distribute the environment application 122 to any computing device among various computing devices. As another example, the digital environment service 104 can render at least a portion of the digital environment, which can be provided to the environment application 122 of the computing device 106 for display accordingly. Although the system 100 is shown to include a single computing device 106 with an associated environment application 122, it should be understood that any number of computing devices and associated environment applications can be used in other examples.

[0030] The digital environment service 104 is shown to include a saliency processor 118 and a content data repository 120. In an example, the saliency processor 118 generates telemetry data and / or obtains telemetry data (e.g., from the computing device 106), at least some of which can be provided for aggregation by the digital environment platform 102. In addition, the saliency processor 118 can request saliency metrics from the digital environment platform (e.g., as can be processed by the request processor 110 as described above). For example, the saliency processor 118 can request saliency metrics for content from the content data repository 120 (e.g., can be associated with the digital environment) and / or can request external content from the digital environment platform 102 (e.g., can be stored by the content data repository 116).

[0031] Although significance metric generation is described above with respect to the significance metric engine 112, it should be understood that in other examples, the digital environment service 104 may obtain a machine learning model and / or weighting function from the digital environment platform 102 that may generate significance metrics, such that at least some significance metrics may be generated at the digital environment service 104.

[0032] The computing device 106 can be any of a variety of computing devices, including but not limited to a mobile computing device, a tablet computing device, a laptop computing device, or a desktop computing device. In an example, the computing device 106 is or can otherwise be communicatively coupled to an AR and / or VR headset, as well as other examples. As shown, the computing device 106 includes an environment application 122, a saliency processor 124, and a content data repository 126. Aspects of the saliency processor 124 and the content data repository 126 can be similar to the saliency processor 118 and the content data repository 120 of the digital environment service 104, respectively, and therefore need not be described in detail below.

[0033] As described above, the environment application 122 can generate a digital environment for presentation to a user of the computing device 106. As another example, at least a portion of the digital environment can be rendered by the digital environment service 104. Thus, the saliency processor 124 and / or the saliency processor 118 can determine the content to be utilized to adapt the digital environment in accordance with the various aspects described herein. For example, the saliency processor 124 can request a saliency metric and / or content from the digital environment platform 102. In an example, the request includes at least a portion of a user profile, such as can be stored by the computing device 106. In other examples, such information can be stored by the digital environment service 104 and / or the digital environment platform 102. Thus, the environment application 122 can determine the spatial location of the presented content, can determine the content to be presented to the user from a set of content, and / or can adapt the environment mechanism based on the determined content, etc.

[0034] Additionally or alternatively, in examples where the digital environment is streamed, recorded, or otherwise captured, the representation of the digital environment may be adapted according to the determined content, e.g., to include such content. In some examples, the content determined by saliency processor 124 may be overlaid or otherwise used to replace content used to adapt the digital environment when rendered for display to a user.

[0035] Similar to the digital environment service 104, it should be understood that in other examples, a machine learning model and / or weighting function that generates a saliency metric may be obtained from the digital environment platform 102 so that at least some saliency metrics and associated content and / or location determinations may be generated at the computing device 106.

[0036] Additionally, as described above, any number of computing devices may be used in other examples. In such examples, a digital environment may be applicable to each respective user of a computing device such that the same digital environment has different associated representations presented to each user. For example, a first user may view a digital environment as including a first content item, while a second user may view the digital environment as including a second content item. Similarly, different environmental mechanisms may be applicable to different users, such as where the first user likes the environmental mechanism, while the second user does not like the environmental mechanism. As another example, an external first content instance may be presented to a first user based on a first set of interests associated with the first user, while an external second content instance may be presented to a second user based on a second set of interests associated with the second user.

[0037] Figure 2 An overview diagram of an example method 200 for generating a content saliency metric for adapting a digital environment according to aspects described herein. In an example, aspects of the method 200 are performed by a saliency metric engine (e.g., as described above with respect to Figure 1 The discussed saliency measurement engine 112) is used to perform.

[0038] As shown, method 200 begins at operation 202, where a set of content attributes is obtained. For example, the attribute set may include the relative desirability and / or scarcity of the content (e.g., as may be based on, for example, the above description of Figure 1 Such attributes may be relative, such as relative to other content of the digital environment and / or content of one or more other digital environments. As another example, the attribute set may include one or more attributes related to the content itself, such as a type of gameplay mechanic, an item type, a spatial location of an item within a digital environment, and / or an associated difficulty level, etc.

[0039] The process proceeds to operation 204, where a set of environmental attributes is obtained. In an example, a set of environmental attributes relates to a spatial location within a digital environment where content can be presented, such as a spatial location close to a user (e.g., within a predetermined distance and / or within the user's field of view). As another example, a set of environmental attributes may include an indication of the user's progress within a storyline of the digital environment. Therefore, it should be understood that the set of environmental attributes may relate to the digital environment itself (e.g., a spatial location and / or one or more associated environmental assets, such as items and / or sounds) as well as the user's perception of and / or interaction with the digital environment.

[0040] Moving to operation 206, a set of user profile attributes is obtained. For example, the set of user profile attributes may relate to the user's play or interaction style, the number of user interactions with a given instance of content and / or with similar types of content, the user's attention habits (e.g., based on the user's field of view determined from an AR / VR headset), and / or explicit and / or implicit user-specific preferences, etc.

[0041] At operation 208, a set of group attributes is obtained. In an example, the set of group attributes includes attributes similar to the attributes obtained at operation 206, but aggregated according to one or more group demographics. For example, the set of group attributes can be determined based on telemetry data associated with one or more digital environments (e.g., as can be stored by an interaction data store (e.g., interaction data repository 114)). As an example, the group attributes can indicate a difficulty level and / or popularity associated with a game mechanic and / or spatial location in which a user is located and / or whose content can be determined.

[0042] Although example attributes are described about operations 202-208, it should be understood that any attribute of various additional and / or alternative attributes can be used in other examples. As an example, one or more of operations 202-208 can be omitted in other examples. Such attributes can be based on telemetry data of an interactive data repository (e.g., interactive data repository 114), as part of a request for a saliency measure or content to an adapted digital environment, and / or based on a user profile (e.g., can be stored by a computing device, a digital environment service, and / or a digital environment platform), and any data source in various other data sources to obtain. In some cases, one or more attributes can be obtained based at least in part on other attributes. For example, user profile attributes can be obtained based on environmental attributes (e.g., it may be the case when user profile attributes are associated with the spatial location of a digital environment), or population attributes can be obtained based on user attributes.

[0043] Flow proceeds to operation 210, where a content saliency metric is generated based on the attributes obtained at operations 202-208. As described above, aspects of operation 210 may include using a machine learning model to generate a saliency metric (e.g., as may have been trained based on user interactions with the content of the digital environment and associated attributes). As another example, operation 210 may include generating a saliency metric based on a set of weights associated with each attribute.

[0044] At operation 212, an indication of the generated saliency metric is provided. For example, the saliency metric may be provided in response to a request for the saliency metric (e.g., as may be obtained from a saliency processor (e.g., as described above with respect to Figure 1 As another example, a saliency metric may be provided and used to rank a set of content, an example of which is described below with respect to Figure 3 The method 300 is described.

[0045] Although method 200 is provided as an example of generating a saliency metric for content, it should be understood that, in accordance with various aspects described herein, a saliency metric can be similarly generated for a spatial location based on location attributes (e.g., rarity and / or desirability of the location, proximity to other rare and / or desired locations, and transportation, visibility, and / or ease of access to the location), environmental attributes, user profile attributes, and / or group attributes. Method 200 terminates at operation 212.

[0046] Figure 3 An overview diagram of an example method 300 for adapting a digital environment according to various aspects described herein is shown. In an example, various aspects of the method 300 are performed by a saliency metric engine or a saliency processor (e.g., as described above with respect to Figure 1 The method 300 may be performed by a saliency metric engine 112 or a saliency processor 118 or 124 as discussed above. For example, aspects of the method 300 may be performed by a digital environment service (e.g., digital environment service 104) to adapt a digital environment for presentation to a user of a computing device, or in other examples, may be similarly performed by a computing device (e.g., computing device 106). As another example, similar aspects may be performed by a digital environment platform to identify content with which to adapt a digital environment.

[0047] As shown, method 300 begins at operation 302, where a set of candidate content is obtained. For example, at least a portion of the set of content may be associated with a digital environment for which content is being determined. In another example, at least a portion of the set of content may include external content (e.g., may be associated with one or more other digital environments). The set of content may be obtained from various sources (e.g., the above-mentioned Figure 1Any of the content data stores 116, 120, and / or 126 of the discussed digital environment platform 102, digital environment service 104, and computing device 106 are obtained.

[0048] Flow proceeds to operation 304, where a content saliency metric is determined for the set of content. For example, operation 304 may include generating a saliency metric for each content instance in the set of content obtained at operation 302. Aspects of operation 304 may be similar to those described above with respect to Figure 2 Those aspects discussed in connection with method 200.

[0049] At operation 306, the set of content is ranked according to the prominence metric generated at operation 304. As described above, the prominence metric may indicate relative prominence in some examples, or absolute prominence in other examples. In some cases, operation 306 also includes filtering out one or more instances of content (e.g., having a prominence metric below a predetermined threshold or generating a ranked set of content having a predetermined number of items).

[0050] Moving to operation 308, content is determined from the ranked set of content. In an example, one or more highest ranked instances of content are selected, or as another example, content may be randomly selected from content instances having a significance metric above a predetermined threshold. Thus, it should be appreciated that any of a variety of techniques may be used to determine content from the ranked set of content.

[0051] The process proceeds to operation 310, where the digital environment is adapted according to the content determined at operation 308. For example, operation 310 may include adapting the digital environment to include 2D or 3D assets or NPCs for presentation to the user. As another example, the environmental mechanics of the digital environment may be incorporated or modified, such as to present a set of different RPG story options to the user and / or have different difficulty levels, significance, and / or lengths. Although example adaptations are described, it is understood that the digital environment may be adapted according to any of a variety of alternative or additional techniques. In some cases, operation 310 may include providing an indication of the determined content, such as when presenting the digital environment or otherwise creating a representation by a remote computing device (e.g., when aspects of method 300 are performed by a digital environment platform or digital environment service). Method 300 terminates at operation 310.

[0052] Figure 4 An overview diagram of another example method 400 for adapting a digital environment according to aspects described herein is shown. In an example, aspects of the method 400 are performed by a saliency metric engine or a saliency processor (e.g., Figure 1The method 400 may be performed by a saliency metric engine 112 or a saliency processor 118 or 124 as discussed above. For example, aspects of the method 400 may be performed by a digital environment service (e.g., digital environment service 104) to determine a location to present content within a digital environment. As another example, such aspects may similarly be performed by a computing device (e.g., computing device 106) or by a digital environment platform (e.g., digital environment platform 102).

[0053] Method 400 begins at operation 402, where a set of locations is determined. In an example, the set of locations may be determined based on the location of the user within the digital environment. For example, the set of locations may include surfaces that are close to the user (e.g., within a predetermined radius or area or within the user's field of view). It should be understood that the spatial locations may have corresponding locations in the user's physical environment, such as when the digital environment is overlaid on the physical environment by an AR headset.

[0054] Flow proceeds to operation 404, where a content saliency metric is determined for the set of locations. For example, operation 404 may include generating a saliency metric for each location in the set of locations determined at operation 402. Aspects of operation 404 may be similar to those described above with respect to Figure 2 The aspects discussed in the method 200 of , such that a saliency metric is generated based on one or more location attributes, environmental attributes, user profile attributes, and / or demographic attributes according to the aspects described herein.

[0055] At operation 406, the set of positions is ranked based on the significance metric generated at operation 404. As described above, the significance metric may indicate relative significance in some examples, or may indicate absolute significance in other examples. In some cases, operation 406 also includes filtering out one or more positions (e.g., having a significance metric below a predetermined threshold or generating a ranked set of positions having a predetermined number of items).

[0056] Moving to operation 408, a position is determined from the ranked set of positions. In an example, one or more of the highest ranked positions are selected, or as another example, a position may be randomly selected from positions having a significance metric above a predetermined threshold. Thus, it should be appreciated that any of a variety of techniques may be used to determine a position from the ranked set of positions.

[0057] Flow proceeds to operation 410, where content is determined to be used to adapt the digital environment (e.g., at the location determined at operation 408). For example, aspects of operation 410 may be similar to those described above with respect to Figure 34. The present invention relates to the aspects discussed in connection with operations 302-308 of method 300 in the embodiment of the present invention and therefore need not be described in detail below. In an example, a set of content is generated based at least in part on the location determined at operation 408. Thus, a different set of content can be generated for different locations within the digital environment.

[0058] At operation 412, the digital environment is adapted according to the location determined at operation 408 using the content determined at operation 410. For example, operation 412 may include adapting the digital environment to include a 2D or 3D asset or NPC to present to the user at the determined location. As another example, an environment mechanic may be incorporated or modified in conjunction with the determined location. In some cases, the content determined at operation 410 is overlaid within the user's physical environment (e.g., as may be the case when the user is wearing an AR headset).

[0059] Although example adaptations are described, it is understood that the digital environment may be adapted according to any of a variety of alternative or additional techniques. In some cases, operation 412 may include providing an indication of the determined location and associated content, such as may be the case when the digital environment is presented or otherwise created by a remote computing device (e.g., when aspects of method 400 are performed by a digital environment platform or digital environment service). In addition, although method 400 is described as an example of subsequently determining content based on a location having a high saliency metric, it is understood that in other examples, a location may be subsequently determined based on content having a high saliency metric. Method 400 terminates at operation 412.

[0060] Figure 5-Figure 7 The and associated descriptions provide a discussion of various operating environments in which aspects of the present disclosure may be implemented. Figure 5-Figure 7 The devices and systems shown and discussed are for purposes of example and explanation and are not limiting of the wide variety of computing device configurations that may be used to implement the various aspects of the present disclosure described herein.

[0061] Figure 5 5 is a block diagram illustrating the physical components (eg, hardware) of a computing device 500 that may be utilized to implement various aspects of the present disclosure. The computing device components described below may be applicable to the computing devices described above, including those described above with respect to Figure 1 The digital environment platform 102, digital environment service 104, and / or computing device 106 are discussed. In a basic configuration, the computing device 500 may include at least one processing unit 502 and a system memory 504. Depending on the configuration and type of the computing device, the system memory 504 may include, but is not limited to, volatile storage (e.g., random access memory), non-volatile storage (e.g., read-only memory), flash memory, or any combination of such memories.

[0062] The system memory 504 may include an operating system 505 and one or more program modules 506 suitable for running software applications 520, such as one or more components supported by the system described herein. As an example, the system memory 504 may store an environment application 524 and a saliency metric engine 526. The operating system 505 may be suitable for controlling the operation of the computing device 500, for example.

[0063] Furthermore, embodiments of the present disclosure may be implemented in conjunction with a graphics library, other operating systems, or any other application program, and are not limited to any particular application or system. Figure 5 508. The computing device 500 may have additional features or functionality. For example, the computing device 500 may also include additional data storage devices (removable and / or non-removable), such as magnetic disks, optical disks, or tapes. Such additional storage Figure 5 , illustrated by a removable storage device 509 and a non-removable storage device 510.

[0064] As described above, a number of program modules and data files may be stored in system memory 504. When executed on processing unit 502, program modules 506 (e.g., applications 520) may perform processes including, but not limited to, various aspects as described herein. Other program modules that may be used in accordance with various aspects of the present disclosure may include email and contact applications, word processing applications, spreadsheet applications, database applications, slide presentation applications, drawing or computer-assisted applications, and the like.

[0065] In addition, embodiments of the present disclosure may be implemented on a single chip including discrete electronic components, a packaged or integrated electronic chip containing logic gates, a circuit utilizing a microprocessor, or containing electronic components or a microprocessor. For example, embodiments of the present disclosure may be implemented by a system on a chip (SOC), wherein Figure 5 Each or many components shown in can be integrated into a single integrated circuit. Such a SOC device may include one or more processing units, a graphics unit, a communication unit, a system virtualization unit, and various application functions, all of which are integrated (or "burned") into a chip substrate as a single integrated circuit. When operated by the SOC, the functions described herein regarding the ability of the client to switch protocols can be operated by dedicated logic integrated with other components of the computing device 500 on a single integrated circuit (chip). Embodiments of the present disclosure may also be implemented using other technologies capable of performing logical operations, such as AND, OR, and NOT, including but not limited to mechanical, optical, fluid, and quantum technologies. In addition, embodiments of the present disclosure may be implemented in a general-purpose computer or in any other circuit or system.

[0066] The computing device 500 may also have one or more input devices 512, such as a keyboard, a mouse, a pen, a sound or voice input device, a touch or sliding input device, etc. Output devices 514 may also be included, such as a display, a speaker, a printer, etc. The aforementioned devices are examples, and other devices may be used. The computing device 500 may include one or more communication connections 516 that allow communication with other computing devices 550. Examples of suitable communication connections 516 include, but are not limited to, radio frequency (RF) transmitters, receivers, and / or transceiver circuits; universal serial bus (USB), parallel and / or serial ports.

[0067] The term computer-readable media as used herein may include computer storage media. Computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures or program modules. System memory 504, removable storage device 509 and non-removable storage device 510 are all examples of computer storage media (e.g., memory storage). Computer storage media may include RAM, ROM, electrically erasable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage, cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other products that can be used to store information and can be accessed by computing device 500. Any such computer storage media may be part of computing device 500. Computer storage media do not include carrier waves or other propagated or modulated data signals.

[0068] Communication media may be embodied by computer readable instructions, data structures, program modules, or other data in a modulated data signal (e.g., a carrier wave or other transport mechanism), and includes any information delivery media. The term "modulated data signal" may describe a signal that has one or more characteristics set or changed in a manner that encodes information in the signal. By way of example, and not limitation, communication media may include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media.

[0069] Fig. 6A and Figure 6B A mobile computing device 600 is shown, with which embodiments of the present disclosure may be implemented, such as a mobile phone, a smart phone, a wearable computer (e.g., a smart watch), a tablet computer, a laptop computer, etc. In some aspects, the client may be a mobile computing device. Fig. 6A, illustrates one aspect of a mobile computing device 600 for implementing various aspects. In a basic configuration, the mobile computing device 600 is a handheld computer having both input elements and output elements. The mobile computing device 600 typically includes a display 605 and one or more input buttons 610 that allow a user to enter information into the mobile computing device 600. The display 605 of the mobile computing device 600 may also be used as an input device (e.g., a touch screen display).

[0070] If included, the optional side input element 615 allows further user input. The side input element 615 can be a rotary switch, a button, or any other type of manual input element. In alternative aspects, the mobile computing device 600 can include more or fewer input elements. For example, in some embodiments, the display 605 may not be a touch screen.

[0071] In yet another alternative embodiment, the mobile computing device 600 is a portable telephone system, such as a cellular phone. The mobile computing device 600 may also include an optional keyboard 635. The optional keyboard 635 may be a physical keyboard or a "soft" keyboard generated on a touch screen display.

[0072] In various embodiments, the output elements include a display 605 for showing a graphical user interface (GUI), a visual indicator 620 (e.g., a light emitting diode), and / or an audio transducer 625 (e.g., a speaker). In some aspects, the mobile computing device 600 includes a vibration transducer for providing tactile feedback to the user. In yet another aspect, the mobile computing device 600 includes input and / or output ports for sending or receiving signals to or from external devices, such as an audio input (e.g., a microphone jack), an audio output (e.g., a headphone jack), and a video output (e.g., an HDMI port).

[0073] Figure 6B 6 is a block diagram illustrating an architecture of one aspect of a mobile computing device. That is, a mobile computing device 600 can incorporate a system (e.g., architecture) 602 to implement some aspects. In one embodiment, the system 602 is implemented as a "smart phone" capable of running one or more applications (e.g., a browser, email, calendar, contact manager, messaging client, game, and media client / player). In some aspects, the system 602 is integrated as a computing device, such as an integrated personal digital assistant (PDA) and a wireless phone.

[0074] One or more application programs 666 may be loaded into the memory 662 and run on or in association with the operating system 664. Examples of application programs include a phone dialer, an email program, a personal information management (PIM) program, a word processing program, a spreadsheet program, an Internet browser program, a messaging program, and the like. The system 602 also includes a storage device 668 within the memory 662. The storage device 668 may be used to store persistent information that should not be lost if the system 602 loses power. The application programs 666 may use and store information in the storage device 668, such as emails or other messages used by the email application, and the like. A synchronization application (not shown) also resides on the system 602 and is programmed to interact with a corresponding synchronization application residing on a host computer to keep information stored in the storage device 668 synchronized with corresponding information stored at the host computer. It should be understood that other applications may be loaded into the memory 662 and run on the mobile computing device 600 described herein.

[0075] System 602 has a power source 670 that can be implemented as one or more batteries. Power source 670 can also include an external power source, such as an AC adapter or a powered docking cradle to replenish or recharge the batteries.

[0076] The system 602 may also include a radio interface layer 672 that performs the functions of sending and receiving radio frequency communications. The radio interface layer 672 facilitates wireless connections between the system 602 and the "outside world" by a communications carrier or service provider. Transmissions to and from the radio interface layer 672 are performed under the control of the operating system 664. In other words, communications received by the radio interface layer 672 may be propagated by the operating system 664 to the application 666, and vice versa.

[0077] The visual indicator 620 can be used to provide visual notifications, and / or the audio interface 674 can be used to generate audible notifications by the audio transducer 625. In the illustrated embodiment, the visual indicator 620 is a light emitting diode (LED), and the audio transducer 625 is a speaker. These devices can be directly coupled to the power supply 670 so that when they are activated, they remain turned on for a duration specified by the notification mechanism even if the processor 660 and other components may be turned off to save battery power. The LED can be programmed to remain turned on indefinitely until the user takes action to indicate the power-on state of the device. The audio interface 674 is used to provide audible signals to the user and receive audible signals from the user. For example, in addition to being coupled to the audio transducer 625, the audio interface 674 can also be coupled to a microphone to receive audible input, such as to facilitate a telephone conversation. According to an embodiment of the present disclosure, the microphone can also be used as an audio sensor to facilitate the control of notifications, as described below. The system 602 can also include a video interface 676, which enables the operation of the onboard camera 630 to record still images, video streams, etc.

[0078] The mobile computing device 600 implementing the system 602 may have additional features or functionality. For example, the mobile computing device 600 may also include additional data storage devices (removable and / or non-removable), such as magnetic disks, optical disks, or tapes. Such additional storage Figure 6B is shown by storage device 668.

[0079] As described above, data / information generated or captured by the mobile computing device 600 and stored by the system 602 may be stored locally on the mobile computing device 600, or the data may be stored on any number of storage media that can be accessed by the device by the radio interface layer 672 or by a wired connection between the mobile computing device 600 and a separate computing device associated with the mobile computing device 600 (e.g., a server computer in a distributed computing network such as the Internet). It should be understood that such data / information can be accessed by the mobile computing device 600 by the radio interface layer 672 or by a distributed computing network. Similarly, such data / information can be easily transferred between computing devices for storage and use according to well-known data / information transmission and storage means, including electronic mail and collaborative data / information sharing systems.

[0080] Figure 7One aspect of the architecture of a system for processing data received at a computing system from a remote source (e.g., a general purpose computing device 704, a tablet computing device 706, or a mobile computing device 708 as described above) is shown. The content displayed at the server 702 can be stored in different communication channels or other storage types. For example, a directory service 722, a web portal 724, a mailbox service 726, an instant messaging repository 728, or a social networking service 730 can be used to store various documents.

[0081] The environment application 720 may be employed by a client communicating with the server 702, and / or the saliency metric engine 721 may be employed by the server 702. The server 702 may provide data to and from client computing devices such as the general purpose computing device 704, the tablet computing device 706, and / or the mobile computing device 708 (e.g., a smart phone) via the network 715. As an example, the computer system described above may be embodied in the general purpose computing device 704, the tablet computing device 706, and / or the mobile computing device 708 (e.g., a smart phone). In addition to receiving graphics data that may be used for pre-processing at the graphics originating system or post-processing at the receiving computing system, any of these embodiments of the computing device may obtain content from the repository 716.

[0082] In addition, it should be understood that the various aspects and functions described herein can be operated on a distributed system (e.g., a cloud-based computing system), where application functions, memory, data storage and retrieval, and various processing functions can be remotely operated from each other through a distributed computing network (e.g., the Internet or an intranet). Various types of user interfaces and information can be displayed by an onboard computing device display or by a remote display unit associated with one or more computing devices. For example, various types of user interfaces and information can be displayed and interacted on a wall surface onto which the user interface and various types of information are projected. Interactions with multiple computing systems that can implement embodiments of the present invention include keystroke input, touch screen input, voice or other audio input, gesture input, where the associated computing device is equipped with a detection (e.g., camera) function for capturing and interpreting user gestures for controlling the functions of the computing device, and the like.

[0083] As will be understood from the foregoing disclosure, one aspect of the present technology relates to a system comprising: at least one processor; and a memory storing instructions that, when executed by the at least one processor, cause the system to perform a set of operations. The set of operations includes: obtaining a set of factors associated with a digital environment, wherein the digital environment is an augmented reality digital environment; based on the set of factors, determining content from a set of candidate content based on a saliency metric associated with each content instance in a set of candidate content; and adapting the digital environment according to the determined content, wherein adapting the digital environment includes at least one of the following: including the determined content in the digital environment for presentation to the user; or modifying the environment mechanism of the digital environment based on the determined content. In an example, the set of candidate content includes content associated with another digital environment or content that is not related to the digital environment. In another example, the saliency metric defines the relative saliency of the determined content associated with the user compared to another content instance in the set of candidate content. In another example, the set of operations includes determining a location from a set of candidate locations based on a saliency metric for each candidate location in a set of candidate locations for the digital environment; and including the determined content includes the determined content included at the determined location within the digital environment. In yet another example, a saliency measure associated with an instance of candidate content is generated using a machine learning model, wherein the machine learning model is trained using training data including user interactions with content of a digital environment, and wherein the user interactions are annotated with one or more associated factors. In yet another example, a set of factors includes: content attributes associated with the determined content; environment attributes associated with the digital environment; user profile attributes associated with the user; and group attributes associated with a group of the digital environment. In another example, a set of factors is obtained based on telemetry data associated with the digital environment.

[0084] In another aspect, the present technology relates to a method for adapting a digital environment based on a saliency metric. The method includes: generating a representation of a digital environment for presentation to a user; requesting content to adapt the digital environment for the user from a digital environment platform; receiving an indication of content from the digital environment platform; and adapting the digital environment according to the indication of the received content, wherein adapting the digital environment includes at least one of: including the content in the representation of the digital environment; or modifying an environment mechanism of the digital environment based on the content. In an example, requesting content to adapt the digital environment for the user includes providing an indication of a user profile associated with the user to the digital environment platform. In another example, the received content is associated with another digital environment. In another example, the indication of the received content is a first content instance, and the method also includes generating a stream of the representation of the digital environment, wherein generating the stream includes: determining a second content instance to adapt the stream; and adapting the stream to include the second content instance. In yet another example, the stream is adapted to include a second content instance in place of the first content instance. In yet another example, the first content instance is determined at least in part based on the user; and the second content instance is determined at least in part based on a group associated with the digital environment.

[0085] In another aspect, the technology relates to another method for adapting a digital environment based on a saliency metric. The method includes: obtaining a set of factors associated with a digital environment, wherein the digital environment is an augmented reality digital environment; determining content from a set of candidate content based on a saliency metric associated with each content instance in a set of candidate content based on the set of factors; and adapting the digital environment according to the determined content, wherein adapting the digital environment includes at least one of: including the determined content in the digital environment for presentation to a user; or modifying an environmental mechanism of the digital environment based on the determined content. In an example, the set of candidate content includes content associated with another digital environment or content that is not related to the digital environment. In another example, the saliency metric defines a relative saliency of the determined content associated with the user compared to another content instance in the candidate content. In another example, the method includes determining a location from a set of candidate locations based on a saliency metric for each candidate location in a set of candidate locations for the digital environment; and including the determined content at the determined location within the digital environment. In yet another example, a saliency measure associated with an instance of candidate content is generated using a machine learning model, wherein the machine learning model is trained using training data including user interactions with content of a digital environment, and wherein the user interactions are annotated with one or more associated factors. In yet another example, a set of factors includes: content attributes associated with the determined content; environment attributes associated with the digital environment; user profile attributes associated with the user; and group attributes associated with a group of the digital environment. In another example, a set of factors is obtained based on telemetry data associated with the digital environment.

[0086] Aspects of the present disclosure are described, for example, above with reference to the block diagrams and / or operational explanations of the methods, systems, and computer program products of the various aspects of the present disclosure. The functions / actions marked in the boxes may not occur in the order shown in any flow chart. For example, depending on the functions / actions involved, two boxes shown in succession may actually be executed substantially simultaneously, or the boxes may sometimes be executed in reverse order.

[0087] The description and explanation of one or more aspects provided in this application are not intended to limit or restrict the scope of the present disclosure claimed in any way. The various aspects, examples and details provided in this application are considered to be sufficient to convey ownership and enable others to manufacture and use the various aspects claimed for protection of this disclosure. The disclosure claimed for protection should not be interpreted as being limited to any aspect, example or detail provided in this application. Whether or not shown and described in combination, various features (both structure and method) are intended to be selectively included or omitted to produce an embodiment with a specific feature set. Description and explanation of the application have been provided, and those skilled in the art can envision the changes, modifications and alternative aspects of the broader scope of the disclosure that fall within the spirit of the broader aspects of the overall inventive concept embodied in this application without departing from the claimed protection.

Claims

1. A system comprising: at least one processor; as well as A memory storing instructions, wherein when the instructions are executed by the at least one processor, the system performs a set of operations, the set of operations comprising: obtaining a set of factors associated with a digital environment, wherein the digital environment is an augmented reality digital environment; determining content from the set of candidate content based on a saliency metric associated with each instance of content in the set of candidate content based on the set of factors; and Adapting the digital environment according to the determined content, wherein adapting the digital environment comprises at least one of the following: including the determined content in the digital environment for presentation to a user; or An environmental mechanism of the digital environment is modified based on the determined content.

2. The system of claim 1, wherein: The set of operations includes determining a location from a set of candidate locations for the digital environment based on a saliency measure for each candidate location in the set of candidate locations; and Including the determined content includes including the determined content at the determined location within the digital environment.

3. A system according to claim 1, wherein the saliency measure associated with the instance of the candidate content is generated using a machine learning model, wherein the machine learning model is trained using training data including user interactions with the content of the digital environment, and wherein the user interactions are annotated with one or more associated factors.

4. A method for adapting a digital environment based on a saliency metric, the method comprising: generating a representation of the digital environment for presentation to a user; Requesting content from a digital environment platform to adapt the digital environment for the user; receiving an indication of content from the digital environment platform; as well as Adapting the digital environment according to the indication of the received content, wherein adapting the digital environment comprises at least one of: including said content in said representation of said digital environment; or An environmental mechanism for modifying the digital environment based on the content.

5. The method of claim 4, wherein requesting the content to adapt the digital environment for the user comprises providing an indication of a user profile associated with the user to the digital environment platform.

6. The method of claim 4, wherein the received indication of content is a first content instance, and the method further comprises generating a stream of the representation of the digital environment, wherein generating the stream comprises: determining a second content instance to adapt the stream; as well as The stream is adapted to include the second content instance.

7. A method for adapting a digital environment based on a saliency metric, the method comprising: obtaining a set of factors associated with a digital environment, wherein the digital environment is an augmented reality digital environment; determining content from the set of candidate content based on a saliency metric associated with each instance of content in the set of candidate content based on the set of factors; as well as Adapting the digital environment according to the determined content, wherein adapting the digital environment comprises at least one of the following: including the determined content in the digital environment for presentation to a user; or An environmental mechanism of the digital environment is modified based on the determined content.

8. The method of claim 7, wherein the set of candidate content includes content associated with another digital environment or content unrelated to the digital environment.

9. The method of claim 7, wherein the prominence metric defines a relative prominence of the determined content associated with the user compared to another content instance in the set of candidate content.

10. The method of claim 7, wherein the set of factors comprises: content attributes associated with the determined content; environmental attributes associated with the digital environment; user profile attributes associated with the user; as well as A community attribute associated with the community of the digital environment.

11. The system of claim 1, wherein the set of candidate content includes content associated with another digital environment or content unrelated to the digital environment.

12. The system of claim 1, wherein the set of factors is obtained based on telemetry data associated with the digital environment.

13. The method of claim 6, wherein the stream is adapted to include the second instance of content in place of the first instance of content.

14. The method according to claim 6, wherein: The first content instance is determined based at least in part on the user; and The second content instance is determined based at least in part on a community associated with the digital environment.

15. A method according to claim 7, wherein the significance measure associated with the instance of the candidate content is generated using a machine learning model, wherein the machine learning model is trained using training data including user interactions with the content of the digital environment, and wherein the user interactions are annotated with one or more associated factors.