Adaptive virtual content for generating spatial characteristics of a physical set

Abstract

In some specific implementations, a method includes: identifying a plurality of subsets associated with a physical scene; determining a set of spatial characteristics for each of the plurality of subsets, wherein a first set of spatial characteristics characterizes the size of a first subset and a second set of spatial characteristics characterizes the size of a second subset; generating an adaptive first augmented reality (AR) content portion for the first subset based at least in part on the first set of spatial characteristics; generating an adaptive second AR content portion for the second subset based at least in part on the second set of spatial characteristics; and generating one or more navigation options that allow a user to traverse between the first subset and the second subset based on the first set of spatial characteristics and the second set of spatial characteristics.

Description

Technical Field

[0001] The present disclosure generally relates to generating virtual content (sometimes also referred to herein as "augmented reality (AR) content"), and more particularly to generating adaptive virtual content based on spatial features of multiple subsets within a physical scene. Background Art

[0002] In some cases, the creation of augmented reality (AR) content does not take into account the spatial features of the physical scene (e.g., volume space, shape, etc.). For example, when the physical scene corresponds to a large room with sufficient space, a user may be presented with AR content and interact with the AR content by walking around the physical scene. However, in another example, when the physical scene corresponds to a small room with limited space, the user cannot navigate the AR content by walking through the physical scene in the same manner as the user can walk through the large room. Brief Description of the Drawings

[0003] Thus, the present disclosure can be understood by those of ordinary skill in the art, and a more detailed description can be referred to some exemplary aspects of specific implementations, some of which are shown in the drawings.

[0004] Figure 1 is a block diagram of an exemplary operating architecture according to some specific implementations.

[0005] Figures 2A to 2D shows a comparison between a first augmented reality (AR) presentation scene sequence and a second AR presentation scene sequence according to some specific implementations.

[0006] Figures 3A to 3F shows an exemplary AR presentation scene sequence for generating adaptive AR content based on spatial features of multiple subsets within a physical scene according to some specific implementations.

[0007] Figures 4A to 4C shows another exemplary AR presentation scene sequence for generating adaptive AR content based on spatial features of multiple subsets within a physical scene according to some specific implementations.

[0008] Figure 5 is a flowchart representation of a method for generating adaptive AR content based on spatial features of multiple subsets within a physical scene according to some specific implementations.

[0009] Figure 6 is a block diagram of an exemplary controller according to some specific implementations.

[0010] Figure 7 is a block diagram of an exemplary device according to some specific implementations.

[0011] In accordance with common practice, the various feature portions shown in the drawings may not be drawn to scale. Thus, for clarity, the dimensions of the various feature portions may be arbitrarily expanded or reduced. Additionally, some of the drawings may not depict all of the components of a given system, method, or apparatus. Finally, throughout the specification and drawings, like reference numerals may be used to denote like feature portions. Summary of the Invention

[0012] The various specific implementations disclosed herein include devices, systems, and methods for generating adaptive enhanced reality (ER) content based on spatial features of multiple subsets within a physical scene. According to some specific implementations, the method is performed at an electronic device including an image sensor, one or more processors, non-transitory memory, and a display. The method includes: identifying a plurality of subsets associated with the physical scene; determining a set of spatial features for each of the plurality of subsets within the physical scene, wherein a first set of spatial features characterizes one or more dimensions of a first subset of the plurality of subsets, and a second set of spatial features characterizes one or more dimensions of a second subset of the plurality of subsets; generating an adaptive first ER content portion for the first subset of the plurality of subsets based at least in part on the first set of spatial features; generating an adaptive second ER content portion for the second subset of the plurality of subsets based at least in part on the second set of spatial features; and generating one or more navigation options that allow a user to traverse between the first subset and the second subset of the plurality of subsets based on the first set of spatial features and the second set of spatial features.

[0013] According to some specific implementations, a device includes an image sensor, a display, one or more processors, non-transitory memory, and one or more programs; the one or more programs are stored in the non-transitory memory and configured to be executed by the one or more processors, and the one or more programs include instructions for performing or causing to be performed any of the methods described herein. According to some specific implementations, a non-transitory computer-readable storage medium stores instructions that, when executed by one or more processors of a device, cause the device to perform or cause to be performed any of the methods described herein. According to some specific implementations, a device includes: an image sensor, a display, one or more processors, non-transitory memory, and means for performing or causing to be performed any of the methods described herein. Detailed Description

[0014] Numerous details are described to provide a thorough understanding of the example embodiments shown in the figures. However, the figures merely illustrate some example aspects of the present disclosure and should not be considered limiting. A person of ordinary skill in the art will understand that other effective aspects and / or variations do not include all of the specific details described herein. Additionally, well-known systems, methods, components, devices, and circuits have not been described in exhaustive detail so as not to obscure more relevant aspects of the example embodiments described herein. Various examples of electronic systems and techniques for using such systems in connection with various augmented reality technologies are described.

[0015] A physical setting refers to the world that individuals can sense and / or interact with without using an electronic system. Physical settings such as a physical park include physical elements such as physical wildlife, physical trees, and physical plants. People can directly sense and / or otherwise interact with the physical setting, for example, using one or more senses including vision, smell, touch, taste, and hearing.

[0016] In contrast to a physical setting, an augmented reality (AR) setting refers to a fully (or partially) computer-generated setting that various people can sense and / or otherwise interact with by using an electronic system. In AR, the movement of a person is partially monitored, and in response thereto, at least one attribute corresponding to at least one virtual object in the AR setting is changed in a manner consistent with one or more physical laws. For example, in response to the AR system detecting that a person looks up, the AR system can adjust various audio and graphics presented to the person in a manner consistent with how such sounds and appearances would change in a physical setting. Adjustments to the attributes of virtual objects in the AR setting can also be made, for example, in response to a representation of movement (e.g., a voice command).

[0017] A person can utilize one or more senses such as vision, smell, taste, touch, and hearing to sense and / or interact with AR objects. For example, a person can sense and / or interact with an object that creates a multi-dimensional or spatial acoustic setting. A multi-dimensional or spatial acoustic setting provides an individual with the perception of discrete sound sources in a multi-dimensional space. Such objects can also achieve acoustic transparency, which can selectively incorporate audio from the physical setting with or without computer-generated audio. In some AR settings, a person can sense and / or interact with only audio objects.

[0018] Virtual reality (VR) is an example of ER. A VR scene refers to an augmented scene that is configured to include only computer-generated sensory inputs for one or more senses. A VR scene includes multiple virtual objects that a person can sense and / or interact with. A person can sense and / or interact with the virtual objects in a VR scene by simulating at least some of the movements of a person's actions within the computer-generated scene and / or by simulating the person or their presence within the computer-generated scene.

[0019] Mixed reality (MR) is another example of ER. An MR scene refers to an augmented scene that is configured to integrate computer-generated sensory inputs (e.g., virtual objects) with sensory inputs from a physical scene or a representation of sensory inputs from a physical scene. On the reality spectrum, an MR scene lies between a fully physical scene at one end and a VR scene at the other end and does not include these scenes.

[0020] In some MR scenes, the computer-generated sensory inputs can be adjusted based on changes in the sensory inputs from the physical scene. Additionally, some electronic systems for presenting an MR scene can detect the position and / or orientation relative to the physical scene to enable interaction between real objects (i.e., physical elements from the physical scene or their representations) and virtual objects. For example, the system can detect movement and adjust the computer-generated sensory inputs accordingly, such that, for example, a virtual tree appears fixed relative to a physical structure.

[0021] Augmented reality (AR) is an example of MR. An AR scene refers to an augmented scene in which one or more virtual objects are superimposed on a physical scene (or a representation thereof). For example, an electronic system can include an opaque display and one or more imaging sensors for capturing video and / or images of the physical scene. For example, such video and / or images can be a representation of the physical scene. The video and / or images are combined with the virtual objects, and the combination is then displayed on the opaque display. The physical scene can be viewed indirectly by a person via the images and / or video of the physical scene. Thus, a person can observe the virtual objects superimposed on the physical scene. When the system captures an image of the physical scene and uses the captured image to display the AR scene on the opaque display, the displayed image is referred to as video pass-through. Alternatively, a transparent or semi-transparent display can be included in the electronic system for displaying the AR scene, such that an individual can directly view the physical scene through the transparent or semi-transparent display. The virtual objects can be displayed on the semi-transparent or transparent display, such that an individual observes the virtual objects superimposed on the physical scene. In another example, a projection system can be utilized to project virtual objects onto the physical scene. For example, the virtual objects can be projected onto a physical surface or as a hologram, such that an individual observes the virtual objects superimposed on the physical scene.

[0022] An AR scene may also refer to an augmented scene in which the representation of the physical scene is modified by computer-generated sensory data. For example, at least a portion of the representation of the physical scene may be modified graphically (e.g., magnified) such that the modified portion still represents the initially captured image (but not an exact copy). Alternatively, when providing video passthrough, one or more sensor images may be modified in order to impose a particular viewpoint different from the viewpoint captured by the image sensor. As another example, a portion of the representation of the physical scene may be altered by graphically blurring or removing that portion.

[0023] Augmented virtual (AV) is another example of MR. An AV scene refers to an augmented scene in which a virtual or computer-generated scene is combined with one or more sensory inputs from a physical scene. Such sensory inputs may include representations of one or more features of the physical scene. A virtual object may, for example, be combined with a color associated with a physical element captured by an imaging sensor. Alternatively, a virtual object may adopt features consistent with, for example, the current weather conditions of the physical scene, such as weather conditions identified via imaging, online weather information, and / or weather-related sensors. As another example, an AR park may include virtual structures, plants, and trees, although the animals within the AR park scene may include features accurately replicated from images of physical animals.

[0024] Various systems allow people to sense and / or interact with an ER scene. For example, a head-mounted system may include one or more speakers and an opaque display. As another example, an external display (e.g., a smart phone) may be incorporated into the head-mounted system. The head-mounted system may include a microphone for capturing audio of the physical scene and / or an image sensor for capturing images / videos of the physical scene. A transparent or translucent display may also be included in the head-mounted system. The translucent or transparent display may, for example, include a substrate through which light (representing an image) is directed to a person's eyes. The display may also include LEDs, OLEDs, liquid crystal on silicon, laser scanning light sources, digital light projectors, or any combination thereof. The substrate through which light is transmitted may be an optical reflector, a holographic substrate, an optical waveguide, a light combiner, or any combination thereof. The transparent or translucent display may, for example, selectively transition between a transparent / translucent state and an opaque state. As another example, an electronic system may be a projection-based system. In a projection-based system, retinal projection may be used to project an image onto a person's retina. Alternatively, a projection-based system may also project virtual objects into the physical scene, for example, such as projecting a virtual object as a hologram or onto a physical surface. Other examples of ER systems include windows configured to display graphics, head-mounted earphones, earphones, speaker arrangements, lenses configured to display graphics, head-up displays, automobile windshields configured to display graphics, input mechanisms (e.g., controllers with or without haptic functionality), desktop or laptop computers, tablets, or smart phones.

[0025] Figure 1 is a block diagram of an exemplary operational architecture 100 according to some specific implementations. Although related features are shown, those of ordinary skill in the art will recognize from this disclosure that various other features are not shown for the sake of brevity and to not obscure more relevant aspects of the exemplary specific implementations disclosed herein. To that end, as a non-limiting example, the operational architecture 100 includes an optional controller 102 and an electronic device 124 (e.g., a tablet computer, a mobile phone, a laptop computer, a wearable computing device, etc.).

[0026] In some specific implementations, the controller 102 is configured to manage and coordinate the ER experience of a user 150 (sometimes also referred to herein as a “virtual environment” or “graphical environment”) and zero or more other users. In some specific implementations, the controller 102 includes a suitable combination of software, firmware, and / or hardware. The controller 102 is described in more detail below with reference to Figure 6 In some specific implementations, the controller 102 is a computing device that is local or remote with respect to the physical set 105. For example, the controller 102 is a local server located within the physical set 105. In another example, the controller 102 is a remote server (e.g., a cloud server, a central server, etc.) located outside of the physical set 105. In some specific implementations, the controller 102 is communicatively coupled to the electronic device 124 via one or more wired or wireless communication channels 144 (e.g., Bluetooth, IEEE 802.11x, IEEE 802.16x, IEEE 802.3x, etc.). In some specific implementations, the functions of the controller 102 are provided by the electronic device 124. Thus, in some specific implementations, the components of the controller 102 are integrated into the electronic device 124.

[0027] In some specific implementations, the electronic device 124 is configured to present audio and / or video content to the user 150. In some specific implementations, the electronic device 124 is configured to present an ER experience to the user 150. In some specific implementations, the electronic device 124 includes a suitable combination of software, firmware, and / or hardware. The electronic device 124 is described in more detail below with reference to Figure 7 more detail.

[0028] According to some specific implementations, when user 150 is physically present within physical setting 105, electronic device 124 presents an enhanced reality (ER) experience to user 150, and the physical setting includes table 107 within the field of view 111 of electronic device 124. Thus, in some specific implementations, user 150 holds electronic device 124 in his / her hand. In some specific implementations, when presenting the ER experience, electronic device 124 is configured to present ER content (e.g., ER cylinder 109) and implement video passthrough of physical setting 105 (e.g., including table 107) on display 122. For example, electronic device 124 corresponds to a mobile phone, a tablet computer, a laptop computer, a wearable computing device, etc.

[0029] In some specific implementations, display 122 corresponds to an additive display that implements optical passthrough of physical setting 105 (including table 107). For example, display 122 corresponds to a transparent lens, and electronic device 124 corresponds to a pair of glasses worn by user 150. Thus, in some specific implementations, electronic device 124 presents a user interface by projecting ER content (e.g., ER cylinder 109) onto the additive display, and the ER content is then superimposed on physical setting 105 from the perspective of user 150. In some specific implementations, electronic device 124 presents a user interface by displaying ER content (e.g., ER cylinder 109) on the additive display, and the ER content is then superimposed on physical setting 105 from the perspective of user 150.

[0030] In some specific implementations, user 150 wears electronic device 124, such as a near-eye system. Thus, electronic device 124 includes one or more displays (e.g., a single display or one display for each eye) provided to display ER content. For example, electronic device 124 surrounds the field of view of user 150. In such specific implementations, electronic device 124 presents an ER scene by displaying data corresponding to the ER scene on one or more displays or by projecting data corresponding to the ER scene onto the retina of user 150.

[0031] In some specific implementations, the electronic device 124 includes an integrated display (e.g., a built-in display) that shows the ER scene. In some specific implementations, the electronic device 124 includes a head-mounted housing. In various specific implementations, the head-mounted housing includes an attachment area to which another device having a display can be attached. For example, in some specific implementations, the electronic device 124 can be attached to the head-mounted housing. In various specific implementations, the head-mounted housing is shaped to form a receiver for receiving another device (e.g., the electronic device 124) that includes a display. For example, in some specific implementations, the electronic device 124 slides / snaps onto the head-mounted housing or is otherwise attached to the head-mounted housing. In some specific implementations, the display of the device attached to the head-mounted housing presents (e.g., shows) the ER scene. In some specific implementations, the electronic device 124 is replaced with an ER chamber, housing, or room configured to present ER content, in which the user 150 does not wear the electronic device 124.

[0032] In some specific implementations, the controller 102 and / or the electronic device 124 causes the ER representation of the user 150 to move within the ER scene based on movement information (e.g., body pose data, eye tracking data, hand tracking data, etc.) from the electronic device 124 and / or optional remote input devices within the physical scene 105. In some specific implementations, the optional remote input devices correspond to fixed or movable sensory devices (e.g., image sensors, depth sensors, infrared (IR) sensors, event cameras, microphones, etc.) within the physical scene 105. In some specific implementations, each of these remote input devices is configured to collect / capture input data when the user 150 is physically within the physical scene 105 and provide the input data to the controller 102 and / or the electronic device 124. In some specific implementations, the remote input device includes a microphone, and the input data includes audio data (e.g., voice samples) associated with the user 150. In some specific implementations, the remote input device includes an image sensor (e.g., a camera), and the input data includes an image of the user 150. In some specific implementations, the input data characterizes the body pose of the user 150 at different times. In some specific implementations, the input data characterizes the head pose of the user 150 at different times. In some specific implementations, the input data characterizes hand tracking information associated with the hands of the user 150 at different times. In some specific implementations, the input data characterizes the speed and / or acceleration of a body part (such as his / her hand) of the user 150. In some specific implementations, the input data indicates the joint positions and / or joint orientations of the user 150. In some specific implementations, the remote input device includes feedback devices such as speakers, lights, etc.

[0033] Figures 2A to 2DShows a comparison between a first ER presentation scene 200a sequence and a second ER presentation scene 200b sequence according to some specific implementations. Although related features are shown, those of ordinary skill in the art will recognize from this disclosure that various other features are not shown for the sake of brevity and to not obscure more relevant aspects of the exemplary specific implementations disclosed herein.

[0034] Figures 2A to 2D The shown ER presentation scene sequences depict a comparison between an electronic device 124 and / or a controller (e.g., Figure 1 and Figure 6 the shown controller 102) that adjusts the ER content in the first ER presentation scene 200a and the second ER presentation scene 200b. This comparison highlights the similarities and differences in generating adaptive ER content based on the spatial characteristics of a first physical set 202 corresponding to a large empty room and the spatial characteristics of a second physical set 204 corresponding to a small room filled with objects.

[0035] Figure 2A Shows a first state 201a (e.g., associated with T1 or a first time period) of a first ER presentation scene 200a associated with a first physical set 202 and a second ER presentation scene 200b associated with a second physical set 204. Referring to Figure 2A the first ER presentation scene 200a in, the first physical set 202 includes an empty room having a first x - dimension 203a, a first y - dimension 205a, and a first z - dimension 206a. As shown in the second ER presentation scene 200b, the second physical set 204 includes a room having a second x - dimension 203b, a second y - dimension 205b, and a second z - dimension 206b. For example, the second x - dimension 203b, the second y - dimension 205b, and the second z - dimension 206b of the second physical set 204 are significantly smaller than the first x - dimension 203a, the first y - dimension 205a, and the first z - dimension 206a of the first physical set 202. Thus, compared to the first physical set 202, the second physical set 204 corresponds to a much smaller volume size. Further, compared to the empty room in the first physical set 202, the second ER presentation scene 200b includes physical objects within the second physical set 204, such as chairs 220a, 220c, tableware 220b, coffee tables 220d, 220e, and a sofa 210.

[0036] In some specific implementations, when the user's field of view is surrounded, the electronic device 124 is configured to implement video passthrough of the first physical scene 202 on the display 122. In some specific implementations, the electronic device 124 is further configured to present the first ER presentation scene 200a on the display 122. In some specific implementations, the display 122 corresponds to an additive display that implements optical passthrough of the first physical scene 202. For example, the display 122 corresponds to a transparent lens, and the electronic device 124 corresponds to a pair of glasses worn by the user. In some specific implementations, the electronic device 124 presents the first ER presentation scene 200a by displaying adaptive ER content on the additive display, and the adaptive ER content is then superimposed on the first physical scene 202 from the user's perspective. In some specific implementations, the electronic device 124 presents the first ER presentation scene 200a by reproducing the adaptive ER on the additive display, and the adaptive ER is then superimposed on the first physical scene 202 from the user's perspective.

[0037] In some specific implementations, when the user's field of view is surrounded, the electronic device 124 is configured to implement video passthrough of the second physical scene 204 (including physical objects) on the display 122. In some specific implementations, the electronic device 124 is further configured to present the second ER presentation scene 200b on the display 122. In some specific implementations, the display 122 corresponds to an additive display that implements optical passthrough of the second physical scene 204 (including physical objects). For example, the display 122 corresponds to a transparent lens, and the electronic device 124 corresponds to a pair of glasses worn by the user. In some specific implementations, the electronic device 124 presents the second ER presentation scene 200b by displaying adaptive ER content on the additive display, and the adaptive ER content is then superimposed on the second physical scene 204 from the user's perspective. In some specific implementations, the electronic device 124 presents the second ER presentation scene 200b by reproducing the adaptive ER on the additive display, and the adaptive ER content is then superimposed on the second physical scene 204 from the user's perspective.

[0038] Figure 2BShows a second state 201b of the first ER presentation scene 200a and the second ER presentation scene 200b (e.g., associated with T2 or the second time period). In some embodiments, the electronic device 124 and / or the controller identify multiple subsets associated with the physical scene. In some embodiments, the electronic device 124 obtains image data corresponding to the physical scene via an outward-facing image sensor of the electronic device. Subsequently, the electronic device 124 identifies multiple subsets within the physical scene at least in part based on the image data, and determines a set of spatial characteristics of the multiple subsets at least in part based on the image data and / or depth information. In some embodiments, the electronic device 124 obtains depth information, image data, etc. from one or more optional remote input devices.

[0039] In some embodiments, the electronic device 124 identifies multiple subsets of the first physical scene 202, including a first subset 230a and a second subset 232a. In some embodiments, the electronic device 124 determines the multiple subsets based on metrics not associated with physical partitioning. For example, if the physical scene does not include physical partitioning, the electronic device 124 identifies the multiple subsets by dividing the physical scene in half and identifying the first half of the physical scene as the first subset and the second half of the physical scene as the second subset. Similarly, as shown in the second ER presentation scene 200b, the electronic device 124 also identifies multiple subsets of the second physical scene 204 as a first subset 230b and a second subset 232b. In some embodiments, the electronic device 124 determines the multiple subsets based on physical partitioning. For example, referring to Figure 3A , if the physical scene corresponds to a house 301, the electronic device 124 identifies the multiple subsets based on each individual room (e.g., basement 310, kitchen 320, bedroom 330, and living room 340). As another example, if the physical scene corresponds to a single room, the electronic device 124 identifies the multiple subsets based on the different walls of the single room. As yet another example, if the physical scene corresponds to a single room, the electronic device 124 identifies the multiple subsets based on the corners of the room.

[0040] In some embodiments, the electronic device 124 determines a set of spatial characteristics for each of the multiple subsets within the physical scene. As shown in the first ER presentation scene 200a, the electronic device 124 determines a first set of spatial characteristics for the first subset 230a, including, for example: the volume size of the first subset 230a based on the x-dimension 207a, y-dimension 209a, and z-dimension 216a; an indication that there are no physical objects within the first subset 230a; and so on. Additionally, the electronic device 124 also determines a second set of spatial characteristics for the second subset 232a, including, for example: the volume size of the second subset 232a based on the x-dimension 211a, y-dimension 213a, and z-dimension 215a; an indication that there are no physical objects within the first physical scene 202; and so on.

[0041] Similarly, as shown in the second ER presentation scene 200b, the electronic device 124 also determines a first set of spatial features of the first subset 230b, including, for example: the volume size of the first subset 230b based on the x-dimension 207b, y-dimension 209b, and z-dimension 216b; an indication that there are physical objects (e.g., chair 220c, bookcase 220b, coffee table 220e, and sofa 210) within the second physical setting 204; and so on. Additionally, the electronic device 124 also determines a second set of spatial features of the second subset 232b, including, for example: the volume size of the second subset 232b based on the x-dimension 211b, y-dimension 213b, and z-dimension 215b; an indication that there are physical objects (e.g., chair 220a, bookcase 220b, coffee tables 220d, 220e, and sofa 210) within the second physical setting 204; and so on. As Figure 2B shown, the sizes of the first subset 230b and the second subset 232b of the second physical setting 204 are much smaller than the sizes of the first subset 230a and the second subset 232a of the first physical setting 202.

[0042] Figure 2C A third state 201c (e.g., associated with T3 or the third time period) of the first ER presentation scene 200a and the second ER presentation scene 200b is shown. In some specific implementations, after logically mapping the first ER portion to the first subset 230a, the electronic device 124 and / or the controller 102 generate a first ER content portion adjusted from pre-determined ER content at least partially based on the first set of spatial features. Referring Figure 2C to the first ER presentation scene 200a in Figure 2C as shown, the electronic device 124 generates adaptive first ER contents 240a-1, 240a-2, and 240a-3 based on the first set of spatial features of the first subset 230a of the first physical setting 202.

[0043] Similarly, in some specific implementations, after logically mapping the first ER portion to the first subset 230b, the electronic device 124 and / or the controller 102 generate an adaptive first ER content portion by adjusting pre-determined ER content based on the first set of spatial features. Referring Figure 2C to the second ER presentation scene 200b in Figure 2CAs shown, the electronic device 124 presents adaptive first ER content 240b-1, 240b-2, and 240b-3 within the first subset 230b of the second physical scene 204. However, compared to the first ER presentation scene 200a, due to the size difference between the first subset 230a of the first physical scene 202 and the first subset 230b of the second physical scene 204, the adaptive ER content 240b-1, 240b-2, and 240b-3 in the first subset 230b is scaled down to a smaller size than the adaptive ER content 240a-1, 240a-2, 240a-3 in the first subset 230a.

[0044] In some specific implementations, after logically mapping the second ER portion to the second subset 232a, the electronic device 124 and / or the controller 102 generate a second ER content portion by adjusting a pre-determined ER content based on the second set of spatial features. Refer to Figure 2C the first ER presentation scene 200a in, the electronic device 124 generates adaptive second ER content 242a-1, 242a-2, 242a-3, and 242a-4 based on the second set of spatial features of the second subset 232a of the first physical scene 202. As Figure 2C shown, the electronic device 124 presents adaptive second ER content 242a-1, 242a-2, 242a-3, and 242a-4 within the second subset 232ba of the first physical scene 202.

[0045] Similarly, in some specific implementations, after logically mapping the second ER portion to the second subset 232b, the electronic device 124 and / or the controller 102 generate a second ER content portion by adjusting a pre-determined ER content based on the second set of spatial features. Refer to Figure 2C the second ER presentation scene 200b in, the electronic device 124 generates adaptive second ER content 242b-1 and 242b-3 based on the second set of spatial features of the second subset 232b of the second physical scene 204. As Figure 2CAs shown, the electronic device 124 presents adaptives 242b-1 and 242b-3 within the second subset 232b of the second physical scene 204. However, compared to the first ER presentation scene 200a, when generating the adaptive second ER content in the second subset 232b, due to the volume size of the second subset 232b and the indication of objects (e.g., sofa 210, chair 220a, and coffee table 220d) within the second subset 232b of the second physical scene 204, the electronic device 124 removes the corresponding adaptive ER contents 242a-2 and 242a-4 (e.g., shown in the second subset 232b of the first physical scene 202). Additionally, compared to the first ER presentation scene 200a, due to the size difference between the second subset 232a of the first physical scene 202 and the second subset 232b of the second physical scene 204, the adaptive ER contents 242b-1 and 242b-3 in the second subset 232b are also scaled down to a smaller size than the adaptive ER contents 242a-1 and 242a-3 in the second subset 232a. Thus, in the second ER presentation scene 200b, the electronic device 124 presents a smaller scaled-down version of the second ER contents 242b-1 and 242b-3 in the second subset 232b and abandons presenting the versions of the adaptive ER contents 242a-2 and 242a-4 in the second subset 232b.

[0046] Figure 2D The fourth state 201d (e.g., associated with T4 or the fourth time period) of the first exemplary ER presentation scene 200a and the second ER presentation scene 200b is shown. In some specific implementations, the electronic device 124 and / or the controller generate one or more navigation options based on the first set of spatial features and the second set of spatial features that allow a user to travel between the first subset and the second subset.

[0047] Referring Figure 2D to the first ER presentation scene 200a in Figure 2DAs shown, the first navigation path 217a winds between the first adaptive ER contents 240a-1, 240a-2, and 240a-3 in the first subset 230a and between the second adaptive ER contents 242a-4, 242a-3, 242a-2, and 242a-1 in the second subset 232a.

[0048] Referring Figure 2D to the second ER presentation scene 200b in [reference], the electronic device 124 generates navigation options including a second navigation path 217b to allow the user 10 to move between a first subset 230b (not shown) and a second subset 232b (not shown) of the second physical set 204. However, compared to the first ER presentation scene 200a, due to the smaller size of the second physical set 204 and the indication of objects (e.g., chairs 220a, 220c, bookcase 220b, coffee tables 220d, 220e, and sofa 210) within the second physical set 204, the electronic device 124 determines that the second navigation path 217b corresponds to a shorter circular navigation path rather than a long winding path. As Figure 2D shown, the second navigation path 217b surrounds the adaptive first ER contents 240b-1, 240b-2, and 240b-3 in the first subset 230b and the adaptive second ER contents 242b-1, 242b-3 in the second subset 232b.

[0049] Figures 3A to 3F An exemplary sequence of ER presentation scenes 300 for generating adaptive ER content based on spatial characteristics of multiple subsets within a physical set is shown. Although relevant features are shown, those of ordinary skill in the art will recognize from this disclosure that, for the sake of brevity and to not obscure more relevant aspects of the exemplary embodiments disclosed herein, various other features are not shown. Figures 3A to 3F The sequence shown depicts the electronic device 124 generating adaptive ER content associated with a time linear graph based on a set of spatial characteristics for each room within a house 301.

[0050] Figure 3AShows a first state 301a of the ER presentation scene 300 (e.g., associated with T1 or the first time period). However, before the first state 301a, the electronic device 124 acquires image data corresponding to the house 301 via the outward-facing image sensor of the electronic device 124 to identify multiple subsets (e.g., rooms) within the house 301 and determine a set of spatial characteristics of the multiple subsets at least partially based on the image data. In some specific implementations, the electronic device 124 uses sensor information from one or more remote input devices (e.g., optional remote input devices) within each room to collect / capture input data and provide the input data to the electronic device 124 and / or the controller 102 with the informed consent of the user. In some specific implementations, the electronic device 124 may extract a set of spatial characteristics of the multiple subsets at least partially based on data acquired from a depth sensor associated with the electronic device 124. In some specific implementations, the electronic device 124 and / or the controller 102 obtain a known set of spatial characteristics of the multiple subsets from an external database or repository. Those of ordinary skill in the art will understand that there are many ways to acquire image data corresponding to a physical setting or obtain a known set of spatial characteristics from an external database. For the sake of brevity, an exhaustive list of all such methods is not provided herein.

[0051] In some specific implementations, the first ER content portion corresponds to a first thematic scene within pre-determined content, and the second ER content portion corresponds to a second thematic scene within the pre-determined content. In some specific implementations, the pre-determined content corresponds to ER content associated with real-life events, stories, movies, TV drama series, etc. For example, as Figures 3A to 3F shown, the ER content portion is associated with a thematic scene within the pre-determined content, and the pre-determined content corresponds to a story having five parts. Thus, the pre-determined content is divided into five ER content portions that are linked together and presented to the user 10 in a time-ordered sequence to convey a linear story to the user 10. Thus, in Figures 3A to 3F , the pre-determined content corresponds to a time-linear graph having a start, rising action, climax, falling action, and resolution. To this end, the electronic device 124 presents an adaptive first ER content (e.g., start), then presents an adaptive second ER content portion (e.g., rising action), then presents an adaptive third ER content portion (e.g., climax), and so on.

[0052] As Figure 3A shown, the house 301 includes at least a basement 310, a kitchen 320, a bedroom 330, and a living room 340. In the first state 301a, the user 10 wearing the electronic device 124 is located in the basement 310. As previously mentioned, in Figure 3ABefore time T1, the electronic device 124 identifies each room as a subset of the house 301. Additionally, the electronic device 124 obtains an ER content portion from predetermined content for presentation to the user 10 within the user interface of the electronic device 124. In some specific implementations, the electronic device 124 obtains the predetermined content from an ER library or database.

[0053] In some specific implementations, the electronic device 124 may determine that a particular ER content portion should be presented in a particular room of the house 301 based on a set of spatial features characterizing one or more dimensions of the particular room. For example, a first ER content portion may be more suitable for a large room because it includes the most ER content compared to other ER content portions. Subsequently, the electronic device 124 logically maps the first ER content portion to the basement 310 rather than to other rooms in the house 301 because the basement 310 has the largest volume size among the rooms within the house 301. As another example, a second ER content portion may be more suitable for the smallest room in the house 301 because it includes less ER content compared to other ER content portions. Thus, the electronic device 124 logically maps the second ER content portion to the kitchen 320 because the kitchen 320 is the smallest room in the house 301.

[0054] In some specific implementations, after the electronic device 124 logically maps the ER content portions to multiple subsets, the electronic device 124 generates one or more navigation options that allow the user 10 to traverse between subsets of the house 301 (e.g., the basement 310, the kitchen 320, the bedroom 330, and the living room 340) in an ordered sequence. In some specific implementations, the user 10 navigates between subsets by physically walking between the rooms of the house 301. To this end, the electronic device 124 determines a navigation path 303 that links the rooms to the ER content portions that will be presented to the user 10 in a time-ordered sequence (e.g., first presenting the adaptive first ER content portion 314, then presenting the adaptive second ER content portion 316, and so on). In Figures 3A to 3F an example, the electronic device 124 determines that five ER content portions should be presented in such a way that the user traverses from the bottom of the house 301 to the second floor of the house 301 along the navigation path 303. As will be described below in Figure 3E and Figure 3FAs explained in more detail, since there are five ER content portions in the pre-determined content to be presented to the user, but only four subsets are associated with the house 301, the electronic device 124 presents the adaptive fourth ER content portion 318 and the adaptive fifth ER content portion 319 in the living room 340. Thus, the electronic device 124 starts the story by presenting the adaptive first ER content portion (e.g., start) to the user 10 in the basement 310, and completes the story by presenting the adaptive fifth ER content portion (e.g., resolution) to the user in the living room 340.

[0055] Figure 3B Shows a second state 301b of the ER presentation scene 300 (e.g., associated with T2 or the second time period). In the second state 301b, at least a portion of the basement 310 is within the field of view 311a of the electronic device 124. As Figure 3B shown, the basement 310 includes a set of stairs. In other words, while holding or wearing the electronic device 124, the user is observing the set of stairs 312 from a side view or perspective orientation through the electronic device 124. In some specific embodiments, after logically mapping the first ER content portion to the basement 310, the electronic device 124 generates the adaptive first ER content portion 314 based at least in part on a first set of spatial features of the basement 310. As Figure 3B shown, in response to determining that a first presentation criterion is met, the electronic device 124 presents a user interface 302 on the display 122, the user interface including the set of stairs 312, a navigation path 303 (optional), and the adaptive first ER content portion 314. Here, the first presentation criterion may correspond to the position of the electronic device 124. At time T3, as Figure 3C shown, the electronic device 124 moves to the second subset (e.g., the kitchen 320) by following the navigation path 303 in order to meet a second presentation criterion (e.g., the position of the electronic device 124) for displaying the adaptive second ER content portion 316.

[0056] Figure 3C Shows a third state 301c of the ER presentation scene 300 (e.g., associated with T3 or the third time period). Compared with Figure 3B the field of view of the electronic device 124 changes due to the translational movement of the electronic device 124 from the basement 310 (e.g., the first subset of the house 301) to the kitchen 320 (e.g., the second subset of the house 301). As Figure 3CAs shown, in the third state 301c, the field of view 311b of the electronic device 124 in the kitchen 320 includes a partial view of the dining table 322 and the refrigerator 324. In some specific embodiments, after logically mapping the second ER content portion to the kitchen 320, the electronic device 124 generates an adaptive second ER content portion 316 at least in part based on the second set of spatial features of the kitchen 320. As Figure 3C shown, in response to determining that the second presentation criteria are met, the electronic device 124 presents a user interface 302 on the display 122, the user interface including partial views of the dining table 322, the refrigerator 324, the navigation path 303 (optional), and the adaptive second ER content portion 316. At time T4, as Figure 3D shown, the electronic device 124 moves to the third subset (e.g., the bedroom 330) by following the navigation path 303 in order to meet the third presentation criteria for displaying the adaptive third ER content portion 317.

[0057] Figure 3D The fourth state 301d of the ER presentation scene 300 is shown (e.g., associated with T4 or the fourth time period). As compared with Figure 3C the field of view of the electronic device 124 changes due to the translational movement of the electronic device 124 from the kitchen 320 (e.g., the second subset of the house 301) to the bedroom 330 (e.g., the third subset of the house 301). As Figure 3D shown, in the fourth state 301d, the field of view 311c of the electronic device 124 in the bedroom 330 includes a partial view of the bed 332. In some specific embodiments, after logically mapping the third ER content portion to the bedroom 330, the electronic device 124 generates an adaptive third ER content portion 317 at least in part based on the third set of spatial features of the bedroom 330. As Figure 3D shown, in response to determining that the third presentation criteria are met, the electronic device 124 presents a user interface 302 on the display 122, the user interface including partial views of the bed 332, the navigation path 303 (optional), and the adaptive third ER content portion 317. At time T5, as Figure 3E shown, the electronic device 124 moves to the fourth subset (e.g., the living room 340) by following the navigation path 303 in order to meet the fourth presentation criteria for displaying the adaptive fourth ER content portion 318.

[0058] Figure 3E The fifth state 301e of the ER presentation scene 300 is shown (e.g., associated with T5 or the fifth time period). As compared with Figure 3D the field of view of the electronic device 124 changes due to the translational movement of the electronic device 124 from the bedroom 330 (e.g., the third subset of the house 301) to the living room 340 associated with the house 301 (e.g., the fourth subset of the house 301). AsFigure 3E As shown, in the fifth state 301e, the field of view 311d of the electronic device 124 includes the sofa 342. In some specific implementations, after logically mapping the fourth ER content portion to the living room 340, the electronic device 124 generates the adaptive fourth ER content portion 318 at least in part based on the fourth set of spatial characteristics of the living room 340. As Figure 3E shown, in response to determining that the fourth presentation criteria are met, the electronic device 124 presents a user interface 302 on the display 122, the user interface including the sofa 342, the navigation path 303 (optional), and the adaptive fourth ER content portion 318 for the time parameter.

[0059] Compared with Figures 3B to 3D the first presentation criteria, the second presentation criteria, and the third presentation criteria in, the fourth presentation criteria and the fifth presentation criteria correspond to a time criterion associated with pre-determined content and a coordinate-based or location-based criterion. The time criterion is associated with the adaptive fourth ER content portion 318 and the fifth ER content portion 319 because the electronic device 124 presents both the adaptive fourth ER content portion 318 and the adaptive fifth ER content portion 319 in the fourth subset (e.g., the living room 340). Thus, the electronic device 124 presents the adaptive fourth ER content portion 318 at least in part based on the time criterion (e.g., playback time from "9:30" to "10:00") before presenting the adaptive fifth ER content portion 319. In this example, for reference, the ER presentation scene 300 depicts a playback time 344 of "9:45", and the electronic device 124 presents the adaptive fourth ER content portion 318 until "10:00". In some specific implementations, the user 10 can navigate between different ER content portions in the same subset at least in part based on physical movement (such as turning his or her head towards a specific area within the physical set).

[0060] Figure 3F shows a sixth state 301f of the ER presentation scene 300 (e.g., associated with T6 or the sixth time period). Compared with Figures 3A to 3D in, Figure 3F the field of view 311d of the electronic device 124 shown in is the same as that in Figure 3E because the electronic device 124 is stationary. However, compared with Figure 3E in, Figure 3F at time T6 in, the passage of time causes the playback time 346 to now be "10:15". In some specific implementations, after logically mapping the fifth ER content portion to the living room 340, the electronic device 124 generates the adaptive fifth ER content portion 319 at least in part based on the fifth set of spatial characteristics of the living room 340. Thus, in Figure 3FAt time T6, the fifth presentation criterion is met because the adaptive fourth ER content portion 318 exceeds its time limit. As Figure 3F shown by time T6 in, in response to determining that the fifth presentation criterion (e.g., a time criterion) is met, the electronic device 124 presents a user interface 302 on the display 122, the user interface including a sofa 342 and an adaptive fifth ER content portion 319.

[0061] In some specific implementations, the predetermined content may be non-linear such that the order in which the electronic device 124 presents the ER content portions from the predetermined content does not matter. In some specific implementations, the electronic device 124 obtains a first set of environmental characteristics associated with a first subset and a second set of environmental characteristics associated with a second subset, wherein generating the adaptive first ER content portion includes logically mapping the first ER content portion to the first subset based at least in part on a first set of spatial characteristics of the first subset and the first set of environmental characteristics associated with the first subset and generating the adaptive second ER content portion includes logically mapping the second ER content portion to the second subset based at least in part on a second set of spatial characteristics of the second subset and the second set of environmental characteristics associated with the second subset. In some specific implementations, adjusting the ER content portion may be based on environmental characteristics such as room type, temperature information, lighting information, objects within the physical setting, time of day, background color of the physical setting, etc. Those of ordinary skill in the art will understand that there are many different types of environmental characteristics. For the sake of brevity, an exhaustive list of all such types is not provided herein.

[0062] As a non-limiting example, the electronic device 124 may be configured to present portions of the predetermined content corresponding to different ecosystems based on the environmental characteristics of the subset associated with the physical setting. Continuing with the previous non-limiting example, a first portion of the predetermined content may correspond to ER content associated with an underground ecosystem, a second portion of the predetermined content may correspond to ER content associated with a grassland ecosystem, a third portion of the predetermined content may correspond to ER content associated with a forest ecosystem, and a fourth portion of the predetermined content may correspond to ER content associated with a desert ecosystem.

[0063] Reference Figure 3A, for example, the electronic device 124 obtains environmental characteristics associated with different rooms within the house 301 in order to determine which rooms meet the mapping criteria for a portion of the predetermined content. For example, the first portion of the predetermined content associated with the underground ecosystem includes: a set of mapping criteria for a subset that is located underground, has the coldest temperature conditions among multiple subsets, and has the darkest lighting conditions among multiple subsets; and a first position-based presentation criterion corresponding to the position of the electronic device 124. Thus, the electronic device 124 logically maps the first portion of the predetermined content associated with the underground ecosystem to the basement 310 because the basement 310 meets the set of mapping criteria of being located underground and having the coldest temperature and the darkest lighting conditions compared to the remaining rooms in the house 301.

[0064] Again, the second portion of the predetermined content associated with the grassland ecosystem includes: a set of mapping criteria for a subset that has hot temperature conditions and has the brightest lighting conditions among multiple subsets; and a second position-based presentation criterion corresponding to the position of the electronic device 124. Thus, the electronic device 124 logically maps the second portion of the predetermined content associated with the grassland ecosystem to the kitchen 320 because the kitchen 320 meets the set of mapping criteria of having the brightest lighting conditions compared to the remaining rooms in the house 301.

[0065] Again, the third portion of the predetermined content associated with the forest ecosystem includes: a set of mapping criteria for a subset that is located on the second floor; and a third position-based presentation criterion corresponding to the position of the electronic device 124. Thus, the electronic device 124 logically maps the third portion of the predetermined content associated with the forest ecosystem to the bedroom 330 because the bedroom 330 meets the mapping criterion of being located on the second floor of the house 301.

[0066] Again, the fourth portion of the predetermined content associated with the desert ecosystem includes: a set of mapping criteria for a subset that has the hottest temperature among multiple subsets; and a fourth position-based presentation criterion corresponding to the position of the electronic device 124. Thus, the electronic device 124 logically maps the fourth portion of the predetermined content associated with the desert ecosystem to the living room 340 because the living room 340 has the hottest temperature compared to the remaining rooms in the house 301.

[0067] Continuing with the above example, at time T2, as Figure 3B shown, the electronic device 124 is located in the basement. Thus, in response to determining that the electronic device 124 meets the first position-based presentation criterion, the electronic device 124 displays an adaptive first portion of the predetermined content associated with the underground ecosystem that is superimposed on the first field of view 311a of the electronic device 124 corresponding to the basement 310. At time T3, as Figure 3CAs shown, the electronic device 124 is located in the kitchen 320. Accordingly, in response to determining that the electronic device 124 meets the presentation criteria based on the second location, the electronic device 124 displays an adaptive second portion of pre-determined content associated with a grassland ecosystem superimposed on a second field of view 311b of the electronic device 124 corresponding to the kitchen 320. At time T4, as Figure 3D As shown, the electronic device 124 is located in the bedroom 330. Accordingly, in response to determining that the electronic device 124 meets the presentation criteria based on the third location, the electronic device 124 displays an adaptive third portion of pre-determined content associated with a forest ecosystem superimposed on a third field of view 311c of the electronic device 124 corresponding to the bedroom 330. At time T5, as Figure 3E As shown, the electronic device 124 is located in the living room 340. Accordingly, in response to determining that the electronic device 124 meets the presentation criteria based on the fourth location, the electronic device 124 displays an adaptive fourth portion of pre-determined content associated with a desert ecosystem superimposed on a fourth field of view 311d of the electronic device 124 corresponding to the living room 340.

[0068] Figures 4A to 4C Another ER presentation scene sequence 400 for generating adaptive ER content based on spatial features of multiple subsets within a physical setting according to some specific implementations is shown. Although related features are shown, those of ordinary skill in the art will recognize from this disclosure that various other features are not shown for the sake of brevity and to not obscure more relevant aspects of the exemplary specific implementations disclosed herein.

[0069] Figure 4A A first state 401a (e.g., associated with T1 or a first time period) of an exemplary ER presentation scene 400 is shown. In the first state 401a, at least a portion of the physical setting is within the field of view 411a of the electronic device 124. In some specific implementations, the electronic device 124 determines a connectivity matrix between multiple subsets within the physical setting and generates additional ER content at least in part based on the connectivity matrix between the multiple subsets within the physical setting for presenting the additional ER content. In this example, the corridor 404 is not part of the multiple subsets within the physical setting but corresponds to the space between the multiple subsets.

[0070] As Figure 4AAs shown, the corridor 404 includes doors leading to rooms (e.g., a first subset of the physical set). Thus, the electronic device 124 generates additional ER content to bridge multiple subsets within the physical set. In some embodiments, the additional ER content corresponds to emergent ER content. In some embodiments, the additional ER content corresponds to ER content selected from pre-determined content. In some embodiments, the additional ER content corresponds to emergent ER content generated based on logically mapping ER content portions to previous and subsequent subsets. In some embodiments, one or more navigation options are generated at least partially based on a connectivity matrix. Here, the electronic device 124 generates a navigation option that includes a navigation path 412 based at least partially on the corridor 404. As Figure 4A shown, the electronic device 124 presents a user interface 402 on the display 122, which includes the corridor 404, the navigation path 412 (optional), and additional ER content 410a, 410b. At time T2, as Figure 4B shown, the electronic device 124 moves to the first subset (e.g., the room 406) by following the navigation path 412 in order to meet the presentation criteria for displaying adaptive ER content.

[0071] Figure 4B shows a second state 401b of an exemplary ER presentation scene 400 (e.g., associated with T2 or a second time period). Compared with Figure 4A , the field of view of the electronic device 124 changes due to the translational movement of the electronic device 124 from the corridor 404 to the interior of the room 406. As Figure 4B shown, in the second state 401b, the field of view 411b of the electronic device 124 includes a painting 416 and a door 420. In some embodiments, after logically mapping a first ER content portion to the room 406, the electronic device 124 generates adaptive ER content (e.g., a virtual agent 418) at least partially based on a first set of spatial features of the room 406. In this example, the first presentation criteria associated with the adaptive ER content are location-based. Thus, in response to determining that the first presentation criteria are met, the electronic device 124 presents a user interface 402 on the display 122, which includes the painting 416, the door 420, and a virtual agent 418 that appears to be standing in the room 406.

[0072] In some embodiments, the electronic device 124 presents one or more ER content items associated with ER content moving from a first subset to a second subset. Figure 4B and Figure 4C show a sequence depicting the virtual agent 418 associated with ER content moving from the room 406 to another subset within the physical set while the field of view 411b of the electronic device 124 is stationary.

[0073] Figure 4C shows the third state 401c of the ER presentation scene 400 (e.g., associated with T3 or the third time period). Compared with FIG. 4D, the field of view 411b of the electronic device is the same, but at Figure 4C time T3, the virtual agent 418 appears to move from the room 406 (e.g., the first subset) towards another room outside the door 420 within the physical set. Accordingly, the electronic device 124 generates a path indicator 414 that allows the user to follow the virtual agent 418 across the door 420 within the physical set to reach another room. In some embodiments, following the path indicator 414 enables the user to move between the room 406 (e.g., the first subset) and another room across the door 420 (e.g., the second subset within the physical set). Accordingly, at Figure 4C time, the electronic device 124 presents a user interface 402 on the display 122, which includes the painting 416, the door 420, the virtual agent 418 moving from the room 406 to the second subset, and the path indicator 414 (optional).

[0074] Figure 5 is a flowchart representation of a method 500 for generating adaptive ER content based on spatial features of multiple subsets within a physical set according to some embodiments. In various embodiments, the method 500 is executed at an electronic device having an image sensor, one or more processors, non-transitory memory, and a display (e.g., Figure 1 and Figure 7 the electronic device 124 shown in Figure 1 and Figure 6 the controller 102 shown in

[0075] Figure 1 or a suitable combination thereof). In some embodiments, the method 500 is executed by processing logic (including hardware, firmware, software, or a combination thereof). In some embodiments, the method 500 is executed by a processor executing code stored in a non-transitory computer-readable medium (e.g., memory). As shown in block 510, the method 500 includes identifying multiple subsets associated with a physical set (e.g., Figure 1 the physical set 105 shown in Figures 2A to 2D or Figures 2A to 2D the first physical set 202 and the second physical set 204 shown in Figures 3A to 3E In some embodiments, the multiple subsets may correspond to portions of a room associated with the physical set. For example, as shown in Figures 3A to 3E the multiple subsets correspond to portions of a single room within the first physical set 202 and the second physical set 204. In some embodiments, the multiple subsets may correspond to rooms within the physical set. For example, as shown in Figures 3A to 3E the multiple subsets correspond to separate rooms associated with the house 301 (e.g., the basement 310, the kitchen 320, the bedroom 330, and the living room 340).

[0076] In some specific implementations, the electronic device determines the multiple subsets based on a physical division. For example, referring to Figure 3A , if the physical scene corresponds to House 301, the electronic device 124 identifies multiple subsets (e.g., basement 310, kitchen 320, bedroom 330, and living room 340) based on each individual room. As another example, if the physical scene corresponds to a single room, the electronic device 124 identifies multiple subsets based on the different walls of the single room. As yet another example, if the physical scene corresponds to a single room, the electronic device 124 identifies multiple subsets based on the corners of the room.

[0077] In some specific implementations, the electronic device 124 determines the multiple subsets based on a metric not associated with a physical division. In some specific implementations, the electronic device determines the multiple subsets based on a metric not associated with a physical division. For example, if the physical scene does not include a physical division, the electronic device 124 identifies multiple subsets by dividing the physical scene into two halves and identifying the first half of the physical scene as the first subset and the second half of the physical scene as the second subset. For example, referring to Figure 2B , in the first ER presentation scene 200a, the electronic device 124 divides the first physical scene 202 into two halves and identifies the first half of the first physical scene 202 as the first subset 230a and the second subset 232a. As another example, also referring to Figure 2B , in the second ER presentation scene 200b, the electronic device 124 divides the second physical scene 204 into two halves and identifies the first half of the second physical scene 204 as the first subset 230b and the second half of the second physical scene 204 as the second subset 232b.

[0078] As shown in block 520, method 500 includes determining a set of spatial features for each of the multiple subsets within the physical scene, where the first set of spatial features characterizes one or more dimensions of the first subset of the multiple subsets, and the second set of spatial features characterizes one or more dimensions of the second subset of the multiple subsets. In some specific implementations, the first set of spatial features includes at least one of the following: the volume size of the first subset of the physical scene, an indication that there are physical objects within the first subset of the physical scene, the shape of the first subset of the physical scene, etc. In some specific implementations, one or more dimensions may correspond to the width, length, height, etc. of the subset within the multiple subsets.

[0079] For example, as Figure 2BThe first ER in [scene 200a] is shown. The first set of spatial features of the first subset 230a includes the volume size of the first subset 230a based on the x-dimension 207a, y-dimension 209a, and z-dimension 216a within the first physical setting 202, and an indication that there are no physical objects within the first physical setting 202; and the second set of spatial features of the second subset 232a includes the volume size of the second subset 232a based on the x-dimension 211a, y-dimension 213a, and z-dimension 215a, and an indication that there are no physical objects within the first physical setting 202.

[0080] In another example, as Figure 2B The second ER in [scene 200b] is shown. The first set of spatial features of the first subset 230b includes the volume size of the first subset 230b based on the x-dimension 207b, y-dimension 209b, and z-dimension 216b within the second physical setting 204, and an indication that there are physical objects (e.g., chair 220c, bookcase 220b, coffee table 220e, and sofa 210) within the second physical setting 204; and the second set of spatial features of the second subset 232b includes the volume size of the second subset 232b based on the x-dimension 211b, y-dimension 213b, and z-dimension (215b), and an indication that there are physical objects (e.g., chair 220a, bookcase 220b, coffee table 220d, 220e, and sofa 210) within the second physical setting 204. In some specific implementations, the spatial features may include a point cloud of the physical setting marked with objects. In some specific implementations, the electronic device determines the set of spatial features by performing semantic segmentation or instance segmentation. In some specific implementations, semantic segmentation corresponds to detecting and labeling the objects that appear within the image data. In some specific implementations, instance segmentation corresponds to detecting and depicting the different objects that appear within the image data.

[0081] As shown in block 530, method 500 includes generating an adaptive first augmented reality (ER) content portion of a first subset of the plurality of subsets based at least in part on the first set of spatial features. Specifically, in some specific implementations, the electronic device 124 logically maps the first portion of the ER content to the first subset that satisfies the first mapping criterion associated with the first portion of the ER content. Next, in some specific implementations, the electronic device 124 generates the adaptive first ER content portion by adjusting a reference ER content (e.g., the first portion of the ER content) based on the spatial features of the first subset. Finally, in some specific implementations, in response to determining that the presentation criterion associated with the adaptive first ER content portion is satisfied, the electronic device 124 presents the adaptive first ER content portion within the first subset.

[0082] For example, in Figure 2B and Figure 2CIn the first ER presentation scene 200a shown, after logically mapping the first ER content portion to the first subset 230a, the electronic device 124 generates the adaptive first ER content portions 240a-1, 240a-2, 240a-3 at least in part based on the x-dimension 207a, y-dimension 209a, and z-dimension 216a of the first subset 230a within the first physical setting 202. Similarly, as also shown in Figure 2B and Figure 2C the second ER presentation scene 200b shown, after logically mapping the first ER content portion to the first subset 230b, the electronic device 124 generates the adaptive first ER content portions 240b-1, 240b-2, 240b-3 at least in part based on the x-dimension 207b, y-dimension 209b, and z-dimension 216b of the first subset 230b within the second physical setting 204.

[0083] In some specific implementations, the ER content corresponds to emergent ER content or pre-determined ER content. In some specific implementations, the method 500 further includes generating emergent ER content including the first ER content portion and the second ER content portion. In some specific implementations, the ER content corresponds to a three-dimensional model of an object, person, or landscape associated with the emergent content. For example, the emergent content may correspond to one or more target executors performing actions to achieve a specific goal. Continuing with this example, as shown in Figure 4C shown, the emergent content corresponds to the virtual agent 418 (e.g., the target executor) performing an action of searching for a painting within the physical setting. Another example, as shown in Figure 4A shown, the electronic device 124 generates emergent ER content to bridge multiple subsets within the physical setting.

[0084] In some specific implementations, the method 500 further includes obtaining pre-determined content including the first ER content portion and the second ER content portion, where the first ER content portion and the second ER content portion are obtained from a database or library. In some specific implementations, the ER content corresponds to a three-dimensional model of an object, person, or landscape associated with the pre-determined content. In some specific implementations, the pre-determined content corresponds to content associated with real-life events, stories, movies, TV drama series, etc. In some specific implementations, the first ER content portion is associated with a first theme scene within the pre-determined content, and the second ER content portion is associated with a second theme scene within the pre-determined content. In some specific implementations, the pre-determined content corresponds to a time-linear graph, and the adaptive first ER content portion is presented before presenting the adaptive second ER content portion. For example, as shown in Figures 3A to 3FAs shown, the adaptive first ER content portion 314 corresponds to the beginning, the adaptive second ER content portion 316 corresponds to the rising action, the adaptive third ER content portion 317 corresponds to the climax, the adaptive fourth ER content portion 318 corresponds to the falling action, and the adaptive fifth ER content portion 319 corresponds to the resolution associated with the line graph, such that the adaptive ER content portions are linked in an orderly manner. Thus, continuing Figures 3A to 3F In the example shown, the electronic device 124 presents the adaptive first ER content portion 314, the adaptive second ER content portion 316, the adaptive third ER content portion 317, the adaptive fourth ER content portion 318, and the adaptive fifth ER content portion 319 in an ordered sequence to give meaning to the story or drawing.

[0085] In some specific implementations, the method 500 further includes generating the adaptive first ER content portion at least in part based on determining whether a first set of spatial features associated with the first subset satisfies a mapping criterion for the first ER content portion of the predetermined content. In some specific implementations, the mapping criterion corresponds to a spatial criterion for finding the best fit for the adaptive first ER content portion. In some specific implementations, the method 500 further includes, in response to determining that the first set of spatial features associated with the first subset satisfies the first mapping criterion, determining the placement of the adaptive first ER content portion within the first subset at least in part based on the first set of spatial features of the first subset, wherein presenting the adaptive first ER content portion synthesized with the first pass-through image data includes presenting the adaptive ER content according to the position. For example, as Figure 3B shown, the electronic device 124 generates the adaptive first ER content portion 314 at least in part based on determining that the dimensions associated with the basement 310 satisfy the first mapping criterion for the first ER content portion of the predetermined content as the largest room in the house 301. Thus, continuing Figure 3B In the example of, in response to determining that the first set of spatial features associated with the basement 310 satisfies the first mapping criterion as the largest room in the house 301, the electronic device 124 determines the placement of the adaptive first ER content portion 314 within the basement 310 by placing the adaptive first ER content portion 314 next to the staircase group 312. Finally, continuing Figure 3B In the example of, the electronic device 124 presents a user interface 302 on the display 122, the user interface including the staircase group 312 and the adaptive first ER content portion 314 synthesized with the first pass-through image data according to the placement.

[0086] As shown in block 540, method 500 includes generating an adaptive second ER content portion of a second subset of a plurality of subsets based at least in part on a second set of spatial features. Specifically, in some implementations, electronic device 124 logically maps a second portion of the ER content to a second subset that meets a second mapping criterion associated with the second portion of the ER content. Next, in some implementations, electronic device 124 generates the adaptive second ER content portion by adjusting a reference ER content (e.g., the second portion of the ER content) based on the spatial features of the second subset. Finally, in some implementations, in response to determining that a presentation criterion associated with the adaptive second ER content portion is met, electronic device 124 presents the adaptive second ER content portion within the second subset.

[0087] For example, in Figure 2B and Figure 2C in the first ER presentation scenario 200a shown, after logically mapping the adaptive second ER content portion to the second subset 232a, electronic device 124 generates adaptive second ER content portions 242a-1, 242a-2, 242a-3, and 242-4 based at least in part on the x-dimension 211a, y-dimension 213a, and z-dimension 215a of the second subset 232a within the first physical setting 204. Similarly, as also shown in Figure 2B and Figure 2C in the second ER presentation scenario 200b in, after logically mapping the second ER content portion to the second subset 232b, electronic device 124 generates adaptive second ER content portions 242b-1, 242b-3 based at least in part on the x-dimension 211b, y-dimension 213b, and z-dimension 215b of the second subset 232b within the second physical setting 204.

[0088] As shown in block 550, method 500 includes generating one or more navigation options that allow a user to travel between the first subset and the second subset based on a first set of spatial features and a second set of spatial features. In some implementations, the navigation options include at least one of the following: a path indicator between the first subset and the second subset, a circular path between the first subset and the second subset, and a determination of a navigation path between the first subset and the second subset. For example, as shown in Figure 2D in the first ER presentation scenario 200a in, electronic device 124 generates a first navigation path 217a in the first physical setting 202 to travel between the first subset 230a and the second subset 232a within the first physical setting 202. Also, as shown in Figure 2D in the second ER presentation scenario 200b in, electronic device 124 generates a second navigation path 217b corresponding to a circular path in the second physical setting 204 to travel between the first subset 230b and the second subset 232b within the second physical setting 204. Again, as shown inFigure 3A As shown, the electronic device 124 generates a navigation path 303 to allow a user to move between multiple subsets within a house 301 (e.g., basement 310, kitchen 320, bedroom 330, and living room 340). Finally, for another example, as Figure 4C shown, the electronic device 124 generates a path indicator 414 to allow the user to move to a subsequent subset within the physical scene. In some embodiments, the navigation options include detecting user input, such as the user's physical movement or the rotation of the user's head. In some embodiments, the navigation options include the user navigating between subsets by turning his head or physically walking to a subsequent subset.

[0089] In some embodiments, the method 500 further includes presenting, on a display, an adaptive first ER content portion superimposed on a first field of view of a device corresponding to a first subset of the physical scene in response to determining that a first presentation criterion is met; and presenting, on the display, an adaptive second ER content portion superimposed on a second field of view of the device corresponding to a second subset within the physical scene in response to determining that a second presentation criterion is met. In some embodiments, the presentation criterion may correspond to at least one of the following: a time criterion associated with the adaptive first ER content portion, coordinates of the first subset, coordinates of the second subset, adjacency between the first ER content portion and the second ER content portion, and the location of the electronic device. For example, if the first subset mapped to the first ER content portion and the second subset mapped to the second ER content portion are located close to each other, the presentation criterion may correspond to the adjacency between the first ER content portion and the second ER content portion. Continuing with this example, the adjacency presentation criterion enables the electronic device 124 to display a transition between presenting the first ER content portion and the second ER content portion that is effectively immediate and does not include an extended delay between presenting the first ER content portion and the second ER content portion. As another non-limiting example, if there are more than one subset within a single room, or if there are not enough rooms to present all the ER content, the presentation criterion may correspond to a time limit. For example, as Figure 3E and Figure 3F shown, the electronic device 124 determines whether the presentation criterion (e.g., a time parameter associated with pre-determined content) is met by checking the current time before transitioning between presenting the adaptive fourth ER content portion 318 and the adaptive fifth ER content portion 319. Continuing Figure 3E and Figure 3F with the example shown, after the passage of a period of time associated with the adaptive fourth ER content portion 318, the electronic device 124 presents the adaptive fifth ER content portion 319 in the living room 340.

[0090] As a non - limiting example, the first presentation criterion may be satisfied at least in part based on when the electronic device or the user is proximate to the first subset. For example, in Figure 3B , the presentation criterion is satisfied when the electronic device 124 is proximate to the first subset (e.g., the basement 310). As another example, in Figure 4B , the presentation criterion is satisfied when the electronic device 124 is located within the first subset (e.g., room 406). In some embodiments, method 500 further includes presenting on a display one or more ER content items associated with a first ER content portion that moves from the first subset to the second subset. For example, as shown in Figure 4B and Figure 4C , the virtual agent 418 is initially presented in the first subset (e.g., room 406) at time T2 in Figure 4B , and then moves from the first subset to the second subset at time T3 in Figure 4C .

[0091] In some embodiments, the first ER content portion includes a default appearance for which adaptation occurs. In some embodiments, generating adaptive ER content includes at least one of the following actions: adding, removing, scaling, or modifying the set of available interactions associated with the ER content to the spatial characteristics of multiple subsets.

[0092] In some embodiments, method 500 further includes adding one or more ER content items to the first ER content portion at least in part based on a first set of spatial characteristics of the first subset. For example, after adjusting the first ER content portion to the first subset, if there is sufficient room in the first subset, the electronic device may add one or more ER content items to the first ER content portion to fill the first subset.

[0093] In some embodiments, method 500 further includes removing one or more ER content items from the first ER content portion at least in part based on a first set of spatial characteristics of the first subset. In some embodiments, the ER content portion may be critical or non - critical, such that the electronic device can decide to remove the non - critical ER content portion rather than the critical ER portion. In some embodiments, the owner or developer of the ER content determines which ER content portions are critical or non - critical. For example, as shown in the first ER presentation scenario 200a in Figure 2C , the electronic device 124 generates adaptive second ER content 242a - 1, 242a - 2, 242a - 3, 242a - 4 in a second subset 232a of a first physical setting 202. Similarly, as shown in Figure 2CThe second ER rendering scenario 200b is shown in [Figure 0]. The electronic device 124 also generates adaptive second ER content 242b-1, 242b-3 in a second subset 232b of the second physical set 204. However, due to the smaller sizes of the objects (e.g., chairs 220a, bookcases 220b, coffee tables 220d, 220e, and sofa 210) in the second subset 232b and the second physical set 204, the electronic device 124 removes the adaptive ER content 242a-2, 242a-4 from the second physical set 204.

[0094] In some specific implementations, the method 500 further includes scaling one or more ER content items associated with a first ER content portion based at least in part on a first set of spatial features of the first subset. For example, as Figure 2C shown in the first ER rendering scenario 200a of [Figure 0], the electronic device 124 generates adaptive first ER content 240a-1, 240a-2, 240a-3 in a first subset 230a of the first physical set 202. Similarly, as Figure 2C shown in the second ER rendering scenario 200b of [Figure 0], the electronic device 124 also generates adaptive first ER content 240b-1, 240b-2, 240b-3 in a first subset 230b of the second physical set 204. However, compared to the first ER rendering scenario 200a, due to the size difference between the first physical set 202 and the second physical set 204, the adaptive first ER content 240-b1, 240b-2, 240b-3 in the second ER rendering scenario 200b is scaled down to a smaller size than the adaptive first ER content 240a-1, 240a-2, 240a-3 in the first ER rendering scenario 200a.

[0095] In some specific implementations, method 500 further includes modifying the available interaction set associated with the first ER content portion at least in part based on the first set of spatial features of the first subset. For example, as shown in the first ER presentation scenario 200a, user 10 interacts with the adaptive first ER content portions 240a-2, 240a-3 by walking 360 degrees around the adaptive first ER content portions 240a-2, 240a-3 in order to view each angle of the adaptive first ER content portions 240a-2, 240a-3. However, as shown in the second ER presentation scenario 200b, user 10 may not be able to interact with the adaptive first ER content portions 240b-2, 240b-3 by walking 360 degrees around the adaptive first ER content portions 240b-2, 240b-3 in the same manner as in the first ER presentation scenario 200a. Instead, in the second ER presentation scenario 200b, due to the smaller size of the second physical set 204 and the presence of multiple objects within the second physical set 204, user 10 is restricted to walking around certain portions of the adaptive first ER content portions 240b-2, 240b-3.

[0096] In some specific implementations, method 500 further includes identifying an object within the first subset that meets the object presentation criteria; and in response to determining that the object within the first subset meets the object presentation criteria, placing the adaptive first ER content portion within the identified object in the first subset. In some specific implementations, the object presentation criteria correspond to a volume or access threshold. For example, an electronic device may identify a suitcase in a living room that meets the object presentation criteria of a specific volume size and an access threshold that can be opened at the top; and in response to determining that the suitcase in the living room meets the object presentation criteria of a specific volume size and an access threshold that can be opened at the top, place the adaptive first ER content portion within the identified suitcase in the living room.

[0097] In some specific implementations, method 500 further includes determining a connectivity matrix between multiple subsets within the physical set; and generating additional ER content at least in part based on the connectivity matrix between multiple subsets within the physical set for presenting the additional ER content. In some specific implementations, one or more navigation options are generated at least in part based on the connectivity matrix. For example, as Figure 4A shown, the electronic device determines the connectivity matrix between rooms within the physical set, and generates a navigation path 412 at least in part based on the connectivity matrix. Continuing Figure 4A with the example in, the electronic device generates additional ER content 410a, 410b by presenting additional ER content in the corridor 404 in order to bridge multiple subsets within the physical set.

[0098] In some specific implementations, method 500 further includes obtaining a first set of environmental features associated with the first subset and a second set of environmental features associated with the second subset, where: generating the adaptive first ER content portion is at least partially based on the first set of spatial features of the first subset and the first set of environmental features associated with the first subset, and generating the adaptive second ER content portion is at least partially based on the second set of spatial features of the second subset and the second set of environmental features associated with the second subset. In some specific implementations, adjusting the ER content portion may be based on environmental features such as room type, temperature information, lighting information, objects within the physical set, time of day, or the background color of the physical set.

[0099] As a non-limiting example and referring to Figure 3A , the electronic device 124 may be configured to present portions of pre-determined content corresponding to different ecosystems based on the environmental features of subsets within the physical set. Continuing with the non-limiting example, a first portion of the pre-determined content corresponds to an underground ecosystem, a second portion of the pre-determined content corresponds to a grassland ecosystem, a third portion of the pre-determined content corresponds to a forest ecosystem, and a fourth portion of the pre-determined content corresponds to a desert ecosystem. Referring to Figure 3A , the electronic device 124 may obtain environmental features associated with different rooms within the house 301. In this example, the environmental features include room type, temperature information, or lighting information.

[0100] Thus, the electronic device 124 presents the adaptive first ER content portion 314 corresponding to the underground ecosystem in the user interface 302 of the field of view 311a of the basement 310 at least partially based on environmental features such as the basement 310 being underground, the cold temperature of the basement 310, or the dark lighting conditions of the basement 310. As another example, the electronic device 124 presents the adaptive second ER content portion 316 corresponding to the grassland ecosystem in the user interface 302 of the field of view 311b of the kitchen 320 at least partially based on environmental features such as the lighting features of the kitchen 320, the warm temperature of the kitchen 320, or the fact that the room is the kitchen 320. As yet another example, the electronic device 124 presents the adaptive third ER content portion 317 corresponding to the forest ecosystem in the user interface 302 of the field of view 311c of the bedroom 330 at least partially based on environmental features such as the bedroom being on the second floor of the house 301 or the lighting conditions of the bedroom 330. As another example, the electronic device 124 may present the adaptive fourth ER content portion 318 corresponding to the desert ecosystem in the user interface 302 of the field of view 311d of the living room 340 at least partially based on environmental features such as the hot temperature of the living room 340.

[0101] In some specific implementations, method 500 further includes obtaining image data corresponding to a physical scene via an outward-facing image sensor of an electronic device, wherein identifying multiple subsets within the physical scene is at least partially based on the image data, and determining a set of spatial features of the multiple subsets within the physical scene is at least partially based on the image data frames. In some specific implementations, the image data corresponds to video passthrough of other optical information. In some specific implementations, the image data corresponds to first video passthrough image data or second video passthrough image data. In some specific implementations, if the image data corresponds to video passthrough, an adaptive ER content portion can be synthesized into the video passthrough content of the real-time scene. In some specific implementations, if the display 122 corresponds to an additive display that implements optical passthrough of the physical scene, the electronic device 124 presents the ER content by projecting or displaying the adaptive ER content on the additive display, which is then superimposed on the physical scene from the user's perspective.

[0102] Figure 6 is a block diagram of an exemplary controller (e.g., Figure 1 controller 102 shown in ). Although some specific features are shown, those of ordinary skill in the art will recognize from this disclosure that various other features are not shown for the sake of brevity and to not obscure more relevant aspects of the specific implementations disclosed herein. For this purpose, as a non-limiting example, in some specific implementations, controller 102 includes one or more processing units 602 (e.g., microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), graphics processing units (GPUs), central processing units (CPUs), processing cores, etc.), one or more input / output (I / O) devices 606, one or more communication interfaces 608 (e.g., types of interfaces such as universal serial bus (USB), IEEE 802.3x, IEEE 802.11x, IEEE 802.16x, global system for mobile communications (GSM), code division multiple access (CDMA), time division multiple access (TDMA), global positioning system (GPS), infrared (IR), BLUETOOTH, ZIGBEE, etc.), one or more programming (e.g., I / O) interfaces 610, a memory 620, and one or more communication buses 604 for interconnecting these components and various other components.

[0103] In some specific implementations, one or more communication buses 604 include circuitry that interconnects and controls communication between system components. In some specific implementations, one or more I / O devices and sensors include at least one of a keyboard, a mouse, a touchpad, a joystick, one or more microphones, one or more speakers, one or more image sensors, one or more displays, etc.

[0104] The memory 620 includes high-speed random access memory, such as DRAM, SRAM, DDRRAM, or other random access solid-state memory devices, and may include non-volatile memory, such as one or more disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. The memory 620 optionally includes one or more storage devices that are remotely located from one or more processing units 602. The memory 620 includes non-transitory computer-readable storage media. In some specific embodiments, the memory 620 or the non-transitory computer-readable storage media of the memory 620 stores the following programs, modules, and data structures or subsets thereof, including an operating system 630, a management module 640, an identification module 650, a content adapter module 660, and a navigation module 670. In some specific embodiments, one or more instructions are included in a combination of logic components and non-transitory memory.

[0105] The operating system 630 includes procedures for handling various basic system services and for performing hardware-related tasks.

[0106] In some specific embodiments, the management module 640 is configured to reproduce, manage, and / or coordinate one or more user interfaces for one or more devices associated with different users (e.g., Figure 1 the user interface 128 shown, Figures 3B to 3F the user interface 302 shown, or Figures 4A to 4C the user interface 402 shown). To this end, in various specific embodiments, the management module 640 includes a data acquisition unit 642, a content manager unit 644, and a data transmission unit 646.

[0107] In some specific embodiments, the data acquisition unit 642 is configured to acquire data (e.g., presentation data, user interaction data, sensor data, location data, etc.) at least from Figure 1 and Figure 7 the electronic device 124 shown. To this end, in various specific embodiments, the data acquisition unit 642 includes instructions and / or logic components for instructions, as well as heuristics and metadata for heuristics.

[0108] In some specific embodiments, the content manager unit 644 is configured to manage and coordinate the user interfaces presented to the user by Figure 1 and Figure 7 the electronic device 124 shown. To this end, in various specific embodiments, the content manager unit 644 includes instructions and / or logic components for instructions, as well as heuristics and metadata for heuristics.

[0109] In some specific embodiments, the data transmission unit 646 is configured to transmit data at least to Figure 1 and Figure 7The illustrated electronic device 124 transmits data (e.g., presentation data, location data, etc.). To this end, in various embodiments, the data transmission unit 646 includes instructions and / or logic components for the instructions, as well as heuristics and metadata for the heuristics.

[0110] In some embodiments, the recognition module 650 is configured to recognize a plurality of subsets associated with a physical scene and determine a set of spatial characteristics for each of the plurality of subsets within the physical scene. To this end, in various embodiments, the recognition unit 650 includes instructions and / or logic for the instructions, as well as heuristics and metadata for the heuristics.

[0111] In some embodiments, the content adapter module 660 is configured to logically map a portion of the ER content to a subset that meets the mapping criteria by identifying the subset that is most suitable for each content portion, and to adapt that portion of the ER content from the reference ER content to the spatial characteristics of the identified subset. To this end, in various embodiments, the content adapter module 660 includes instructions and / or logic components for these instructions, as well as heuristics and metadata for the heuristics.

[0112] In some embodiments, the navigation module 670 is configured to generate one or more navigation options that allow a user to traverse between a first subset and a second subset. To this end, in various embodiments, the navigation module 670 includes instructions and / or logic components for these instructions, as well as heuristics and metadata for the heuristics.

[0113] Although the management module 640, the recognition module 650, the content adapter module 660, and the navigation module 670 are shown as residing on a single device (e.g., the controller 102), it should be understood that in some embodiments, any combination of the management module 640, the recognition module 650, the content adapter module 660, and the navigation module 670 may be located in separate computing devices.

[0114] In some embodiments, the functionality of the controller 102 is provided by and / or in conjunction with the electronic device 124 shown below in Figure 7 Moreover, Figure 6 This is more of a functional description of the various features that may be present in a particular embodiment, as opposed to a structural schematic of the specific embodiments described herein. As will be recognized by those of ordinary skill in the art, items shown separately may be combined, and some items may be separated. For example, Figure 6Some of the functional modules shown separately in the [specific implementation] can be implemented in a single module, and the various functions of a single functional block can be implemented by one or more functional blocks in various specific implementations. The actual number of modules and the specific division of functions, as well as how to allocate features therein, will vary according to specific implementations, and in some specific implementations, it depends in part on the specific combination of hardware, software, and / or firmware selected for a particular implementation.

[0115] Figure 7 is a block diagram of an exemplary electronic device 124 (e.g., mobile phone, tablet computer, laptop computer, near-eye system, etc.) according to some specific implementations. Although some specific features are shown, those of ordinary skill in the art will recognize from this disclosure that various other features are not shown for the sake of brevity and to not obscure more relevant aspects of the specific implementations disclosed herein. For this purpose, as a non-limiting example, in some specific implementations, the electronic device 124 includes one or more processing units 702 (e.g., microprocessor, ASIC, FPGA, GPU, CPU, processing core, etc.), one or more I / O devices and sensors 706, one or more communication interfaces 708 (e.g., USB, IEEE 802.3x, IEEE 802.11x, IEEE802.16x, GSM, CDMA, TDMA, GPS, IR, BLUETOOTH, ZIGBEE, and / or similar types of interfaces), one or more programming interfaces 710, one or more displays 712, one or more image sensors 714, a memory 720, and one or more communication buses 704 for interconnecting these components and various other components.

[0116] In some specific implementations, the one or more communication buses 704 include circuitry for interconnecting communication between system components and control components. In some specific implementations, the one or more I / O devices and sensors 706 include at least one of the following: inertial measurement unit (IMU), accelerometer, gyroscope, thermometer, one or more physiological sensors (e.g., blood pressure monitor, heart rate monitor, blood oxygen sensor, blood glucose sensor, etc.), one or more microphones, one or more speakers, a haptic engine, a heating and / or cooling unit, a skin shear engine, etc.

[0117] In some specific implementations, one or more displays 712 are capable of presenting a user interface (e.g., Figure 1 the user interface 128 shown, Figures 3B to 3F the user interface 302 shown, or Figures 4A to 4CThe user interface 402) or ER content shown. In some specific embodiments, one or more displays 712 are also configured to present planar video content to the user (e.g., two-dimensional or "flat" audio-video interleaved (AVI), Flash video (FLV), Windows Media Video (WMV), etc. files associated with a TV drama or movie, or real-time video passthrough of the operating environment). In some specific embodiments, one or more displays 712 correspond to holographic, digital light processing (DLP), liquid crystal display (LCD), liquid crystal on silicon (LCoS), organic light-emitting field-effect transistor (OLET), organic light-emitting diode (OLED), surface-conduction electron-emitter display (SED), field-emission display (FED), quantum dot light-emitting diode (QD-LED), microelectromechanical systems (MEMS), and / or similar display types. In some specific embodiments, one or more displays 712 correspond to diffractive, reflective, polarization, holographic, etc. waveguide displays. For example, the electronic device 124 includes a single display. As another example, the electronic device 124 includes a display for each eye of the user.

[0118] In some specific embodiments, one or more image sensors 714 are configured to acquire image data frames. For example, one or more image sensors 714 correspond to one or more RGB cameras (e.g., having a CMOS image sensor or a CCD image sensor), infrared (IR) image sensors, event-based cameras, etc.

[0119] The memory 720 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices. In some specific embodiments, the memory 720 includes non-volatile memory, such as one or more disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. The memory 720 optionally includes one or more storage devices remotely located from one or more processing units 702. The memory 720 includes non-transitory computer-readable storage media. In some specific embodiments, the memory 720 or the non-transitory computer-readable storage media of the memory 720 stores the following programs, modules, and data structures or subsets thereof, including an optional operating system 730 and a presentation module 740.

[0120] The optional operating system 730 includes procedures for handling various basic system services and for performing hardware-related tasks. In some specific embodiments, the presentation module 740 is configured to present a user interface or ER content to the user via one or more displays 712. To this end, in various specific embodiments, the presentation module 740 includes a data acquisition unit 742, a presentation unit 744, and a data transmission unit 746.

[0121] In some specific implementations, the data acquisition unit 742 is configured to obtain data (e.g., presentation data, interaction data, location data, etc.) from at least one of one or more I / O devices and sensors 706 associated with the electronic device 124 or the controller 102 shown in Figure 1 and Figure 6 . To this end, in various specific implementations, the data acquisition unit 742 includes instructions and / or logic components for the instructions, as well as heuristics and metadata for the heuristics.

[0122] In some specific implementations, the presentation unit 744 is configured to present a user interface (e.g., the user interface 128 shown in Figure 1 , the user interface 302 shown in Figures 3B to 3F , or the user interface 402 shown in Figures 4A to 4C ) or an ER experience via one or more displays 712. To this end, in various specific implementations, the presentation unit 744 includes instructions and / or logic components for the instructions, as well as heuristics and metadata for the heuristics.

[0123] In some specific implementations, the data transmission unit 746 is configured to transmit data (e.g., presentation data, location data, etc.) to the controller 102 shown in Figure 1 and Figure 6 . To this end, in various specific implementations, the data transmission unit 746 includes instructions and / or logic components for the instructions, as well as heuristics and metadata for the heuristics.

[0124] Although the data acquisition unit 742, the presentation unit 744, and the data transmission unit 746 are shown as residing in a single device (e.g., the electronic device 124 shown in Figure 1 or Figure 7 ), it should be understood that in some specific implementations, any combination of the data acquisition unit 742, the presentation unit 744, and the data transmission unit 746 may be located in separate computing devices. In some specific implementations, the functions and / or components of the controller 102 are combined with or provided by the electronic device 124.

[0125] Furthermore, Figure 7 is more used as a functional description of various features that may exist in a specific implementation, different from the structural schematic diagram of the specific implementation described herein. As those of ordinary skill in the art will recognize, the items shown separately can be combined, and some items can be separated. For example, Figure 7Some of the functional modules shown separately in the [Chinese context] can be implemented in a single module, and the various functions of a single functional block can be implemented by one or more functional blocks in various embodiments. The actual number of modules and the specific division of functions, as well as how features are allocated therein, will vary according to the specific implementation, and in some specific implementations, it depends in part on the specific combination of hardware, software, and / or firmware selected for a particular embodiment.

[0126] Although the various aspects of specific implementations within the scope of the appended claims have been described above, it should be apparent that the various features of the above specific implementations can be embodied in a wide variety of forms, and any specific structure and / or function described above is merely illustrative. Based on this disclosure, those skilled in the art should understand that the aspects described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of the aspects described herein can be used to implement a device and / or a method can be practiced. Additionally, other structures and / or functions can be used to implement such a device and / or a method can be practiced in addition to or different from one or more of the aspects described herein.

[0127] It will also be understood that although terms such as "first," "second," etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first subset can be referred to as a second subset, and similarly, a second subset can be referred to as a first subset, which changes the meaning of the description, as long as the "first subset" that appears is consistently renamed and the "second subset" that appears is consistently renamed. Both the first subset and the second subset are subsets, but they are not the same subset.

[0128] The terms used herein are merely for the purpose of describing specific specific implementations and are not intended to limit the claims. As used in the description of this specific implementation and the appended claims, the singular forms "a," "an," and "the" are intended to also cover the plural forms unless the context clearly indicates otherwise. It will also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will also be understood that the term "comprising," when used in this specification, specifies the presence of the stated features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or their groupings.

[0129] As used herein, the term "if" can be interpreted to mean "when the precondition is true" or "while the precondition is true" or "in response to determining" or "in accordance with determining" or "in response to detecting" that the precondition is true, depending on the context. Similarly, the phrases "if it is determined [that the precondition is true]" or "if [the precondition is true]" or "when [the precondition is true]" are interpreted to mean "when it is determined that the precondition is true" or "in response to determining" or "in accordance with determining" that the precondition is true or "when it is detected that the precondition is true" or "in response to detecting" that the precondition is true, depending on the context.

Claims

1. A method, the method comprising: at an electronic device including an image sensor, one or more processors, non-transitory memory, and a display: identifying a plurality of subsets associated with a physical environment; determining a set of spatial characteristics for each of the plurality of subsets within the physical environment, wherein a first set of spatial characteristics characterizes one or more dimensions of a first subset of the plurality of subsets, and a second set of spatial characteristics characterizes one or more dimensions of a second subset of the plurality of subsets; generating an adaptive first extended reality (XR) content portion of the first subset of the plurality of subsets based on the one or more dimensions of the first subset; generating an adaptive second XR content portion of the second subset of the plurality of subsets at least in part based on the one or more dimensions of the first subset; generating one or more navigation options that allow a user to traverse between the first subset and the second subset based on the first set of spatial characteristics and the second set of spatial characteristics; and indicating the one or more navigation options by displaying movement of an XR content item associated with the adaptive first extended reality (XR) content portion from the first subset to the second subset.

2. The method of claim 1, wherein the first set of spatial characteristics includes at least one of: a volume size of the first subset of the plurality of subsets, an indication that there is a physical object within the first subset of the plurality of subsets, and a shape of the first subset of the plurality of subsets.

3. The method of claim 1, wherein the one or more navigation options includes at least one of: a path indicator between the first subset and the second subset of the plurality of subsets and a circular path between the first subset and the second subset of the plurality of subsets.

4. The method of claim 1, the method further comprising: determining a connectivity matrix between the plurality of subsets within the physical environment; generating additional XR content based on the connectivity matrix; and presenting the additional XR content at least in part based on the connectivity matrix between the plurality of subsets within the physical environment.

5. The method of claim 4, wherein the one or more navigation options are generated at least in part based on the connectivity matrix.

6. The method of claim 1, wherein generating the adaptive first extended reality (XR) content portion is at least in part based on determining whether the first set of spatial characteristics associated with the first subset of the plurality of subsets meets a first mapping criterion for a first extended reality (XR) content portion of a pre-determined XR content.

7. The method of claim 6, the method further comprising: In response to determining that the first set of spatial features associated with the first subset of the plurality of subsets satisfies the first mapping criterion, determining the placement of the adaptive first extended reality (XR) content portion within the first subset of the plurality of subsets based at least in part on the first set of spatial features of the first subset of the plurality of subsets, wherein presenting the adaptive first extended reality (XR) content portion includes compositing the adaptive first extended reality (XR) content with first passthrough image data according to the placement.

8. The method according to claim 1, the method further comprising: In response to determining that a first presentation criterion is satisfied, presenting, on the display, the adaptive first extended reality (XR) content portion superimposed on a first field of view of the device, the first field of view corresponding to the first subset of the plurality of subsets of the physical environment; and In response to determining that a second presentation criterion is satisfied, presenting, on the display, the adaptive second XR content portion superimposed on a second field of view of the device, the second field of view corresponding to the second subset of the plurality of subsets of the physical environment.

9. The method according to claim 8, wherein the first presentation criterion corresponds to at least one of the following: a time criterion associated with the adaptive first extended reality (XR) content portion, coordinates of the first subset of the plurality of subsets, coordinates of the second subset of the plurality of subsets, an adjacency degree between the first extended reality (XR) content portion and the second XR content portion, and a location of the electronic device.

10. The method according to claim 1, the method further comprising: Obtaining a first set of environmental features associated with the first subset of the plurality of subsets and a second set of environmental features associated with the second subset of the plurality of subsets, wherein: Generating the adaptive first extended reality (XR) content portion is at least partially based on the first set of spatial features of the first subset of the plurality of subsets and the first set of environmental features associated with the first subset of the plurality of subsets, and Generating the adaptive second XR content portion is at least partially based on the second set of spatial features of the second subset of the plurality of subsets and the second set of environmental features associated with the second subset of the plurality of subsets.

11. The method according to claim 10, wherein the environmental features in the first set of environmental features correspond to at least one of the following: room type, temperature information, lighting information, objects within the physical environment, time of day, and background color of the physical environment.

12. The method according to claim 1, wherein generating the adaptive first extended reality (XR) content portion includes: Adding one or more XR content items to a first extended reality (XR) content portion of a predetermined XR content based at least in part on the first set of spatial features of the first subset of the plurality of subsets.

13. The method according to claim 1, wherein generating the adaptive first extended reality (XR) content portion includes removing one or more XR content items from a first extended reality (XR) content portion of a predetermined XR content based at least in part on the first set of spatial features of the first subset of the plurality of subsets.

14. The method according to claim 1, wherein generating the adaptive first extended reality (XR) content portion includes modifying an available interaction set associated with a first extended reality (XR) content portion of a predetermined XR content based at least in part on the first set of spatial features of the first subset of the plurality of subsets.

15. The method according to claim 1, wherein the first extended reality (XR) content portion is associated with a first thematic scene within the predetermined XR content, and the second XR content portion is associated with a second thematic scene within the predetermined XR content.

16. The method according to claim 15, wherein the predetermined XR content corresponds to a time linear graph, and the adaptive first extended reality (XR) content portion is presented before presenting the adaptive second XR content portion.

17. The method according to claim 1, the method further comprising: identifying an object within the first subset of the plurality of subsets that meets an object presentation criterion; and responsive to determining that the object within the first subset of the plurality of subsets meets the object presentation criterion, placing the adaptive first extended reality (XR) content portion within the identified object within the first subset of the plurality of subsets.

18. The method according to claim 1, the method further comprising: obtaining a predetermined XR content, the predetermined XR content including: a first extended reality (XR) content portion and a second XR content portion, the first extended reality (XR) content portion being adjusted to the adaptive first extended reality (XR) content portion, the second XR content portion being adjusted to the adaptive second XR content portion, wherein the first extended reality (XR) content portion and the second XR content portion are obtained from a database.

19. The method according to claim 1, the method further comprising: obtaining image data corresponding to the physical environment via an outward-facing image sensor of the electronic device, wherein identifying the plurality of subsets within the physical environment is at least partially based on the image data, and determining the first set of spatial features and the second set of spatial features of the plurality of subsets within the physical environment is at least partially based on the image data.

20. The method according to claim 1, wherein the XR content item represents a virtual agent, and displaying the movement of the XR content item includes displaying the movement of the virtual agent from the first subset to the second subset.

21. An electronic device, the electronic device comprising: an image sensor; a display; one or more processors; a non-transitory memory; and One or more programs stored in the non-transitory memory, the one or more programs, when executed by the one or more processors, cause the electronic device to perform or cause to be performed any one of the methods according to claims 1 to 20.

22. A non-transitory memory storing one or more programs, the one or more programs, when executed by one or more processors of an electronic device having an image sensor and a display, cause the electronic device to perform any one of the methods according to claims 1 to 20.

Images