Controlling user selection queues

Abstract

The subject matter of the invention is to control a user to select a queue. Various implementations disclosed herein include apparatuses, systems, and methods for synthesizing affordance associated with computer generated reality (CGR) objects representing physical items. In various implementations, an apparatus includes a display, a non-transitory memory, and one or more processors coupled to the display and the non-transitory memory. In some implementations, a method includes displaying a CGR object in a CGR environment. In some implementations, the CGR object represents a physical item. In some implementations, the method includes synthesizing an affordance associated with the CGR object. In some implementations, the method includes detecting an input involving the affordance. In some implementations, the method includes adding an identifier identifying the physical item to a user selection queue in response to detecting the input.

Description

[0001] This application is a divisional application of the invention patent application with the application date of May 28, 2020, application number 202010470149.1, and invention title "Control User Selection Queue".

[0002] Cross - Reference to Related Applications

[0003] This application claims the benefit of U.S. Provisional Patent Application No. 62 / 855,920, filed on Jun. 1, 2019, which is hereby incorporated by reference in its entirety. Technical Field

[0004] The present disclosure generally relates to controlling a user selection queue. Background Art

[0005] Some devices are capable of generating and presenting computer-generated reality (CGR) environments. Some CGR environments include virtual environments, which are simulated replacements of physical environments. Some CGR environments include augmented environments, which are modified versions of physical environments. Some devices for presenting CGR environments include mobile communication devices such as smart phones, head-mounted displays (HMDs), glasses, head-up displays (HUDs), and optical projection systems. However, most previously available devices for presenting CGR environments are ineffective in controlling a user selection queue. Brief Description of the Drawings

[0006] Thus, the present disclosure can be understood by those of ordinary skill in the art, and a more detailed description can be referred to in some illustrative aspects of specific implementations, some of which are shown in the drawings.

[0007] Figures 1A to 1L is an illustration of an example operating environment for controlling a user selection queue according to some specific implementations.

[0008] Figures 2A to 2G is an illustration of an example operating environment for displaying a CGR object representing a wearable physical item according to some specific implementations.

[0009] Figures 3A to 3I is an illustration of an example operating environment for simultaneously controlling multiple user selection queues according to some specific implementations.

[0010] Figures 4A to 4O is an illustration of an example operating environment for synthesizing a masking element according to some specific implementations.

[0011] Figures 5A to 5C is a flowchart representation of a method for controlling a user selection queue according to some specific implementations.

[0012] Figures 6A to 6CIt is a flowchart representation of a method for displaying CGR objects representing wearable physical items according to some specific implementations.

[0013] Figures 7A to 7C It is a flowchart representation of a method for simultaneously controlling multiple user selection queues according to some specific implementations.

[0014] Figures 8A to 8C It is a flowchart representation of a method for synthesizing masking elements according to some specific implementations.

[0015] Figure 9 It is a block diagram of a device for presenting a CGR environment according to some specific implementations.

[0016] According to common practice, the various feature parts shown in the drawings may not be drawn to scale. Therefore, for clarity, the sizes of the various feature parts can be arbitrarily enlarged or reduced. Additionally, some drawings may not depict all the components of a given system, method, or device. Finally, throughout the specification and the drawings, like reference numerals may be used to represent like feature parts. Summary of the Invention

[0017] The various specific implementations disclosed herein include devices, systems, and methods for synthesizing affordance representations associated with computer-generated reality (CGR) objects representing physical items. In various specific implementations, the device includes a display, a non-transitory memory, and one or more processors coupled to the display and the non-transitory memory. In some specific implementations, the method includes displaying a CGR object in a CGR environment. In some specific implementations, the CGR object represents a physical item. In some specific implementations, the method includes synthesizing an affordance representation associated with the CGR object. In some specific implementations, the method includes detecting an input related to the affordance representation. In some specific implementations, the method includes adding an identifier identifying the physical item to a user selection queue in response to detecting the input.

[0018] The various embodiments disclosed herein include devices, systems, and methods for displaying a computer-generated reality (CGR) object representing a wearable physical item according to a deformation model of the wearable physical item. In various embodiments, the device includes a display, a non-transitory memory, and one or more processors coupled to the display and the non-transitory memory. In some embodiments, the method includes obtaining a computer-generated reality (CGR) representation of a person. In some embodiments, at least a portion of the CGR representation is proportional to the corresponding portion of the person. In some embodiments, the method includes obtaining a CGR object representing the wearable physical item. In some embodiments, the CGR object is associated with a deformation model characterizing one or more material properties of the wearable physical item. In some embodiments, the method includes displaying the CGR object associated with the CGR representation of the person. In some embodiments, the CGR object interacts with the CGR representation of the person according to the deformation model.

[0019] The various embodiments disclosed herein include devices, systems, and methods for adding an identifier of a physical item to a source-specific user selection queue. In various embodiments, the device includes a display, a non-transitory memory, and one or more processors coupled to the display and the non-transitory memory. In some embodiments, the method includes displaying a plurality of computer-generated reality (CGR) objects representing respective physical items from a plurality of sources including a first source and a second source. In some embodiments, the method includes detecting an input selecting a first CGR object of the plurality of CGR objects and a second CGR object of the plurality of CGR objects. In some embodiments, the first CGR object represents a first physical item from the first source, and the second CGR object represents a second physical item from the second source. In some embodiments, the method includes adding an identifier of the first physical item to a first user selection queue associated with the first source. In some embodiments, the method includes adding an identifier of the second physical item to a second user selection queue associated with the second source.

[0020] The various embodiments disclosed herein include devices, systems, and methods for masking physical items that occlude a physical surface. In various embodiments, the device includes a display, an environmental sensor, a non-transitory memory, and one or more processors coupled to the display, the environmental sensor, and the non-transitory memory. In some embodiments, the method includes detecting a physical surface in a physical environment around the device. In some embodiments, the method includes detecting one or more physical items that occlude respective portions of the physical surface. In some embodiments, the method includes synthesizing a masking element to mask the one or more physical items located on the physical surface.

[0021] According to some specific embodiments, the device includes one or more processors, non-transitory memory, and one or more programs. In some specific embodiments, the one or more programs are stored in the non-transitory memory and executed by the one or more processors. In some specific embodiments, the one or more programs include instructions for performing or causing to perform any of the methods described herein. According to some specific embodiments, instructions are stored in a non-transitory computer-readable storage medium that, when executed by one or more processors of the device, cause the device to perform or cause to perform any of the methods described herein. According to some specific embodiments, a device includes one or more processors, non-transitory memory, and means for performing or causing to perform any of the methods described herein. Detailed Description

[0022] Numerous details are described in order to provide a thorough understanding of the example embodiments shown in the figures. However, the figures merely illustrate some example aspects of the present disclosure and should not be considered limiting. Those of ordinary skill in the art will understand that other effective aspects and / or variations do not include all of the specific details described herein. Additionally, well-known systems, methods, components, devices, and circuits have not been described in detail so as not to obscure more relevant aspects of the exemplary embodiments described herein.

[0023] The physical environment refers to the physical world that people can sense and / or interact with without the assistance of an electronic system. The physical environment such as a physical park includes physical items such as physical trees, physical buildings, and physical people. People can directly sense and / or interact with the physical environment, such as through vision, touch, hearing, taste, and smell.

[0024] In contrast, a computer-generated reality (CGR) environment refers to a fully or partially simulated environment that people perceive and / or interact with via an electronic system. In CGR, a subset of a person's physical movements or their representations are tracked, and in response, one or more characteristics of one or more virtual objects simulated in the CGR environment are adjusted in a manner that complies with at least one physical law. For example, a CGR system can detect a person's head rotation, and in response, adjust the graphical content and sound field presented to the person in a manner similar to how such views and sounds would change in the physical environment. In some cases (e.g., for accessibility reasons), the adjustment of the characteristics of virtual objects in the CGR environment can be made in response to a representation of physical movement (e.g., a voice command).

[0025] A person can use any of their senses to sense and / or interact with a CGR object, including vision, hearing, touch, taste, and smell. For example, a person can sense and / or interact with an audio object that creates a 3D or spatial audio environment that provides the perception of point audio sources in 3D space. As another example, the audio object can enable audio transparency that selectively introduces ambient sounds from the physical environment with or without computer-generated audio. In some CGR environments, a person can sense and / or interact only with audio objects.

[0026] Examples of CGR include virtual reality and mixed reality.

[0027] A virtual reality (VR) environment is an artificial environment designed to be based entirely on computer-generated sensory input for one or more senses. A VR environment includes multiple virtual objects with which a person can sense and / or interact. For example, computer-generated images of trees, buildings, and avatars representing people are examples of virtual objects. A person can sense and / or interact with the virtual objects in a VR environment through a simulation of the person's presence within the computer-generated environment and / or through a simulation of a subset of the person's physical movements within the computer-generated environment.

[0028] Compared to a VR environment that is designed to be based entirely on computer-generated sensory input, a mixed reality (MR) environment is an artificial environment designed to include sensory input or a representation thereof from the physical environment in addition to computer-generated sensory input (e.g., virtual objects). On the virtual continuum, a mixed reality environment is any condition between a fully physical environment at one end and a virtual reality environment at the other end, but excluding these two ends.

[0029] In some MR environments, the computer-generated sensory input can respond to changes in the sensory input from the physical environment. Additionally, some electronic systems for presenting an MR environment can track the position and / or orientation relative to the physical environment so that virtual objects can interact with real objects (i.e., physical items from the physical environment or their representations). For example, the system can cause movement so that a virtual tree appears stationary relative to the physical ground.

[0030] Examples of mixed reality include augmented reality and augmented virtuality.

[0031] An augmented reality (AR) environment is a simulated environment in which one or more virtual objects are superimposed on a physical environment or a representation thereof. For example, an electronic system for presenting an AR environment may have a transparent or translucent display through which a person can directly view the physical environment. The system can be configured to present virtual objects on the transparent or translucent display such that the person using the system perceives the virtual objects superimposed on the physical environment. Alternatively, the system can have an opaque display and one or more imaging sensors that capture images or video of the physical environment, which are representations of the physical environment. The system combines the images or video with virtual objects and presents the combination on the opaque display. The person uses the system to indirectly view the physical environment via the images or video of the physical environment and perceives the virtual objects superimposed on the physical environment. As used herein, the video of the physical environment displayed on the opaque display is referred to as "passthrough video," meaning that the system uses one or more image sensors to capture images of the physical environment and uses those images when presenting the AR environment on the opaque display. Further alternatively, the system can have a projection system that projects virtual objects into the physical environment, such as as a hologram or on a physical surface, such that the person using the system perceives the virtual objects superimposed on the physical environment.

[0032] An augmented reality environment is also a simulated environment in which a representation of the physical environment is transformed by computer-generated sensory information. For example, in providing passthrough video, the system can transform one or more sensor images to impose an alternative perspective (e.g., viewpoint) different from the perspective captured by the imaging sensor. As another example, a representation of the physical environment can be transformed by graphically modifying (e.g., magnifying) portions thereof such that the modified portions can be a representative but not a true version of the originally captured image. As yet another example, a representation of the physical environment can be transformed by graphically removing portions thereof or blurring portions thereof.

[0033] An augmented virtual (AV) environment is a simulated environment in which a virtual or computer-generated environment incorporates one or more sensory inputs from a physical environment. The sensory inputs can be representations of one or more features of the physical environment. For example, an AV park can have virtual trees and virtual buildings, but a person's face is realistically reproduced from an image of a physical person. As another example, a virtual object can adopt the shape or color of a physical item imaged by one or more imaging sensors. As yet another example, a virtual object can adopt a shadow that conforms to the position of the sun in the physical environment.

[0034] There are many different types of electronic systems that enable a person to sense and / or interact with various CGR environments. Examples include head-mounted systems, projection-based systems, head-up displays (HUDs), vehicle windshields integrated with display capabilities, windows integrated with display capabilities, displays formed as lenses designed to be placed on a person's eyes (e.g., similar to contact lenses), headphones / earpieces, speaker arrays, input systems (e.g., wearable or handheld controllers with or without haptic feedback), smartphones, tablets, and desktop / laptop computers. A head-mounted system can have one or more speakers and an integrated opaque display. Alternatively, a head-mounted system can be configured to receive an external opaque display (e.g., a smartphone). A head-mounted system can incorporate one or more imaging sensors for capturing images or video of the physical environment and / or one or more microphones for capturing audio of the physical environment. A head-mounted system can have a transparent or translucent display instead of an opaque display. The transparent or translucent display can have a medium through which light representing an image is directed to a person's eyes. The display can utilize digital light projection, OLED, LED, uLED, liquid crystal on silicon, laser scanning light sources, or any combination of these technologies. The medium can be an optical waveguide, a holographic medium, an optical combiner, an optical reflector, or any combination thereof. In a particular implementation, the transparent or translucent display can be configured to selectively become opaque. A projection-based system can employ retinal projection technology that projects a graphical image onto a person's retina. The projection system can also be configured to project virtual objects into the physical environment, such as as a hologram or on a physical surface.

[0035] Some CGR devices allow a user to view a CGR representation of a physical item. However, if the user wants to add the physical item to a user selection queue, the user has to navigate back to the web page corresponding to the physical item in order to populate the user selection queue with the identifier of the physical item. This degrades the user experience and requires excessive user input, which unnecessarily drains the battery of the device. The present disclosure provides methods, systems, and / or devices for displaying a CGR object representing a physical item and synthesizing an affordance representation therewith. When the affordance representation is activated, the physical item is added to the user selection queue. The affordance representation reduces the need for unnecessary user input corresponding to the user navigating back to the web page of the physical item to add the physical item to the user selection queue.

[0036] Figure 1Ais a block diagram of an exemplary physical environment 10 according to some embodiments. Although relevant features are shown, those of ordinary skill in the art will recognize from this disclosure that various other features are not shown for the sake of brevity and to not obscure more relevant aspects of the exemplary embodiments disclosed herein. To that end, by way of non-limiting example, the physical environment 10 includes a first floor lamp 12, a second floor lamp 14, and an electronic device 100. In some embodiments, the electronic device 100 is held by a person (not shown). In some embodiments, the electronic device 100 includes a smart phone, a tablet computer, a laptop computer, etc.

[0037] In Figure 1A the example of, the electronic device 100 displays a web page 102 corresponding to a sofa. The web page 102 includes a two-dimensional (2D) representation 104 of the sofa (e.g., an image of the sofa). The web page 102 also includes an affordance representation 106 for viewing a CGR representation of the sofa. Referring to Figure 1B , the electronic device 100 detects a user input 108 that activates the affordance representation 106 (e.g., a contact at a location corresponding to the affordance representation 106, such as a tap or a press). As Figure 1C shown, in response to detecting the user input 108, the electronic device 100 presents a CGR environment 110.

[0038] In Figure 1C the example of, the CGR environment 110 includes a first CGR floor lamp 112, a second CGR floor lamp 114, and a CGR sofa 116. The first CGR floor lamp 112 is a CGR representation of the first floor lamp 12 in the physical environment 10. The second CGR floor lamp 114 is a CGR representation of the second floor lamp 14 in the physical environment 10. The CGR sofa 116 is a CGR representation of the sofa represented by the 2D representation 104 on the web page 102 shown by Figures 1A to 1B . In some embodiments, the sofa is associated with an identifier (ID) 118 that identifies the sofa. In some embodiments, the ID 118 includes one or more alphanumeric character sets (e.g., a serial number, an item number, a barcode number, a name, a title, a description, etc.). In some embodiments, the ID 118 includes a machine-readable representation of data (e.g., an optical machine-readable representation of data, such as a barcode or a QR code). In some embodiments, the electronic device 100 does not display the ID 118 in the CGR environment 110. In some embodiments, the CGR environment 110 is a perspective of the physical environment 10 (e.g., a video perspective or an optical perspective).

[0039] As Figure 1CAs shown, in some specific embodiments, the electronic device 100 synthesizes the affordance representation 120 associated with the CGR sofa 116. In response to the activation of the affordance representation 120, the electronic device 100 adds (e.g., writes) the ID 118 of the sofa to the user selection queue 150. In some specific embodiments, the electronic device 100 displays the affordance representation 120 adjacent to the CGR sofa 116. For example, as Figure 1C shown, the electronic device 100 displays the affordance representation 120 below the CGR sofa 116. However, in some specific embodiments, the electronic device 100 displays the affordance representation 120 at another location. For example, in some specific embodiments, the electronic device 100 overlays the affordance representation 120 onto the CGR sofa 116. In Figure 1C the example of, the affordance representation 120 is visible. However, in some specific embodiments, the affordance representation 120 is invisible. For example, in some specific embodiments, the CGR sofa 116 serves as the affordance representation 120. In such specific embodiments, in response to detecting the selection of the CGR sofa 116, the electronic device 100 adds the ID of the sofa to the user selection queue.

[0040] Referring to Figure 1D , the electronic device 100 detects the user input 130 related to the affordance representation 120. In Figure 1D the example of, the user input 130 includes a contact (e.g., a tap or a press) at the position corresponding to the affordance representation 120. In some specific embodiments, the electronic device 100 detects a gaze input related to the affordance representation 120. In some specific embodiments, the electronic device 100 detects a speech input related to the affordance representation 120. In some specific embodiments, the electronic device 100 detects a three-dimensional (3D) gesture corresponding to the selection of the affordance representation 120. For example, the electronic device 100 uses hand tracking to detect that the person controlling the electronic device 100 has performed a 3D gesture. In some specific embodiments, the electronic device 100 obtains an input corresponding to the selection of the affordance representation 120 from a controller (not shown). For example, the electronic device 100 detects the activation of a controller button corresponding to the affordance representation 120. As Figure 1E shown, in response to detecting the user input 130, the electronic device 100 adds the ID 118 of the sofa to the user selection queue 150.

[0041] Referring to Figure 1E , in some specific embodiments, the electronic device 100 displays a notification 140 in response to detecting Figure 1D the user input 130 shown. In some specific embodiments, the notification 140 indicates that the electronic device 100 has added the ID 118 of the sofa to the user selection queue 150. For example, as Figure 1EAs shown, in some specific embodiments, the notification 140 includes text 142 (e.g., "Sofa added to the queue"). In some specific embodiments, the notification 140 includes an enabling representation 144 for displaying a visual representation of the user-selected queue 150, and an enabling representation 146 for continuing browsing.

[0042] Reference Figure 1F , the electronic device 100 detects a user input 130 related to the enabling representation 144 for displaying a visual representation of the user-selected queue 150. As Figure 1G shown, in response to detecting the user input 130, the electronic device 100 displays a visual representation 151 of the user-selected queue 150. In Figure 1G the example, the visual representation 151 of the user-selected queue 150 includes a 2D representation 104 of the sofa, a description 152 of the sofa, a deletion enabling representation 154 for removing the ID 118 from the user-selected queue 150, a modification enabling representation 156, and a confirmation enabling representation 158. In some specific embodiments, the modification enabling representation 156 allows a user of the electronic device 100 to modify the quantity associated with the sofa (e.g., increasing the quantity from the default quantity of 1 to a higher quantity). In some specific embodiments, the confirmation enabling representation 158 allows a user of the electronic device 100 to confirm the user-selected queue 150. In some specific embodiments, detecting the activation of the confirmation enabling representation 158 triggers the electronic device 100 to perform an operation (e.g., the electronic device 100 triggers the transfer of points to another device, e.g., to a device associated with the sofa seller).

[0043] Reference Figure 1H , in some specific embodiments, the electronic device 100 displays the enabling representation 120 in a designated portion 160 of the CGR environment 110. In Figure 1H the example, the designated portion 160 faces the lower right corner of the CGR environment 110. However, in some specific embodiments, the designated portion 160 faces the left side of the CGR environment 110, the right side of the CGR environment 110, the top of the CGR environment 110, the bottom of the CGR environment 110, or the center of the CGR environment 110.

[0044] Reference Figure 1I , in some specific embodiments, the electronic device 100 displays a CGR representation of multiple items not in the physical environment 10. In Figure 1I the example, the electronic device 100 displays a CGR painting 170 representing a physical painting. In some specific embodiments, the electronic device 100 displays an enabling representation 172 to add an ID (not shown) identifying the physical painting to the user-selected queue 150. In some specific embodiments, the electronic device 100 displays an enabling representation 180 to add the IDs of multiple physical items to the user-selected queue 150 simultaneously. In Figure 1IIn the example, in response to detecting the activation of the enabling representation 180, the electronic device 100 adds the ID 118 of the sofa to the user selection queue 150, and adds the ID of the physical painting to the user selection queue 150. Thus, the user of the electronic device 100 does not need to separately activate the enabling representations 120 and 172, thereby reducing the amount of user input and enhancing the user experience of the electronic device 100.

[0045] In some specific embodiments, a head-mounted device (HMD) (not shown) worn by a person presents (e.g., displays) the CGR environment 110 according to various specific embodiments. In some specific embodiments, the HMD includes an integrated display (e.g., a built-in display) that displays the CGR environment 110. In some specific embodiments, the HMD includes a head-mounted housing. In various specific embodiments, the head-mounted housing includes an attachment area to which another device having a display can be attached. For example, in some specific embodiments, the electronic device 100 can be attached to the head-mounted housing. In various specific embodiments, the head-mounted housing is shaped to form a receiver for receiving another device (e.g., the electronic device 100) including a display. For example, in some specific embodiments, the electronic device 100 slides / snaps onto the head-mounted housing or is otherwise attached to the head-mounted housing. In some specific embodiments, the display of the device attached to the head-mounted housing presents (e.g., displays) the CGR environment 110. In various specific embodiments, examples of the electronic device 100 include smartphones, tablets, media players, laptops, etc.

[0046] Reference Figure 1J , in some specific embodiments, the electronic device 100 displays replacement enabling representations to replace the CGR object with a replacement CGR object. In Figure 1J the example, the electronic device 100 displays a first replacement enabling representation 194a (e.g., a left arrow) and a second replacement enabling representation 194b (e.g., a right arrow), which allow the user of the electronic device 100 to replace the CGR sofa 116 with a replacement CGR object (e.g., another CGR sofa, e.g., a different CGR sofa representing a different physical sofa).

[0047] Reference Figure 1K , the electronic device 100 detects a user input 196 related to the second replacement enabling representation 194b. As Figure 1L shown, in response to detecting the user input 196, the electronic device 100 replaces the CGR sofa 116 with a second CGR sofa 198 representing a second physical sofa, which is different from the physical sofa represented by the Figure 1K shown CGR sofa 116. In Figure 1L the example, the enabling representation 120 is configured to add the ID of the second physical sofa to the user selection queue 150.

[0048] Sometimes it is difficult to select a wearable physical item via a device because size charts are often inaccurate and wearable physical items often do not fit the user well. As a result, users often have to return the wearable physical item, which results in unnecessary user input that causes wear and tear on the device and / or excessive power usage. The present disclosure provides methods, systems, and / or devices for generating a CGR object representing a wearable physical item and displaying the CGR object as being worn by a CGR representation of the user. The CGR object is associated with a deformation model that defines how the CGR object deforms when worn by the CGR representation of the user.

[0049] The deformation model of the CGR object is a function of one or more material properties of the wearable physical item. For example, the deformation model is a function of the material type, texture, stiffness, elasticity, color, and / or size of the wearable physical item. The CGR representation of the user can be generated by scanning the user. For example, by taking a photo of the user and determining the dimensions of the user based on photogrammetry. The dimensions of the user can also be determined based on depth data captured by a depth camera. The device can obtain the CGR object (e.g., from the manufacturer of the wearable physical item) or generate the CGR object based on the material properties of the wearable physical item. For example, the device can use size values and material composition information associated with the wearable physical item to generate the CGR object.

[0050] Figure 2A is a block diagram of an example physical environment 20 according to some specific implementations. Although related features are shown, those of ordinary skill in the art will recognize from the present disclosure that, for the sake of brevity and in order not to obscure more relevant aspects of the exemplary specific implementations disclosed herein, various other features are not shown. To that end, as a non-limiting example, the physical environment 20 includes a person 22 and an electronic device 200. As Figure 2A shown, in some specific implementations, the electronic device 200 is held by the person 22. In some specific implementations, the electronic device 200 includes a smart phone, a tablet computer, a laptop computer, etc.

[0051] In Figure 2AIn the example, the electronic device 200 displays a CGR environment 210. The CGR environment 210 includes a CGR representation 212 of a person 22. In some embodiments, the CGR representation 212 is proportional to the person 22. For example, the ratio between the head and the torso of the CGR representation 212 matches the ratio between the head and the torso of the person 22. In various embodiments, the electronic device 200 generates the CGR representation 212 of the person 22 based on a body model of the person 22. In some embodiments, the electronic device 200 captures images of the person 22, generates a body model of the person 22 based on these images, and generates the CGR representation 212 according to the body model. In some embodiments, the electronic device 100 receives user input corresponding to body measurements of the person 22, and the electronic device 200 generates the CGR representation 212 based on these body measurements.

[0052] In Figure 2A the example, the CGR environment 210 includes a CGR object configuration panel 220 (hereinafter referred to as "panel 220" for brevity). The panel 220 includes a two-dimensional (2D) representation 222 of a T-shirt (e.g., an image of a T-shirt). In some embodiments, the panel 220 includes an affordance representation for selecting a specific configuration of the T-shirt represented by the 2D representation 222. In Figure 2A the example, the panel 220 includes a size affordance representation 224 for selecting the size of the T-shirt, a style affordance representation 226 for selecting the style of the T-shirt, and a material type affordance representation 228 for selecting the material type of the T-shirt. The size affordance representation 224 includes a small size affordance representation 224a for selecting a small-sized T-shirt, a medium size affordance representation 224b for selecting a medium-sized T-shirt, and a large size affordance representation 224c for selecting a large-sized T-shirt. The style affordance representation 226 includes a fitted style affordance representation 226a for selecting a fitted style of the T-shirt, and a classic style affordance representation 226b for selecting a classic style of the T-shirt. The material type affordance representation 228 includes a cotton affordance representation 228a for selecting a cotton type of the T-shirt, a polyester affordance representation 228b for selecting a polyester type of the T-shirt, a blend affordance representation 228c for selecting a blend material type of the T-shirt, and a merino wool affordance representation 228d for selecting a merino wool type of the T-shirt.

[0053] Referring Figure 2B , the electronic device 200 detects that the small size affordance representation 224a, the fitted style affordance representation 226a, and the cotton affordance representation 228a have been selected. The electronic device 200 generates and displays a CGR T-shirt 230 associated with the CGR representation 212 of the person 22. For example, the electronic device 200 displays the CGR T-shirt 230 as being worn by the CGR representation 212 of the person 22. In some embodiments, the electronic device 200 overlays the CGR T-shirt 230 on the CGR representation 212 of the person 22.

[0054] The CGR T-shirt 230 is associated with a deformation model 232. The deformation model 232 defines how the CGR T-shirt 230 interacts with the CGR representation 212 of the person 22, which indicates how the corresponding physical T-shirt (e.g., a small and fitted cotton T-shirt) will fit on the person 22. In some embodiments, the deformation model 232 is a function of the selected affordance representations 224, 226, and 228. For example, the deformation model 232 is a function of the small size, fitted style, and cotton material of the T-shirt. More generally, in various embodiments, the deformation model 232 is a function of one or more material properties of the T-shirt (e.g., size, material type, style, stiffness, elasticity, etc.). The deformation model 232 defines the deformation of the CGR T-shirt 230 on the CGR representation 212 of the person 22. The deformation model 232 defines how the physical T-shirt represented by the CGR T-shirt 230 will fit on the person 22.

[0055] Since the deformation model 232 characterizes one or more material properties of the T-shirt, the CGR T-shirt 230 is within a certain similarity of the corresponding physical T-shirt. For example, the CGR T-shirt 230 is within a certain similarity of a physical T-shirt that is small, fitted, and made of cotton. Thus, the deformation of the CGR T-shirt 230 on the CGR representation 212 of the person 22 is within a certain similarity of the deformation of the corresponding physical T-shirt on the person 22. For example, the fit of the CGR T-shirt 230 on the CGR representation 212 is within a certain similarity of the fit of the corresponding physical T-shirt on the person 22. In Figure 2B the example, placing the CGR T-shirt 230 on the CGR representation 212 produces CGR stretch lines 234. Since the deformation of the CGR T-shirt 230 on the CGR representation 212 is within a certain similarity of the deformation of the corresponding physical T-shirt on the person 22, a person 22 wearing the corresponding physical T-shirt will likely result in physical stretch lines similar to the CGR stretch lines 234.

[0056] Refer to Figure 2C , the electronic device 200 detects that the large-size affordance representation 224c, classic-style affordance representation 226b, and blended-fabric affordance representation 228c have been selected. In response to detecting the selection of the large-size affordance representation 224c, classic-style affordance representation 226b, and blended-fabric affordance representation 228c, the electronic device 200 generates and displays the CGR T-shirt 240 as being worn by the CGR representation 212 of the person 22. The CGR T-shirt 240 is associated with a deformation model 242. The deformation model 242 is a function of the large size, classic style, and blended-fabric material of the T-shirt. The deformation model 242 defines the deformation of the CGR T-shirt 240 on the CGR representation 212 of the person 22. In Figure 2CIn the example, placing the CGR T-shirt 240 on the CGR representation 212 causes the CGR sag 244 to be located at the neck portion of the CGR T-shirt 240, and the CGR sag 246 to face the bottom of the CGR T-shirt 240. Since the deformation model 242 models the deformation of the corresponding physical T-shirt on the person 22, when the person 22 wears the corresponding physical T-shirt, the corresponding physical T-shirt will likely sag at the neck of the physical T-shirt and sag towards the bottom of the physical T-shirt.

[0057] Reference Figure 2D , the electronic device 200 detects that the medium size affordance representation 224b, the slim fit style affordance representation 226a, and the merino wool affordance representation 228d have been selected. In response to detecting the selection of the medium size affordance representation 224b, the slim fit style affordance representation 226a, and the merino wool affordance representation 228d, the electronic device 200 generates and displays the CGR T-shirt 250 as being worn by the CGR representation 212 of the person 22. The CGR T-shirt 250 is associated with a deformation model 252. The deformation model 252 is a function of the medium size, the slim fit style, and the merino wool material of the T-shirt. The deformation model 252 defines the deformation of the CGR T-shirt 250 on the CGR representation 212 of the person 22. In Figure 2D the example, placing the CGR T-shirt 250 on the CGR representation 212 does not cause stretch lines or sag. Since the deformation model 252 models the deformation of the corresponding physical T-shirt on the person 22, when the person 22 wears the corresponding physical T-shirt, the corresponding physical T-shirt will likely not sag or cause stretch lines. The CGR T-shirt 250 appears to fit better on the CGR representation 212 than Figure 2B and Figure 2C the CGR T-shirts 230 and 240 shown respectively. Thus, the physical T-shirt corresponding to the CGR T-shirt 250 will likely fit better on the person 22 than the physical T-shirts corresponding to the CGR T-shirts 230 and 240.

[0058] Figure 2E shows a graphical user interface (GUI) 260 that allows the person 22 to search for wearable physical items based on the body model 270 of the person 22. In some specific implementations, the GUI 260 allows the person 22 to generate the body model 270. In Figure 2EIn the example of [[ID=]], the GUI 260 includes an enabling representation 262 that, when activated, triggers the capture of an image of the person 22. In some specific implementations, the GUI 260 includes an upload image enabling representation 264 for uploading an image (e.g., an image of the person 22). In some specific implementations, the electronic device 220 generates a body model 270 based on the captured image and / or the uploaded image. In some specific implementations, the electronic device 200 extracts the dimensions of the person 22 from the captured image and / or the uploaded image using methods, devices, and / or systems associated with photogrammetry, and the electronic device 200 generates a body model 270 based on the dimensions of the person 22.

[0059] In some specific implementations, the GUI 260 includes a measurement result enabling representation 266 that allows the person 22 to input body measurement results (e.g., the body dimensions of the person 22, such as waist size, arm size, thigh size, arm size, etc.). In such specific implementations, the electronic device 200 generates a body model 270 based on the body measurement results obtained by the electronic device 200. In some specific implementations, the GUI 260 includes an upload model enabling representation 268 that, when activated, allows the person 22 to upload the body model 270. In Figure 2E the example of [[ID=]], the GUI 260 includes a search enabling representation 272 that, when activated, triggers a search for wearable physical items based on the body model 270.

[0060] In some specific implementations, the CGR environment 210 simultaneously displays CGR objects representing multiple wearable physical items associated with the CGR representation 212 of the person 22. For example, the CGR environment 210 simultaneously displays the CGR representation 212 wearing multiple CGR objects representing the corresponding wearable physical items. Referring to Figure 2F , the panel 220 displays a 2D representation 278 of the physical shorts. The CGR environment 210 displays the CGR shorts 280, which represent the physical shorts worn by the CGR representation 212 of the person 22. The CGR shorts 280 are associated with a deformation model 282 that models the deformation of the corresponding physical shorts on the person 22.

[0061] Referring to Figure 2G , in some specific implementations, the CGR environment 210 includes an enabling representation 290 to add an identifier identifying the physical T-shirt represented by the CGR T-shirt 250 to the user selection queue (e.g., Figure 1C the user selection queue 150 shown). In some specific implementations, the electronic device 200 detects the activation of the enabling representation 290 and adds the identifier to the user selection queue.

[0062] In some embodiments, a head - mountable device (HMD) (not shown) worn by a person 22 presents (e.g., displays) a CGR environment 210 according to various embodiments. In some embodiments, the HMD includes an integrated display (e.g., a built - in display) that displays the CGR environment 210. In some embodiments, the HMD includes a head - mountable housing. In various embodiments, the head - mountable housing includes an attachment area to which another device having a display can be attached. For example, in some embodiments, the electronic device 200 can be attached to the head - mountable housing. In various embodiments, the head - mountable housing is shaped to form a receptacle for receiving another device (e.g., the electronic device 200) that includes a display. For example, in some embodiments, the electronic device 200 slides / snaps onto the head - mountable housing or is otherwise attached to the head - mountable housing. In some embodiments, the display of the device attached to the head - mountable housing presents (e.g., displays) the CGR environment 210. In various embodiments, examples of the electronic device 200 include a smart phone, a tablet computer, a media player, a laptop computer, etc.

[0063] Adding identifiers of physical items from different sources to the respective user - selection queues typically requires a sequence of user inputs that degrade the user experience. For example, the user may have to navigate to the web page of each source and add the identifier of the physical item provided by that source to a source - specific user - selection queue. Excessive user input results in unnecessary wear on the device and unnecessary battery usage. The present disclosure provides methods, systems, and / or devices for allowing a user to view CGR objects representing physical items provided by different sources and add identifiers that identify some or all of the physical items to source - specific user - selection queues. The present disclosure provides methods, systems, and / or devices that reduce the need to navigate to different pages corresponding to each source. Thus, the present disclosure provides an enhanced user experience, reduces wear on the device, and / or reduces power consumption by reducing unnecessary user input.

[0064] The present invention discloses a device that simultaneously displays CGR objects representing physical items from different sources. The device allows a user to provide input to select some or all of the CGR objects. The device adds the identifiers of the physical items corresponding to the selected CGR objects to source - specific user - selection queues. For example, if a user selects physical items provided by five different sources, the device populates five source - specific user - selection queues corresponding to those five sources.

[0065] Figure 3Ais a block diagram of an example environment according to some illustrative embodiments. Although related features are shown, those of ordinary skill in the art will recognize from this disclosure that various other features are not shown for the sake of brevity and in order not to obscure more relevant aspects of the illustrative embodiments disclosed herein. To that end, as a non-limiting example, the environment includes an electronic device 300. In some illustrative embodiments, the electronic device 300 is held by a person (not shown). In some illustrative embodiments, the electronic device 300 includes a smart phone, a tablet computer, a laptop computer, and the like.

[0066] In some illustrative embodiments, the electronic device 300 presents a CGR environment 310. In Figure 3A the example, the CGR environment 310 includes a first CGR lamp 312, a second CGR lamp 314, a CGR sofa 316, a CGR painting 318, and a CGR coffee table 320. In some illustrative embodiments, the CGR environment 310 represents a physical environment, and some of the CGR objects in the CGR environment 310 represent physical items in the physical environment. For example, the first CGR lamp 312 represents a first physical lamp in the physical environment, and the second CGR lamp 314 represents a second physical lamp in the physical environment.

[0067] In some illustrative embodiments, some of the CGR objects in the CGR environment 310 represent physical items that are not in the physical environment represented by the CGR environment 310. For example, the CGR sofa 316 represents a physical sofa that is not in the physical environment corresponding to the CGR environment 310. Similarly, the CGR painting 318 represents a physical painting that is not in the corresponding physical environment. The CGR coffee table 320 represents a physical coffee table that is not in the corresponding physical environment.

[0068] In some illustrative embodiments, some of the CGR objects in the CGR environment 310 represent physical items purchased from a source (e.g., purchased from a store such as a physical store or an online store). In Figure 3A the example, the CGR sofa 316 represents a physical sofa purchased from a first source 330 (e.g., purchased from Ron's Furniture Store). The CGR painting 318 represents a physical painting purchased from a second source 340 (e.g., purchased from Jacob's Art Gallery). The CGR coffee table 320 represents a physical coffee table purchased from the first source 330.

[0069] In some specific implementations, the physical item represented by a CGR object is associated with a corresponding identifier (ID) that identifies the physical item. For example, the physical sofa represented by the CGR sofa 316 is associated with the sofa ID 317 that identifies the physical sofa. In some specific implementations, the physical painting represented by the CGR painting 318 is associated with the painting ID 319 that identifies the physical painting. In some specific implementations, the physical coffee table represented by the CGR coffee table 320 is associated with the coffee table ID 321 that identifies the physical coffee table. In some specific implementations, the ID includes a serial number, a barcode number, an item number, a model number, a product number, a manufacturer code, and / or a machine-readable representation of data (e.g., an optically machine-readable representation of data, such as a barcode or a QR code).

[0070] The first source 330 is associated with the first user selection queue 332, and the second source 340 is associated with a second user selection queue 342 that is different from the first user selection queue 332. More generally, in various specific implementations, each source is associated with a corresponding user selection queue. When the electronic device 300 detects a selection of a specific CGR object that represents a specific physical item from a specific source, the electronic device 300 adds the identifier (ID) that identifies the specific physical item to the source-specific user selection queue associated with the specific source.

[0071] Reference Figure 3B , the electronic device 300 detects a user input 350 at a location corresponding to the CGR sofa 316. In some specific implementations, the user input 350 corresponds to a request to associate the physical sofa represented by the CGR sofa 316 with the first user selection queue 332. For example, in some specific implementations, the user input 350 corresponds to a request to add the sofa ID 317 that identifies the physical sofa represented by the CGR sofa 316 to the first user selection queue 332.

[0072] As Figure 3C shown, in some specific implementations, the electronic device 300 adds the sofa ID 317 to the first user selection queue 332 in response to detecting the user input 350. In some specific implementations, the electronic device 300 displays a notification 352 in response to adding the sofa ID 317 to the first user selection queue 332. In some specific implementations, the notification 352 includes text 354 indicating that the electronic device 300 has added the sofa ID 317 to the first user selection queue 332. In some specific implementations, the notification 352 includes a first queue enabling representation 356 that triggers the display of a visual representation of the first user selection queue 332 when activated. In some specific implementations, the notification 352 includes a return enabling representation 358 that causes the electronic device 300 to redisplay the CGR environment 310 when activated.

[0073] Reference Figure 3D , the electronic device 300 detects a user input 360 related to the return affordance indication 358. As Figure 3E shown, in response to detecting the user input 360, the electronic device 300 redisplay the CGR environment 310. In some specific implementations, the electronic device 300 modifies the visual attributes of the CGR sofa 316 to indicate that the sofa ID 317 has been added to the first user selection queue 332. In Figure 3E the example of, the electronic device 300 has displayed the CGR sofa 316 with a shadow effect. More generally, in some specific implementations, the electronic device 300 changes the visual attributes of the CGR object to indicate that the ID of the corresponding physical item has been added to the source-specific user selection queue.

[0074] In Figure 3E the example of, the electronic device 300 displays a notation 362 (e.g., a checkmark) in the queue to indicate that the CGR sofa 316 has been selected and / or to indicate that the ID identifying the corresponding physical sofa has been added to the user selection queue. In some specific implementations, the electronic device 300 displays text 364 to indicate that the sofa ID 317 has been added to the first user selection queue 332. In some specific implementations, the electronic device 300 displays a remove affordance indication 366 that, when activated, removes the sofa ID 317 from the first user selection queue 332.

[0075] Reference Figure 3F , the electronic device 300 detects a user input 370 at a position corresponding to the CGR painting 318. In some specific implementations, the user input 370 corresponds to a request to associate the physical painting represented by the CGR painting 318 with the second user selection queue 342. For example, in some specific implementations, the user input 370 corresponds to a request to add the painting ID 319 identifying the physical painting represented by the CGR painting 318 to the second user selection queue 342.

[0076] As Figure 3DAs shown, in some specific embodiments, the electronic device 300 adds the painting ID 319 to the second user selection queue 342 in response to detecting the user input 370. In some specific embodiments, the electronic device 300 displays the notification 372 in response to adding the painting ID 319 to the second user selection queue 342. In some specific embodiments, the notification 372 includes the text 374 indicating that the electronic device 300 has added the painting ID 319 to the second user selection queue 342. In some specific embodiments, the notification 372 includes the second queue enabling representation 376, which triggers the display of the visual representation of the second user selection queue 342 when activated. In some specific embodiments, the notification 372 includes the return enabling representation 378, which causes the electronic device 300 to redisplay the CGR experience 310 when activated. In some specific embodiments, the notification 372 includes the multi-queue enabling representation 378 (e.g., all queue enabling representations), which triggers the display of the visual representations of multiple user selection queues (e.g., the visual representations of all user selection queues, e.g., the visual representation of the first user selection queue 332 and the visual representation of the second user selection queue 342) when activated. The notification 372 includes the return enabling representation 380, which causes the electronic device 300 to redisplay the CGR environment 310 when activated.

[0077] Reference Figure 3H , the electronic device 300 detects the user input 382 related to the return enabling representation 380. As Figure 3I shown, in response to detecting the user input 382, the electronic device 300 displays the first visual representation 332R of the first user selection queue 332 and the second visual representation 342R of the second user selection queue 342. The first visual representation 332R includes the 2D representation 333 of the physical sofa, the description 334 of the sofa, the first delete enabling representation 335 for removing the sofa ID 317 from the first user selection queue 332, the first modification enabling representation 336 for modifying the number of sofas, and the first confirmation enabling representation 337 for confirming the first user selection queue 332. The second visual representation 342R includes the 2D representation 343 of the physical painting, the description 344 of the painting, the second delete enabling representation 345 for removing the painting ID 319 from the second user selection queue 342, the second modification enabling representation 346 for modifying the number of paintings, and the second confirmation enabling representation 347 for confirming the second user selection queue 342. In some specific embodiments, the electronic device 300 displays the third confirmation enabling representation 384, which simultaneously confirms the first user selection queue 332 and the second user selection queue 342 when activated.

[0078] In some specific implementations, detecting the activation of the first confirmation enabling indication 337 triggers the placement of the physical sofa in the physical environment around the electronic device 300. For example, in some specific implementations, in response to detecting the activation of the first confirmation enabling indication 337, the electronic device 300 transmits a message to a device associated with the first source 330. In some specific implementations, the message includes a request to deliver the physical sofa to the physical environment around the electronic device 300.

[0079] In some specific implementations, a head-mounted device (HMD) (not shown) worn by a person presents (e.g., displays) the CGR environment 310 according to various specific implementations. In some specific implementations, the HMD includes an integrated display (e.g., a built-in display) that displays the CGR environment 310. In some specific implementations, the HMD includes a head-wearable housing. In various specific implementations, the head-wearable housing includes an attachment area to which another device having a display can be attached. For example, in some specific implementations, the electronic device 300 can be attached to the head-wearable housing. In various specific implementations, the head-wearable housing is shaped to form a receiver for receiving another device (e.g., the electronic device 300) including a display. For example, in some specific implementations, the electronic device 300 slides / snaps onto the head-wearable housing or is otherwise attached to the head-wearable housing. In some specific implementations, the display of the device attached to the head-wearable housing presents (e.g., displays) the CGR environment 310. In various specific implementations, examples of the electronic device 300 include a smart phone, a tablet computer, a media player, a laptop computer, etc.

[0080] In some cases, a person may want to see how a physical item would look in the person's physical environment. Some devices generate and present a CGR environment similar to the person's physical environment and place CGR objects representing new physical items, thereby allowing the person to see how the new physical items would look in the physical environment. However, if the physical environment is too cluttered, the corresponding CGR environment is similarly cluttered. Thus, there may be no space in the CGR environment to place CGR objects representing new physical items. The present disclosure provides methods, devices, and / or systems for masking physical items in a physical environment to make space for CGR objects representing new physical items. The device detects a physical surface, detects a physical item that obscures a portion of the physical surface, and synthesizes a masking element to mask the physical item and make space for the new CGR object.

[0081] Figure 4Ais a block diagram of an example physical environment 40 according to some specific implementations. Although related features are shown, those of ordinary skill in the art will recognize from this disclosure that various other features are not shown for the sake of brevity and to not obscure more relevant aspects of the exemplary specific implementations disclosed herein. To that end, as a non-limiting example, the physical environment 40 includes a physical surface 42. In some specific implementations, the physical surface 42 represents the top surface of a physical table. In some specific implementations, the physical surface 42 represents the floor.

[0082] In some specific implementations, the physical surface 42 is associated with visual attributes 43. In some specific implementations, the visual attributes 43 indicate the material properties of the physical surface 42. For example, in some specific implementations, the visual attributes 43 indicate the color of the physical surface 42. In some specific implementations, the visual attributes 43 indicate the material type of the physical surface 42. In some specific implementations, the visual attributes 43 indicate the texture of the physical surface 42. In some specific implementations, the visual attributes 43 indicate the reflectivity of the physical surface 42.

[0083] In Figure 4A the example of, there are physical items that occlude corresponding portions of the physical surface 42. For example, a first physical item 44 (e.g., a package), a second physical item 46 (e.g., a box), and a third physical item 48 (e.g., a speaker) are placed on the physical surface 42. As can be seen in Figure 4A the first physical items 44, 46, and 48 occlude (e.g., cover) a significant portion (e.g., most of) the physical surface 42.

[0084] Referring to Figure 4B , in some specific implementations, an electronic device 400 presents (e.g., displays) a CGR environment 410 corresponding to (e.g., representing) the physical environment 40. The CGR environment 410 includes a CGR surface 412 representing the physical surface 42. The CGR surface 412 is associated with visual attributes 413 within a certain similarity of the visual attributes 43 of the physical surface. For example, in some specific implementations, the CGR surface 412 has the same material properties as the physical surface 42. For example, the CGR surface 412 has the same color as the physical surface 42, uses the same material type, has the same texture, and / or has the same texture.

[0085] In Figure 4B the example of, the CGR environment 410 includes a first CGR object 414 representing the first physical item 44, a second CGR object 416 representing the second physical item 46, and a third CGR object 418 representing the third physical item 48. As in Figure 4BAs can be seen, CGR objects 414, 416, and 418 occupy a significant portion of the CGR surface 412. Therefore, there is not enough space on the CGR surface 412 to place additional CGR objects. In some specific implementations, the electronic device 400 is held by a person (not shown). In some specific implementations, the electronic device 400 includes a smart phone, a tablet computer, a laptop computer, etc.

[0086] In some specific implementations, the CGR environment 410 is a perspective of the physical environment 40. For example, in some specific implementations, the CGR environment 410 is a video perspective of the physical environment 40. In some specific implementations, the CGR environment 410 is an optical perspective of the physical environment 40. In such specific implementations, the CGR surface 412 and the CGR objects 414, 416, and 418 are the same as the physical surface 42 and the physical items 44, 46, and 48, respectively.

[0087] Reference Figure 4C , in some specific implementations, the electronic device 400 synthesizes a masking element 422 to mask the physical items 44, 46, and 48 located on the physical surface 42. In some specific implementations, the electronic device 400 synthesizes a masking element 422 to mask the CGR objects 414, 416, and 418 on the CGR surface 412. As can be seen in Figure 4C , in some specific implementations, the synthesized masking element 422 provides the appearance that there are no CGR objects on the CGR surface 412. Since the CGR surface 412 represents the physical surface 42, the synthesized masking element 422 provides the appearance that there are no physical items on the physical surface 42. Therefore, the synthesized masking element 422 provides the appearance of a less cluttered physical surface 42 without requiring an operator (e.g., a person or a robot) to physically remove the physical items 44, 46, and 48 from the physical surface 42.

[0088] Reference Figure 4D , in some specific implementations, the masking element 422 is associated with visual attributes 423. In some specific implementations, the visual attributes 423 of the masking element 422 are within a certain similarity of the visual attributes 413 of the CGR surface 412. Since the visual attributes 413 of the CGR surface 412 are within a certain similarity of the visual attributes 43 of the physical surface, in some specific implementations, the visual attributes 423 of the masking element 422 are within a certain similarity of the visual attributes 43 of the physical surface 42. Therefore, in some specific implementations, the masking element 422 has the same color, material type, texture, and / or reflectivity as the physical surface 42. The matching of the visual attributes 423 of the masking element 422 with the visual attributes 43 of the physical surface 42 provides the appearance that there are no physical items on the physical surface 42.

[0089] Figure 4EAnother masking element 422a associated with the visual attribute 423a is shown. In Figure 4E the example, the visual attribute 423a of the masking element 422a is different from (e.g., significantly different from) the visual attribute 413 of the CGR surface 412. Since the visual attribute 413 of the CGR surface 412 is within a certain similarity of the visual attribute 43 of the physical surface 42, in some embodiments, the visual attribute 423a of the masking element 422a is not within a certain similarity of the visual attribute 43 of the physical surface 42. Thus, in some embodiments, the masking element 422a has a different color, material type, texture, and / or reflectivity from the physical surface 42.

[0090] Referring Figure 4F , in some embodiments, the electronic device 400 synthesizes a plurality of masking elements. In some embodiments, each masking element masks a respective one of the physical items. In some embodiments, each masking element conforms to the shape of the physical item masked by the masking element. In some embodiments, each masking element masks a respective one of the CGR objects. In some embodiments, each masking element conforms to the shape of the CGR object to which the masking element conforms. In Figure 4F the example, the electronic device 400 generates a first masking element 424 to mask a first CGR object 414 representing a first physical item 44, a second masking element 426 to mask a second CGR object 416 representing a second physical item 46, and a third masking element 428 to mask a third CGR object 418 representing a third physical item 48. In some embodiments, the first masking element 424 conforms to the shape of the first CGR object 414 and / or the first physical item 44. In some embodiments, the second masking element 426 conforms to the shape of the second CGR object 416 and / or the second physical item 46. In some embodiments, the third masking element 428 conforms to the shape of the third CGR object 418 and / or the third physical item 48.

[0091] Referring Figure 4G , in some embodiments, the electronic device 400 displays a CGR object store 430. The CGR object store 430 includes CGR objects representing physical items. In Figure 4G the example, the CGR object store 430 includes a fourth CGR object 440 (e.g., a CGR table lamp) representing a fourth physical item (e.g., a physical table lamp). As Figure 4H shown, in some embodiments, the electronic device 400 detects a user input 442 corresponding to a request to place the fourth CGR object 440 on the CGR surface 412. In some embodiments, the user input 442 includes a drag gesture starting at the fourth CGR object 440 and ending at the CGR surface 412.

[0092] As Figure 4I shown, in some specific implementations, the electronic device 400 synthesizes a masking element 422 in response to detecting a user input 442. The electronic device 400 overlays the fourth CGR object 440 on the masking element 422 to provide the appearance that the fourth CGR object 440 is placed on the CGR surface 412. In Figure 4I the example of, the CGR object storage 430 displays a fifth CGR object 444.

[0093] Referring Figure 4J , in some specific implementations, the electronic device 400 displays a web page 450 corresponding to a fourth physical item (e.g., a physical table lamp). In some specific implementations, the web page 450 includes a 2D representation 452 of the fourth physical item (e.g., an image of the fourth physical item), a description 454 of the fourth physical item, and an enabling representation 456 that, when activated, causes the electronic device 400 to display a fourth CGR object 440 representing the fourth physical item. As Figure 4K shown, the electronic device 400 detects a user input 458 related to the enabling representation 456. As Figure 4L shown, in response to detecting the user input 458, the electronic device 400 synthesizes a masking element 422 and displays the fourth CGR object 440 on top of the masking element 422.

[0094] Referring Figure 4M , in some specific implementations, the electronic device 400 displays a clearing enabling representation 460 that, when activated, causes the electronic device 400 to synthesize a masking element onto the CGR surface 412 to mask the CGR objects 414, 416, and 418 on the CGR surface 412. As Figure 4N shown, the electronic device 400 detects a user input 462 related to the clearing enabling representation 460. As Figure 4O shown, in response to detecting the user input 462, the electronic device 400 synthesizes the masking element 422 on the CGR surface 412. In some specific implementations, the electronic device 400 displays a removing enabling representation 464 that, when activated, causes the electronic device 400 to remove the masking element 422 and redisplay the CGR objects 414, 416, and 418 on the CGR surface 412.

[0095] In some specific implementations, a head-mounted device (HMD) (not shown) worn by a person presents (e.g., displays) a CGR environment 410 according to various specific implementations. In some specific implementations, the HMD includes an integrated display (e.g., a built-in display) for displaying the CGR environment 410. In some specific implementations, the HMD includes a head-wearable housing. In various specific implementations, the head-wearable housing includes an attachment area to which another device having a display can be attached. For example, in some specific implementations, the electronic device 400 can be attached to the head-wearable housing. In various specific implementations, the head-wearable housing is shaped to form a receiver for receiving another device (e.g., the electronic device 400) including a display. For example, in some specific implementations, the electronic device 400 slides / snaps onto the head-wearable housing or is otherwise attached to the head-wearable housing. In some specific implementations, the display of the device attached to the head-wearable housing presents (e.g., displays) the CGR environment 410. In various specific implementations, examples of the electronic device 400 include a smart phone, a tablet computer, a media player, a laptop computer, etc.

[0096] Figure 5A is a flowchart representation of a method 500 for controlling a user selection queue. In various specific implementations, the method 500 is performed by a device (e.g., an electronic device 100 as Figures 1A to 1L shown) having a display, a non-transitory memory, and one or more processors coupled to the display and the non-transitory memory. In some specific implementations, the method 500 is performed by processing logic (including hardware, firmware, software, or a combination thereof). In some specific implementations, the method 500 is performed by a processor executing code stored in a non-transitory computer-readable medium (e.g., a memory).

[0097] As shown in block 510, in various specific implementations, the method 500 includes displaying a computer-generated reality (CGR) object in a CGR environment. In some specific implementations, the CGR object represents a physical item. For example, as Figure 1C shown, the electronic device 100 displays a CGR sofa 116 in a CGR environment 110. As described with respect to Figure 1C the CGR sofa 116 represents a physical sofa (e.g., the sofa represented by the 2D representation 104 as Figure 1A shown).

[0098] As shown in block 520, in various specific implementations, the method 500 includes synthesizing an affordance representation associated with the CGR object. For example, as Figure 1C shown, the electronic device 100 synthesizes an affordance representation 120 associated with the CGR sofa 116. In some specific implementations, the method 500 includes displaying the affordance representation adjacent to the CGR object. For example, as Figure 1CAs shown, the electronic device 100 displays an enabling representation 120 adjacent to the CGR sofa 116.

[0099] As shown in block 530, in various embodiments, the method 500 includes detecting an input related to the enabling representation. For example, as Figure 1D shown, the electronic device 100 detects a user input 130 related to the enabling representation 120. In some embodiments, the method 500 includes detecting a contact at a position corresponding to the enabling representation.

[0100] As shown in block 540, in various embodiments, the method 500 includes adding an identifier identifying a physical item to a user selection queue in response to detecting the input. For example, as Figures 1D to 1E shown, the electronic device 100 adds an identifier 118 of the sofa to the user selection queue 150 in response to detecting the user input 130. In some embodiments, the method 500 includes writing an identifier identifying a physical item to the user selection queue. In some embodiments, the method 500 includes associating an identifier identifying a physical item with the user selection queue. In various embodiments, adding an identifier of a physical item to the user selection queue while the device is displaying a CGR object representing the physical item reduces the need to manually navigate to a web page corresponding to the physical item in order to add an identifier identifying the physical item to the user selection queue.

[0101] Referring Figure 5B , as shown in block 510a, in some embodiments, the CGR environment includes representations of physical items in a physical environment located around the device. For example, as Figure 1C shown, the CGR environment 110 includes a first CGR floor lamp 112 representing a first floor lamp 12 in the physical environment 10 around the electronic device 100, and a second CGR floor lamp 114 representing a second floor lamp 14 in the physical environment 10 around the electronic device 100.

[0102] As shown in block 510b, in some embodiments, the method includes displaying a web page including a two-dimensional (2D) representation of a physical item, and obtaining an input to switch from the web page to a CGR mode in which a CGR object representing the physical item is displayed. For example, as Figure 1A shown, the electronic device 100 displays a web page 102 including a 2D representation 104 of a sofa. Additionally, as Figure 1B shown, the electronic device 100 detects a user input 108 related to an enabling representation 106 corresponding to a request to switch to the CGR mode in which a CGR sofa 116 representing the sofa is displayed.

[0103] As shown in block 520a, in some embodiments, the synthetic affordance representation can include a synthetic affordance representation within a threshold distance of the CGR object. For example, as Figure 1C shown, the electronic device 100 synthesizes an affordance representation 120 adjacent to or near the CGR couch 116. In some embodiments, method 500 includes obtaining a threshold distance.

[0104] As shown in block 520b, in some embodiments, method 500 includes synthesizing an affordance representation at a specified portion of the CGR environment. For example, as Figure 1H shown, the electronic device 100 synthesizes an affordance representation 120 at the specified portion 160. In some embodiments, method 500 includes selecting the specified portion. For example, as Figure 1H shown, the electronic device 100 selects the lower right corner of the CGR environment 110 as the specified portion 160.

[0105] As shown in block 520c, in some embodiments, method 500 includes configuring the affordance representation. As shown in block 520d, in some embodiments, method 500 includes changing a visual attribute of the affordance representation. For example, in some embodiments, method 500 includes changing the text string displayed by the affordance representation. In some embodiments, method 500 includes replacing the default text of the affordance representation with a replacement string. In some embodiments, the replacement string is a function of the physical item represented by the CGR object (e.g., changing "Add to queue" to "Add couch to queue").

[0106] As shown in block 520e, in some embodiments, method 500 includes changing an operation associated with the affordance representation. In some embodiments, method 500 includes changing the operation to add an identifier to a user-selected queue and confirm the user-selected queue. In some embodiments, method 500 includes changing the operation to add an identifier to a favorites list. In some embodiments, method 500 includes changing the operation to save an identifier. In some embodiments, method 500 includes bookmarking a web page associated with the physical item represented by the CGR object. In some embodiments, method 500 includes sharing the identifier with another device (e.g., another device associated with a contact in the device's contacts application).

[0107] As shown in block 520f, in some embodiments, method 500 includes restricting the network connectivity of a device when displaying an enabling representation. In some embodiments, method 500 includes preventing malicious code from executing on a device when displaying an enabling representation. In some embodiments, restricting network connectivity prevents malicious code from executing on a device. In some embodiments, method 500 includes turning off a transceiver (e.g., a radio component) of a device when displaying an enabling representation. In some embodiments, method 500 includes switching a device from a communication mode to a non-communication mode (e.g., switching to airplane mode) when displaying an enabling representation.

[0108] As shown in block 520g, in some embodiments, method 500 includes determining whether a CGR object occupies at least a threshold number of pixels and synthesizing an enabling representation in response to determining that the CGR object occupies at least the threshold number of pixels. In some embodiments, method 500 includes determining whether a CGR object occupies at least a threshold portion of a CGR environment and synthesizing an enabling representation in response to determining that the CGR object occupies at least the threshold portion of the CGR environment.

[0109] As shown in block 530a, in some embodiments, an input related to an enabling representation includes a user selection of the enabling representation. For example, as Figure 1D shown, electronic device 100 detects user input 130 related to enabling representation 120. As shown in block 530b, in some embodiments, an input related to an enabling representation includes a gaze input related to the enabling representation. For example, in some embodiments, electronic device 100 uses eye tracking to detect a gaze input. As shown in block 530c, in some embodiments, an input related to an enabling representation includes a speech input. For example, in some embodiments, electronic device 100 detects speech via a microphone, and the speech corresponds to an input that activates the enabling representation.

[0110] Referring Figure 5C to, as shown in block 540, in some embodiments, method 500 includes modifying visual attributes of a CGR object to indicate that the CGR object is selectable. For example, in some embodiments, method 500 includes displaying a CGR object with a raised appearance to indicate that the CGR object can be pressed. In some embodiments, method 500 includes displaying text adjacent to or overlapping the CGR object to indicate the selectability of the CGR object.

[0111] As shown in block 550, in some embodiments, method 500 includes detecting an input related to a CGR object and manipulating the CGR object based on the input related to the CGR object. For example, in some embodiments, electronic device 100 changes the size of a CGR object in response to detecting a pinch gesture at a location corresponding to the CGR object. In some embodiments, electronic device 100 moves the CGR object in response to detecting a drag gesture or a swipe gesture at a location corresponding to the CGR object.

[0112] As shown in block 560, in some embodiments, method 500 includes displaying a second CGR object representing a second physical item when a CGR object is displayed in a CGR environment. In some embodiments, method 500 includes adding an identifier identifying the second physical item to a user selection queue in response to detecting an input related to an affordance representation. In some embodiments, the affordance representation is associated with the CGR object and the second CGR object. For example, as Figure 1I shown, in addition to CGR sofa 116, electronic device 100 also displays CGR painting 170.

[0113] As shown in block 560a, in some embodiments, method 500 includes identifying a second physical item by a recommendation engine based on the physical item. For example, in response to electronic device 100 displaying CGR sofa 116, the recommendation engine identifies the painting represented by CGR painting 170.

[0114] As shown in block 570, in some embodiments, method 500 includes displaying a replacement affordance representation associated with the CGR object. In some embodiments, the replacement affordance representation allows the CGR object to be replaced with a third CGR object representing a third physical item. For example, as Figure 1J shown, electronic device 100 displays second replacement affordance representation 194b. In some embodiments, method 500 includes detecting an input related to the replacement affordance representation. For example, as Figure 1K shown, electronic device 100 detects user input 196 related to second replacement affordance representation 194b. In some embodiments, method 500 includes replacing the CGR object with a third CGR object in response to detecting an input related to the replacement affordance representation. For example, as Figure 1L shown, electronic device 100 displays second CGR sofa 198 in response to detecting user input 196.

[0115] Figure 6A is a flowchart representation of a method 600 for displaying a CGR object representing a wearable physical item. In various embodiments, method 600 is performed by a device having a display, a non-transitory memory, and one or more processors coupled to the display and the non-transitory memory (e.g., as Figures 2A to 2Gperformed by the electronic device 200 shown. In some specific implementations, method 600 is performed by processing logic components (including hardware, firmware, software, or combinations thereof). In some specific implementations, method 600 is performed by a processor executing code stored in a non-transitory computer-readable medium (e.g., memory).

[0116] As shown in block 610, in various specific implementations, method 600 includes obtaining a computer-generated reality (CGR) representation of a person. In some specific implementations, at least a portion of the CGR representation is proportional to the corresponding portion of the person. For example, as Figure 2A shown, the electronic device 200 obtains (e.g., generates) and presents a CGR representation 212 of person 22. As with respect to Figure 2A described, in some specific implementations, the CGR representation 212 is proportional to person 22.

[0117] As shown in block 620, in various specific implementations, method 600 includes obtaining a CGR object representing a wearable physical item. In some specific implementations, the CGR object is associated with a deformation model characterizing one or more material properties of the wearable physical item. For example, as Figure 2B shown, the electronic device 200 obtains (e.g., generates or receives) and presents a CGR T-shirt 230 that represents a physical T-shirt that is small in size, has a fitted style, and is made of cotton. As Figure 2B shown, the CGR T-shirt 230 is associated with a deformation model 232 that characterizes the material properties of the physical T-shirt (e.g., small size, fitted style, and cotton material composition).

[0118] As shown in block 630, in various specific implementations, method 600 includes displaying a CGR object associated with the CGR representation of a person. In some specific implementations, the CGR object interacts with the CGR representation of the person according to the deformation model. For example, as Figure 2B shown, the electronic device 200 displays the CGR T-shirt 230 as being worn by the CGR representation 212 of person 22. As Figure 2B shown, the CGR T-shirt 230 deforms on the CGR representation 212 according to the deformation model 232. For example, the CGR T-shirt 230 deforms to form a CGR stretch line 234. As described herein, since the deformation model models the deformation of the wearable physical item on a person, displaying the CGR object associated with the CGR representation of a person allows the person to see how the wearable physical item will fit on the person, thereby enhancing the user experience of the device.

[0119] Referring to Figure 6B , as shown in block 610a, in some specific implementations, method 600 includes obtaining a body model of a person and generating a CGR representation of the person based on the body model of the person. For example, as Figure 2EAs shown, in some specific embodiments, the electronic device 200 obtains (e.g., generates or receives) the body model 270 of the person 22, and the electronic device 200 generates (e.g., synthesizes) the CGR representation 212 of the person 22 based on the body model 270.

[0120] As shown in block 610b, in some specific embodiments, the method 600 includes obtaining the physical dimensions of a person and generating a body model based on the physical dimensions of the person. For example, as Figure 2E shown, in some specific embodiments, the electronic device 200 displays a GUI 260 with a measurement result enabling representation 266. In some specific embodiments, in response to detecting the activation of the measurement result enabling representation 266, the electronic device 200 displays user interface elements (e.g., text boxes, drop-down menus, etc.) that allow the person 22 to input physical dimensions. In some specific embodiments, the electronic device 200 generates the body model 270 based on the physical dimensions, and the electronic device 200 generates the CGR representation 212 based on the body model 270.

[0121] As shown in block 610c, in some specific embodiments, the method 600 includes capturing one or more images of a person and generating a CGR representation of the person based on the one or more images of the person. For example, as Figure 2E shown, in some specific embodiments, the electronic device 200 displays an enabling representation 262 that triggers the capture of a photo when activated. In some specific embodiments, the electronic device 200 uses the captured picture to generate the CGR representation 212 of the person 22. In some specific embodiments, the method 600 includes using methods, devices, and / or systems associated with photogrammetry to extract the physical dimensions of the person 22 from the captured photo and generating the body model 270 and / or the CGR representation 212 based on the extracted physical dimensions.

[0122] As shown in block 610d, in some specific embodiments, the method 600 includes obtaining depth data associated with a person and generating a CGR representation of the person based on the depth data. In some specific embodiments, the method 600 includes capturing depth data from a depth sensor (e.g., depth camera) of the device. In some specific embodiments, the method 600 includes determining the physical dimensions of the person based on the depth data and generating the body model and / or the CGR representation of the person based on the physical dimensions.

[0123] As shown in block 620a, in some specific embodiments, the method 600 includes generating a deformation model of a CGR object based on one or more material properties of a wearable physical item. For example, as Figure 2B shown, the electronic device 200 generates a deformation model 232 of the CGR T-shirt 230 based on the material properties of the physical T-shirt (e.g., small size, fitted style, and / or cotton composition).

[0124] As shown in block 620b, in some embodiments, method 600 includes generating a deformation model of the CGR object based on the material type of the wearable physical item. For example, as Figure 2B shown, electronic device 200 generates a deformation model 232 of CGR T-shirt 230 based on the cotton composition of the physical T-shirt.

[0125] As shown in block 620c, in some embodiments, method 600 includes generating a deformation model of the CGR object based on the texture of the wearable physical item. In some embodiments, the texture is a function of the material type. Thus, in some embodiments, generating a deformation model based on the material type of the wearable physical item includes generating a deformation model based on the texture of the wearable physical item.

[0126] As shown in block 620d, in some embodiments, method 600 includes generating a deformation model of the CGR object based on the stiffness of the wearable physical item. In some embodiments, the stiffness is a function of the material type and / or style of the wearable physical item. Thus, in some embodiments, generating a deformation model based on the material type and / or style of the wearable physical item includes generating a deformation model based on the stiffness of the wearable physical item. In some embodiments, method 600 includes obtaining a stiffness value and generating a deformation model based on the stiffness value. In some embodiments, the stiffness value relates to the amount of sizing applied to the wearable physical item.

[0127] As shown in block 620e, in some embodiments, method 600 includes generating a deformation model of the CGR object based on the size of the wearable physical item. For example, as Figure 2B shown, electronic device 200 generates a deformation model 232 based on the small size selected by person 22. As Figure 2B shown, the small size helps in the formation of CGR stretch lines 234. Similarly, as Figure 2C shown, electronic device 200 generates a deformation model 242 based on the large size selected by person 22. As Figure 2C shown, the large size helps in CGR sagging 244 and 246.

[0128] As shown in block 620f, in some embodiments, the wearable physical item includes a clothing item (e.g., a T-shirt as Figures 2A to 2D shown, or as Figure 2FThe shorts shown). In various specific implementations, method 600 allows a person to see how a particular clothing item would fit the person without trying on the clothing item. In some specific implementations, the clothing item includes undergarments (e.g., bras or panties). Since many stores do not allow people to try on undergarments, in various specific implementations, method 600 allows a person to see how a particular undergarment would fit the person without trying on the undergarment. In various specific implementations, method 600 enables a person to find clothing items that fit the person in a reduced amount of time, thereby reducing the amount of time the device's display remains on. Reducing the amount of time the device's display remains on reduces the power consumption of the device.

[0129] Reference Figure 6C , as shown in block 630a, in some specific implementations, method 600 includes displaying a CGR object as being worn by a CGR representation of a person. For example, as Figure 2B shown, electronic device 200 displays CGR T-shirt 230 as being worn by CGR representation 212 of person 22.

[0130] As shown in block 630b, in some specific implementations, method 600 includes displaying a second CGR object associated with a second portion of the CGR representation of a person. In some specific implementations, the second CGR object represents a second wearable physical item. For example, as Figure 2F shown, electronic device 200 displays CGR shorts 280 as being worn by CGR representation 212 of person 22. Displaying CGR objects representing multiple wearable physical items simultaneously reduces the amount of time the device's display remains on, thereby reducing the power consumption of the device. Displaying CGR objects representing multiple wearable physical items simultaneously allows a person to select multiple physical items simultaneously, thereby enhancing the user experience of the device.

[0131] As shown in block 640, in some specific implementations, method 600 includes searching a data store for a set of wearable physical items that fit a person's body model and selecting a wearable physical item from the set of wearable physical items that fit the person's body model. For example, as Figure 2E shown, in some specific implementations, electronic device 200 displays GUI 260, which allows person 22 to search for clothing that fits body model 270 of person 22.

[0132] As shown in block 650, in some specific implementations, method 600 includes, after displaying a CGR object associated with a CGR representation of a person, displaying an affordance for adding an identifier that identifies a wearable physical item to a user selection queue. For example, as Figure 2GAs shown, the electronic device 200 displays an enabling representation 290 that, when activated, causes the electronic device 200 to add an ID identifying the physical T-shirt represented by the CGR T-shirt 250 to a user selection queue (e.g., Figure 1C the user selection queue 150 shown). In some embodiments, method 600 includes detecting an input related to the enabling representation (e.g., detecting a user input that activates Figure 2G the enabling representation 290 shown). In some embodiments, method 600 includes adding an identifier identifying a wearable physical item to a user selection queue (e.g., writing the identifier to Figure 1C the user selection queue 150 shown).

[0133] As shown in block 660, in some embodiments, method 600 includes erasing source material to identify a wearable physical item. For example, in some embodiments, method 600 includes erasing a movie to identify clothing items worn by fictional characters in the movie and searching a clothing data store to find physical clothing items within a certain similarity to the clothing items worn by fictional characters in the movie.

[0134] Figure 7A is a flowchart representation of a method 700 for simultaneously controlling multiple user selection queues. In various embodiments, method 700 is performed by a device having a display, a non-transitory memory, and one or more processors coupled to the display and the non-transitory memory (e.g., an electronic device 300 such as Figures 3A to 3I shown). In some embodiments, method 700 is performed by processing logic (including hardware, firmware, software, or a combination thereof). In some embodiments, method 700 is performed by a processor executing code stored in a non-transitory computer-readable medium (e.g., a memory).

[0135] As shown in block 710, in various embodiments, method 700 includes displaying multiple computer-generated reality (CGR) objects representing corresponding physical items from multiple sources including a first source and a second source. For example, as Figure 3A shown, the electronic device 300 displays a CGR sofa 316 representing a physical sofa from a first source 330 and a CGR painting 318 representing a physical painting from a second source 340.

[0136] As shown in block 720, in various embodiments, method 700 includes detecting an input selecting a first CGR object among the multiple CGR objects and a second CGR object among the multiple CGR objects. In some embodiments, the first CGR object represents a first physical item from a first source and the second CGR object represents a second physical item from a second source. For example, as Figure 3B and Figure 3FAs shown, the electronic device 300 detects user inputs 350 and 370 that respectively select the CGR sofa 316 and the CGR painting 318.

[0137] As shown in block 730, in various embodiments, method 700 includes adding an identifier of a first physical item to a first user selection queue associated with a first source. For example, as Figure 3B shown, the electronic device 300 adds the sofa ID 317 to the first user selection queue 332.

[0138] As shown in block 740, in various embodiments, method 700 includes adding an identifier of a second physical item to a second user selection queue associated with a second source. For example, as Figure 3G shown, the electronic device 300 adds the painting ID 319 to the second user selection queue 342. In various embodiments, method 700 allows the IDs of physical items from different sources to be added simultaneously to the corresponding source-specific user selection queues, thereby reducing the user input corresponding to manually navigating to a web page associated with the source in order to add these IDs to the corresponding source-specific user selection queues. Thus, method 700 enhances the user experience of the device and / or extends the battery life of the device by reducing user input and / or by reducing the amount of time the device's display remains on.

[0139] Referring Figure 7B , as shown in block 710a, in some embodiments, method 700 includes switching from a web page including a two-dimensional (2D) representation of a physical item to a CGR environment in response to an input corresponding to a request to display a plurality of CGR objects. For example, in some embodiments, the electronic device 300 displays a web page similar to the web page 102 shown in Figure 1A . In some embodiments, method 700 includes detecting a user input corresponding to a request to enter a CGR mode, and displaying a CGR environment including a plurality of CGR objects in response to detecting the user input.

[0140] As shown in block 710b, in some embodiments, method 700 includes identifying at least a portion of the displayed CGR objects by a recommendation engine. In some embodiments, method 700 includes identifying physical items in the physical environment around the device, recommending new physical items based on the physical items in the physical environment, and displaying CGR objects representing the new physical items. For example, in some embodiments, the electronic device 300 identifies two floor lamps in the physical environment around the electronic device 300, recommends placing a sofa between the two floor lamps, and displays the CGR sofa 316 between the CGR lamps 312 and 314.

[0141] As shown in block 720a, in some embodiments, method 700 includes detecting respective contacts with a first CGR object and a second CGR object. For example, as Figure 3B and Figure 3F shown, the electronic device 300 detects user inputs 350 and 370 on the CGR sofa 316 and the CGR painting 318, respectively.

[0142] As shown in block 720b, in some embodiments, method 700 includes detecting that a CGR representation of a person has touched a first CGR object and a second CGR object. For example, in some embodiments, the CGR environment 310 includes a CGR representation of a person, and the CGR representation provides the user inputs 350 and 370 in Figure 3B and Figure 3F respectively.

[0143] As shown in block 720c, in some embodiments, method 700 includes detecting a gaze input involving a first CGR object and a second CGR object. In some embodiments, method 700 includes tracking the movement of a person's eyes using the device, and determining that the person has selected the first CGR object and the second CGR object in response to the person's gaze being fixed at the first CGR object and the second CGR object for a threshold amount of time.

[0144] As shown in block 720d, in some embodiments, method 700 includes detecting a speech input involving a first CGR object and a second CGR object. In some embodiments, method 700 includes detecting a voice input corresponding to the selection of the first CGR object and the second CGR object.

[0145] As shown in block 720e, in some embodiments, method 700 includes modifying visual attributes of the first CGR object and the second CGR object to indicate the selection of the first CGR object and the second CGR object. For example, as Figure 3E shown, the electronic device 300 displays the CGR sofa 316 with a shadow effect to indicate that the CGR sofa 316 has been selected. In some embodiments, method 700 includes highlighting the selected CGR object, making the selected CGR object brighter, and / or placing a polygon around the selected CGR object.

[0146] As shown in block 720f, in some embodiments, method 700 includes modifying visual attributes of the remainder of the plurality of CGR objects to indicate that the remainder of the plurality of CGR objects has not been selected. For example, in some embodiments, method 700 includes graying out the remainder of the plurality of CGR objects, and / or making the remainder of the plurality of CGR objects less bright.

[0147] Refer to Figure 7C, as shown in block 730a, in some specific implementations, the first user selection queue includes a first virtual basket. As shown in block 740a, in some specific implementations, the second user selection queue includes a second virtual basket. As shown in block 730b, in some specific implementations, the first user selection queue includes a first list of favorite CGR objects. As shown in block 740b, in some specific implementations, the second user selection queue includes a second list of favorite CGR objects.

[0148] As shown in block 750, in some specific implementations, method 700 includes replacing one CGR object among a plurality of CGR objects with a replacement CGR object in response to detecting a replacement input, while maintaining the display of the remaining portions of the plurality of CGR objects. For example, replacing CGR sofa 116 with another CGR sofa representing a physical sofa different from the physical sofa represented by CGR sofa 116, while maintaining the display of CGR painting 318.

[0149] Figure 8A is a flowchart representation of method 800 for synthesizing a masking element. In various specific implementations, method 800 is performed by a device (e.g., Figures 4A to 4O the electronic device 400 shown) having a display, an environmental sensor, a non-transitory memory, and one or more processors coupled to the display, the environmental sensor, and the non-transitory memory. In some specific implementations, method 800 is performed by processing logic (including hardware, firmware, software, or a combination thereof). In some specific implementations, method 800 is performed by a processor executing code stored in a non-transitory computer-readable medium (e.g., a memory).

[0150] As shown in block 810, in various specific implementations, method 800 includes detecting a physical surface in the physical environment around the device. For example, as with respect to Figure 4A and Figure 4B described, the electronic device 400 detects the physical surface 42 in the physical environment 40. In some specific implementations, method 800 includes obtaining a grid map of the physical environment and determining based on the grid map that the physical environment includes a physical surface. In some specific implementations, method 800 includes capturing environmental data corresponding to the physical environment (e.g., an image of the physical environment and / or depth data corresponding to the physical environment), and determining based on the environmental data that the physical environment includes a physical surface.

[0151] As shown in block 820, in various specific implementations, method 800 includes detecting one or more physical items that occlude corresponding portions of the physical surface. For example, as with respect to Figure 4A and Figure 4BAs described above, the electronic device 400 detects that the physical items 44, 46, and 48 occlude corresponding portions of the physical surface 42. In some specific embodiments, the method 800 includes detecting physical items based on environmental data. In some specific embodiments, the method 800 includes detecting physical items based on a grid map of the physical environment.

[0152] As shown in block 830, in various specific embodiments, the method 800 includes synthesizing a masking element to mask one or more physical items located on the physical surface. For example, as Figure 4C shown, the electronic device 400 synthesizes the masking element 422 onto the CGR surface 412 to mask the CGR objects 414, 416, and 418.

[0153] Referring Figure 8B , as shown in block 830a, in some specific embodiments, the method 800 includes determining one or more visual attributes of the physical surface and configuring a masking element with the one or more visual attributes of the physical surface. For example, in some specific embodiments, the electronic device 400 determines the visual attribute 43 of the physical surface 42 and configures the visual attribute 423 of the masking element 422 to be within a certain similarity of the visual attribute 43 of the physical surface 42.

[0154] As shown in block 830b, in some specific embodiments, the method 800 includes setting the color value of the masking element based on the color of the physical surface such that the color of the masking element is within a certain similarity of the color of the physical surface. For example, in some specific embodiments, the electronic device 400 sets the color value of the masking element 422 such that the color of the masking element 422 matches the color of the physical surface 42.

[0155] As shown in block 830c, in some specific embodiments, the method 800 includes determining the texture of the physical surface and setting the texture value of the masking element based on the texture of the physical surface such that the texture of the masking element is within a certain similarity of the texture of the physical surface. For example, in some specific embodiments, the electronic device 400 sets the texture value of the masking element 422 such that the texture of the masking element 422 matches the texture of the physical surface 42.

[0156] As shown in block 830d, in some specific embodiments, one or more visual attributes of the masking element are different from one or more visual attributes of the physical surface. For example, as Figure 4E shown, the visual attribute 423a of the masking element 422a is different from the visual attribute 413 of the CGR surface 412.

[0157] As shown in block 830e, in some specific embodiments, the method 800 includes blurring the entire physical surface. For example, in some specific embodiments, the masking element 422 masks the entire CGR surface 412.

[0158] As shown in block 830f, in some embodiments, method 800 includes determining the shape of one or more physical items and synthesizing a masking element based on the shape of the one or more physical items. In some embodiments, the shape of the masking element corresponds to the shape of the one or more physical items. For example, in some embodiments, masking element 422 conforms to the shape of CGR objects 414, 416, and / or 418.

[0159] As shown in block 830g, in some embodiments, the masking element includes a single masking element that masks multiple physical items among the one or more physical items. For example, as Figure 4C shown, masking element 422 masks CGR objects 414, 416, and 418 that respectively represent physical items 44, 46, and 48.

[0160] As shown in block 830h, in some embodiments, the masking element includes multiple masking elements, each of which masks a corresponding one of the one or more physical items. For example, as Figure 4F shown, electronic device 400 generates masking elements 424, 426, and 428 to respectively mask CGR objects 414, 416, and 418 that respectively represent physical items 44, 46, and 48.

[0161] Referring Figure 8C , as shown in block 840, in some embodiments, method 800 includes obtaining an input that identifies one or more physical elements to be masked. For example, as Figure 4N shown, electronic device 400 detects user input 462 corresponding to a request to mask CGR objects 414, 416, and 418 that respectively represent physical items 44, 46, and 48.

[0162] As shown in block 850, in some embodiments, method 800 includes obtaining an indication to place a computer-generated reality (CGR) object at a location corresponding to one or more physical items, synthesizing a masking element in response to obtaining the indication, and covering the CGR object with the masking element. For example, as Figure 4H shown, electronic device 400 obtains a request to place a fourth CGR object 440 onto CGR surface 412. As Figure 4I shown, electronic device 400 synthesizes masking element 422 in response to obtaining the request to place the fourth CGR object 440 on CGR surface 412, and covers the fourth CGR object 440 with masking element 422.

[0163] As shown in block 860, in some embodiments, method 800 includes detecting an input for moving a CGR object and simultaneously moving the masking element and the CGR object in the direction indicated by the input. For example, referring Figure 4IWhen the electronic device 400 detects an input to move the fourth CGR object 440 to an area of the CGR environment 410 that has other CGR objects that are currently unmasked, the electronic device 400 moves the masking element 422 to that area to mask the CGR objects in that area.

[0164] As shown in block 870, in some embodiments, method 800 includes displaying a CGR environment corresponding to a physical environment. In some embodiments, the CGR environment includes CGR representations of one or more physical items. In some embodiments, method 800 includes compositing a masking element onto the CGR representations of one or more physical items. For example, as Figure 4B and Figure 4C shown, the electronic device 400 displays a CGR environment 410 corresponding to the physical environment 40. As Figure 4B shown, the CGR environment 410 includes CGR objects 414, 416, and 418 that respectively represent physical items 44, 46, and 48. As Figure 4C shown, the electronic device 400 composites the masking element 422 to mask the CGR objects 414, 416, and 418.

[0165] As shown in block 880, in some embodiments, method 800 includes displaying a perspective of the physical environment. For example, as discussed with respect to Figure 4B in some embodiments, the CGR environment 410 includes a perspective of the physical environment 40. In some embodiments, method 800 includes displaying a video perspective of the physical environment. In some embodiments, method 800 includes presenting an optical perspective of the physical environment.

[0166] Figure 9 is a block diagram of a device 900 for presenting / masking communication data according to some embodiments. Although some specific features are shown, those of ordinary skill in the art will recognize from this disclosure that various other features are not shown for the sake of brevity and to not obscure more relevant aspects of the embodiments disclosed herein. To that end, as a non-limiting example, in some embodiments, device 900 includes one or more processing units (CPUs) 901, a network interface 902, a programming interface 903, a memory 904, an environmental sensor 907, one or more input / output (I / O) devices 908, and one or more communication buses 905 for interconnecting these and various other components.

[0167] In some specific implementations, a network interface 902 is provided to establish and maintain a metadata tunnel between, among other things, a cloud-hosted network management system and at least one private network including one or more compatible devices. In some specific implementations, one or more communication buses 905 include circuitry that interconnects and controls communication between system components. Memory 904 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices, and may include non-volatile memory, such as one or more disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 904 optionally includes one or more storage devices that are remotely located from one or more CPUs 901. Memory 904 includes a non-transitory computer-readable storage medium.

[0168] In various specific implementations, the environmental sensor 907 includes an image sensor. For example, in some specific implementations, the environmental sensor 907 includes a camera (e.g., a scene-facing camera, an outward-facing camera, or a rear-facing camera). In some specific implementations, the environmental sensor 907 includes a depth sensor. For example, in some specific implementations, the environmental sensor 907 includes a depth camera.

[0169] In some specific implementations, one or more I / O devices 908 include a display for displaying a CGR environment (e.g., Figures 1C to 1F and Figures 1H to 1L the CGR environment 110 shown, Figures 2A to 2D and Figures 2F to 2G the CGR environment 210 shown, Figures 3A to 3H the CGR environment 310 shown, or Figures 4B to 4O the CGR environment 410 shown). In some specific implementations, the display includes a video see-through display that displays at least a portion of the physical environment around the device 900 as an image captured by a scene camera. In various specific implementations, the display includes an optical see-through display that is at least partially transparent and allows light emitted or reflected by the physical environment to pass through.

[0170] In some specific implementations, memory 904 or the non-transitory computer-readable storage medium of memory 904 stores the following programs, modules, and data structures or subsets thereof, including an optional operating system 906, a data acquirer 910, and a CGR experience generator 920. In various specific implementations, the device 900 executes methods 500, 600, 700, and / or 800. In various specific implementations, the device 900 implements electronic devices 100, 200, 300, and / or 400.

[0171] In some specific implementations, the data acquirer 910 acquires data. In some specific implementations, the data acquirer 910 acquires an input (e.g., detects a user input). In some specific implementations, the data acquirer 910 acquires environmental data from the environmental sensor 907. To this end, the data acquirer 910 includes instructions 910a as well as heuristics and metadata 910b. In some specific implementations, the CGR experience generator 920 generates and presents the CGR environments 110, 210, 310, and / or 410. To this end, the CGR experience generator 920 includes instructions 920a as well as heuristics and metadata 920b.

[0172] While aspects of specific implementations within the scope of the appended claims have been described above, it should be apparent that the various features of the above-described specific implementations may be embodied in a variety of forms, and any specific structure and / or function described above is merely illustrative. Based on this disclosure, those skilled in the art should understand that the aspects described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of the aspects described herein may be used to implement a device and / or a method may be practiced. Additionally, other structures and / or functions may be used to implement such a device and / or a method may be practiced other than or different from one or more of the aspects described herein.

[0173] It will also be understood that although terms such as "first," "second," etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first node may be referred to as a second node, and similarly, a second node may be referred to as a first node, which changes the meaning of the description, provided that all occurrences of "first node" are consistently renamed and all occurrences of "second node" are consistently renamed. The first node and the second node are both nodes, but they are not the same node.

[0174] The terms used herein are merely for the purpose of describing specific implementations and are not intended to limit the claims. As used in the description of this specific implementation and the appended claims, the singular forms "a," "an," and "the" are intended to also cover the plural forms unless the context clearly indicates otherwise. It will also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will also be understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or their groups.

[0175] As used herein, the term "if" can be interpreted to mean "when the precondition is true" or "while the precondition is true" or "in response to determining" or "in accordance with determining" or "in response to detecting" that the precondition is true, depending on the context. Similarly, the phrase "if it is determined that [the precondition is true]" or "if [the precondition is true]" or "when [the precondition is true]" is interpreted to mean "when it is determined that the precondition is true" or "in response to determining" or "in accordance with determining" the precondition is true or "when it is detected that the precondition is true" or "in response to detecting" that the precondition is true, depending on the context.

Claims

1. A method, comprising: At a device including a display, a non-transitory memory, and one or more processors coupled to the display and the non-transitory memory: Display a plurality of computer-generated reality (CGR) objects, the plurality of CGR objects representing respective physical items from a plurality of sources, the plurality of sources including a first source and a second source; Detect an input that selects a first CGR object among the plurality of CGR objects and a second CGR object among the plurality of CGR objects, wherein the first CGR object represents a first physical item from the first source and the second CGR object represents a second physical item from the second source; Add an identifier of the first physical item to a first user selection queue associated with the first source; and Add an identifier of the second physical item to a second user selection queue associated with the second source.

2. The method according to claim 1, wherein detecting the input comprises: Detect respective contacts with the first CGR object and the second CGR object.

3. The method according to claim 1, wherein detecting the input comprises: Detect that a CGR representation of a person has touched the first CGR object and the second CGR object.

4. The method according to claim 1, wherein detecting the input comprises: Detect a gaze input involving the first CGR object and the second CGR object.

5. The method according to claim 1, wherein detecting the input comprises: Detect a speech input involving the first CGR object and the second CGR object.

6. The method according to claim 1, further comprising: Modify visual attributes of the first CGR object and the second CGR object to indicate the selection of the first CGR object and the second CGR object.

7. The method according to claim 1, further comprising: Modify visual attributes of the remainder of the plurality of CGR objects to indicate that the remainder of the plurality of CGR objects is not selected.

8. The method according to claim 1, further comprising: In response to an input corresponding to a request to display the plurality of CGR objects, switch from a web page including two-dimensional (2D) representations of physical items to the CGR environment.

9. The method according to claim 1, further comprising: Identify at least a portion of the displayed CGR objects by a recommendation engine.

10. The method according to claim 1, further comprising: In response to detecting a replacement input, replace a CGR object among the plurality of CGR objects with a replacement CGR object while maintaining the display of the remainder of the plurality of CGR objects.

11. The method according to claim 1, wherein the first user selection queue includes a first virtual basket, and the second user selection queue includes a second virtual basket.

12. The method according to claim 1, wherein the first user selection queue includes a first list of favorite CGR objects, and the second user selection queue includes a second list of favorite CGR objects.

13. An apparatus, comprising: One or more processors; A display; A non-transitory memory; And One or more programs stored in the non-transitory memory, the one or more programs, when executed by the one or more processors, cause the device to: Display a plurality of computer-generated reality (CGR) objects, the plurality of CGR objects representing respective physical items from a plurality of sources, the plurality of sources including a first source and a second source; Detect an input that selects a first CGR object among the plurality of CGR objects and a second CGR object among the plurality of CGR objects, wherein the first CGR object represents a first physical item from the first source and the second CGR object represents a second physical item from the second source; Add an identifier of the first physical item to a first user selection queue associated with the first source; and Add an identifier of the second physical item to a second user selection queue associated with the second source.

14. The apparatus according to claim 13, wherein detecting the input comprises: Detect respective contacts with the first CGR object and the second CGR object.

15. The apparatus according to claim 13, wherein detecting the input comprises: Detect that a CGR representation of a person has touched the first CGR object and the second CGR object.

16. The apparatus according to claim 13, wherein detecting the input comprises: Detect a gaze input related to the first CGR object and the second CGR object.

17. The apparatus according to claim 13, wherein the one or more programs further cause the apparatus to: Modify visual attributes of the first CGR object and the second CGR object to indicate the selection of the first CGR object and the second CGR object; and Modify visual attributes of the remainder of the plurality of CGR objects to indicate that the remainder of the plurality of CGR objects is not selected.

18. The apparatus according to claim 13, wherein the one or more programs further cause the apparatus to: In response to an input corresponding to a request to display the plurality of CGR objects, switch from a web page including a two-dimensional 2D representation of physical items to the CGR environment.

19. The apparatus according to claim 13, wherein the first user selection queue includes a first list of favorite CGR objects, and the second user selection queue includes a second list of favorite CGR objects.

20. A non-transitory memory storing one or more programs, the one or more programs when executed by one or more processors of a device having a display cause the device to: Display a plurality of computer-generated reality CGR objects, the plurality of CGR objects representing corresponding physical items from a plurality of sources, the plurality of sources including a first source and a second source; Detect an input selecting a first CGR object among the plurality of CGR objects and a second CGR object among the plurality of CGR objects, wherein the first CGR object represents a first physical item from the first source, and the second CGR object represents a second physical item from the second source; Add an identifier of the first physical item to a first user selection queue associated with the first source; and Add the identifier of the second physical item to a second user selection queue associated with the second source.

Images