Controlling user selection queues

Abstract

Various embodiments disclosed herein include devices, systems, and methods for synthesizing affordances associated with computer-generated reality (CGR) objects representing physical items. In various embodiments, a device includes a display, a non-volatile memory, and one or more processors coupled to the display and the non-volatile memory. In some embodiments, the method includes displaying a CGR object in a CGR environment. In some embodiments, the CGR object represents a physical item. In some embodiments, the method includes synthesizing affordances associated with the CGR object. In some embodiments, the method includes detecting an input involving the affordance. In some embodiments, the method includes adding an identifier identifying the physical item to a user selection queue in response to detecting the input.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Patent Application No. 62 / 855,920, filed on June 1, 2019, which is hereby incorporated by reference in its entirety. Technical Field

[0003] The present disclosure generally relates to controlling a user selection queue. Background Art

[0004] Some devices are capable of generating and presenting computer generated reality (CGR) environments. Some CGR environments include virtual environments, which are simulated replacements for physical environments. Some CGR environments include augmented environments, which are modified versions of physical environments. Some devices that present CGR environments include mobile communication devices, such as smart phones, head mounted displays (HMDs), glasses, head up displays (HUDs), and optical projection systems. However, most previously available devices that present CGR environments are ineffective in controlling user selection queues. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] So that the present disclosure may be understood by those of ordinary skill in the art, a more detailed description may be obtained with reference to aspects of some exemplary implementations, some of which are illustrated in the accompanying drawings.

[0006] Figures 1A to 1L is an illustration of an example operating environment for controlling a user selection queue according to some implementations.

[0007] Figures 2A to 2G is an illustration of an example operating environment for displaying CGR objects representing wearable physical items according to some implementations.

[0008] FIG. 3A to FIG. 3I is an illustration of an example operating environment for simultaneously controlling multiple user selection queues according to some implementations.

[0009] FIG. 4A to FIG. 4O is an illustration of an example operating environment for a synthetic masking element according to some implementations.

[0010] FIG. 5A to FIG. 5C is a flowchart representation of a method of controlling a user selection queue according to some specific implementations.

[0011] FIG. 6A to FIG. 6C is a flowchart representation of a method of displaying a CGR object representing a wearable physical item according to some specific implementations.

[0012] 7A to 7Cis a flowchart representation of a method of simultaneously controlling multiple user selection queues according to some specific implementations.

[0013] Figures 8A to 8C is a flowchart representation of a method of synthesizing a masking element according to some specific implementations.

[0014] Fig. 9 is a block diagram of a device for presenting a CGR environment according to some specific implementations.

[0015] As is common practice, the various features shown in the drawings may not be drawn to scale. Therefore, the sizes of the various features may be arbitrarily expanded or reduced for clarity. In addition, some drawings may not depict all components of a given system, method, or device. Finally, throughout the specification and drawings, similar reference numerals may be used to represent similar features. Summary of the invention

[0016] Various embodiments disclosed herein include devices, systems, and methods for synthesizing affordances associated with computer-generated reality (CGR) objects representing physical items. In various embodiments, the device includes a display, a non-volatile memory, and one or more processors coupled to the display and the non-volatile memory. In some embodiments, the method includes displaying a CGR object in a CGR environment. In some embodiments, the CGR object represents a physical item. In some embodiments, the method includes synthesizing affordances associated with the CGR object. In some embodiments, the method includes detecting input involving affordances. In some embodiments, the method includes adding an identifier identifying a physical item to a user selection queue in response to detecting the input.

[0017] Various embodiments disclosed herein include devices, systems, and methods for displaying a CGR object representing a wearable physical item based on a deformation model of the wearable physical item. In various embodiments, the device includes a display, a non-volatile memory, and one or more processors coupled to the display and the non-volatile memory. In some embodiments, the method includes obtaining a computer-generated reality (CGR) representation of a person. In some embodiments, at least a portion of the CGR representation is proportional to a corresponding portion of the person. In some embodiments, the method includes obtaining a CGR object representing a wearable physical item. In some embodiments, the CGR object is associated with a deformation model that characterizes one or more material properties of the wearable physical item. In some embodiments, the method includes displaying a CGR object associated with the CGR representation of the person. In some embodiments, the CGR object interacts with the CGR representation of the person based on the deformation model.

[0018] Various embodiments disclosed herein include devices, systems, and methods for adding identifiers of physical items to source-specific user selection queues. In various embodiments, the device includes a display, a non-transitory memory, and one or more processors coupled to the display and the non-transitory memory. In some embodiments, the method includes displaying multiple computer-generated reality (CGR) objects representing corresponding physical items from multiple sources including a first source and a second source. In some embodiments, the method includes detecting an input of selecting a first CGR object in a plurality of CGR objects and a second CGR object in a plurality of CGR objects. In some embodiments, the first CGR object represents a first physical item from a first source, and the second CGR object represents a second physical item from a second source. In some embodiments, the method includes adding an identifier of the first physical item to a first user selection queue associated with the first source. In some embodiments, the method includes adding an identifier of the second physical item to a second user selection queue associated with the second source.

[0019] Various embodiments disclosed herein include devices, systems, and methods for masking physical items that obstruct a physical surface. In various embodiments, the device includes a display, an environmental sensor, a non-transitory memory, and one or more processors coupled to the display, the environmental sensor, and the non-transitory memory. In some embodiments, the method includes detecting a physical surface in a physical environment surrounding the device. In some embodiments, the method includes detecting one or more physical items that obstruct a corresponding portion of the physical surface. In some embodiments, the method includes synthesizing a masking element to mask one or more physical items located on the physical surface.

[0020] According to some specific implementations, the device includes one or more processors, non-transitory memory, and one or more programs. In some specific implementations, one or more programs are stored in non-transitory memory and executed by one or more processors. In some specific implementations, one or more programs include instructions for executing or causing the execution of any method described herein. According to some specific implementations, a non-transitory computer-readable storage medium stores instructions that, when executed by one or more processors of a device, cause the device to execute or cause the execution of any of the methods described herein. According to some specific implementations, a device includes one or more processors, non-transitory memory, and a device for executing or causing the execution of any of the methods described herein. DETAILED DESCRIPTION

[0021] Many details are described in order to provide a thorough understanding of the example implementations shown in the accompanying drawings. However, the accompanying drawings only illustrate some example aspects of the present disclosure and should not be considered limiting. One of ordinary skill in the art will appreciate that other effective aspects and / or variations do not include all of the specific details described herein. In addition, well-known systems, methods, components, devices, and circuits are not described in detail in order to avoid obscuring more relevant aspects of the exemplary implementations described herein.

[0022] The physical environment refers to the physical world that people can sense and / or interact with without the help of electronic systems. A physical environment, such as a physical park, includes physical objects, such as physical trees, physical buildings, and physical people. People can directly sense and / or interact with the physical environment, such as through vision, touch, hearing, taste, and smell.

[0023] In contrast, a computer generated reality (CGR) environment refers to a fully or partially simulated environment that a person perceives and / or interacts with via an electronic system. In CGR, a subset of a person's physical movements, or representations thereof, are tracked, and in response, one or more features of one or more virtual objects simulated in the CGR environment are adjusted in a manner that complies with at least one law of physics. For example, a CGR system may detect a person's head turning, and in response, adjust the graphical content and sound field presented to the person in a manner similar to the way such views and sounds change in a physical environment. In some cases (e.g., for accessibility reasons), adjustments to features of virtual objects in a CGR environment may be made in response to representations of physical movement (e.g., voice commands).

[0024] A person may sense and / or interact with a CGR object using any of their senses, including vision, hearing, touch, taste, and smell. For example, a person may sense and / or interact with an audio object that creates a 3D or spatial audio environment that provides the perception of a point audio source in 3D space. As another example, an audio object may enable audio transparency that selectively introduces ambient sounds from the physical environment with or without computer-generated audio. In some CGR environments, a person may sense and / or interact only with an audio object.

[0025] Examples of CGR include virtual reality and mixed reality.

[0026] A virtual reality (VR) environment refers to a simulated environment designed to be based entirely on computer-generated sensory input to one or more senses. A VR environment includes a plurality of virtual objects that a person can sense and / or interact with. For example, trees, buildings, and computer-generated images representing avatars of people are examples of virtual objects. A person can sense and / or interact with virtual objects in a VR environment through a simulation of the person's presence within the computer-generated environment, and / or through a simulation of a subset of the person's physical movements within the computer-generated environment.

[0027] In contrast to VR environments, which are designed to be based entirely on computer-generated sensory input, a mixed reality (MR) environment refers to a simulated environment designed to include sensory input from the physical environment, or representations thereof, in addition to computer-generated sensory input (e.g., virtual objects). On the virtuality continuum, a mixed reality environment is anything between a fully physical environment at one end and a virtual reality environment at the other end, but not including both ends.

[0028] In some MR environments, computer-generated sensory input can respond to changes in sensory input from the physical environment. In addition, some electronic systems used to render the MR environment can track position and / or orientation relative to the physical environment to enable virtual objects to interact with real objects (i.e., physical items from the physical environment or representations thereof). For example, the system can cause motion so that virtual trees appear stationary relative to the physical ground.

[0029] Examples of mixed reality include augmented reality and augmented virtuality.

[0030] An augmented reality (AR) environment refers to a simulated environment in which one or more virtual objects are superimposed on a physical environment or a representation thereof. For example, an electronic system for presenting an AR environment may have a transparent or translucent display through which a person can directly view the physical environment. The system may be configured to present virtual objects on a transparent or translucent display so that a person uses the system to perceive virtual objects superimposed on a physical environment. Alternatively, the system may have an opaque display and one or more imaging sensors that capture images or videos of a physical environment that are representations of the physical environment. The system combines the image or video with the virtual object and presents the composition on an opaque display. People use the system to indirectly view the physical environment via an image or video of the physical environment and perceive virtual objects superimposed on the physical environment. As used herein, a video of a physical environment displayed on an opaque display is referred to as a "transparent video," meaning that the system uses one or more image sensors to capture images of the physical environment and uses those images when presenting an AR environment on an opaque display. Further alternatively, the system may have a projection system that projects virtual objects into a physical environment, for example as a hologram or on a physical surface, so that a person using the system perceives the virtual objects superimposed on the physical environment.

[0031] An augmented reality environment also refers to a simulated environment in which a representation of a physical environment is transformed by computer-generated sensory information. For example, in providing a pass-through video, the system may transform one or more sensor images to impose a selected perspective (e.g., a viewpoint) that is different from the perspective captured by the imaging sensor. For another example, a representation of a physical environment may be transformed by graphically modifying (e.g., enlarging) a portion thereof so that the modified portion may be a representative but not true version of the original captured image. For another example, a representation of a physical environment may be transformed by graphically eliminating or blurring a portion thereof.

[0032] An augmented virtual (AV) environment refers to a simulated environment in which a virtual or computer-generated environment is combined with one or more sensory inputs from a physical environment. The sensory input may be a representation of one or more features of the physical environment. For example, an AV park may have virtual trees and virtual buildings, but the faces of people are realistically reproduced from images taken of physical people. For another example, a virtual object may take the shape or color of a physical object imaged by one or more imaging sensors. For another example, a virtual object may take a shadow that conforms to the position of the sun in the physical environment.

[0033] There are many different types of electronic systems that enable people to sense and / or interact with various CGR environments. Examples include head-mounted systems, projection-based systems, head-up displays (HUDs), vehicle windshields with integrated display capabilities, windows with integrated display capabilities, displays formed as lenses designed to be placed on people's eyes (e.g., similar to contact lenses), headphones / earpieces, speaker arrays, input systems (e.g., wearable or handheld controllers with or without tactile feedback), smart phones, tablets, and desktop / laptop computers. A head-mounted system can have one or more speakers and an integrated opaque display. Alternatively, a head-mounted system can be configured to accept an external opaque display (e.g., a smart phone). A head-mounted system can incorporate one or more imaging sensors for capturing images or videos of a physical environment, and / or one or more microphones for capturing audio of a physical environment. Instead of an opaque display, a head-mounted system can have a transparent or translucent display. A transparent or translucent display can have a medium through which light representing an image is directed to a person's eyes. The display can utilize digital light projection, OLED, LED, uLED, liquid crystal on silicon, laser scanning light source, or any combination of these technologies. The medium can be an optical waveguide, a hologram medium, an optical combiner, an optical reflector, or any combination thereof. In a specific implementation, a transparent or translucent display can be configured to selectively become opaque. A projection-based system can employ retinal projection technology that projects a graphic image onto a person's retina. The projection system can also be configured to project a virtual object into a physical environment, such as a hologram or onto a physical surface.

[0034] Some CGR devices allow users to view CGR representations of physical items. However, if the user wants to add the physical item to a user selection queue, the user has to navigate back to a web page corresponding to the physical item in order to populate the user selection queue with an identifier of the physical item. This impairs the user experience and requires excessive user input, which can unnecessarily drain the device's battery. The present disclosure provides methods, systems, and / or devices for displaying a CGR object representing a physical item and synthesizing an affordance with the CGR object. When the affordance is activated, the physical item is added to the user selection queue. The affordance reduces the need for unnecessary user input corresponding to the user navigating back to the web page for the physical item in order to add the physical item to the user selection queue.

[0035] Figure 1A1 is a block diagram of an example physical environment 10 according to some implementations. Although relevant features are shown, one of ordinary skill in the art will recognize from this disclosure that various other features are not shown for the sake of brevity and so as not to obscure more relevant aspects of the exemplary implementations disclosed herein. To this end, as a non-limiting example, the physical environment 10 includes a first floor lamp 12, a second floor lamp 14, and an electronic device 100. In some implementations, the electronic device 100 is held by a person (not shown). In some implementations, the electronic device 100 includes a smart phone, a tablet computer, a laptop computer, etc.

[0036] exist Figure 1A In the example of FIG. 1 , electronic device 100 displays web page 102 corresponding to a sofa. Web page 102 includes a two-dimensional (2D) representation 104 of the sofa (eg, an image of the sofa). Web page 102 also includes an affordance 106 for viewing a CGR representation of the sofa. Figure 1B , electronic device 100 detects user input 108 (e.g., contact at a location corresponding to affordance 106, such as a tap or press) that activates affordance 106. Figure 1C As shown, in response to detecting user input 108 , electronic device 100 presents CGR environment 110 .

[0037] exist Figure 1C In the example of FIG. 1 , the CGR environment 110 includes a first CGR floor lamp 112, a second CGR floor lamp 114, and a CGR sofa 116. The first CGR floor lamp 112 is a CGR representation of the first floor lamp 12 in the physical environment 10. The second CGR floor lamp 114 is a CGR representation of the second floor lamp 14 in the physical environment 10. The CGR sofa 116 is a CGR representation of the first floor lamp 12 in the physical environment 10. Figure 1A to Figure 1B 104 on the web page 102 shown. In some implementations, the sofa is associated with an identifier (ID) 118 that identifies the sofa. In some implementations, the ID 118 includes one or more alphanumeric character sets (e.g., a serial number, an item number, a barcode number, a name, a title, a description, etc.). In some implementations, the ID 118 includes a machine-readable representation of data (e.g., an optical machine-readable representation of data, such as a barcode or a QR code). In some implementations, the electronic device 100 does not display the ID 118 in the CGR environment 110. In some implementations, the CGR environment 110 is a perspective (e.g., a video perspective or an optical perspective) of the physical environment 10.

[0038] like Figure 1CAs shown, in some implementations, electronic device 100 synthesizes affordance 120 associated with CGR sofa 116. In response to activation of affordance 120, electronic device 100 adds (e.g., writes) sofa ID 118 to user selection queue 150. In some implementations, electronic device 100 displays affordance 120 adjacent to CGR sofa 116. For example, Figure 1C As shown, the electronic device 100 displays the enable indication 120 below the CGR sofa 116. However, in some implementations, the electronic device 100 displays the enable indication 120 at another location. For example, in some implementations, the electronic device 100 overlays the enable indication 120 onto the CGR sofa 116. In the example of FIG. 1C , the enable indication 120 is visible. However, in some implementations, the enable indication 120 is not visible. For example, in some implementations, the CGR sofa 116 is used as the enable indication 120. In such implementations, in response to detecting a selection of the CGR sofa 116, the electronic device 100 adds the ID of the sofa to the user selection queue.

[0039] refer to Figure 1D , electronic device 100 detects user input 130 involving affordance 120. Figure 1D In the example of , user input 130 includes contact (e.g., a tap or press) at a location corresponding to enable representation 120. In some implementations, electronic device 100 detects gaze input involving enable representation 120. In some implementations, electronic device 100 detects speech input involving enable representation 120. In some implementations, electronic device 100 detects a three-dimensional (3D) gesture corresponding to a selection of enable representation 120. For example, electronic device 100 utilizes hand tracking to detect that a person controlling electronic device 100 has performed a 3D gesture. In some implementations, electronic device 100 obtains input corresponding to a selection of enable representation 120 from a controller (not shown). For example, electronic device 100 detects activation of a controller button corresponding to enable representation 120. Figure 1E As shown, in response to detecting the user input 130 , the electronic device 100 adds the ID 118 of the sofa to the user selection queue 150 .

[0040] refer to Figure 1E In some embodiments, the electronic device 100 responds to the detection of Figure 1D The user input 130 shown is displayed and notification 140 is displayed. In some specific implementations, notification 140 indicates that the electronic device 100 has added the ID 118 of the sofa to the user selection queue 150. For example, Figure 1EAs shown, in some implementations, notification 140 includes text 142 (e.g., "Sofa added to queue"). In some implementations, notification 140 includes an affordance 144 for displaying a visual representation of user selection queue 150, and an affordance 146 for continuing browsing.

[0041] refer to Figure 1F , electronic device 100 detects user input 130 involving affordance 144 for displaying a visual representation of user selection queue 150. Figure 1G As shown, in response to detecting the user input 130, the electronic device 100 displays a visual representation 151 of the user selection queue 150. Figure 1G In the example of , the visual representation 151 of the user selection queue 150 includes a 2D representation 104 of a sofa, a description 152 of the sofa, a delete enable indication 154 for removing ID 118 from the user selection queue 150, a modify enable indication 156, and a confirm enable indication 158. In some implementations, the modify enable indication 156 allows the user of the electronic device 100 to modify the quantity associated with the sofa (e.g., increase the quantity from a default quantity of 1' to a higher quantity). In some implementations, the confirm enable indication 158 allows the user of the electronic device 100 to confirm the user selection queue 150. In some implementations, detecting activation of the confirm enable indication 158 triggers the electronic device 100 to perform an operation (e.g., the electronic device 100 triggers the transfer of points to another device, e.g., to a device associated with the seller of the sofa).

[0042] refer to Figure 1H In some implementations, the electronic device 100 displays the affordance 120 in a designated portion 160 of the CGR environment 110. Figure 1H In the example of FIG. 1 , the designated portion 160 is toward the lower right corner of the CGR environment 110. However, in some implementations, the designated portion 160 is toward the left side of the CGR environment 110, the right side of the CGR environment 110, the top of the CGR environment 110, the bottom of the CGR environment 110, or the center of the CGR environment 110.

[0043] refer to Fig. 1I In some implementations, the electronic device 100 displays CGR representations of multiple items that are not in the physical environment 10. Fig. 1I In the example of , the electronic device 100 displays a CGR drawing 170 representing a physical drawing. In some implementations, the electronic device 100 displays an affordance 172 to add an ID (not shown) identifying the physical drawing to the user selection queue 150. In some implementations, the electronic device 100 displays an affordance 180 to add the IDs of multiple physical items to the user selection queue 150 at the same time. Fig. 1I In the example of , in response to detecting activation of affordance 180, electronic device 100 adds ID 118 of the sofa to user selection queue 150, and adds the ID of the physical painting to user selection queue 150. Therefore, the user of electronic device 100 does not need to activate affordances 120 and 172 separately, thereby reducing the amount of user input and enhancing the user experience of electronic device 100.

[0044] In some implementations, a head-mountable device (HMD) (not shown) worn by a person presents (e.g., displays) a CGR environment 110 according to various implementations. In some implementations, the HMD includes an integrated display (e.g., a built-in display) that displays the CGR environment 110. In some implementations, the HMD includes a head-mountable housing. In various implementations, the head-mountable housing includes an attachment area to which another device having a display can be attached. For example, in some implementations, the electronic device 100 can be attached to the head-mountable housing. In various implementations, the head-mountable housing is shaped to form a receiver for receiving another device (e.g., the electronic device 100) including a display. For example, in some implementations, the electronic device 100 slides / snaps onto the head-mountable housing or is otherwise attached to the head-mountable housing. In some implementations, the display of the device attached to the head-mountable housing presents (e.g., displays) the CGR environment 110. In various implementations, examples of the electronic device 100 include smart phones, tablet computers, media players, laptop computers, etc.

[0045] refer to Figure 1J In some implementations, the electronic device 100 displays a replacement affordance to replace the CGR object with a replacement CGR object. Figure 1J In the example, the electronic device 100 displays a first replacement enable indication 194a (e.g., a left arrow) and a second replacement enable indication 194b (e.g., a right arrow) that allow a user of the electronic device 100 to replace the CGR sofa 116 with a replacement CGR object (e.g., another CGR sofa, e.g., a different CGR sofa representing a different physical sofa).

[0046] refer to Figure 1K , electronic device 100 detects user input 196 involving second replacement affordance 194b. Figure 1L As shown, in response to detecting user input 196, electronic device 100 replaces CGR sofa 116 with a second CGR sofa 198 representing a second physical sofa, which is different from the second physical sofa represented by Figure 1K The CGR sofa 116 shown represents a physical sofa. Figure 1LIn the example of , the affordance 120 is configured to add the ID of the second physical sofa to the user selection queue 150 .

[0047] It is sometimes difficult to select a wearable physical item via a device because size charts are often inaccurate and the wearable physical item often does not fit the user well. As a result, the user often has to return the wearable physical item, which results in unnecessary user input that causes wear and / or excessive power usage on the device. The present disclosure provides methods, systems, and / or devices for generating CGR objects representing wearable physical items and displaying the CGR objects as worn by the CGR representation of the user. The CGR object is associated with a deformation model that defines how the CGR object deforms when the CGR object is worn by the CGR representation of the user.

[0048] The deformation model of the CGR object is a function of one or more material properties of the wearable physical item. For example, the deformation model is a function of the material type, texture, stiffness, elasticity, color and / or size of the wearable physical item. The CGR representation of the user can be generated by scanning the user. For example, by taking a photo of the user and determining the dimensions of the user based on photogrammetry. The dimensions of the user can also be determined based on depth data captured by a depth camera. The device can obtain the CGR object (for example, from the manufacturer of the wearable physical item), or generate the CGR object based on the material properties of the wearable physical item. For example, the device can generate the CGR object using dimensional values ​​and material composition information associated with the wearable physical item.

[0049] Figure 2A 2 is a block diagram of an example physical environment 20 according to some implementations. Although relevant features are shown, one of ordinary skill in the art will recognize from this disclosure that various other features are not shown for the sake of brevity and so as not to obscure more relevant aspects of the exemplary implementations disclosed herein. To this end, as a non-limiting example, the physical environment 20 includes a person 22 and an electronic device 200. Figure 2A As shown, in some implementations, the electronic device 200 is held by a person 22. In some implementations, the electronic device 200 includes a smart phone, a tablet computer, a laptop computer, etc.

[0050] exist Figure 2A, the electronic device 200 displays a CGR environment 210. The CGR environment 210 includes a CGR representation 212 of a person 22. In some implementations, the CGR representation 212 is proportional to the person 22. For example, the ratio between the head and the torso of the CGR representation 212 matches the ratio between the head and the torso of the person 22. In various implementations, the electronic device 200 generates the CGR representation 212 of the person 22 based on a body model of the person 22. In some implementations, the electronic device 200 captures images of the person 22, generates a body model of the person 22 based on these images, and generates the CGR representation 212 based on the body model. In some implementations, the electronic device 100 receives user input corresponding to body measurements of the person 22, and the electronic device 200 generates the CGR representation 212 based on these body measurements.

[0051] exist Figure 2A In the example of FIG. 1 , CGR environment 210 includes CGR object configuration panel 220 (hereinafter referred to as “panel 220” for brevity). Panel 220 includes a two-dimensional (2D) representation 222 of a T-shirt (e.g., an image of a T-shirt). In some implementations, panel 220 includes an affordance for selecting a particular configuration of the T-shirt represented by 2D representation 222. Figure 2A In the example of , the panel 220 includes a size indication 224 for selecting the size of a T-shirt, a style indication 226 for selecting the style of the T-shirt, and a material indication 228 for selecting the material type of the T-shirt. The size indication 224 includes a small size indication 224a for selecting a small size T-shirt, a medium size indication 224b for selecting a medium size T-shirt, and a large size indication 224c for selecting a large size T-shirt. The style indication 226 includes a slim style indication 226a for selecting a slim style of the T-shirt, and a classic style indication 226b for selecting a classic style of the T-shirt. The material indication 228 includes a cotton indication 228a for selecting a cotton type of a T-shirt, a polyester indication 228b for selecting a polyester type of a T-shirt, a blended indication 228c for selecting a blended material type of a T-shirt, and a merino wool indication 228d for selecting a merino wool type of a T-shirt.

[0052] refer to Figure 2B, the electronic device 200 detects that the small size affordance 224a, the slim fit affordance 226a, and the cotton affordance 228a have been selected. The electronic device 200 generates and displays a CGR T-shirt 230 associated with the CGR representation 212 of the person 22. For example, the electronic device 200 displays the CGR T-shirt 230 as being worn by the CGR representation 212 of the person 22. In some specific implementations, the electronic device 200 overlays the CGR T-shirt 230 on the CGR representation 212 of the person 22.

[0053] CGR T-shirt 230 is associated with deformation model 232. Deformation model 232 defines how CGR T-shirt 230 interacts with CGR representation 212 of person 22, which represents how a corresponding physical T-shirt (e.g., a small and slim-fitting cotton T-shirt) will fit person 22. In some implementations, deformation model 232 is a function of the selected affordances 224, 226, and 228. For example, deformation model 232 is a function of the small size, slim-fitting style, and cotton material of the T-shirt. More generally, in various implementations, deformation model 232 is a function of one or more material properties of the T-shirt (e.g., size, material type, style, stiffness, elasticity, etc.). Deformation model 232 defines the deformation of CGR T-shirt 230 on CGR representation 212 of person 22. Deformation model 232 defines how a physical T-shirt represented by CGR T-shirt 230 will fit person 22.

[0054] Because deformation model 232 characterizes one or more material properties of a t-shirt, CGR t-shirt 230 is within a certain similarity to a corresponding physical t-shirt. For example, CGR t-shirt 230 is within a certain similarity to a physical t-shirt that is small, slim-fitting, and made of cotton. Therefore, the deformation of CGR t-shirt 230 on CGR representation 212 of person 22 is within a certain similarity to the deformation of the corresponding physical t-shirt on person 22. For example, the fit of CGR t-shirt 230 on CGR representation 212 is within a certain similarity to the fit of the corresponding physical t-shirt on person 22. Figure 2B In the example of , placing CGR T-shirt 230 on CGR representation 212 produces CGR stretch line 234. Since the deformation of CGR T-shirt 230 on CGR representation 212 is within a certain similarity of the deformation of the corresponding physical T-shirt on person 22, person 22 wearing the corresponding physical T-shirt will likely result in a physical stretch line similar to CGR stretch line 234.

[0055] refer to Figure 2C, the electronic device 200 detects that the large size enable representation 224c, the classic style enable representation 226b, and the blended enable representation 228c have been selected. In response to detecting the selection of the large size enable representation 224c, the classic style enable representation 226b, and the blended enable representation 228c, the electronic device 200 generates and displays the CGR T-shirt 240 as being worn by the CGR representation 212 of the person 22. The CGR T-shirt 240 is associated with a deformation model 242. The deformation model 242 is a function of the large size, classic style, and blended material of the T-shirt. The deformation model 242 defines the deformation of the CGR T-shirt 240 on the CGR representation 212 of the person 22. Figure 2C In the example of , placing CGR T-shirt 240 on CGR representation 212 results in CGR droop 244 being located at the neck portion of CGR T-shirt 240, and CGR droop 246 being towards the bottom of CGR T-shirt 240. Because deformation model 242 models the deformation of the corresponding physical T-shirt on person 22, when person 22 wears the corresponding physical T-shirt, the corresponding physical T-shirt will likely droop at the neck of the physical T-shirt and droop towards the bottom of the physical T-shirt.

[0056] refer to Figure 2D , the electronic device 200 detects that the medium size enable representation 224b, the slim style enable representation 226a, and the merino wool enable representation 228d have been selected. In response to detecting the selection of the medium size enable representation 224b, the slim style enable representation 226a, and the merino wool enable representation 228d, the electronic device 200 generates and displays the CGR T-shirt 250 as being worn by the CGR representation 212 of the person 22. The CGR T-shirt 250 is associated with a deformation model 252. The deformation model 252 is a function of the medium size, slim style, and merino wool material of the T-shirt. The deformation model 252 defines the deformation of the CGR T-shirt 250 on the CGR representation 212 of the person 22. Figure 2D In the example of FIG. 2 , placing CGR T-shirt 250 on CGR representation 212 will not result in stretch lines or sagging. Since deformation model 252 models the deformation of the corresponding physical T-shirt on person 22, the corresponding physical T-shirt will likely not sag or result in stretch lines when person 22 wears it. CGR T-shirt 250 looks better than Figure 2B and Figure 2C CGR T-shirts 230 and 240, respectively, are shown to fit better on CGR representation 212. Therefore, a physical T-shirt corresponding to CGR T-shirt 250 will likely fit better on person 22 than the physical T-shirts corresponding to CGR T-shirts 230 and 240.

[0057] Figure 2EA graphical user interface (GUI) 260 is shown that allows person 22 to search for wearable physical items based on a body model 270 of person 22. In some implementations, GUI 260 allows person 22 to generate body model 270. Figure 2E In the example of , GUI 260 includes an enable representation 262, which, when activated, triggers capturing an image of person 22. In some implementations, GUI 260 includes an upload image enable representation 264 for uploading an image (e.g., an image of person 22). In some implementations, electronic device 220 generates body model 270 based on the captured image and / or the uploaded image. In some implementations, electronic device 200 utilizes methods, devices, and / or systems associated with photogrammetry to extract dimensions of person 22 from the captured image and / or the uploaded image, and electronic device 200 generates body model 270 based on the dimensions of person 22.

[0058] In some implementations, GUI 260 includes a measurement enable representation 266 that allows person 22 to input body measurements (e.g., body dimensions of person 22, such as waist size, arm size, thigh size, arm size, etc.). In such implementations, electronic device 200 generates body model 270 based on the body measurements obtained by electronic device 200. In some implementations, GUI 260 includes an upload model enable representation 268 that, when activated, allows person 22 to upload body model 270. Figure 2E In the example of , GUI 260 includes a search affordance 272 that, when activated, triggers a search for wearable physical items based on body model 270.

[0059] In some implementations, the CGR environment 210 simultaneously displays CGR objects representing multiple wearable physical items associated with the CGR representation 212 of the person 22. For example, the CGR environment 210 simultaneously displays the CGR representation 212 of the person 22 wearing multiple CGR objects representing corresponding wearable physical items. Figure 2F , panel 220 displays a 2D representation 278 of physical shorts. CGR environment 210 displays CGR shorts 280, which represents the physical shorts worn by CGR representation 212 of person 22. CGR shorts 280 are associated with a deformation model 282 that models the deformation of the corresponding physical shorts on person 22.

[0060] refer to Figure 2GIn some implementations, the CGR environment 210 includes an affordance 290 to add an identifier identifying a physical T-shirt represented by the CGR T-shirt 250 to a user selection queue (e.g., Figure 1C 150 shown). In some implementations, electronic device 200 detects activation of affordance 290 and adds the identifier to the user selection queue.

[0061] In some implementations, a head-mountable device (HMD) (not shown) worn by person 22 presents (e.g., displays) a CGR environment 210 according to various implementations. In some implementations, the HMD includes an integrated display (e.g., a built-in display) that displays the CGR environment 210. In some implementations, the HMD includes a head-mountable housing. In various implementations, the head-mountable housing includes an attachment area to which another device having a display can be attached. For example, in some implementations, the electronic device 200 can be attached to the head-mountable housing. In various implementations, the head-mountable housing is shaped to form a receiver for receiving another device (e.g., the electronic device 200) including a display. For example, in some implementations, the electronic device 200 slides / snaps onto the head-mountable housing or is otherwise attached to the head-mountable housing. In some implementations, the display of the device attached to the head-mountable housing presents (e.g., displays) the CGR environment 210. In various implementations, examples of the electronic device 200 include smart phones, tablet computers, media players, laptop computers, etc.

[0062] Adding identifiers of physical items from different sources to corresponding user selection queues typically requires a sequence of user inputs that impairs the user experience. For example, a user may have to navigate to a web page for each source and add identifiers of physical items provided by that source to a source-specific user selection queue. Excessive user input causes unnecessary wear and tear on the device and unnecessary battery usage. The present disclosure provides methods, systems, and / or devices for allowing a user to view CGR objects representing physical items provided by different sources and add identifiers identifying some or all of the physical items to a source-specific user selection queue. The present disclosure provides methods, systems, and / or devices that reduce the need to navigate to different pages corresponding to each source. Therefore, the present disclosure provides an enhanced user experience, reduces wear and tear on the device, and / or reduces power consumption by reducing unnecessary user input.

[0063] The present invention discloses a device that simultaneously displays CGR objects representing physical items from different sources. The device allows a user to provide input to select some or all of the CGR objects. The device adds identifiers of physical items corresponding to the selected CGR objects to a source-specific user selection queue. For example, if a user selects physical items provided by five different sources, the device populates five source-specific user selection queues corresponding to the five sources.

[0064] Figure 3A 300 is a block diagram of an example environment according to some implementations. Although relevant features are shown, those of ordinary skill in the art will recognize from this disclosure that various other features are not shown for the sake of brevity and in order not to obscure more relevant aspects of the exemplary implementations disclosed herein. To this end, as a non-limiting example, the environment includes an electronic device 300. In some implementations, the electronic device 300 is held by a person (not shown). In some implementations, the electronic device 300 includes a smart phone, a tablet computer, a laptop computer, etc.

[0065] In some implementations, the electronic device 300 presents a CGR environment 310. Figure 3A , CGR environment 310 includes first CGR lamp 312, second CGR lamp 314, CGR sofa 316, CGR painting 318, and CGR coffee table 320. In some implementations, CGR environment 310 represents a physical environment, and some CGR objects in CGR environment 310 represent physical items in the physical environment. For example, first CGR lamp 312 represents a first physical lamp in the physical environment, and second CGR lamp 314 represents a second physical lamp in the physical environment.

[0066] In some implementations, some CGR objects in CGR environment 310 represent physical items that are not in the physical environment represented by CGR environment 310. For example, CGR sofa 316 represents a physical sofa that is not in the physical environment corresponding to CGR environment 310. Similarly, CGR painting 318 represents a physical painting that is not in the corresponding physical environment. CGR coffee table 320 represents a physical coffee table that is not in the corresponding physical environment.

[0067] In some implementations, some CGR objects in the CGR environment 310 represent physical items purchased from a source (e.g., purchased from a store such as a physical store or an online store). Figure 3AIn the example of , CGR sofa 316 represents a physical sofa purchased from a first source 330 (e.g., purchased from Ron's furniture store). CGR painting 318 represents a physical painting purchased from a second source 340 (e.g., purchased from Jacob Art Gallery). CGR coffee table 320 represents a physical coffee table purchased from the first source 330.

[0068] In some implementations, the physical items represented by the CGR objects are associated with corresponding identifiers (IDs) that identify the physical items. For example, the physical sofa represented by CGR sofa 316 is associated with sofa ID 317 that identifies the physical sofa. In some implementations, the physical painting represented by CGR painting 318 is associated with painting ID 319 that identifies the physical painting. In some implementations, the physical coffee table represented by CGR coffee table 320 is associated with coffee table ID 321 that identifies the physical coffee table. In some implementations, the ID includes a serial number, a barcode number, an item number, a model number, a product number, a manufacturer code, and / or a machine-readable representation of data (e.g., an optical machine-readable representation of data, such as a barcode or QR code).

[0069] The first source 330 is associated with a first user selection queue 332, and the second source 340 is associated with a second user selection queue 342 that is different from the first user selection queue 332. More generally, in various implementations, each source is associated with a respective user selection queue. When the electronic device 300 detects a selection of a particular CGR object representing a particular physical item from a particular source, the electronic device 300 adds an identifier (ID) identifying the particular physical item to a source-specific user selection queue associated with the particular source.

[0070] refer to Figure 3B , the electronic device 300 detects a user input 350 at a location corresponding to the CGR sofa 316. In some implementations, the user input 350 corresponds to a request to associate the physical sofa represented by the CGR sofa 316 with the first user selection queue 332. For example, in some implementations, the user input 350 corresponds to a request to add the sofa ID 317 identifying the physical sofa represented by the CGR sofa 316 to the first user selection queue 332.

[0071] like Figure 3CAs shown, in some implementations, the electronic device 300 adds the sofa ID 317 to the first user selection queue 332 in response to detecting the user input 350. In some implementations, the electronic device 300 displays a notification 352 in response to adding the sofa ID 317 to the first user selection queue 332. In some implementations, the notification 352 includes text 354 indicating that the electronic device 300 has added the sofa ID 317 to the first user selection queue 332. In some implementations, the notification 352 includes a first queue enable representation 356, which triggers the display of a visual representation of the first user selection queue 332 when activated. In some implementations, the notification 352 includes a return enable representation 358, which causes the electronic device 300 to redisplay the CGR environment 310 when activated.

[0072] refer to Figure 3D , the electronic device 300 detects a user input 360 related to returning the enabling indication 358. Figure 3E As shown, in response to detecting the user input 360, the electronic device 300 redisplays the CGR environment 310. In some implementations, the electronic device 300 modifies the visual attributes of the CGR sofa 316 to indicate that the sofa ID 317 has been added to the first user selection queue 332. Figure 3E In the example of , the electronic device 300 has displayed a CGR sofa 316 with a shadow effect. More generally, in some implementations, the electronic device 300 changes the visual attributes of the CGR object to indicate that the ID of the corresponding physical item has been added to the source-specific user selection queue.

[0073] exist Figure 3E In the example of , the electronic device 300 displays a queue mark 362 (e.g., a check mark) to indicate that the CGR sofa 316 has been selected and / or to indicate that the ID identifying the corresponding physical sofa has been added to the user selection queue. In some specific implementations, the electronic device 300 displays text 364 to indicate that the sofa ID 317 has been added to the first user selection queue 332. In some specific implementations, the electronic device 300 displays a removal enable indication 366, which removes the sofa ID 317 from the first user selection queue 332 when activated.

[0074] refer to Figure 3F, the electronic device 300 detects a user input 370 at a location corresponding to the CGR drawing 318. In some implementations, the user input 370 corresponds to a request to associate the physical drawing represented by the CGR drawing 318 with the second user selection queue 342. For example, in some implementations, the user input 370 corresponds to a request to add a drawing ID 319 identifying the physical drawing represented by the CGR drawing 318 to the second user selection queue 342.

[0075] like Figure 3D As shown, in some implementations, electronic device 300 adds drawing ID 319 to second user selection queue 342 in response to detecting user input 370. In some implementations, electronic device 300 displays notification 372 in response to adding drawing ID 319 to second user selection queue 342. In some implementations, notification 372 includes text 374 indicating that electronic device 300 has added drawing ID 319 to second user selection queue 342. In some implementations, notification 372 includes second queue enable representation 376, which, when activated, triggers display of a visual representation of second user selection queue 342. In some implementations, notification 372 includes return enable representation 378, which, when activated, causes electronic device 300 to redisplay CGR experience 310. In some implementations, notification 372 includes a multiple queue enable representation 378 (e.g., an all queues enable representation) that, when activated, triggers display of visual representations of multiple user selection queues (e.g., visual representations of all user selection queues, such as a visual representation of first user selection queue 332 and a visual representation of second user selection queue 342). Notification 372 includes a return enable representation 380 that, when activated, causes electronic device 300 to redisplay CGR environment 310.

[0076] refer to Figure 3H , the electronic device 300 detects a user input 382 related to returning the enabling indication 380. Fig. 3IAs shown, in response to detecting user input 382, ​​electronic device 300 displays a first visual representation 332R of first user selection queue 332 and a second visual representation 342R of second user selection queue 342. First visual representation 332R includes a 2D representation 333 of a physical sofa, a description 334 of the sofa, a first delete enable indication 335 for removing sofa ID 317 from first user selection queue 332, a first modify enable indication 336 for modifying the number of sofas, and a first confirm enable indication 337 for confirming first user selection queue 332. Second visual representation 342R includes a 2D representation 343 of a physical painting, a description 344 of the painting, a second delete enable indication 345 for removing painting ID 319 from second user selection queue 342, a second modify enable indication 346 for modifying the number of paintings, and a second confirm enable indication 347 for confirming second user selection queue 342. In some embodiments, the electronic device 300 displays a third confirmation enable indication 384, which, when activated, simultaneously confirms the first user selection queue 332 and the second user selection queue 342.

[0077] In some implementations, detecting activation of first confirmation enable indication 337 triggers placement of a physical couch in the physical environment around electronic device 300. For example, in some implementations, in response to detecting activation of first confirmation enable indication 337, electronic device 300 transmits a message to a device associated with first source 330. In some implementations, the message includes a request to deliver the physical couch to the physical environment around electronic device 300.

[0078] In some implementations, a head-mountable device (HMD) (not shown) worn by a person presents (e.g., displays) a CGR environment 310 according to various implementations. In some implementations, the HMD includes an integrated display (e.g., a built-in display) that displays the CGR environment 310. In some implementations, the HMD includes a head-mountable shell. In various implementations, the head-mountable shell includes an attachment area to which another device having a display can be attached. For example, in some implementations, the electronic device 300 can be attached to the head-mountable shell. In various implementations, the head-mountable shell is shaped to form a receiver for receiving another device (e.g., the electronic device 300) including a display. For example, in some implementations, the electronic device 300 slides / snaps onto the head-mountable shell or is otherwise attached to the head-mountable shell. In some implementations, the display of the device attached to the head-mountable shell presents (e.g., displays) the CGR environment 310. In various implementations, examples of the electronic device 300 include smart phones, tablet computers, media players, laptop computers, etc.

[0079] In some cases, a person may want to see how a physical item would look in the person's physical environment. Some devices generate and present a CGR environment that is similar to the person's physical environment, and place CGR objects representing new physical items, thereby allowing the person to see how the new physical item will look in the physical environment. However, if the physical environment is too cluttered, the corresponding CGR environment is similarly cluttered. Therefore, there may be no space in the CGR environment to place the CGR object representing the new physical item. The present disclosure provides methods, devices, and / or systems for masking physical items in a physical environment to leave space for CGR objects representing the new physical item. The device detects a physical surface, detects the presence of a physical item that obscures a portion of the physical surface, and synthesizes a masking element to mask the physical item and leave space for the new CGR object.

[0080] Figure 4A 4 is a block diagram of an example physical environment 40 according to some implementations. Although relevant features are shown, those of ordinary skill in the art will recognize from this disclosure that various other features are not shown for the sake of brevity and so as not to obscure more relevant aspects of the exemplary implementations disclosed herein. To this end, as a non-limiting example, the physical environment 40 includes a physical surface 42. In some implementations, the physical surface 42 represents a top surface of a physical table. In some implementations, the physical surface 42 represents a floor.

[0081] In some implementations, physical surface 42 is associated with visual attribute 43. In some implementations, visual attribute 43 indicates a material attribute of physical surface 42. For example, in some implementations, visual attribute 43 indicates a color of physical surface 42. In some implementations, visual attribute 43 indicates a material type of physical surface 42. In some implementations, visual attribute 43 indicates a texture of physical surface 42. In some implementations, visual attribute 43 indicates a reflectivity of physical surface 42.

[0082] exist Figure 4A In the example of , there are physical items that obstruct corresponding portions of physical surface 42. For example, a first physical item 44 (e.g., a package), a second physical item 46 (e.g., a box), and a third physical item 48 (e.g., a speaker) are placed on physical surface 42. Figure 4A As can be seen in FIG. 4 , first physical objects 44 , 46 , and 48 occlude (eg, cover) a significant portion of physical surface 42 (eg, a majority of physical surface 42 ).

[0083] refer to Figure 4BIn some implementations, the electronic device 400 presents (e.g., displays) a CGR environment 410 that corresponds to (e.g., represents) the physical environment 40. The CGR environment 410 includes a CGR surface 412 that represents the physical surface 42. The CGR surface 412 is associated with visual properties 413 that are within a certain similarity of the visual properties 43 of the physical surface. For example, in some implementations, the CGR surface 412 has the same material properties as the physical surface 42. For example, the CGR surface 412 has the same color as the physical surface 42, uses the same material type, has the same texture, and / or has the same texture.

[0084] exist Figure 4B In the example of FIG. 4 , the CGR environment 410 includes a first CGR object 414 representing the first physical item 44, a second CGR object 416 representing the second physical item 46, and a third CGR object 418 representing the third physical item 48. Figure 4B As can be seen in FIG. 4 , CGR objects 414, 416, and 418 occupy a significant portion of CGR surface 412. Therefore, there is not enough space on CGR surface 412 to place additional CGR objects. In some implementations, electronic device 400 is held by a person (not shown). In some implementations, electronic device 400 includes a smart phone, a tablet computer, a laptop computer, etc.

[0085] In some implementations, CGR environment 410 is a perspective of physical environment 40. For example, in some implementations, CGR environment 410 is a video perspective of physical environment 40. In some implementations, CGR environment 410 is an optical perspective of physical environment 40. In such implementations, CGR surface 412 and CGR objects 414, 416, and 418 are the same as physical surface 42 and physical items 44, 46, and 48, respectively.

[0086] refer to Figure 4C In some implementations, the electronic device 400 synthesizes the masking element 422 to mask the physical items 44, 46, and 48 located on the physical surface 42. In some implementations, the electronic device 400 synthesizes the masking element 422 to mask the CGR objects 414, 416, and 418 on the CGR surface 412. Figure 4CAs can be seen in FIG. 4 , in some implementations, synthetic masking element 422 provides the appearance that no CGR objects are present on CGR surface 412. Since CGR surface 412 represents physical surface 42, synthetic masking element 422 provides the appearance that no physical items are present on physical surface 42. Thus, synthetic masking element 422 provides the appearance of a less cluttered physical surface 42 without requiring an operator (e.g., a person or a robot) to physically remove physical items 44, 46, and 48 from physical surface 42.

[0087] refer to Figure 4D , in some implementations, the masking element 422 is associated with a visual attribute 423. In some implementations, the visual attribute 423 of the masking element 422 is within a certain similarity of the visual attribute 413 of the CGR surface 412. Because the visual attribute 413 of the CGR surface 412 is within a certain similarity of the visual attribute 413 of the physical surface, in some implementations, the visual attribute 423 of the masking element 422 is within a certain similarity of the visual attribute 43 of the physical surface. Thus, in some implementations, the masking element 422 has the same color, material type, texture, and / or reflectivity as the physical surface 42. The visual attribute 423 of the masking element 422 matches the visual attribute 43 of the physical surface 42 to provide the appearance that no physical item is present on the physical surface 42.

[0088] Figure 4E Another masking element 422a is shown that is associated with a visual attribute 423a. Figure 4E In the example of FIG. 4 , the visual attribute 423a of the masking element 422a is different (e.g., significantly different) from the visual attribute 413 of the CGR surface 412. Because the visual attribute 413 of the CGR surface 412 is within a certain similarity of the visual attribute 43 of the physical surface 42, in some implementations, the visual attribute 423a of the masking element 422a is not within a certain similarity of the visual attribute 43 of the physical surface 42. Therefore, in some implementations, the masking element 422a has a different color, material type, texture, and / or reflectivity than the physical surface 42.

[0089] refer to Figure 4F In some implementations, the electronic device 400 synthesizes a plurality of masking elements. In some implementations, each masking element masks a corresponding one of the physical items. In some implementations, each masking element conforms to the shape of the physical item that the masking element masks. In some implementations, each masking element masks a corresponding one of the CGR objects. In some implementations, each masking element conforms to the shape of the CGR object that the masking element conforms to. Figure 4FIn the example of FIG. 4 , the electronic device 400 generates a first masking element 424 to mask a first CGR object 414 representing a first physical item 44, a second masking element 426 to mask a second CGR object 416 representing a second physical item 46, and a third masking element 428 to mask a third CGR object 418 representing a third physical item 48. In some implementations, the first masking element 424 conforms to the shape of the first CGR object 414 and / or the first physical item 44. In some implementations, the second masking element 426 conforms to the shape of the second CGR object 416 and / or the second physical item 46. In some implementations, the third masking element 428 conforms to the shape of the third CGR object 418 and / or the third physical item 48.

[0090] refer to Figure 4G In some implementations, the electronic device 400 displays a CGR object store 430. The CGR object store 430 includes CGR objects representing physical items. Figure 4G In the example of , the CGR object store 430 includes a fourth CGR object 440 (eg, a CGR desk lamp) representing a fourth physical item (eg, a physical desk lamp). Figure 4H As shown, in some implementations, the electronic device 400 detects a user input 442 corresponding to a request to place a fourth CGR object 440 on the CGR surface 412. In some implementations, the user input 442 includes a drag gesture starting at the fourth CGR object 440 and ending at the CGR surface 412.

[0091] like Fig. 4I As shown, in some implementations, the electronic device 400 synthesizes the masking element 422 in response to detecting the user input 442. The electronic device 400 overlays the fourth CGR object 440 on the masking element 422 to provide the appearance that the fourth CGR object 440 is placed on the CGR surface 412. Fig. 4I In the example of FIG. 4 , the CGR object storage 430 displays a fifth CGR object 444 .

[0092] refer to Figure 4J In some implementations, the electronic device 400 displays a webpage 450 corresponding to a fourth physical item (e.g., a physical desk lamp). In some implementations, the webpage 450 includes a 2D representation 452 of the fourth physical item (e.g., an image of the fourth physical item), a description 454 of the fourth physical item, and an affordance 456 that, when activated, causes the electronic device 400 to display a fourth CGR object 440 representing the fourth physical item. Figure 4K As shown, electronic device 400 detects user input 458 involving affordance 456. Figure 4LAs shown, in response to detecting the user input 458 , the electronic device 400 synthesizes the masking element 422 and displays the fourth CGR object 440 on top of the masking element 422 .

[0093] refer to Figure 4M In some implementations, the electronic device 400 displays a clear enable indication 460, which, when activated, causes the electronic device 400 to synthesize a masking element onto the CGR surface 412 so as to mask the CGR objects 414, 416, and 418 on the CGR surface 412. As shown in FIG. 4N, the electronic device 400 detects a user input 462 related to the clear enable indication 460. As shown in FIG. 4O, in response to detecting the user input 462, the electronic device 400 synthesizes the masking element 422 onto the CGR surface 412. In some implementations, the electronic device 400 displays a remove enable indication 464, which, when activated, causes the electronic device 400 to remove the masking element 422 and redisplay the CGR objects 414, 416, and 418 on the CGR surface 412.

[0094] In some implementations, a head-mountable device (HMD) (not shown) worn by a person presents (e.g., displays) a CGR environment 410 according to various implementations. In some implementations, the HMD includes an integrated display (e.g., a built-in display) that displays the CGR environment 410. In some implementations, the HMD includes a head-mountable shell. In various implementations, the head-mountable shell includes an attachment area to which another device having a display can be attached. For example, in some implementations, the electronic device 400 can be attached to the head-mountable shell. In various implementations, the head-mountable shell is shaped to form a receiver for receiving another device (e.g., the electronic device 400) including a display. For example, in some implementations, the electronic device 400 slides / snaps onto the head-mountable shell or is otherwise attached to the head-mountable shell. In some implementations, the display of the device attached to the head-mountable shell presents (e.g., displays) the CGR environment 410. In various implementations, examples of the electronic device 400 include smart phones, tablet computers, media players, laptop computers, etc.

[0095] Figure 5A 5 is a flowchart representation of a method 500 for controlling a user selection queue. In various implementations, the method 500 is performed by a device having a display, a non-volatile memory, and one or more processors coupled to the display and the non-volatile memory (e.g., Figures 1A to 1LIn some implementations, the method 500 is performed by a processing logic component (including hardware, firmware, software, or a combination thereof). In some implementations, the method 500 is performed by a processor that executes code stored in a non-transitory computer-readable medium (e.g., a memory).

[0096] As shown in block 510, in various implementations, method 500 includes displaying a computer generated reality (CGR) object in a CGR environment. In some implementations, the CGR object represents a physical object. For example, Figure 1C As shown, the electronic device 100 displays a CGR sofa 116 in the CGR environment 110. Figure 1C As described above, CGR sofa 116 represents a physical sofa (e.g., Figure 1A The 2D representation 104 shown is a sofa).

[0097] As indicated at block 520, in various implementations, method 500 includes synthesizing an affordance associated with a CGR object. Figure 1C As shown, electronic device 100 synthesizes affordance 120 associated with CGR couch 116. In some implementations, method 500 includes displaying affordance adjacent to a CGR object. Figure 1C As shown, the electronic device 100 displays an enable indication 120 adjacent to the CGR sofa 116.

[0098] As indicated at block 530, in various implementations, method 500 includes detecting input involving an affordance. Figure 1D As shown, electronic device 100 detects user input 130 involving affordance 120. In some implementations, method 500 includes detecting contact at a location corresponding to the affordance.

[0099] As indicated at block 540, in various implementations, method 500 includes adding an identifier identifying the physical item to a user selection queue in response to detecting the input. Figure 1D to Figure 1E As shown, the electronic device 100 adds the identifier 118 of the sofa to the user selection queue 150 in response to detecting the user input 130. In some implementations, the method 500 includes writing the identifier identifying the physical item to the user selection queue. In some implementations, the method 500 includes associating the identifier identifying the physical item with the user selection queue. In various implementations, adding the identifier of the physical item to the user selection queue while the device is displaying a CGR object representing the physical item reduces the need to manually navigate to a web page corresponding to the physical item in order to add the identifier identifying the physical item to the user selection queue.

[0100] refer to Figure 5B As shown in block 510a, in some implementations, the CGR environment includes representations of physical items located in the physical environment surrounding the device. Figure 1C As shown, the CGR environment 110 includes a first CGR floor lamp 112 representing the first floor lamp 12 in the physical environment 10 surrounding the electronic device 100 , and a second CGR floor lamp 114 representing the second floor lamp 14 in the physical environment 10 surrounding the electronic device 100 .

[0101] As shown in block 510b, in some implementations, the method includes displaying a webpage including a two-dimensional (2D) representation of a physical item, and obtaining input to switch from the webpage to a CGR mode in which a CGR object representing the physical item is displayed. Figure 1A As shown, the electronic device 100 displays a web page 102 including a 2D representation 104 of a sofa. Figure 1B As shown, electronic device 100 detects user input 108 involving affordance 106 corresponding to a request to switch to a CGR mode in which CGR sofa 116 representing a sofa is displayed.

[0102] As shown in block 520a, in some implementations, the synthesized affordance includes synthesizing the affordance within a threshold distance of the CGR object. Figure 1C As shown, electronic device 100 is adjacent to or in proximity to CGR couch 116 synthetic affordance 120. In some implementations, method 500 includes obtaining a threshold distance.

[0103] As indicated at block 520b, in some implementations, method 500 includes synthesizing affordances at designated portions of the CGR environment. Figure 1H As shown, electronic device 100 synthesizes affordance 120 at designated portion 160. In some implementations, method 500 includes selecting a designated portion. Figure 1H As shown, the electronic device 100 selects the lower right corner of the CGR environment 110 as the designated portion 160 .

[0104] As shown in box 520c, in some embodiments, method 500 includes configuring the enable representation. As shown in box 520d, in some embodiments, method 500 includes changing the visual attributes of the enable representation. For example, in some embodiments, method 500 includes changing the text string displayed by the enable representation. In some embodiments, method 500 includes replacing the default text of the enable representation with the replacement string. In some embodiments, the replacement string is a function of the physical item represented by the CGR object (e.g., changing "add to queue" to "add sofa to queue").

[0105] As shown in box 520e, in some embodiments, method 500 includes changing the operation associated with the enabling representation. In some embodiments, method 500 includes changing the operation to add the identifier to a user selection queue and confirming the user selection queue. In some embodiments, method 500 includes changing the operation to add the identifier to a favorites list. In some embodiments, method 500 includes changing the operation to save the identifier. In some embodiments, method 500 includes bookmarking a web page associated with a physical item represented by the CGR object. In some embodiments, method 500 includes sharing the identifier with another device (e.g., another device associated with a contact in a contact application of the device).

[0106] As shown in box 520f, in some embodiments, method 500 includes constraining the network connectivity of the device when the enable indication is displayed. In some embodiments, method 500 includes preventing malicious code from executing on the device when the enable indication is displayed. In some embodiments, constraining network connectivity prevents malicious code from executing on the device. In some embodiments, method 500 includes turning off a transceiver (e.g., a radio component) of the device when the enable indication is displayed. In some embodiments, method 500 includes switching the device from a communication mode to a non-communication mode (e.g., switching to airplane mode) when the enable indication is displayed.

[0107] As shown in block 520g, in some implementations, method 500 includes determining whether a CGR object occupies at least a threshold number of pixels, and synthesizing an affordance in response to determining that the CGR object occupies at least the threshold number of pixels. In some implementations, method 500 includes determining whether a CGR object occupies at least a threshold portion of a CGR environment, and synthesizing an affordance in response to determining that the CGR object occupies at least a threshold portion of the CGR environment.

[0108] As shown in block 530a, in some implementations, the input related to the affordance includes a user selection of the affordance. Figure 1D As shown, the electronic device 100 detects user input 130 related to the enabling representation 120. As shown in box 530b, in some specific implementations, the input related to the enabling representation includes a gaze input related to the enabling representation. For example, in some specific implementations, the electronic device 100 uses eye tracking to detect the gaze input. As shown in box 530c, in some specific implementations, the input related to the enabling representation includes a verbal input. For example, in some specific implementations, the electronic device 100 detects a voice via a microphone, and the voice corresponds to the input of activating the enabling representation.

[0109] refer to Figure 5CIn some implementations, method 500 includes modifying a visual attribute of the CGR object to indicate that the CGR object is selectable, as shown in block 540. For example, in some implementations, method 500 includes displaying the CGR object with a raised appearance to indicate that the CGR object is pressable. In some implementations, method 500 includes displaying text adjacent to or overlapping the CGR object to indicate selectability of the CGR object.

[0110] As shown in block 550, in some implementations, the method 500 includes detecting input related to the CGR object, and manipulating the CGR object based on the input related to the CGR object. For example, in some implementations, the electronic device 100 changes the size of the CGR object in response to detecting a pinch gesture at a location corresponding to the CGR object. In some implementations, the electronic device 100 moves the CGR object in response to detecting a drag gesture or a swipe gesture at a location corresponding to the CGR object.

[0111] As shown in block 560, in some implementations, method 500 includes displaying a second CGR object representing the second physical item when the CGR object is displayed in the CGR environment. In some implementations, method 500 includes adding an identifier identifying the second physical item to a user selection queue in response to detecting an input related to an affordance. In some implementations, the affordance is associated with the CGR object and the second CGR object. For example, Fig. 1I As shown, in addition to the CGR sofa 116 , the electronic device 100 also displays a CGR painting 170 .

[0112] As shown in block 560a, in some implementations, the method 500 includes identifying, by the recommendation engine, a second physical item based on the physical item. For example, in response to the electronic device 100 displaying the CGR sofa 116, the recommendation engine identifies a painting represented by the CGR painting 170.

[0113] As indicated at block 570, in some implementations, method 500 includes displaying a replacement affordance associated with the CGR object. In some implementations, the replacement affordance allows replacement of the CGR object with a third CGR object representing a third physical item. For example, Figure 1J As shown, electronic device 100 displays second replacement affordance 194b. In some implementations, method 500 includes detecting input involving replacement affordances. For example, Figure 1K As shown, electronic device 100 detects user input 196 involving second replacement affordance 194b. In some implementations, method 500 includes replacing the CGR object with a third CGR object in response to detecting the input involving the replacement affordance. Figure 1LAs shown, the electronic device 100 displays a second CGR sofa 198 in response to detecting the user input 196 .

[0114] Fig. 6A 6 is a flowchart representation of a method 600 for displaying a CGR object representing a wearable physical item. In various implementations, the method 600 is performed by a device (e.g., a computer program product) having a display, a non-transitory memory, and one or more processors coupled to the display and the non-transitory memory. Figure 2A 2G ) to perform. In some specific implementations, method 600 is performed by a processing logic component (including hardware, firmware, software, or a combination thereof). In some specific implementations, method 600 is performed by a processor that executes code stored in a non-transitory computer-readable medium (e.g., a memory).

[0115] As shown in block 610, in various implementations, method 600 includes obtaining a computer generated reality (CGR) representation of a person. In some implementations, at least a portion of the CGR representation is proportional to a corresponding portion of the person. For example, Figure 2A As shown, the electronic device 200 obtains (eg, generates) and presents a CGR representation 212 of the person 22. Figure 2A As described, in some implementations, CGR representation 212 is proportional to person 22 .

[0116] As shown in block 620, in various implementations, method 600 includes obtaining a CGR object representing a wearable physical item. In some implementations, the CGR object is associated with a deformation model that characterizes one or more material properties of the wearable physical item. For example, Figure 2B As shown, the electronic device 200 obtains (e.g., generates or receives) and presents a CGR T-shirt 230, which represents a physical T-shirt of small size, slim fit, and made of cotton. Figure 2B As shown, CGR T-shirt 230 is associated with deformation model 232 that characterizes the material properties of the physical T-shirt (eg, small size, slim fit, and cotton material composition).

[0117] As shown in block 630, in various implementations, method 600 includes displaying a CGR object associated with the CGR representation of the person. In some implementations, the CGR object interacts with the CGR representation of the person based on a deformation model. For example, Figure 2B As shown, electronic device 200 displays CGR T-shirt 230 as being worn by CGR representation 212 of person 22. Figure 2BAs shown, CGR T-shirt 230 is deformed on CGR representation 212 according to deformation model 232. For example, CGR T-shirt 230 is deformed to form CGR stretch line 234. As described herein, because the deformation model models the deformation of the wearable physical item on the person, displaying CGR objects associated with the CGR representation of the person allows the person to see how the wearable physical item will fit the person, thereby enhancing the user experience of the device.

[0118] refer to Figure 6B As shown in block 610a, in some implementations, method 600 includes obtaining a body model of a person, and generating a CGR representation of the person based on the body model of the person. For example, as shown in FIG. 2E, in some implementations, electronic device 200 obtains (e.g., generates or receives) a body model 270 of person 22, and electronic device 200 generates (e.g., synthesizes) a CGR representation 212 of person 22 based on body model 270.

[0119] As shown in block 610b, in some implementations, method 600 includes obtaining physical dimensions of a person and generating a body model based on the physical dimensions of the person. Figure 2E As shown, in some implementations, the electronic device 200 displays a GUI 260 having a measurement result affordance 266. In some implementations, in response to detecting activation of the measurement result affordance 266, the electronic device 200 displays a user interface element (e.g., a text box, a drop-down menu, etc.) that allows the person 22 to input the physical dimensions. In some implementations, the electronic device 200 generates a body model 270 based on the physical dimensions, and the electronic device 200 generates a CGR representation 212 based on the body model 270.

[0120] As shown in box 610c, in some embodiments, method 600 includes capturing one or more images of a person and generating a CGR representation of the person based on the one or more images of the person. For example, as shown in Figure 2E, in some embodiments, electronic device 200 displays an enable indication 262, which triggers the capture of a photo when activated. In some embodiments, electronic device 200 uses the captured picture to generate CGR representation 212 of person 22. In some embodiments, method 600 includes extracting physical dimensions of person 22 from the captured photo using methods, devices and / or systems associated with photogrammetry, and generating body model 270 and / or CGR representation 212 based on the extracted physical dimensions.

[0121] As shown in block 610d, in some implementations, method 600 includes obtaining depth data associated with a person, and generating a CGR representation of the person based on the depth data. In some implementations, method 600 includes capturing the depth data from a depth sensor (e.g., a depth camera) of a device. In some implementations, method 600 includes determining physical dimensions of the person based on the depth data, and generating a body model and / or CGR representation of the person based on the physical dimensions.

[0122] As shown in block 620a, in some implementations, method 600 includes generating a deformation model of the CGR object based on one or more material properties of the wearable physical object. Figure 2B As shown, the electronic device 200 generates a deformation model 232 of the CGR T-shirt 230 based on material properties of the physical T-shirt (eg, small size, slim fit, and / or cotton content).

[0123] As shown in block 620b, in some implementations, method 600 includes generating a deformation model of the CGR object based on the material type of the wearable physical item. Figure 2B As shown, the electronic device 200 generates a deformation model 232 of the CGR T-shirt 230 based on the cotton composition of the physical T-shirt.

[0124] As shown in block 620c, in some implementations, method 600 includes generating a deformation model of the CGR object based on a texture of the wearable physical item. In some implementations, the texture is a function of a material type. Thus, in some implementations, generating the deformation model based on a material type of the wearable physical item includes generating the deformation model based on a texture of the wearable physical item.

[0125] As shown in block 620d, in some implementations, method 600 includes generating a deformation model of the CGR object based on a stiffness of the wearable physical item. In some implementations, the stiffness is a function of a material type and / or style of the wearable physical item. Thus, in some implementations, generating the deformation model based on a material type and / or style of the wearable physical item includes generating the deformation model based on the stiffness of the wearable physical item. In some implementations, method 600 includes obtaining a stiffness value, and generating the deformation model based on the stiffness value. In some implementations, the stiffness value relates to an amount of sizing applied to the wearable physical item.

[0126] As shown in block 620e, in some implementations, method 600 includes generating a deformation model of the CGR object based on the size of the wearable physical object. Figure 2B As shown, the electronic device 200 generates a deformed model 232 based on the small size selected by the person 22. Figure 2B As shown, the small size facilitates the formation of the CGR stretch line 234. Similarly, as Figure 2C As shown, the electronic device 200 generates a deformed model 242 based on the large size selected by the person 22. Figure 2C As shown, the large size facilitates CGR droops 244 and 246 .

[0127] As shown in block 620f, in some implementations, the wearable physical item includes an item of clothing (e.g., FIG. 2A to FIG. 2D T-shirt as shown, or Figure 2F In various implementations, method 600 allows a person to see how a particular item of clothing may fit the person without having to try the item of clothing on. In some implementations, the item of clothing includes underwear (e.g., bras or panties). Since many stores do not allow people to try on underwear, in various implementations, method 600 allows a person to see how a particular item of clothing may fit the person without having to try the underwear on. In various implementations, method 600 enables a person to find an item of clothing that fits the person in a reduced amount of time, thereby reducing the amount of time a display of a device remains on. Reducing the amount of time a display of a device remains on reduces power consumption of the device.

[0128] refer to Figure 6C As shown in block 630a, in some implementations, method 600 includes displaying the CGR object as being worn by the CGR representation of the person. Figure 2B As shown, electronic device 200 displays CGR T-shirt 230 as being worn by CGR representation 212 of person 22 .

[0129] As indicated at block 630b, in some implementations, method 600 includes displaying a second CGR object associated with a second portion of the CGR representation of the person. In some implementations, the second CGR object represents a second wearable physical item. For example, Figure 2F As shown, the electronic device 200 displays CGR shorts 280 as being worn by the CGR representation 212 of the person 22. Simultaneously displaying CGR objects representing multiple wearable physical items reduces the amount of time the device's display remains on, thereby reducing power consumption of the device. Simultaneously displaying CGR objects representing multiple wearable physical items allows a person to select multiple physical items simultaneously, thereby enhancing the user experience of the device.

[0130] As shown in block 640, in some implementations, method 600 includes searching a data store for a set of wearable physical items that fit a body model of a person, and selecting a wearable physical item from the set of wearable physical items that fit a body model of a person. Figure 2EAs shown, in some implementations, the electronic device 200 displays a GUI 260 that allows the person 22 to search for clothing that fits the body model 270 of the person 22 .

[0131] As shown in block 650, in some implementations, method 600 includes, after displaying the CGR object associated with the CGR representation of the person, displaying an affordance for adding an identifier identifying the wearable physical item to a user selection queue. Figure 2G As shown, electronic device 200 displays affordance 290, which, when activated, causes electronic device 200 to add an ID identifying the physical T-shirt represented by CGR T-shirt 250 to a user selection queue (e.g., Figure 1C In some implementations, method 600 includes detecting input related to an affordance (e.g., detecting activation of an affordance). Figure 2G In some implementations, method 600 includes adding an identifier identifying the wearable physical item to a user selection queue (e.g., writing the identifier to Figure 1C User selection queue 150 is shown).

[0132] In some implementations, method 600 includes scraping the source material to identify the wearable physical item, as shown in block 660. For example, in some implementations, method 600 includes scraping a movie to identify clothing items worn by fictional characters in the movie, and searching the clothing data store to find physical clothing items within a certain similarity of the clothing items worn by the fictional characters in the movie.

[0133] Fig. 7A 700 is a flowchart representation of a method 700 for controlling multiple user selection queues simultaneously. In various implementations, the method 700 is performed by a device (e.g., a computer) having a display, a non-volatile memory, and one or more processors coupled to the display and the non-volatile memory. FIG. 3A to FIG. 3I In some implementations, the method 700 is performed by a processing logic component (including hardware, firmware, software, or a combination thereof). In some implementations, the method 700 is performed by a processor executing code stored in a non-transitory computer-readable medium (e.g., a memory).

[0134] As shown in block 710, in various implementations, method 700 includes displaying a plurality of computer generated reality (CGR) objects representing respective physical items from a plurality of sources including a first source and a second source. Figure 3AAs shown, the electronic device 300 displays a CGR sofa 316 representing a physical sofa from a first source 330 , and a CGR painting 318 representing a physical painting from a second source 340 .

[0135] As shown in block 720, in various implementations, method 700 includes detecting an input selecting a first CGR object from a plurality of CGR objects and a second CGR object from a plurality of CGR objects. In some implementations, the first CGR object represents a first physical item from a first source, and the second CGR object represents a second physical item from a second source. For example, Figure 3B and Figure 3F As shown, electronic device 300 detects user inputs 350 and 370 selecting CGR sofa 316 and CGR painting 318, respectively.

[0136] As shown in block 730, in various implementations, method 700 includes adding an identifier of the first physical item to a first user selection queue associated with the first source. Figure 3B As shown, the electronic device 300 adds the sofa ID 317 to the first user selection queue 332 .

[0137] As indicated at block 740, in various implementations, method 700 includes adding an identifier of the second physical item to a second user selection queue associated with the second source. Figure 3G As shown, the electronic device 300 adds the painting ID 319 to the second user selection queue 342. In various specific implementations, the method 700 allows the IDs of physical items from different sources to be added to the corresponding source-specific user selection queues at the same time, thereby reducing the user input corresponding to manually navigating to the web page associated with the source in order to add these IDs to the corresponding source-specific user selection queues. Therefore, the method 700 enhances the user experience of the device and / or extends the battery life of the device by reducing user input and / or by reducing the amount of time the display of the device remains on.

[0138] refer to Figure 7B As shown in block 710a, in some implementations, method 700 includes switching from a web page including a two-dimensional (2D) representation of a physical object to a CGR environment in response to an input corresponding to a request to display a plurality of CGR objects. For example, in some implementations, electronic device 300 displays a web page with a two-dimensional (2D) representation of a physical object. Figure 1A In some implementations, the method 700 includes detecting a user input corresponding to a request to enter a CGR mode, and displaying a CGR environment including a plurality of CGR objects in response to detecting the user input.

[0139] As shown in block 710b, in some implementations, method 700 includes identifying, by a recommendation engine, at least a portion of the displayed CGR object. In some implementations, method 700 includes identifying physical items in a physical environment surrounding the device, recommending new physical items based on the physical items in the physical environment, and displaying a CGR object representing the new physical item. For example, in some implementations, electronic device 300 identifies two floor lamps in a physical environment surrounding electronic device 300, recommends placing a sofa between the two floor lamps, and displays CGR sofa 316 between CGR lamps 312 and 314.

[0140] As shown in block 720a, in some implementations, method 700 includes detecting respective contacts with the first CGR object and the second CGR object. Figure 3B and Figure 3F As shown, the electronic device 300 detects user inputs 350 and 370 on the CGR sofa 316 and the CGR painting 318, respectively.

[0141] As shown in block 720b, in some implementations, method 700 includes detecting that the CGR representation of the person has touched the first CGR object and the second CGR object. For example, in some implementations, the CGR environment 310 includes a CGR representation of the person, and the CGR representation provides Figure 3B and Figure 3F The user enters 350 and 370 in .

[0142] As shown in block 720c, in some implementations, method 700 includes detecting gaze input involving the first CGR object and the second CGR object. In some implementations, method 700 includes tracking movement of eyes of a person using the device, and determining that the person has selected the first CGR object and the second CGR object in response to the gaze of the eyes being fixed at the first CGR object and the second CGR object for a threshold amount of time.

[0143] As shown in block 720d, in some implementations, the method 700 includes detecting speech input related to the first CGR object and the second CGR object. In some implementations, the method 700 includes detecting speech input corresponding to selection of the first CGR object and the second CGR object.

[0144] As shown in block 720e, in some implementations, method 700 includes modifying visual attributes of the first CGR object and the second CGR object to indicate selection of the first CGR object and the second CGR object. Figure 3EAs shown, the electronic device 300 displays the CGR sofa 316 with a shadow effect to indicate that the CGR sofa 316 has been selected. In some implementations, the method 700 includes highlighting the selected CGR object, making the selected CGR object brighter, and / or placing a polygon around the selected CGR object.

[0145] As shown in block 720f, in some implementations, the method 700 includes modifying a visual attribute of the remaining portion of the plurality of CGR objects to indicate that the remaining portion of the plurality of CGR objects is not selected. For example, in some implementations, the method 700 includes graying out the remaining portion of the plurality of CGR objects and / or making the remaining portion of the plurality of CGR objects less bright.

[0146] refer to Figure 7C As shown in block 730a, in some implementations, the first user selection queue includes a first virtual basket. As shown in block 740a, in some implementations, the second user selection queue includes a second virtual basket. As shown in block 730b, in some implementations, the first user selection queue includes a first list of favorite CGR objects. As shown in block 740b, in some implementations, the second user selection queue includes a second list of favorite CGR objects.

[0147] As shown in block 750, in some implementations, method 700 includes replacing one of the plurality of CGR objects with a replacement CGR object in response to detecting a replacement input, while maintaining display of the remaining portion of the plurality of CGR objects. For example, replacing CGR sofa 116 with another CGR sofa representing a physical sofa different from the physical sofa represented by CGR sofa 116, while maintaining display of CGR painting 318.

[0148] Fig. 8A 8 is a flowchart representation of a method 800 for synthesizing a masking element. In various implementations, the method 800 is performed by a device (e.g., Figure 4A 400). In some implementations, method 800 is performed by a processing logic component (including hardware, firmware, software, or a combination thereof). In some implementations, method 800 is performed by a processor executing code stored in a non-transitory computer-readable medium (e.g., a memory).

[0149] As shown in block 810, in various implementations, method 800 includes detecting a physical surface in a physical environment surrounding a device. Figure 4A and Figure 4BAs described, the electronic device 400 detects a physical surface 42 in the physical environment 40. In some implementations, the method 800 includes obtaining a grid map of the physical environment, and determining that the physical environment includes a physical surface based on the grid map. In some implementations, the method 800 includes capturing environmental data corresponding to the physical environment (e.g., an image of the physical environment and / or depth data corresponding to the physical environment), and determining that the physical environment includes a physical surface based on the environmental data.

[0150] As shown in block 820, in various implementations, method 800 includes detecting one or more physical objects that occlude corresponding portions of the physical surface. Figure 4A and Figure 4B As described, electronic device 400 detects that physical objects 44, 46, and 48 occlude corresponding portions of physical surface 42. In some implementations, method 800 includes detecting physical objects based on environmental data. In some implementations, method 800 includes detecting physical objects based on a grid map of the physical environment.

[0151] As shown in block 830, in various implementations, method 800 includes synthesizing a masking element to mask one or more physical items located on a physical surface. Figure 4C As shown, the electronic device 400 synthesizes the masking element 422 onto the CGR surface 412 so as to mask the CGR objects 414 , 416 , and 418 .

[0152] refer to Figure 8B As shown in block 830a, in some implementations, method 800 includes determining one or more visual attributes of a physical surface, and configuring a masking element having one or more visual attributes of the physical surface. For example, in some implementations, electronic device 400 determines visual attribute 43 of physical surface 42, and configures visual attribute 423 of masking element 422 to be within a certain similarity of visual attribute 43 of physical surface 42.

[0153] As shown in block 830b, in some implementations, method 800 includes setting a color value of a masking element based on the color of the physical surface so that the color of the masking element is within a certain similarity of the color of the physical surface. For example, in some implementations, electronic device 400 sets a color value of masking element 422 so that the color of masking element 422 matches the color of physical surface 42.

[0154] As shown in block 830c, in some implementations, method 800 includes determining a texture of the physical surface, and setting a texture value of the masking element based on the texture of the physical surface so that the texture of the masking element is within a certain similarity of the texture of the physical surface. For example, in some implementations, electronic device 400 sets a texture value of masking element 422 so that the texture of masking element 422 matches the texture of physical surface 42.

[0155] As shown in block 830d, in some implementations, one or more visual attributes of the masking element are different from one or more visual attributes of the physical surface. Figure 4E As shown, the visual property 423a of the masking element 422a is different from the visual property 413 of the CGR surface 412.

[0156] As shown in block 830e, in some implementations, the method 800 includes blurring the entire physical surface. For example, in some implementations, the masking element 422 masks the entire CGR surface 412.

[0157] As shown in block 830f, in some implementations, method 800 includes determining the shape of one or more physical items, and synthesizing a masking element based on the shape of the one or more physical items. In some implementations, the shape of the masking element corresponds to the shape of the one or more physical items. For example, in some implementations, the masking element 422 conforms to the shape of the CGR objects 414, 416, and / or 418.

[0158] As shown in block 830g, in some implementations, the masking element includes a single masking element that masks multiple physical items in the one or more physical items. Figure 4C As shown, masking element 422 masks CGR objects 414, 416, and 418 that represent physical items 44, 46, and 48, respectively.

[0159] As shown in block 830h, in some implementations, the masking element includes a plurality of masking elements, each masking element masking a corresponding one of the one or more physical items. Figure 4F As shown, the electronic device 400 generates masking elements 424, 426, and 428 to respectively mask the CGR objects 414, 416, and 418 representing the physical items 44, 46, and 48, respectively.

[0160] refer to Figure 8C As shown in block 840, in some implementations, method 800 includes obtaining input identifying one or more physical elements to be masked. Figure 4NAs shown, electronic device 400 detects user input 462 corresponding to a request to mask CGR objects 414, 416, and 418 representing physical items 44, 46, and 48, respectively.

[0161] As shown in block 850, in some implementations, method 800 includes obtaining an instruction to place a computer generated reality (CGR) object at a location corresponding to one or more physical items, synthesizing a masking element in response to obtaining the instruction, and overlaying the CGR object on the masking element. Figure 4H As shown, the electronic device 400 obtains a request to place the fourth CGR object 440 on the CGR surface 412. Fig. 4I As shown, the electronic device 400 synthesizes the masking element 422 in response to obtaining a request to place the fourth CGR object 440 on the CGR surface 412 , and covers the fourth CGR object 440 on the masking element 422 .

[0162] As indicated at block 860, in some implementations, method 800 includes detecting an input for moving the CGR object, and simultaneously moving the masking element and the CGR object in a direction indicated by the input. Fig. 4I When the electronic device 400 detects an input to move the fourth CGR object 440 to a region of the CGR environment 410 having other CGR objects that are not currently masked, the electronic device 400 moves the masking element 422 to the region so as to mask the CGR objects in the region.

[0163] As indicated at block 870, in some implementations, method 800 includes displaying a CGR environment corresponding to a physical environment. In some implementations, the CGR environment includes CGR representations of one or more physical items. In some implementations, method 800 includes compositing a masking element onto the CGR representations of the one or more physical items. For example, Figure 4B and Figure 4C As shown, the electronic device 400 displays a CGR environment 410 corresponding to the physical environment 40. Figure 4B As shown, CGR environment 410 includes CGR objects 414, 416, and 418 that represent physical items 44, 46, and 48, respectively. Figure 4C As shown, the electronic device 400 synthesizes the masking element 422 to mask the CGR objects 414 , 416 , and 418 .

[0164] As shown in block 880, in some implementations, method 800 includes displaying a perspective of the physical environment. Figure 4BAs discussed, in some implementations, the CGR environment 410 includes a perspective of the physical environment 40. In some implementations, the method 800 includes displaying a video perspective of the physical environment. In some implementations, the method 800 includes presenting an optical perspective of the physical environment.

[0165] Fig. 9 900 is a block diagram of a device 900 for presenting / masking communication data according to some implementations. Although some specific features are shown, those of ordinary skill in the art will recognize from this disclosure that various other features are not shown for the sake of brevity and so as not to obscure more relevant aspects of the implementations disclosed herein. To this end, as a non-limiting example, in some implementations, the device 900 includes one or more processing units (CPUs) 901, a network interface 902, a programming interface 903, a memory 904, an environmental sensor 907, one or more input / output (I / O) devices 908, and one or more communication buses 905 for interconnecting these and various other components.

[0166] In some implementations, a network interface 902 is provided to, among other purposes, establish and maintain a metadata tunnel between a cloud-hosted network management system and at least one private network including one or more compatible devices. In some implementations, one or more communication buses 905 include circuits for interconnecting and controlling communications between system components. Memory 904 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices, and may include non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 904 optionally includes one or more storage devices remotely located from one or more CPUs 901. Memory 904 includes non-transitory computer-readable storage media.

[0167] In various implementations, the environmental sensor 907 includes an image sensor. For example, in some implementations, the environmental sensor 907 includes a camera (e.g., a scene-facing camera, an outward-facing camera, or a rear-facing camera). In some implementations, the environmental sensor 907 includes a depth sensor. For example, in some implementations, the environmental sensor 907 includes a depth camera.

[0168] In some implementations, one or more I / O devices 908 include a device for displaying a CGR environment (e.g., Figure 1C to Figure 1F and Figure 1H to Figure 1L The CGR environment 110 shown, Figure 2A To Figure 2D and Figure 2F to Figure 2G The CGR environment 210 shown, FIG. 3A to FIG. 3HThe CGR environment 310 shown or FIG. 4B to FIG. 4O In some implementations, the display includes a video see-through display that displays at least a portion of the physical environment surrounding the device 900 as an image captured by a scene camera. In various implementations, the display includes an optical see-through display that is at least partially transparent and passes light emitted or reflected by the physical environment.

[0169] In some implementations, the memory 904 or a non-transitory computer-readable storage medium of the memory 904 stores the following programs, modules, and data structures or a subset thereof, including an optional operating system 906, a data obtainer 910, and a CGR experience generator 920. In various implementations, the device 900 performs the methods 500, 600, 700, and / or 800. In various implementations, the device 900 implements the electronic device 100, 200, 300, and / or 400.

[0170] In some implementations, data obtainer 910 obtains data. In some implementations, data obtainer 910 obtains input (e.g., detects user input). In some implementations, data obtainer 910 obtains environmental data from environmental sensor 907. To this end, data obtainer 910 includes instructions 910a and heuristics and metadata 910b. In some implementations, CGR experience generator 920 generates and presents CGR environment 110, 210, 310 and / or 410. To this end, CGR experience generator 920 includes instructions 920a and heuristics and metadata 920b.

[0171] Although various aspects of specific implementations within the scope of the appended claims are described above, it should be apparent that the various features of the above-mentioned specific implementations can be embodied in a variety of forms, and any specific structures and / or functions described above are merely illustrative. Based on this disclosure, it should be understood by those skilled in the art that the aspects described herein can be implemented independently of any other aspects, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement a device and / or a method can be practiced. In addition, in addition to or different from one or more aspects set forth herein, other structures and / or functions can be used to implement such a device and / or such a method can be practiced.

[0172] It will also be understood that, although the terms "first", "second", etc. may be used to describe various elements in this article, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, a first node may be referred to as a second node, and similarly, a second node may be referred to as a first node, which changes the meaning of the description, as long as all occurrences of the "first node" are consistently renamed and all occurrences of the "second node" are consistently renamed. Both the first node and the second node are nodes, but they are not the same node.

[0173] The terms used herein are only for describing specific implementations and are not intended to limit the claims. As used in the description of this specific implementation and in the appended claims, the singular forms of "a", "an" and "the" are intended to also cover the plural forms, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" used herein refers to and covers any and all possible combinations of one or more items in the associated listed items. It will also be understood that the terms "include" and / or "comprising" when used in this specification specify the presence of stated features, integers, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts, and / or their grouping.

[0174] As used herein, the term “if” may be interpreted to mean “when the antecedent is true” or “when the antecedent is true” or “in response to determining” or “upon determining” or “in response to detecting” that the antecedent is true, depending on the context. Similarly, the phrase “if it is determined that [the antecedent is true]” or “if [the antecedent is true]” or “when [the antecedent is true]” is interpreted to mean “upon determining that the antecedent is true” or “in response to determining” or “upon determining” that the antecedent is true or “when detecting that the antecedent is true” or “in response to detecting” that the antecedent is true, depending on the context.

Claims

1. A method comprising: At a device comprising a display, a non-transitory memory, and one or more processors coupled to the display and the non-transitory memory: displaying a CGR object in a computer generated reality (CGR) environment, wherein the CGR object represents a physical item; synthesizing an affordance associated with the CGR object; detecting input involving the affordance; and In response to detecting the input, an identifier identifying the physical item is added to a user selection queue and the user selection queue is displayed, the user selection queue including two-dimensional (2D) representations of the physical item. 2 . The method of claim 1 , wherein the CGR environment includes representations of physical items located in a physical environment surrounding the device.

3. The method according to claim 1, further comprising: displaying a webpage including the 2D representation of the physical item; as well as An input is obtained for switching from the web page to a CGR mode in which the CGR object representing the physical item is displayed.

4. A method according to claim 1, wherein the input relating to the enabling representation includes a user selection of the enabling representation.

5. A method according to claim 1, wherein the input involving the enabling representation includes a gaze input involving the enabling representation.

6. A method according to claim 1, wherein the input related to the enabling representation includes verbal input.

7. The method of claim 1, wherein synthesizing the enable representation comprises synthesizing the enable representation within a threshold distance of the CGR object.

8. The method of claim 1, wherein synthesizing the enable representation comprises synthesizing the enable representation at a specified portion of the CGR environment.

9. The method according to claim 1, further comprising: Configure the affordance.

10. The method according to claim 1, further comprising: Changing a visual attribute of the affordance.

11. The method according to claim 1, further comprising: Change the operation associated with the affordance.

12. The method according to claim 1, further comprising: Network connectivity of the device is constrained while the affordance is displayed.

13. The method of claim 1 , wherein synthesizing the affordance comprises: determining whether the CGR object occupies at least a threshold number of pixels; as well as In response to determining that the CGR object occupies at least the threshold number of pixels, the affordance representation is synthesized.

14. The method according to claim 1, further comprising: A visual attribute of the CGR object is modified to indicate that the CGR object is selectable.

15. The method according to claim 1, further comprising: detecting an input involving the CGR object; as well as The CGR object is manipulated based on the input related to the CGR object.

16. The method according to claim 1, further comprising: while displaying the CGR object in the CGR environment, displaying a second CGR object representing a second physical item; as well as In response to detecting the input involving the affordance associated with the CGR object and the second CGR object, adding an identifier identifying the second physical item to the user selection queue.

17. The method according to claim 16, further comprising: The second physical item is identified by a recommendation engine based on the physical item.

18. The method according to any one of claims 1 to 17, further comprising: displaying a replacement affordance associated with the CGR object, wherein the replacement affordance allows replacement of the CGR object with a third CGR object representing a third physical item; detecting input involving the replacement affordance; and In response to detecting the input involving the replacement affordance, replacing the CGR object with the third CGR object.

19. A device comprising: one or more processors; monitor; non-transitory memory; as well as one or more programs stored in the non-transitory memory, which, when executed by the one or more processors, cause the device to: displaying a CGR object in a computer generated reality (CGR) environment, wherein the CGR object represents a physical item; synthesizing an affordance associated with the CGR object; detecting input involving the affordance; and In response to detecting the input, an identifier identifying the physical item is added to a user selection queue and the user selection queue is displayed, the user selection queue including two-dimensional (2D) representations of the physical item.

20. A non-transitory memory storing one or more programs that, when executed by one or more processors of a device having a display, cause the device to: displaying a CGR object in a computer generated reality (CGR) environment, wherein the CGR object represents a physical item; synthesizing an affordance associated with the CGR object; detecting input involving the affordance; and In response to detecting the input, an identifier identifying the physical item is added to a user selection queue and the user selection queue is displayed, the user selection queue including two-dimensional (2D) representations of the physical item.

Images