Presenting virtual elements associated with product in environment

Abstract

Some examples of the present disclosure relate to systems and methods for performing one or more actions in response to detecting a product in a physical environment. For example, performing the one or more actions includes presenting the one or more virtual elements in the environment. For example, performing the one or more actions includes storing information associated with the product. In some examples, the one or more actions are performed in accordance with one or more criteria being satisfied. For example, the one or more criteria include criteria that are satisfied when one or more indications of user interest in the product are detected. In some examples, one or more virtual elements indicative of a location of a product within an area of a physical environment are presented in the environment. The method of the present disclosure is optionally performed at an electronic device that includes a head-mounted display.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Application No. 63 / 586,965 filed on September 29, 2023, U.S. Provisional Application No. 63 / 686,612 filed on August 23, 2024, and U.S. Patent Application No. 18 / 890,601 filed on September 19, 2024, the contents of which are incorporated herein by reference in their entirety for all purposes. Technical Field

[0003] The present disclosure generally relates to systems and methods for presenting virtual elements in an environment. Background Art

[0004] Some computer graphics environments provide two-dimensional and / or three-dimensional environments in which at least some of the objects displayed for viewing by a user are virtual and generated by a computer. For example, a virtual object can be viewed simultaneously with one or more objects of a physical environment in the environment. Summary of the invention

[0005] Some examples of the present disclosure relate to systems and methods for performing one or more actions associated with a product in a physical environment. For example, the one or more actions include presenting one or more virtual elements associated with the product in the environment. In some examples, at an electronic device that communicates with one or more displays and one or more input devices, the electronic device detects a first product in the physical environment of a user of the electronic device via one or more input devices. In some examples, based on determining that one or more first criteria are met, the electronic device performs one or more actions, the one or more first criteria including a first criterion that is met when the electronic device detects one or more indications of interest of the user of the electronic device in the first product.

[0006] In some examples, the first product is a commodity included in a physical environment. For example, the first product includes different types of hardware, household goods, and / or groceries. In some examples, detecting one or more indications of interest includes detecting a user's attention to the first product. In some examples, detecting one or more indications of interest includes detecting a gesture performed by a user on the first product. In some examples, detecting one or more indications of interest includes detecting a user interaction with the first product in the physical environment. In some examples, one or more actions include presenting a first virtual element associated with the first product in the environment. In some examples, the first virtual element includes one or more visual indications of one or more areas (e.g., stores / businesses) where the first product is available for purchase. In some examples, one or more actions include adding information associated with the first product to an information repository associated with a corresponding application. For example, the electronic device adds the first product to a shopping list included in a file of the corresponding application.

[0007] In some examples, at an electronic device in communication with one or more displays and one or more input devices, the electronic device detects a location of the electronic device in a physical environment. In some examples, based on determining that the location of the electronic device satisfies one or more first criteria, the electronic device presents in the environment, via the one or more displays, one or more first virtual elements indicating a first location of a first product within a first area of ​​the physical environment, the one or more first criteria including a first criterion that is satisfied when the location of the electronic device is within a first area of ​​the physical environment that includes a first location corresponding to the first product, wherein the first product is associated with one or more indications of a user's intent to purchase the first product, and the one or more indications of the user's intent are detected by the electronic device.

[0008] In some examples, the first area of ​​the physical environment corresponds to a store, and the first location corresponding to the first product corresponds to a location within the store. In some examples, the one or more indications of user intent detected by the electronic device include detected attention of a user of the electronic device to the first product, user interaction with the first product, and / or identified information associated with the first product included in a file of an application. In some examples, the electronic device detects the one or more indications of user intent at a second location outside of the first area of ​​the physical environment. For example, the electronic device detects the one or more indications of user intent at a location of the physical environment that is not within a store where the first product is available for purchase.

[0009] A comprehensive description of these examples is provided in the drawings and detailed description, and it should be understood that this summary does not limit the scope of the present disclosure in any way. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] For a better understanding of the various examples described herein, reference should be made to the following detailed description and the following drawings. Throughout the drawings, like reference numerals generally refer to corresponding parts.

[0011] Figure 1 An electronic device presenting an extended reality environment according to some examples of the present disclosure is illustrated.

[0012] Figure 2 A block diagram illustrating an example architecture of a device according to some examples of the present disclosure.

[0013] FIG. 3A to FIG. 3I An exemplary method for presenting one or more virtual elements associated with a product in an environment according to some examples of the present disclosure is illustrated.

[0014] Figures 4A to 4K An exemplary method for presenting one or more virtual elements associated with one or more products in an environment based on a location of an electronic device according to some examples of the present disclosure is illustrated.

[0015] Figure 5 A flow chart illustrating an example process for presenting one or more virtual elements associated with a product in an environment according to some examples of the present disclosure.

[0016] Figure 6 A flow chart illustrating an example process for presenting one or more virtual elements associated with one or more products in an environment based on a location of an electronic device according to some examples of the present disclosure. DETAILED DESCRIPTION

[0017] Some examples of the present disclosure relate to systems and methods for performing one or more actions associated with a product in a physical environment. For example, the one or more actions include presenting one or more virtual elements associated with the product in the environment. In some examples, at an electronic device that communicates with one or more displays and one or more input devices, the electronic device detects a first product in the physical environment of a user of the electronic device via one or more input devices. In some examples, based on determining that one or more first criteria are met, the electronic device performs one or more actions, the one or more first criteria including a first criterion that is met when the electronic device detects one or more indications of interest of the user of the electronic device in the first product.

[0018] In some examples, the first product is a commodity included in a physical environment. For example, the first product includes different types of hardware, household goods, and / or groceries. In some examples, detecting one or more indications of interest includes detecting a user's attention to the first product. In some examples, detecting one or more indications of interest includes detecting a gesture performed by a user on the first product. In some examples, detecting one or more indications of interest includes detecting a user interaction with the first product in the physical environment. In some examples, one or more actions include presenting a first virtual element associated with the first product in the environment. In some examples, the first virtual element includes one or more visual indications of one or more areas (e.g., stores / businesses) where the first product is available for purchase. In some examples, one or more actions include adding information associated with the first product to an information repository associated with a corresponding application. For example, the electronic device adds the first product to a shopping list included in a file of the corresponding application.

[0019] In some examples, at an electronic device in communication with one or more displays and one or more input devices, the electronic device detects a location of the electronic device in a physical environment. In some examples, based on determining that the location of the electronic device satisfies one or more first criteria, the electronic device presents in the environment, via the one or more displays, one or more first virtual elements indicating a first location of a first product within a first area of ​​the physical environment, the one or more first criteria including a first criterion that is satisfied when the location of the electronic device is within a first area of ​​the physical environment that includes a first location corresponding to the first product, wherein the first product is associated with one or more indications of a user's intent to purchase the first product, and the one or more indications of the user's intent are detected by the electronic device.

[0020] In some examples, the first area of ​​the physical environment corresponds to a store, and the first location corresponding to the first product corresponds to a location within the store. In some examples, the one or more indications of user intent detected by the electronic device include detected attention of a user of the electronic device to the first product, user interaction with the first product, and / or identified information associated with the first product included in a file of an application. In some examples, the electronic device detects the one or more indications of user intent at a second location outside of the first area of ​​the physical environment. For example, the electronic device detects the one or more indications of user intent at a location of the physical environment that is not within a store where the first product is available for purchase.

[0021] In some examples, a three-dimensional object is displayed in a computer-generated three-dimensional environment at a particular orientation that controls one or more behaviors of the three-dimensional object (e.g., when the three-dimensional object moves within the three-dimensional environment). In some examples, the orientation in which the three-dimensional object is displayed in the three-dimensional environment is selected by a user of the electronic device or automatically selected by the electronic device. For example, when initiating presentation of the three-dimensional object in the three-dimensional environment, the user may select a particular orientation of the three-dimensional object, or the electronic device may automatically select the orientation of the three-dimensional object (e.g., based on the type of the three-dimensional object).

[0022] In some examples, a three-dimensional object may be displayed in a three-dimensional environment in a world-locked orientation, a body-locked orientation, a tilt-locked orientation, or a head-locked orientation, as described below. As used herein, an object displayed in a three-dimensional environment in a body-locked orientation has a distance and orientation offset relative to a portion of the user's body (e.g., the user's torso). Alternatively, in some examples, the body-locked object has a fixed distance from the user, and the orientation of the content does not reference any part of the user's body (e.g., may be displayed in the same basic direction relative to the user regardless of head and / or body movement). Additionally or alternatively, in some examples, the body-locked object may be configured to always maintain gravity or horizon (e.g., perpendicular to gravity) alignment, so that changes in the head and / or body in the rolling direction will not cause the body-locked object to move within the three-dimensional environment. Instead, translational movement in either configuration will cause the body-locked object to reposition within the three-dimensional environment to maintain the distance offset.

[0023] As used herein, an object displayed in a head-locked orientation in a three-dimensional environment has a distance and orientation offset relative to the user's head. In some examples, the head-locked object moves within the three-dimensional environment when the user's head moves (when the user's viewpoint changes).

[0024] As used herein, an object displayed in a world-locked orientation in a three-dimensional environment has no distance or orientation offset relative to the user.

[0025] As used herein, an object displayed in a three-dimensional environment with a tilt-locked orientation (referred to herein as a tilt-locked object) has a distance offset relative to a user, such as a portion of a user's body (e.g., a user's torso) or a user's head. In some examples, the tilt-locked object is displayed with a fixed orientation relative to the three-dimensional environment. In some examples, the tilt-locked object moves according to a polar (e.g., spherical) coordinate system centered on a pole passing through the user (e.g., the user's head). For example, the tilt-locked object is moved in the three-dimensional environment based on the movement of the user's head in a spherical space around the user's head (e.g., centered on the user's head). Thus, if the user tilts his head relative to gravity (e.g., up or down in a pitch direction), the tilt-locked object will follow the head tilt and move radially along the sphere, so that the tilt-locked object is repositioned within the three-dimensional environment to the same distance offset relative to the user as before the head tilt, while optionally maintaining the same orientation relative to the three-dimensional environment. In some examples, if the user moves his head in a rolling direction (e.g., clockwise or counterclockwise) relative to gravity, the tilt-locked object is not repositioned within the three-dimensional environment.

[0026] Figure 1 An electronic device 101 is illustrated that presents an extended reality (XR) environment (e.g., a computer-generated environment that optionally includes representations of physical and / or virtual objects) according to some examples of the present disclosure. In some examples, such as Figure 1 As shown, the electronic device 101 is a head mounted display or other head mounted device configured to be worn on the head of the user of the electronic device 101. Figure 2 An example of an electronic device 101 is described by using an architectural block diagram of FIG. Figure 1 As shown, electronic device 101 and table 106 are located in a physical environment. The physical environment may include physical features, such as physical surfaces (e.g., floors, walls) or physical objects (e.g., tables, lights, etc.). In some examples, electronic device 101 may be configured to detect and / or capture an image of the physical environment including table 106 (shown in the field of view of electronic device 101).

[0027] In some examples, such as Figure 1 As shown, the electronic device 101 includes one or more internal image sensors 114a oriented toward the user's face (e.g., as described below with reference to Figure 2101 ). In some examples, internal image sensor 114a is used for eye tracking (e.g., detecting the gaze of a user). Internal image sensor 114a is optionally arranged on the left and right portions of display 120 to enable eye tracking of the left and right eyes of the user. In some examples, electronic device 101 also includes external image sensors 114b and 114c facing outward from the user to detect and / or capture the physical environment of electronic device 101 and / or movement of the user's hands or other body parts.

[0028] In some examples, the display 120 has a field of view visible to the user (e.g., it may or may not correspond to the field of view of the external image sensors 114b and 114c). Because the display 120 is optionally part of the head-mounted device, the field of view of the display 120 is optionally the same as or similar to the field of view of the user's eyes. In other examples, the field of view of the display 120 may be smaller than the field of view of the user's eyes. In some examples, the electronic device 101 may be an optical see-through device, in which the display 120 is a transparent or translucent display through which parts of the physical environment can be directly viewed. In some examples, the display 120 may be included in a transparent lens and may overlap with all or only a portion of the transparent lens. In other examples, the electronic device may be a video pass-through device, in which the display 120 is an opaque display configured to display an image of the physical environment captured by the external image sensors 114b and 114c.

[0029] In some examples, in response to a trigger, the electronic device 101 may be configured to display in the XR environment Figure 1 The cube shown represents a virtual object 104 that does not exist in the physical environment but is displayed in the XR environment positioned on top of a real-world table 106 (or a representation thereof). Optionally, in response to detecting the plane of the table 106 in the physical environment 100, the virtual object 104 can be displayed on the surface of the table 106 in the XR environment displayed via the display 120 of the electronic device 101.

[0030] It should be understood that virtual object 104 is a representative virtual object and may include and render one or more different virtual objects (e.g., virtual objects with various dimensions, such as two-dimensional or other three-dimensional virtual objects) in a three-dimensional XR environment. For example, a virtual object may represent an application or user interface displayed in an XR environment. In some examples, a virtual object may represent content corresponding to an application and / or displayed via a user interface in an XR environment. In some examples, virtual object 104 is optionally configured to be interactive and responsive to user input (e.g., air gestures, such as air pinch gestures, air tap gestures, and / or air touch gestures), so that a user virtually touches, taps, moves, rotates, or otherwise interacts with virtual object 104.

[0031] In some examples, displaying an object in a three-dimensional environment may include interaction with one or more user interface objects in the three-dimensional environment. For example, initiation of display of an object in a three-dimensional environment may include interaction with one or more virtual options / indicators displayed in the three-dimensional environment. In some examples, when display of an object is initiated in a three-dimensional environment, an electronic device may track the user's gaze as an input for identifying one or more virtual options / indicators as targets for selection. For example, gaze may be used to identify one or more virtual options / indicators as targets for selection using another selection input. In some examples, a virtual option / indicator may be selected using hand tracking input detected via an input device that communicates with the electronic device. In some examples, an object displayed in a three-dimensional environment may be moved and / or reoriented in the three-dimensional environment based on movement input detected via an input device.

[0032] In the following discussion, an electronic device that communicates with a display generation component and one or more input devices is described. It should be understood that the electronic device optionally communicates with one or more other physical user interface devices, such as a touch-sensitive surface, a physical keyboard, a mouse, a joystick, a hand tracking device, an eye tracking device, a stylus, etc. In addition, as described above, it should be understood that the described electronic device, display, and touch-sensitive surface are optionally distributed between two or more devices. Therefore, as used in the present disclosure, information displayed on or by an electronic device is optionally used to describe information output by the electronic device to be displayed on an independent display device (touch-sensitive or non-touch-sensitive). Similarly, as used in the present disclosure, input received on an electronic device (e.g., touch input received on a touch-sensitive surface of an electronic device, or touch input received on the surface of a stylus) is optionally used to describe input received on an independent input device, and the electronic device receives input information from the independent input device.

[0033] The device typically supports a variety of applications, such as one or more of the following applications: a drawing application, a presentation application, a word processing application, a website creation application, a disk editing application, a spreadsheet application, a gaming application, a telephony application, a video conferencing application, an email application, an instant messaging application, a fitness support application, a photo management application, a digital camera application, a digital video camera application, a web browsing application, a digital music player application, a television channel browsing application, and / or a digital video player application.

[0034] Figure 2 A block diagram of an example architecture of a device 201 according to some examples of the present disclosure is illustrated. In some examples, the device 201 includes one or more electronic devices. For example, the electronic device 201 can be a portable device, an auxiliary device that communicates with another device, a head-mounted display, etc. In some examples, the electronic device 201 corresponds to the above reference Figure 1 An electronic device 101 is described.

[0035] like Figure 2 As illustrated, the electronic device 201 optionally includes various sensors, such as one or more hand tracking sensors 202, one or more position sensors 204, one or more image sensors 206 (optionally corresponding to Figure 1 114a and / or external image sensors 114b and 114c in the embodiment of the present invention), one or more touch-sensitive surfaces 209, one or more motion and / or orientation sensors 210, one or more eye tracking sensors 212, one or more microphones 213 or other audio sensors, one or more body tracking sensors (e.g., torso and / or head tracking sensors), one or more display generation components 214 (optionally corresponding to Figure 1 The display 120 in the electronic device 201, one or more speakers 216, one or more processors 218, one or more memories 220 and / or communication circuits 222. One or more communication buses 208 are optionally used for communication between the above components of the electronic device 201.

[0036] The communication circuit 222 optionally includes circuits for communicating with electronic devices, networks such as the Internet, intranets, wired networks and / or wireless networks, cellular networks, and wireless local area networks (LANs). The communication circuit 222 optionally includes circuits for using near field communication (NFC) and / or short-range communication such as A circuit for communicating.

[0037] Processor 218 includes one or more general purpose processors, one or more graphics processors, and / or one or more digital signal processors. In some examples, memory 220 is a non-transitory computer-readable storage medium (e.g., flash memory, random access memory, or other volatile or non-volatile memory or storage device) storing computer-readable instructions, which are configured to be executed by processor 218 to perform the techniques, processes, and / or methods described below. In some examples, memory 220 may include more than one non-transitory computer-readable storage medium. A non-transitory computer-readable storage medium may be any medium (e.g., excluding signals) that can tangibly contain or store computer-executable instructions for use by or in combination with an instruction execution system, device, and apparatus. In some examples, the storage medium is a transient computer-readable storage medium. In some examples, the storage medium is a non-transitory computer-readable storage medium. A non-transitory computer-readable storage medium may include, but is not limited to, a magnetic storage device, an optical storage device, and / or a semiconductor storage device. Examples of such storage devices include magnetic disks, optical disks based on compact disc (CD), digital versatile disc (DVD) or Blu-ray technology, and persistent solid-state memory (such as flash memory, solid-state drives, etc.).

[0038] In some examples, display generation component 214 includes a single display (e.g., a liquid crystal display (LCD), an organic light emitting diode (OLED), or other type of display). In some examples, display generation component 214 includes multiple displays. In some examples, display generation component 214 may include a display with touch capabilities (e.g., a touch screen), a projector, a holographic projector, a retinal projector, a transparent or translucent display, etc. In some examples, electronic device 201 includes touch-sensitive surfaces 209 for receiving user inputs (such as tap inputs and swipe inputs or other gestures). In some examples, display generation component 214 and touch-sensitive surface 209 form a touch-sensitive display (e.g., a touch screen integrated with electronic device 201 or a touch screen that communicates with electronic device 201 outside electronic device 201).

[0039] The electronic device 201 optionally includes an image sensor 206. The image sensor 206 optionally includes one or more visible light image sensors (such as a charge coupled device (CCD) sensor) and / or a complementary metal oxide semiconductor (CMOS) sensor operable to obtain an image of a physical object from a real-world environment. The image sensor 206 also optionally includes one or more infrared (IR) sensors, such as a passive IR sensor or an active IR sensor, for detecting infrared light from the real-world environment. For example, an active IR sensor includes an IR emitter for emitting infrared light into the real-world environment. The image sensor 206 also optionally includes one or more cameras configured to capture the movement of physical objects in the real-world environment. The image sensor 206 also optionally includes one or more depth sensors configured to detect the distance of the physical object from the electronic device 201. In some examples, information from the one or more depth sensors may allow the device to identify an object in the real-world environment and distinguish it from other objects in the real-world environment. In some examples, the one or more depth sensors may allow the device to determine the texture and / or topography of an object in the real-world environment.

[0040] In some examples, the electronic device 201 uses a combination of a CCD sensor, an event camera, and a depth sensor to detect the physical environment around the electronic device 201. In some examples, the image sensor 206 includes a first image sensor and a second image sensor. The first image sensor and the second image sensor work in tandem and are optionally configured to capture different information of physical objects in the real-world environment. In some examples, the first image sensor is a visible light image sensor and the second image sensor is a depth sensor. In some examples, the electronic device 201 uses the image sensor 206 to detect the positioning and orientation of the electronic device 201 and / or the display generation component 214 in the real-world environment. For example, the electronic device 201 uses the image sensor 206 to track the positioning and orientation of the display generation component 214 relative to one or more fixed objects in the real-world environment.

[0041] In some examples, the electronic device 201 includes a microphone 213 or other audio sensor. The electronic device 201 optionally uses the microphone 213 to detect sounds from the user and / or the user's real-world environment. In some examples, the microphone 213 includes a microphone array (multiple microphones) that optionally operate in conjunction to identify ambient noise or locate a sound source in the space of the real-world environment.

[0042] The electronic device 201 includes a location sensor 204 for detecting the location of the electronic device 201 and / or the display generating component 214. For example, the location sensor 204 may include a global positioning system (GPS) receiver that receives data from one or more satellites and allows the electronic device 201 to determine the absolute location of the device in the physical world.

[0043] The electronic device 201 includes an orientation sensor 210 for detecting the orientation and / or movement of the electronic device 201 and / or the display generation component 214. For example, the electronic device 201 uses the orientation sensor 210 to track changes in the position and / or orientation of the electronic device 201 and / or the display generation component 214, such as relative to physical objects in the real world environment. The orientation sensor 210 optionally includes one or more gyroscopes and / or one or more accelerometers.

[0044] In some examples, the electronic device 201 includes a hand tracking sensor 202 and / or an eye tracking sensor 212 and / or other body tracking sensors, such as a leg tracking sensor, a torso tracking sensor, and / or a head tracking sensor. The hand tracking sensor 202 is configured to track the position / location of one or more parts of the user's hand, and / or the movement of one or more parts of the user's hand relative to the extended reality environment, relative to the display generation component 214, and / or relative to another defined coordinate system. The eye tracking sensor 212 is configured to track the position and movement of the user's gaze (more generally, eyes, face, or head) relative to the real world or extended reality environment and / or relative to the display generation component 214. In some examples, the hand tracking sensor 202 and / or the eye tracking sensor 212 are implemented together with the display generation component 214. In some examples, the hand tracking sensor 202 and / or the eye tracking sensor 212 are implemented separately from the display generation component 214.

[0045] In some examples, the hand tracking sensor 202 (and / or other body tracking sensors, such as leg, torso, and / or head tracking sensors) may use an image sensor 206 (e.g., one or more IR cameras, 3D cameras, depth cameras, etc.) that captures three-dimensional information from the real world including one or more body parts (e.g., the hands, legs, torso of a human user). In some examples, the hands may be resolved with sufficient resolution to distinguish fingers and their corresponding positioning. In some examples, one or more image sensors 206 are positioned relative to the user to define a field of view and an interaction space of the image sensor 206, in which the finger / hand position, orientation, and / or movement captured by the image sensor is used as input (e.g., to distinguish from the user's resting hand or other hands of other people in the real-world environment). Tracking fingers / hands for input (e.g., gestures, touches, taps, etc.) may be advantageous because it does not require the user to touch, hold, or wear any type of beacon, sensor, or other marker.

[0046] In some examples, eye tracking sensor 212 includes at least one eye tracking camera (e.g., an infrared (IR) camera) and / or an illumination source (e.g., an IR light source such as an LED) that emits light toward the user's eyes. The eye tracking camera may be pointed toward the user's eyes to receive reflected IR light from the light source directly or indirectly from the eyes. In some examples, both eyes are tracked separately by corresponding eye tracking cameras and illumination sources, and focus / gaze may be determined by tracking both eyes. In some examples, one eye (e.g., the dominant eye) is tracked by one or more corresponding eye tracking cameras / illumination sources.

[0047] The electronic device 201 is not limited to Figure 2 Rather than components and configurations of the electronic device 201, it may include fewer components, other components, or additional components in a variety of configurations. In some examples, the electronic device 201 may be implemented between two electronic devices (e.g., as a system). In some such examples, each (or more) electronic device may each include one or more of the same components discussed above, such as various sensors, one or more display generating components, one or more speakers, one or more processors, one or more memories, and / or communication circuits. One or more people using the electronic device 201 are optionally referred to herein as one or more users of the device.

[0048] Now focus on an electronic device (e.g., corresponding to electronic device 201) that performs one or more actions in response to detecting one or more products in a physical environment. For example, the electronic device presents one or more virtual elements associated with one or more products in an environment (e.g., a three-dimensional environment). For example, the electronic device stores information associated with the one or more products in an information repository associated with a corresponding application. In some examples, when one or more indications of interest in a product by a user of the electronic device are detected, the electronic device performs one or more actions. In some examples, when the location of the electronic device is within an area of ​​the physical environment that includes one or more locations corresponding to the one or more products, the electronic device presents one or more virtual elements associated with the product in the environment.

[0049] FIG. 3A to FIG. 3IAn exemplary method for presenting one or more virtual elements associated with a product having a quantity less than a threshold amount in an environment according to some examples of the present disclosure is illustrated. Specifically, the one or more virtual elements are presented in the environment by an electronic device 302. In some examples, the electronic device 302 has one or more characteristics of the electronic device 101 and / or the electronic device 201 as described above. In some examples, the electronic device 302 is a head-mounted display that includes a display generation component 330 (e.g., using a transparent and / or translucent display) that presents the environment 304 to a user (e.g., or optionally, one or more display generation components having one or more characteristics of the display generation component 214 as described above). In some examples, the electronic device 302 includes one or more image sensors 314a-314c (e.g., image sensors 314a-314c include one or more characteristics of image sensors 114a-114c and / or image sensor 206 as described above), which are configured to detect a physical environment (e.g., having one or more characteristics of a real-world environment and / or a physical environment as described above) and / or movement of one or more parts of a user (e.g., hands, head, and / or eyes) and / or a user's attention (e.g., gaze). In some examples, the electronic device 302 includes one or more hardware input devices. For example, the electronic device 302 includes hardware buttons 360a and 360b that can be actuated by a user of the electronic device 302 (e.g., when wearing the electronic device 302). In some examples, the one or more hardware input devices include one or more buttons, knobs, dials, switches, crowns, touch screens (e.g., touch-sensitive displays), trackpads, and / or keyboards. In some examples, the electronic device 302 is a head-mounted camera device that communicates with a display. For example, electronic device 302 includes one or more image sensors 314a-314c configured to detect a physical environment and communicate with a second electronic device having a display component (e.g., a mobile phone, a tablet computer, a wearable device (e.g., a smart watch), and / or a laptop computer).

[0050] In some examples, FIG. 3A to FIG. 3I The environment 304 shown is a three-dimensional environment presented to the user of the electronic device 302 by the display generation component 330 (for example, the display generation component 330 is a transparent or semi-transparent display). In some examples, the environment 304 is an extended reality (XR) environment having one or more characteristics of the above-mentioned XR environment. For example, from the current viewpoint of the user of the electronic device 302, one or more virtual elements (for example, reference elements) from the physical environment of the user of the electronic device 302 Figure 3E The virtual element 318a shown and described above and / or one or more physical objects (e.g., reference Figure 1The real-world environment 106 shown and described is visible (e.g., through video penetration or optical see-through of the physical environment visible to a user of the electronic device 302 via the display generation component 330). In some examples, the environment 304 is a virtual reality environment (e.g., the environment 304 is fully or partially immersive (e.g., the user of the electronic device 302 controls the level of virtual immersion through one or more input devices of the electronic device 302)).

[0051] In some examples, one or more virtual elements (e.g., reference Figure 3E The virtual elements 318a) shown and described are displayed by a display component in communication with the electronic device 302 (e.g., rather than presenting one or more virtual elements in a virtual environment such as the environment 304). For example, the electronic device 302 detects one or more inputs using the image sensors 314a-314c (e.g., by detecting the physical environment), and in response to detecting the one or more inputs, one or more virtual elements (e.g., virtual elements 318a) are displayed on the display component (e.g., content is displayed on a display of a mobile phone in communication with the electronic device 302). For example, the electronic device 302 does not include one or more displays (e.g., display generation component 330), and does not present a three-dimensional environment (e.g., environment 304) to a user of the electronic device 302.

[0052] Figure 3A A product 306 is illustrated that is visible to a user of the electronic device 302 via the display generation component 330. In some examples, the product 306 is included in a physical environment that is visible to the user of the electronic device 302 via video penetration or optical see-through. Figure 3AAs shown, product 306 is a pack of screws. It should be understood that in some examples, products may include different types of hardware (e.g., nails) or other goods, such as household items (e.g., soap, detergent, light bulbs and / or batteries), toiletries (e.g., shampoo, shaving cream, toothpaste and / or medication), beverages, condiments, spices, paper, wrappings, fruits, vegetables and / or other types of groceries. In some examples, electronic device 302 detects product 306 (e.g., product 306 is within the field of view of image sensors 314a-314c of electronic device 302). In some examples, image and / or object recognition (e.g., using image sensors 314a-314c) is used to detect product 306. For example, electronic device 302 identifies the packaging of product 306 (e.g., including the size, shape, color and / or label on the packaging). For example, electronic device 302 identifies product 306 by detecting one or more objects of product 306 (e.g., electronic device 302 detects one or more screws included inside a package and / or identifies product 306 based on the one or more screws). In some examples, electronic device 302 identifies product 306 independently of the package (e.g., electronic device 302 identifies a hardware product by detecting one or more pieces of hardware on a table, and / or identifies a grocery product by one or more pieces of goods arranged in a refrigerator or cabinet). In some examples, electronic device 302 identifies product 306 via a barcode and / or other scannable image (e.g., a QR code) on the package of product 306. In some examples, electronic device 302 can identify that a user of electronic device 302 is interested in product 306 by interacting with the user of product 306 in a physical environment. For example, electronic device 302 identifies that the user is interested in product 306 by detecting that the user of electronic device 302 is paying attention to (e.g., looking at) product 306 (e.g., for more than a threshold period of time, such as 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, 2, 5 seconds, or 10 seconds). In some examples, electronic device 302 identifies that the user is interested in product 306 by detecting hand-based input (e.g., as shown in FIG. 1 ). Figure 3B The electronic device 302 may identify that the user of the electronic device 302 is interested in the product 306 (e.g., after detecting the product 306) based on a user history (e.g., the user of the electronic device 302 has searched for the product 306 in a web browser, included information associated with the product 306 in a file (e.g., a note in a note application), and / or has previously purchased the product 306).

[0053] In some examples, the quantity (e.g., estimated quantity) of the product can be determined by the electronic device 302. For example, when detecting the product 306, the electronic device 302 identifies the number of screws included in the package (e.g., Figure 3A As shown, two screws are visible in the packaging of product 306). In some examples, electronic device 302 identifies an estimated number of screws within the packaging of product 306. In some examples, electronic device 302 identifies the quantity (e.g., estimated quantity) of product 306 independently of the packaging (e.g., electronic device 302 identifies the number of screws by detecting the number of screws arranged on a table). In some examples, based on the quantity and / or estimated quantity of product 306 being less than a threshold amount (e.g., optionally, in addition to determining that the user of electronic device 302 is interested in product 306 as described above), electronic device 302 performs an action, such as adding information associated with product 306 to an information repository and / or file associated with a corresponding application (e.g., electronic device 302 adds product 306 to a shopping list in a notation application, such as the following reference). Figure 3G ) or presenting one or more virtual elements associated with product 306 in environment 304 (e.g., as shown and described in reference Figure 3D-1 to Figure 3F306 and / or other products that include one or more objects within a package). For example, the threshold amount is scaled based on the product detected by the electronic device 302 (e.g., if the complete packaging of the product 306 includes 30 screws, the threshold amount is an amount less than 30 screws (e.g., 5, 10, or 15 screws), and if the detected product is a packaging of batteries and the complete packaging includes 6 batteries, the threshold amount is an amount less than 6 batteries (e.g., 3 or 2 batteries)). Alternatively, the threshold amount is optionally a percentage of the total amount of the product (e.g., corresponding to the number of objects within the packaging and / or the volume of the product within the container) (e.g., less than 1%, 2%, 5%, 10%, 15%, 20%, 25%, 30%, 35%, 40%, 45%, or 50%). For example, the product is tape, and the threshold amount corresponds to the thickness of the tape roll (e.g., the thickness of the tape roll is detected using image sensors 314a-314c). Additionally or alternatively, in some examples, the threshold amount is a threshold sound level (e.g., in decibels) associated with the sound emitted from the product (e.g., as described in reference to Figure 3B In some examples, the electronic device 302 receives an incoming call via one or more user inputs (e.g., an air gesture (e.g., as shown and described in reference Figure 3C shown and described), physical interaction with the product (e.g., as referenced Figure 3B The electronic device 302 may determine that the quantity of the product 306 is less than a threshold amount, based on one or more user inputs used by the electronic device 302 to determine that the quantity of the product 306 is less than the threshold amount. For example, based on one or more user inputs used by the electronic device 302 to determine that the quantity of the product 306 is less than the threshold amount, the electronic device 302 may optionally determine that the user of the electronic device 302 is interested in purchasing the product 306. In some examples, one or more virtual elements presented in the environment 304 include information about a location (e.g., a business) where the product 306 can be purchased (e.g., as referenced in FIG. 1 ). Figure 3E shown and described).

[0054] Adding information associated with one or more products to an information repository and / or file associated with a corresponding application based on detecting one or more indications of user interest in the one or more products (e.g., adding product 306 to a shopping list) minimizes the amount of user input required to add the one or more products to the information repository and / or file (e.g., because the electronic device 302 automatically adds the information in response to detecting the one or more indications of interest), thereby improving user-device interaction and conserving computing resources associated with additional user input. Adding information associated with one or more products to the information repository and / or file based on determining that the quantity (and / or estimated quantity) of the one or more products is less than a threshold amount ensures that the one or more products are added to the information repository and / or file when replenishment of the one or more products is needed, thereby reducing errors in user-device interaction and conserving computing resources associated with user input to correct errors.

[0055] Figure 3B The electronic device 302 is illustrated as detecting an interaction of a user of the electronic device 302 with a product 306 in a physical environment (eg, the user's interaction with the product 306 in the physical environment is visible via a display generation component 330). Figure 3B As shown, electronic device 302 detects that a hand 308 of a user of electronic device 302 lifts (e.g., grabs, holds, and / or moves) product 306. Additionally, the user's interaction with product 306 causes sound 310 to be emitted from product 306. For example, interaction with product 306 causes one or more objects (e.g., screws) within the package to move (e.g., the user shakes the package, which causes one or more screws to contact the interior of the package, thereby generating a sound that can be detected by electronic device 302). In some examples, electronic device 302 uses one or more audio sensors (e.g., with reference to Figure 2 In some examples, the electronic device 302 detects the product 306 using the audio input (e.g., in addition to using the image sensors 314a-314c). In some examples, based on the detected audio input, the electronic device 302 identifies that the quantity of the product 306 is less than a threshold quantity (e.g., the electronic device 302 compares the audio input to one or more audio files (e.g., corresponding to the voice print) associated with the product 306 (e.g., the one or more audio files are stored in a memory of the electronic device 302 and / or retrieved from a network). ... Figure 3B As shown, the user interacts with the product 306, and the electronic device 302 determines the number of objects within the packaging of the product 306. For example, lifting and / or turning the product 306 enables the electronic device 302 to detect the number of screws within the packaging of the product 306 (e.g., Figure 3A Compared with, relative to Figure 3B The product 306 is viewed from different angles and / or distances of the electronic device 302).

[0056] In some examples, the electronic device 302 is based on Figure 3B The detected user interaction with product 306 shown determines that the user of electronic device 302 is interested in product 306. For example, electronic device 302 determines whether the user of electronic device 302 intends to purchase product 306 (e.g., optionally in addition to determining that the quantity of product 306 is less than a threshold amount). Figure 3B , electronic device 302 associates the detected interaction with product 306 as an indication of user interest in product 306. For example, detecting an indication of user interest in product 306 includes detecting attention (e.g., gaze) of a user of electronic device 302 to product 306 (e.g., for a threshold time period (e.g., 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, 2 seconds, 5 seconds, or 10 seconds)). For another example, detecting an indication of user interest in product 306 includes detecting user interaction with product 306 (e.g., corresponding to detecting lifting (e.g., for a threshold time period (e.g., 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, 2 seconds, 5 seconds, or 10 seconds)), rotating, flipping, twisting, shaking, and / or moving product 306). In some examples, detecting an indication of user interest in product 306 includes detecting an air gesture performed by the user (e.g., as described in reference to Figure 3C In some examples, in response to detecting an indication of user interest in product 306 (e.g., detecting user interaction with product 306, such as gaze and / or hand-based input), electronic device 302 stores information associated with product 306 (e.g., in a file of an application).

[0057] Figure 3C A user of electronic device 302 is illustrated performing an air gesture using hand 308. In some examples, the air gesture is detected as being toward product 306. In some examples, the air gesture corresponds to an indication of user interest in product 306. In some examples, the air gesture corresponds to a predefined gesture (e.g., via one or more system settings and / or one or more user settings) for indicating that the user of electronic device 302 is interested in purchasing product 306 and / or for indicating that the quantity of product 306 is less than a threshold amount. In some examples, in response to detecting that the user of electronic device 302 is interested in purchasing product 306, the user of electronic device 302 may be prompted to click on the product 306. Figure 3C The air gesture shown (e.g., and optionally based on a separate determination that the quantity of product 306 is less than a threshold amount), electronic device 302 presents one or more virtual elements in environment 304 (e.g., as shown in reference Figure 3D-1 to Figure 3F In some examples, in response to detecting Figure 3C As shown in the air gesture, the electronic device 302 stores information associated with the product 306 (e.g., in an information repository and / or file of an application). Figure 3C As shown, the air gesture performed using hand 308 corresponds to a pointing gesture toward product 306. It should be understood that in some examples, air gestures may include Figure 3C The gestures shown are different gestures (e.g., air tap, air pinch, long air pinch (e.g., a pinch that lasts for a threshold time period such as 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, 2 seconds, 5 seconds, or 10 seconds), air drag (e.g., proximal movement of the hand while performing a gesture such as a pinch with one or more fingers), two-finger flick, hand wave, and / or tapping a real-world surface). For example, the type of air gesture is customized by a user of the electronic device 302 (e.g., and associated with one or more settings of a user profile).

[0058] In some examples, electronic device 302 presents one or more virtual elements associated with product 306 in environment 304 in response to satisfying one or more criteria. For example, the one or more criteria include criteria that are satisfied when electronic device 302 determines that the quantity of product 306 is less than a threshold amount (e.g., as described above). For example, the one or more criteria include criteria that are satisfied when electronic device 302 detects an interaction (e.g., physical interaction) by a user of electronic device 302 with product 306 (e.g., as described above). For example, the one or more criteria include criteria that are satisfied when electronic device 302 detects one or more indications of user interest in the product (e.g., as described above). In some examples, in response to satisfying one or more criteria, electronic device 302 stores information associated with product 306 (e.g., in a file of an application (e.g., as referenced above). Figure 3G For example, in response to one or more criteria being met, the electronic device 302 (e.g., automatically (e.g., without additional user input) adds the product 306 to a shopping list (e.g., associated with a note application, such as Figure 3G 306). Presenting one or more virtual elements associated with product 306 in environment 304 in response to satisfying one or more criteria enables electronic device 302 to present the one or more virtual elements when a user of electronic device 302 intends to present the one or more virtual elements (e.g., because the user is interested in product 306 and / or product 306 has a quantity that is estimated to be less than a threshold amount), thereby improving user-device interaction and conserving computing resources.

[0059] Figure 3D-1The electronic device 302 is illustrated as presenting a virtual element 311 in the environment 304 in response to satisfying one or more criteria. For example, the electronic device 302 presents the virtual element 311 in the environment 304 in response to detecting a user's attention (e.g., gaze) on the product 306 (e.g., for a threshold time period (e.g., 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, 2 seconds, 5 seconds, or 10 seconds)). For example, the electronic device 302 presents the virtual element 311 in the environment 304 in response to detecting a physical interaction with the product 306 (e.g., as described in reference Figure 3B ) and / or in response to detecting an air gesture (e.g., as shown and described in reference Figure 3C In some examples, the virtual element 311 includes a first selectable option 315a that can be selected to initiate a process for purchasing the product 306 online (e.g., through an e-commerce website or application). For example, in response to detecting the selection of the first selectable option 315a, the electronic device 302 automatically (e.g., without additional user input) orders the product 306 from an online seller (e.g., from a seller preferred by the user based on one or more settings associated with the user profile, and / or using a payment method preferred by the user based on one or more settings associated with the user profile). For example, in response to detecting the selection of the first selectable option 315a, the electronic device 302 presents a list of online sellers from which the product 306 can be purchased (e.g., similar to Figure 3E In some examples, the second selectable option 315b may be selected to stop presenting the virtual element 311 in the environment 304 (e.g., and abandon the online purchase of the product 306). Figure 3D-3 One or more characteristics of the user input are shown and described as user input to select the first selectable option 315a and / or the second selectable option 315b.

[0060] Alternatively, in some examples, electronic device 302 automatically purchases product 306 (e.g., through an online seller) in response to satisfying one or more criteria (e.g., without additional user input and / or without presenting virtual element 311). For example, a user of electronic device 302 selects and / or authorizes electronic device 302 (e.g., through an application (e.g., an e-commerce application) and / or a digital payment service) to automatically order one or more products (e.g., product 306) that electronic device 302 detects as having a quantity and / or estimated quantity that is less than a threshold amount (e.g., the threshold amount described above).

[0061] Figure 3D-2The electronic device 302 is illustrated as presenting a virtual element 313 in the environment 304 in response to satisfying one or more criteria. For example, the electronic device 302 presents the virtual element 313 in the environment 304 in response to detecting a user's attention (e.g., gaze) on the product 306 (e.g., for a threshold time period (e.g., 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, 2 seconds, 5 seconds, or 10 seconds)). For example, the electronic device 302 presents the virtual element 313 in the environment 304 in response to detecting a physical interaction with the product 306 (e.g., as described in reference Figure 3B ) and / or in response to detecting an air gesture (e.g., as shown and described in reference Figure 3C In some examples, the virtual element 313 includes a first selectable option 317a that can be selected to store information associated with the product 306 (e.g., in an information repository associated with the corresponding application and / or in a file of the corresponding application). For example, in response to detecting the selection of the first selectable option 317a, the electronic device 302 adds the product 306 to the shopping list (e.g., similar to the reference 304). Figure 3G The shopping list may include a shopping list included in user interface 328 shown and described above. The shopping list is optionally not store-specific (e.g., the shopping list includes a list of products that can be purchased at different stores). For example, in response to detecting that electronic device 302 (e.g., and a user wearing electronic device 302) is within proximity (e.g., a threshold distance) of a store and / or business from which products on the shopping list (e.g., product 306) can be purchased, electronic device 302 presents, via display generation component 330, a virtual element corresponding to the notification (e.g., as shown in reference to FIG. 1 ). Figure 3H and / or Fig. 3I In some examples, the second selectable option 317b may be selected to stop presenting the virtual element 313 in the environment 304 (e.g., and forgo storing information associated with the product 306). Figure 3D-3 The user input of one or more characteristics of the user input shown and described above selects the first selectable option 317a and / or the second selectable option 317b. Alternatively, in some examples, the electronic device 302 automatically stores information associated with the product 306 (e.g., in a file of the application, such as on a file associated with a shopping list) in response to satisfying one or more criteria (e.g., without presenting the virtual element 313).

[0062] Figure 3D-3The electronic device 302 is illustrated as presenting a virtual element 312 in the environment 304 in response to satisfying one or more criteria. For example, the electronic device 302 presents the virtual element 312 in the environment 304 in response to detecting a user's attention (e.g., gaze) on the product 306 (e.g., for a threshold time period (e.g., 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, 2 seconds, 5 seconds, or 10 seconds)). For example, the electronic device 302 presents the virtual element 312 in the environment 304 in response to detecting a physical interaction with the product 306 (e.g., as described in reference Figure 3B ) and / or in response to detecting an air gesture (e.g., as shown and described in reference Figure 3C In some examples, virtual element 312 includes an option that can be selected by a user of electronic device 302 to present one or more areas (e.g., stores / businesses) where product 306 can be purchased in environment 304. Figure 3D-3 As shown, the virtual element 312 includes a first selectable option 316a and a second selectable option 316b. In some examples, the first selectable option 316a may be selected to present one or more virtual elements of one or more areas (e.g., stores / businesses) including purchasable products 306 in the environment 304. In some examples, the second selectable option 316b may be selected to stop presenting the virtual element 312 in the environment 304 (e.g., the second selectable option 316b may be selected to abandon presenting one or more virtual elements of one or more areas (e.g., stores / businesses) including purchasable products 306 in the environment 304). Presenting the virtual element 312 in the environment 304 provides the user of the electronic device 302 with an opportunity to confirm that it is their intention that the electronic device 302 presents one or more virtual elements in the environment 304, thereby saving computing resources by preventing the presentation of virtual elements that the user is not interested in. In some examples, the virtual element 312 is optionally displayed via a display component (e.g., optionally different from the display generation component 330) that communicates with the electronic device 302. For example, in response to satisfying one or more criteria (e.g., electronic device 302 determines that the one or more criteria are satisfied by using one or more inputs detected by one or more image sensors), virtual element 312 is displayed on a display of a device (e.g., a mobile phone, a tablet computer, a laptop computer, and / or a smart watch) in communication with electronic device 302. In some examples, a user of electronic device 302 interacts with the virtual element through the device in communication with electronic device 302 (e.g., through a touch-sensitive display of the device).

[0063] In some examples, the first selectable option 316a and / or the second selectable option 316b may be selected by user input. Figure 3D-3As shown, a user of electronic device 302 provides input corresponding to a selection of first selectable option 316a (e.g., corresponding to a request to present one or more virtual elements in environment 304 that include information about where product 306 can be purchased). In some examples, Figure 3D-3 As shown, the user input includes a gaze 322 (e.g., by Figure 3D-3 In some examples, such as Figure 3D-3 As shown, the user input includes an air gesture (e.g., an air pinch) performed using hand 308. In some examples, the user input may alternatively include audio input (e.g., a verbal command), input provided through a hardware device (e.g., hardware button 360a and / or hardware button 360b), the launch of an application (e.g., associated with a location where purchasable products are presented), and / or input provided through a touch-sensitive surface.

[0064] In some examples, in response to satisfying one or more criteria (e.g., in response to detecting a quantity of products 306 and / or one or more indications of user interest in products 306), electronic device 302 presents in environment 304 a virtual element (e.g., reference to a location in the environment) that includes one or more areas of the physical environment where products 306 can be purchased (e.g., corresponding to one or more stores and / or businesses). Figure 3E 318a). Specifically, in some examples, the electronic device 302 presents a virtual element of one or more areas including purchasable products 306 in the environment 304 without presenting the virtual element 312. For example, in response to detecting that the quantity of products 306 is below a threshold amount and / or one or more indications of user interest in products 306, the electronic device 302 automatically presents a virtual element of one or more areas including purchasable products 306 in the environment 304 (e.g., discussed below). In some examples, in response to satisfying one or more criteria, the electronic device 302 stores information associated with the products 306 (e.g., in an information repository associated with a corresponding application and / or in a file of the corresponding application) without presenting the one or more virtual elements (e.g., without presenting the virtual element 312). Automatically presenting a virtual element of one or more areas including purchasable products 306 in the environment 304 and / or storing information associated with the products 306 without presenting a qualifying virtual element (e.g., virtual element 312) in the environment 304 minimizes the presentation of unnecessary virtual objects in the environment 304 and conserves computing resources.

[0065] Figure 3EThe electronic device 302 is illustrated as presenting a virtual element 318a of one or more visual indications of one or more areas (e.g., stores / businesses) in the environment 304 that include purchasable products 306 (e.g., in response to one or more criteria being met). Figure 3EAs shown, the virtual element 318 includes visual indications of a first area 324a, a second area 324b, and a third area 324c. For example, the first area 324a, the second area 324b, and the third area 324c correspond to stores / businesses where the product 306 can be purchased. In some examples, the electronic device 302 presents one or more areas in an order based on the proximity of the one or more areas to the location of the electronic device 302 (e.g., the location closest to the electronic device 302 is the first area included / listed in the virtual element 318a (e.g., at the top of the list of stores / businesses)). In addition, the one or more areas included in the virtual element 318a are optionally within a threshold distance (e.g., 0.5 miles, 1 mile, 2 miles, 5 miles, 10 miles, 15 miles, 20 miles, 25 miles, 30 miles, 35 miles, 40 miles, 45 miles, or 50 miles) of the electronic device 302. In some examples, the electronic device 302 ranks one or more areas based on known user history according to user data provided by an application (e.g., a web browsing application, a navigation application, and / or a shopping application) (e.g., the first area 324a, the second area 324b, and / or the third area 324c correspond to stores / businesses previously visited and / or previously searched for by the user of the electronic device 302). In some examples, the electronic device 302 ranks one or more areas based on user preferences (e.g., the electronic device 302 detects that the user frequents one or more stores / businesses, and / or one or more stores / businesses are defined as preferred stores / businesses in a user profile (e.g., set by the user). In some examples, the virtual element 318a is presented with an area corresponding to where the user of the electronic device 302 can purchase the product 306 (e.g., the one area is the user's most preferred store / business, or is the store / business closest to the electronic device 302). In some examples, a user of the electronic device 302 may interact with the virtual element 318a to view alternate areas (e.g., through user input, such as selection of a selectable option included in the virtual element 318a (e.g., causing expansion of the list of areas and / or viewing of a new page of the list of areas) or scrolling of the virtual element 318a (e.g., causing display of additional areas), audio input (e.g., verbal commands), input provided through a hardware device (e.g., hardware buttons 360a and / or 360b), and / or input provided through a touch-sensitive surface). Including one or more areas of purchasable products 306 within the virtual element 318a based on proximity and / or prior user interactions limits the multiple areas presented by the electronic device 302 to those areas that may be of interest to the user of the electronic device 302, thereby improving user-device interaction and conserving computing resources.

[0066] like Figure 3EAs shown, virtual element 318a includes information associated with one or more areas of purchasable product 306. For example, Figure 3E As shown, the first area 324a, the second area 324b, and the third area 324c are presented with a name (e.g., a store name), a current distance from the electronic device 302, a logo (e.g., of a store brand), a price of the product 306, and an indication of the quantity of the product 306 in the store (e.g., current inventory). In some examples, one or more areas are presented with additional information (e.g., or optionally less information) associated with the product 306, such as the name and / or brand of the product, the specifications of the product (e.g., size, material, type, and / or color), and / or the location of the product in each area (e.g., aisle number and / or shelf number). In addition, as Figure 3E 3, a virtual affordance 320 is shown. In some examples, the virtual affordance 320 may be selected to stop presenting the virtual element 318a in the environment 304 (e.g., by a user input having one or more characteristics of a selection of the first area 324a). In some examples, the electronic device 302 stops presenting the virtual element 318a after the user of the electronic device 302 has not interacted with the virtual element 318a for a threshold time period (e.g., 0.1 minutes, 0.2 minutes, 0.5 minutes, 1 minute, 2 minutes, 3 minutes, 4 minutes, 5 minutes, or 10 minutes). In some examples, the electronic device 302 stops presenting the virtual element 318a in response to detecting an air gesture, a verbal input, a hardware device input, and / or an input provided through a touch-sensitive surface.

[0067] In some examples, one or more regions may be selected through user input to cause the electronic device 302 to perform operations associated with the one or more regions. Figure 3E As shown, the user of the electronic device 302 provides input corresponding to the selection of the first area 324a. In some examples, the input includes attention (e.g., gaze 322) directed to the first area 324a and / or an air gesture (e.g., air pinch) performed by the hand 308. In some examples, in response to detecting the input corresponding to the selection of a particular area, the electronic device 302 stores information associated with the product 306 and / or the area where the product 306 can be purchased (e.g., as shown in reference Figure 3G For example, product 306 is added to a shopping list for a store associated with first region 324a.

[0068] Figure 3F The electronic device 302 is shown in response to the Figure 3E The virtual element 318b is presented in the environment 304 in response to the input provided by the user of the electronic device 302 in the environment 304. Figure 3EIn response to the selection input of the first area 324a shown, the electronic device 302 presents a virtual element 318b in the environment 304 to provide the user of the electronic device 302 with an opportunity to confirm whether the electronic device 302 should store information associated with the product 306 and / or the first area 324a. Figure 3F As shown, virtual element 318b includes a first selectable option 326a and a second selectable option 326b. In some examples, first selectable option 326a may be selected to store information associated with product 306 and / or first area 324a. In some examples, second selectable option 326b may be selected to forgo storing information associated with product 306 and / or first area 324a. In some examples, the user input may be selected (e.g., having one or more characteristics of the user input for selecting first selectable option 316a and / or second selectable option 316b, as described in reference to FIG. 1 ). Figure 3D-3 In some examples, in response to the selection of the second selectable option 326b, the electronic device 302 stops presenting the virtual element 318b in the environment 304 (e.g., and optionally stops presenting the virtual element 318a in the environment 304). In some examples, when the virtual element 318b is stopped from being presented in the environment 304 in response to the input corresponding to the selection of the second selectable option 326b, the electronic device 302 maintains the presentation of the virtual element 318a (e.g., so that the user of the electronic device 302 can select a different area among the one or more areas of the purchasable products 306).

[0069] like Figure 3F As shown, the user of the electronic device 302 provides input corresponding to a selection of the first selectable option 326a (e.g., the input includes a gaze 322 directed to the first selectable option 326a while optionally performing an air gesture using the hand 308). In some examples, in response to the selection of the first selectable option 326a, the electronic device 302 adds information associated with the product 306 and the first area 324a to the information repository. For example, the electronic device 302 stores information associated with the user of the electronic device 302's intent to purchase the product 306 at a store associated with the first area 324a in a memory during a first time period. During a second time period, after the first time period, the electronic device 302 determines that the location of the electronic device 302 corresponds to the first area 324a and presents one or more virtual elements associated with the product 306 in the environment 304 (e.g., the one or more virtual elements navigate the user of the electronic device 302 to the location of the product 306 within the store and / or facilitate the purchase of the product 306, as described in reference to Figures 4A-4K306 and the first area 324a. For example, the application is associated with tracking a shopping list (e.g., the electronic device 302 adds the product 306 to a shopping list accessible to the user of the electronic device 302 (e.g., a file of the application)). For example, the application is associated with a note-taking application (e.g., the electronic device 302 creates a note (e.g., a file of the application) that includes a shopping list and / or adds the product 306 to an existing note that includes a shopping list (e.g., a shopping list that is for a store associated with the first area 324a)). In some examples, in response to a reference to Figure 3E In response to the selection of the first area 324a shown and described, the electronic device 302 automatically adds the information associated with the product 306 and the first area 324a to the information repository. For example, after receiving the input corresponding to the selection of the first area 324a and / or before adding the information associated with the product 306 and the first area 324a to the information repository, the electronic device 302 does not present the virtual element 318b in the environment 304. Automatically storing the information associated with the product 306 in the information repository without presenting a qualified virtual element (e.g., virtual element 318b) in the environment 304 minimizes the presentation of virtual objects in the environment 304 and saves computing resources.

[0070] Figure 3G The electronic device 302 is illustrated as presenting a user interface 328 of an application in the environment 304. In some examples, the user interface 328 is associated with an application that includes the electronic device 302 adding information associated with the product 306 and the first area 324a to an information repository. For example, the electronic device 302 adds information associated with the product 306 and the first area 324a to the information repository (e.g., in response to a request by Figure 3F302 ), and the information associated with the product 306 and the first area 324a is accessible to the user of the electronic device 302 through the user interface 328. Adding the information associated with the product 306 and the first area 324a to the user interface accessible by the user of the electronic device 302 provides the user with an opportunity to modify the information (e.g., including removing the information and / or editing the information to include different locations associated with different stores if the user has already purchased the product). In some examples, the user interface 328 is displayed via a display component (e.g., optionally different from the display generation component 330) that communicates with the electronic device 302. For example, the user interface 328 is accessed via a device (e.g., a mobile phone, a tablet computer, a laptop computer, and / or a smart watch) that communicates with the electronic device 302. In some examples, the user of the electronic device 302 interacts with the user interface 328 through the device that communicates with the electronic device 302 (e.g., through a touch-sensitive display of the device).

[0071] like Figure 3G As shown, user interface 328 is associated with a note application that includes a shopping list. For example, a shopping list includes a note that can be used with Figure 3E The shopping list optionally includes one or more products in addition to (and optionally different from) product 306. For example, Figure 3GAs shown, the shopping list included in the user interface 328 includes tape and nails. In some examples, one or more products other than product 306 correspond to one or more products detected and / or determined by the electronic device 302 to have a quantity less than a threshold amount. In some examples, before adding one or more products other than product 306 to the shopping list included in the user interface 328, the electronic device 302 detects one or more indications of user interest in (e.g., purchasing) the one or more products. In some examples, the electronic device 302 automatically adds the one or more products to the shopping list included in the user interface 328 (e.g., in response to satisfying one or more criteria). In some examples, the user of the electronic device 302 adds the one or more products to the user interface 328 (e.g., through one or more user inputs, such as audio input (e.g., verbal commands), input provided through a virtual input element (e.g., a virtual keyboard), and / or input provided through a touch-sensitive surface). Alternatively, in some examples, the shopping list included in the user interface 328 is not associated with one or more areas (e.g., stores or businesses). For example, based on determining that one or more criteria (e.g., as described above) are met, electronic device 302 adds product 306 to the shopping list (e.g., automatically (e.g., without additional user input)) (e.g., electronic device 302 adds product 306 to the shopping list without presenting a request). Figure 3D-1 to Figure 3F For example, a shopping list can be used to purchase products at different stores or businesses (e.g., and is not limited to products found at a particular store). Additionally or alternatively, in some examples, products included in a shopping list are not limited to a particular brand or type (e.g., a shopping list can be used to purchase different brands of products (e.g., different brands of screws, tape, or nails)).

[0072] In some examples, in response to selection of the first region 324a (e.g., as shown in FIG. FIG. 3E to FIG. 3F In some examples, in response to selection of an area 324a, information associated with the first area 324a (e.g., an address and / or coordinates) is stored in the navigation application. For example, in response to launching the navigation application (e.g., on the electronic device 302 and / or on a device in communication with the electronic device 302), the first area 324a is presented as a suggested location for the user to navigate to. In some examples, in response to creation of a shopping list associated with the first area 324a (e.g., as shown and described above), directions to the first area 324a are retrieved and / or provided by a navigation application (e.g., accessible via the electronic device and / or by a device in communication with the electronic device). In some examples, in response to creation of a shopping list associated with the first area 324a (e.g., as shown in FIG. 1 ), directions to the first area 324a are retrieved and / or provided by a navigation application (e.g., accessible via the electronic device and / or by a device in communication with the electronic device). In some examples, in response to selection of an area 324a, information associated with the first area 324a (e.g., an address and / or coordinates) is stored in the navigation application. Figure 3G ), information associated with the first area 324a is stored in the navigation application.

[0073] Figure 3H The electronic device 302 is illustrated as presenting, via the display generation component 330, a virtual element 372 indicating that one or more products of interest to the user 307 are available for purchase in an area of ​​the physical environment (e.g., at a store and / or business). Figure 3H As shown, virtual element 372 indicates to user 307 (e.g., shown in top view 370) of electronic device 302 (e.g., user 307 is wearing electronic device 302) that one or more products stored on the shopping list (e.g., electronic device 302 adds information associated with the one or more products to an information repository, as described above) are available for purchase from nearby store 350 (e.g., store 350 corresponds to an area of ​​the physical environment, as described above with reference to FIG. 1 ). FIG. 3E to FIG. 3F Purchase as described above.

[0074] In some examples, Figure 3H, the electronic device 302 presents the virtual element 372 based on determining that one or more second criteria are met. For example, the one or more criteria include criteria that are met when the electronic device 302 previously stored one or more products in the shopping list. For example, the one or more criteria include criteria that are met when a corresponding area of ​​the physical environment (e.g., store 350) has one or more products from the shopping list available for purchase and the corresponding area is within a threshold distance (e.g., 0.1 km, 0.2 km, 0.5 km, 1 km, 2 km, 5 km, or 10 km) of the current location of the electronic device 302 (e.g., and / or the user 307). For example, the one or more criteria include criteria that are met when the electronic device 302 (e.g., and / or the user 307) moves away from the corresponding location of the physical environment (e.g., the electronic device 302 and / or the user 307 moves away from a location corresponding to the home of the user 307 (e.g., the location corresponding to the home of the user 307 is stored in the user profile), and / or the electronic device 302 and / or the user 307 moves away from an initial location where the one or more products were added to the shopping list). For example, one or more criteria include criteria that are satisfied when user 307 initiates transportation or changes a current mode of transportation (e.g., as detected by orientation sensor 210 (e.g., motion sensor and / or one or more gyroscopes and / or one or more accelerometers) and / or location sensor 204) (e.g., electronic device 302 detects movement of user 307 and / or electronic device 302 via a vehicle (e.g., a car). For example, one or more criteria include criteria that are satisfied when user 307 navigates on a route using a navigation application (e.g., as described above), and a corresponding area of ​​the physical environment (e.g., store 350) on the route has one or more products from a shopping list available for purchase. For example, one or more criteria include criteria that are satisfied when electronic device 302 detects an indication that user 307 is interested in shopping (e.g., user 307 previously created a calendar event associated with shopping (e.g., grocery shopping), and / or the current time of day and / or week corresponds to a time when user 307 previously performed an errand such as shopping). For example, one or more criteria include criteria that are satisfied based on determining that user 307 has no existing events and / or obligations during a period of time (e.g., the calendar indicates that user 307 has no existing events scheduled lasting 1 hour, 2 hours, 6 hours, 12 hours, or 24 hours).

[0075] Fig. 3I The electronic device 302 is illustrated as presenting a display via the display generation component 330 to indicate to the user of the electronic device 302 (eg, Figure 3H The virtual element 352 of one or more products that the user 307 is interested in is shown as available for purchase in an area of ​​the physical environment (e.g., a store and / or a business). Fig. 3IAs shown, virtual element 352 indicates that one or more products stored on a shopping list (e.g., as described above) are available for purchase at store 350. For example, electronic device 302 presents virtual element 352 based on determining that a corresponding area of ​​the physical environment (e.g., store 350) having one or more products available for purchase from the stored shopping list is within a current field of view of electronic device 302 (e.g., and / or a user of electronic device 302).

[0076] Figures 4A to 4K An exemplary method for presenting one or more virtual elements associated with one or more products in an environment based on the location of an electronic device according to some examples of the present disclosure is illustrated. For example, when the location of the electronic device is within an area of ​​the physical environment that includes one or more locations corresponding to the one or more products, the one or more virtual elements are presented in the environment. In some examples, the one or more products are associated with one or more indications of a user's intent to purchase the one or more products detected by the electronic device (e.g., as described in reference to FIG. 1 ). FIG. 3A to FIG. 3I For example, before the electronic device is located within the area of ​​the physical environment, the electronic device pre-detects one or more indications of a user's intent to purchase one or more products. In some examples, the electronic device detects one or more indications of a user's intent to purchase one or more products from a memory stored in the electronic device and / or associated with a corresponding application (e.g., such as a reference Figure 3G An information repository associated with an annotation application (shown and described) identifies one or more products. For example, the information repository includes information associated with one or more products (e.g., the information is used to fill in a shopping list included in a file of the application).

[0077] In some examples, Figures 4A to 4K The environment 404 has a reference FIG. 3A to FIG. 3I One or more characteristics of the environment 304 shown and described. In some examples, the electronic device 402 has a reference FIG. 3A to FIG. 3I One or more characteristics of the electronic device 302 shown and described. In some examples, the display generation component 430 has a reference FIG. 3A to FIG. 3I One or more characteristics of the display generation component 330 shown and described. In some examples, the image sensors 414a-414c have reference FIG. 3A to FIG. 3I One or more characteristics of the image sensors 314a-314c shown and described. In some examples, the hardware buttons 460a-460b have reference FIG. 3A to FIG. 3I One or more characteristics of hardware buttons 360a-360b are shown and described.

[0078] In some examples, one or more virtual elements (e.g., reference Figure 4BThe virtual elements 414 and 416a-416b shown and described are optionally displayed by a display component of a second device in communication with the electronic device 402 (e.g., rather than the electronic device 402 presenting one or more virtual elements in a virtual environment such as the environment 404). For example, the electronic device 402 detects one or more inputs using image sensors 414a-414c (e.g., by detecting a physical environment), and in response to detecting the one or more inputs, one or more virtual elements (e.g., virtual element 418a) are displayed on a display component of the second device (e.g., content is displayed on a display of a mobile phone in communication with the electronic device 402). In some examples, the electronic device 402 optionally does not include one or more displays (e.g., display generation component 430), and optionally does not present a three-dimensional environment (e.g., environment 404) to a user of the electronic device 402.

[0079] Figure 4A The example illustrates that a user of electronic device 402 enters an area of ​​a physical environment that includes one or more locations corresponding to one or more products. For example, the area of ​​the physical environment is a store where the one or more products can be purchased. For example, the area of ​​the physical environment corresponds to FIG. 3E to FIG. 3F 404. Alternatively, the area of ​​the physical environment does not correspond to an area previously selected by the user of the electronic device 402. For example, the electronic device 402 detects that the current location of the electronic device 402 corresponds to a store where one or more products from a shopping list are available for purchase (e.g., the shopping list is optionally not a store-specific shopping list) (e.g., based on determining that the current location of the electronic device 402 corresponds to a store where one or more products from the shopping list are available for purchase, the electronic device 402 presents one or more virtual elements associated with one or more products in the environment 404 (e.g., as described below)). In some examples, the one or more products were previously (e.g., prior to entering the store) detected and / or identified (e.g., by image and / or object recognition) by the electronic device 402 (e.g., using image sensors 414a-414c). For example, the electronic device 402 detects that the user of the electronic device 402 has a quantity of one or more products that is less than a threshold amount (e.g., as described in reference to FIG. 3A to FIG. 3I For example, the electronic device 402 detects a purchased product (eg, the product corresponds to a reference FIG. 3A to FIG. 3F In some examples, information associated with the product is included in a file of an application (e.g., the product is included on a shopping list in an annotation (e.g., a file) of an annotation application (e.g., as shown in FIG. 1 ). Figure 3G shown and described)).

[0080] In some examples, the electronic device 402 presents one or more virtual elements in the environment 404 to virtually enhance the shopping experience of the user of the electronic device 402 based on determining that the location of the electronic device 402 is within a store including one or more products. For example, the electronic device 402 presents one or more virtual elements in the environment 404 to help the user navigate to one or more locations corresponding to one or more products in the store (e.g., the one or more virtual elements include the channel number and / or shelf number of the one or more products in the store). For example, the electronic device 402 presents one or more virtual elements in the environment 404 to facilitate the purchase of one or more products through a digital payment service (e.g., one or more virtual elements guide the user of the electronic device 402 to complete the digital payment process). In some examples, the store supports the presentation of one or more virtual elements and / or the digital payment process. For example, the store (e.g., a third party) provides (e.g., through data and / or a database) information associated with one or more products and / or the location of one or more products in the store (e.g., the information is used by the electronic device 402 to present one or more virtual elements in the environment 404). For example, the store optionally includes one or more devices (e.g., cameras and / or sensors) for verifying that the purchase of one or more products has been successfully completed (e.g., one or more devices are arranged near the exit of the store (e.g., at the checkout counter and / or exit channel)). In some examples, the electronic device 402 is optionally configured to communicate with the one or more devices. Presenting one or more virtual elements associated with one or more products in the environment 404 when (i) the user intends to purchase one or more products (e.g., based on previously detected user interactions associated with the one or more products) and (ii) the user is in a store where the one or more products can be purchased improves user-device interaction and saves computing resources.

[0081] like Figure 4A As shown, virtual element 410 is presented by electronic device 402 in environment 404. In some examples, when electronic device 402 determines that the detected location of electronic device 402 is within an area (e.g., a store) of the physical environment that includes one or more products (e.g., one or more products are associated with one or more indications of user intent (e.g., to purchase one or more products) detected by electronic device 402), virtual element 410 is presented in environment 404. In some examples, virtual element 410 corresponds to a virtual message welcoming the user of electronic device 402 to the store. In some examples, such as Figure 4AAs shown, virtual element 410 includes one or more selectable options for enhancing the shopping experience of the user of electronic device 402. For example, virtual element 410 includes a first selectable option 412a, which can be selected (e.g., through user input having one or more characteristics of the above-mentioned user input) to present one or more virtual elements associated with one or more products in environment 404. For example, virtual element 410 includes a second selectable option 412b, which can be selected to forgo presenting one or more virtual elements associated with one or more products in environment 404. Figure 4A As shown, the electronic device 402 detects that the user of the electronic device 402 provides an input corresponding to the selection of the first selectable option 412a. Figure 4A As shown, gaze 422 is directed to first selectable option 412a while an air gesture is performed by hand 408a (eg, an air pinch).

[0082] Figure 4B The electronic device 402 is shown in response to the Figure 4A Alternatively, in some examples, in response to the electronic device 402 determining that the location of the electronic device 402 is within a store that includes one or more products (e.g., or the location of the electronic device 402 is within a store that was previously identified as a preferred store for purchasing the one or more products (e.g., as shown in the reference example), the virtual elements 414, 416a, and 416b are presented in the environment 404. Figure 3E-3F As shown and described), virtual elements 414, 416a, and 416b are presented in environment 404. For example, based on determining that the location of electronic device 402 is within a store including one or more products, electronic device 402 automatically presents virtual elements 414, 416a, and 416b, and optionally does not present Figure 4A The virtual element 410 is shown. Figure 4B As shown, virtual element 414 includes a logo for identifying a first product (e.g., screws) and a location of the first product within the store. For example, the location of the first product within the store corresponds to an aisle number, a row number, and / or a bin number of the first product within the store. In some examples, virtual element 414 includes the name of the first product and / or other information associated with the first product (e.g., a brand name and / or product specifications (e.g., size, material, color, type, etc.)). In some examples, the first product corresponds to the first product on a shopping list (e.g., from a reference Figure 3GIn some examples, the first product corresponds to the product on the shopping list that is closest to the current location of the electronic device 402 within the store. In some examples, the electronic device 402 determines an order for presenting a set of navigation directions to the user for multiple products on the shopping list based on the location of the products within the store (e.g., products located in the same area of ​​the store are navigated sequentially (e.g., guiding the user of the electronic device 402 along an efficient route in the store to obtain the products)). Figure 4B As shown, virtual element 414 is presented in an upper area of ​​environment 404 from the viewpoint of the user of electronic device 402. In some examples, virtual element 414 is alternatively presented in a different area of ​​environment 404 (e.g., in a bottom area, a side area, a corner of environment 304, and / or an empty space (e.g., not including one or more objects) in environment 404) from the viewpoint of the user of electronic device 402. In some examples, virtual element 414 is presented in a world-locked orientation so that virtual element 414 is not presented in environment 404 at a location corresponding to an aisle (e.g., a third aisle) of a first product in environment 404 at a distance or orientation offset relative to the user of electronic device 402. In some examples, virtual element 414 is alternatively presented in a head-locked and / or body-locked orientation. For example, when the user navigates to the first product, virtual element 414 maintains its presentation position in environment 404 relative to the viewpoint of the user of electronic device 402. In some examples, virtual element 414 is optionally presented simultaneously with virtual elements 416a and 416b, but it should be understood that in other examples, electronic device 402 presents virtual element 414 without simultaneously presenting virtual elements 416a and 416b.

[0083] like Figure 4B As shown, virtual elements 416a and 416b correspond to navigation elements (e.g., arrows) for guiding the user of electronic device 402 to the location corresponding to the first product (e.g., the channel and row included in virtual element 414). Figure 4B, virtual elements 416a and 416b are presented on a visible floor surface in environment 404. In some examples, virtual elements 416a and 416b are alternatively presented in different areas of environment 404 (e.g., not on the surface). In some examples, electronic device 402 presents different numbers of navigation elements in environment 404 (e.g., electronic device 402 presents a single arrow including curvature). In some examples, presenting virtual elements 416a and 416b in environment 404 includes presenting one or more visual and / or audio cues when a user of electronic device 402 navigates to (e.g., walks to or otherwise travels to) a first product. For example, when the user navigates to a location within the store corresponding to the first product, the size, shape, orientation, and / or position of virtual elements 416a and 416b are updated to reflect the updated navigation to the first product (e.g., relative to the current location (e.g., position and / or orientation) of electronic device 402). For example, virtual elements 416a and 416b change brightness and / or color based on whether the user of electronic device 402 correctly navigates toward the first product. For example, the electronic device 402 provides an audio output to the user to assist in navigating to the first product (eg, the audio output includes verbal directions).

[0084] In some examples, virtual elements 414 and / or 416a-416b are displayed via a display component (e.g., optionally different from display generation component 430) of a second device (e.g., the second device corresponds to a mobile phone or a computer tablet) that communicates with electronic device 402. For example, electronic device 402 optionally does not include display generation component 430, and optionally does not include rendering environment 404. In some examples, virtual elements 414 and / or 416a-416b are presented through a corresponding application, which is accessed by a second device that communicates with electronic device 402 (e.g., the corresponding application is accessed by a mobile phone, a tablet computer, and / or a smart watch). In some examples, virtual elements other than virtual elements 414 and / or 416a-416b are displayed via a display component. For example, a guide list and / or a floor map of a store is displayed via a display component that helps a user of electronic device 402 navigate to the location of the first product in the store. In some examples, the electronic device 402 and / or a second device in communication with the electronic device 402 optionally provides audio output (eg, corresponding to verbal instructions) to a user of the electronic device 402 .

[0085] Figure 4CThe electronic device 402 is illustrated as presenting a virtual element 418a in the environment 404 based on the first product 406 being within the field of view of the user of the electronic device 402 (e.g., and within the field of view of the electronic device 402). In some examples, the first product 406 is in the field of view of the user of the electronic device 402 because the user has navigated to the location of the first product 406 (e.g., using Figure 4B 4. Virtual elements 414, 416a, and 416b presented in environment 404. Figure 4C As shown, the user of electronic device 402 is positioned in front of a shelf that includes first product 406 (e.g., the first product is located between products 420a and 420b on the shelf). Figure 4C As shown, when the first product 406 is within the field of view of the user of the electronic device 402, the electronic device 402 stops presenting the virtual elements 414, 416a, and 416b. In some examples, the virtual element 418a (e.g., or virtual content having one or more characteristics of the virtual element 418a) is optionally displayed via a display component (e.g., different from the display generation component 430) in communication with the electronic device 402. For example, the electronic device 402 optionally does not include the display generation component 430 and / or presents the virtual environment 404.

[0086] In some examples, virtual element 418a includes information associated with first product 406. For example, Figure 4C As shown, virtual element 418a includes product specifications (e.g., the number of screws in the package, the type of screws, and the size). Additionally or alternatively, in some examples, virtual element 418a includes a product name, a brand name, a price, and / or customer reviews. Figure 4CAs shown, from the viewpoint of the user of the electronic device 402, the virtual element 418a is presented above the first product 406 in the environment 404. In some examples, the virtual element 418a is alternatively presented at a different location in the environment 404 (e.g., an upper area, a lower area, a side of the first product 406, a corner area, and / or in an empty space of the environment 404). In some examples, electronic device 402 stops presenting virtual element 418a in environment 404 in response to user input (e.g., input selecting a virtual affordance included in virtual element 418a for stopping presentation of virtual element 418a, an air gesture, providing attention (e.g., gaze) to virtual element 418a and / or first product 406 for less than a threshold amount (e.g., less than 0.1 minute, 0.2 minute, 0.5 minute, 1 minute, 2 minutes, or 5 minutes), providing audio input (e.g., a verbal command), providing input through a hardware device (e.g., hardware buttons 460a and / or 460b), and / or providing input through a touch-sensitive surface (e.g., in communication with electronic device 402). In some examples, electronic device 402 does not present virtual element 418a in environment 304, thereby minimizing the number of virtual objects presented in environment 404 and conserving computing resources.

[0087] Figure 4D The electronic device 402 is illustrated as presenting a virtual element 418b in the environment 404. In some examples, the electronic device 402 presents the virtual element 418b in the environment 404 in response to the user's interaction with the first product 406. For example, Figure 4D As shown, the user of the electronic device 402 lifts (e.g., grabs and / or moves) the first product 406 from the shelf with a hand 408a. In some examples, in response to detecting that the user lifts the first product 406 from the shelf, the electronic device 402 presents a virtual element 418b in the environment 404. In some examples, the electronic device 402 alternatively presents the virtual element 418b in the environment 404 in response to detecting that the user of the electronic device 402 pays attention to (e.g., gazes at) the first product 406 (e.g., for more than a threshold time period (e.g., 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, 2 seconds, 5 seconds, or 10 seconds)). In some examples, the electronic device 402 presents the virtual element 418b in the environment 404 in response to detecting that the user of the electronic device 402 lifts the first product 406 from the shelf and moves the first product 406 beyond a threshold distance (e.g., 0.5 meters, 1 meter, 2 meters, 5 meters, 10 meters, 15 meters, 20 meters, 25 meters, or 50 meters) from the shelf. In some examples, electronic device 402 responds to detecting different user inputs, such as audio input (e.g., including spoken commands), air gestures (e.g., as described in reference Figure 4G406a), input provided through a hardware device (e.g., hardware buttons 460a and / or 460b), and / or input provided through a touch-sensitive surface (e.g., in communication with the electronic device 402). Rendering the virtual element 418b in the environment 404 in response to detecting a user interaction with the first product 406 enables the virtual element 418b to be presented when the user of the electronic device 402 intends to add the first product 406 to a digital shopping cart (e.g., as described below) and / or purchase the first product 406a, thereby improving user-device interaction and conserving computing resources.

[0088] In some examples, virtual element 418b includes one or more selectable options for adding first product 406 to a digital shopping cart. Figure 4D As shown, the virtual element 418b includes a first selectable option 424a and a second selectable option 424b. In some examples, the first selectable option 424a can be selected to add the first product 406 to the digital shopping cart. In some examples, the second selectable option 424b can be selected to abandon adding the first product 406 to the digital shopping cart. In some examples, the first selectable option 424a and the second selectable option 424b can be selected by user input (e.g., having one or more characteristics of one or more user inputs described above). In some examples, the electronic device 402 stops presenting the virtual element 418b in the environment 404 in response to detecting an input corresponding to the selection of the second selectable option 424b. In some examples, the electronic device 402 stops presenting the virtual element 418b in the environment 404 after a threshold time period (e.g., 0.5 seconds, 1 second, 2 seconds, 5 seconds, 10 seconds, 15 seconds, 20 seconds, 25 seconds, 30 seconds, 60 seconds, or 90 seconds). Alternatively, in some examples, the electronic device 402 does not present the virtual element 418b in the environment 404 in response to the user interaction with the first product 406 (e.g., and / or in response to one or more of the different user inputs described above). For example, in response to detecting the user interaction with the first product 406, the electronic device 402 automatically adds the first product 406 to the digital shopping cart (e.g., without presenting the virtual element 418b before adding the first product 406 to the digital shopping cart). Automatically adding the first product 406 to the digital shopping cart without presenting a qualifying virtual element (e.g., the virtual element 418b) in the environment 404 minimizes the presentation of virtual objects in the environment 404 and saves computing resources.

[0089] like Figure 4DAs shown, the user of the electronic device 402 provides input corresponding to the selection of the first selectable option 424a (e.g., the user of the electronic device 402 provides a gaze 422 directed to the first selectable option 424a while performing an air gesture with the hand 408a). In some examples, in response to detecting the input corresponding to the selection of the first selectable option 424a, the electronic device 402 adds the first product 406 to the digital shopping cart. In some examples, the electronic device 402 provides a visual and / or audio indication to confirm that the first product 406 is added to the digital shopping cart. For example, Figure 4E As shown, the electronic device 402 presents a virtual element 418c in the environment 404 corresponding to a visual confirmation that the first product 406 has been added to the digital shopping cart. In some examples, in addition to (e.g., or instead of) presenting the virtual element 418c in the environment 404, the electronic device 402 provides an audio output to the user of the electronic device 402 (e.g., through an output device of the electronic device 402 and / or an output device of a device (e.g., a mobile phone, a tablet computer, and / or a smart watch) that communicates with the electronic device 402) to provide feedback that the first product 406 has been added to the digital shopping cart. In some examples, the electronic device 402 stops presenting the virtual element 418c in the environment 404 after 1 second, 2 seconds, 5 seconds, 10 seconds, 15 seconds, 20 seconds, 25 seconds, 30 seconds, 45 seconds, 60 seconds, or 120 seconds of presentation time in the environment 404. In some examples, the electronic device 402 stops presenting the virtual element 418c in response to user input (e.g., input selecting a virtual enable representation included in the virtual element 418c for stopping presenting the virtual element 418c, an air gesture, providing attention (e.g., gaze) to the virtual element 418c for less than a threshold amount (e.g., less than 0.1 minutes, 0.2 minutes, 0.5 minutes, 1 minute, 2 minutes, or 5 minutes), providing audio input (e.g., a verbal command), providing input through a hardware device (e.g., hardware buttons 460a and / or 460b), and / or providing input through a touch-sensitive surface (e.g., in communication with the electronic device 402)).

[0090] Figure 4F The electronic device 402 is shown in response to the Figure 4D One or more virtual elements associated with the second product are presented in the environment 404 based on input provided by a user of the electronic device 402 in the digital shopping cart (e.g., after adding the first product 406 to the digital shopping cart). Figure 4F As shown, the second product is a roll of tape (e.g., as represented by an icon of the second product included in virtual element 426). In some examples, the second product is a roll of tape that is included in the same shopping list (e.g., such as reference 426). Figure 3GIn some examples, electronic device 402 detects (e.g., before entering a store) that a user of electronic device 402 has a quantity (e.g., an estimated quantity) of a second product that is less than a threshold quantity (e.g., such as a threshold quantity). FIG. 3A to FIG. 3C In some examples, electronic device 402 detects (e.g., before entering a store) one or more indications of a user's intent to purchase a second product (e.g., Figure 3D-3 to Figure 3F 3, electronic device 302 detects one or more user inputs corresponding to an intent of a user of electronic device 302 to purchase product 306. In some examples, electronic device 402 presents one or more virtual elements (e.g., virtual elements 426, 428a, and / or 428b) associated with second product 406 in environment 404 after first product 406 based on the order of the products on the shopping list and / or the proximity (e.g., distance) of the second product to the first product in the store. In some examples, presenting virtual element 426 in environment 404 has the same characteristics as described in reference to FIG. Figure 4B One or more characteristics of virtual element 414 are presented in environment 404 as shown and described. In some examples, virtual elements 428a and 428b are presented in environment 404 with reference to Figure 4B One or more characteristics of virtual elements 416 a and 416 b are shown and described as being presented in environment 404 .

[0091] Figure 4G The electronic device 402 is illustrated as presenting a virtual element 432a in the environment 404 in response to the second product 436 being within the field of view of the user of the electronic device 402 (e.g., and the electronic device 402). In some examples, the second product 436 is in the field of view of the user of the electronic device 402 because the user has used Figure 4F The virtual elements 426, 428a, and 428b presented in the environment 404 navigate to the location of the second product 436. Figure 4G As shown, the user of electronic device 402 views a shelf including second product 436 (e.g., second product 436 is located between products 434a and 434b on the shelf). In some examples, virtual element 432a is presented in environment 404 with a configuration similar to that shown in FIG. Figure 4C One or more characteristics of the virtual element 418 a are presented in the environment 404 .

[0092] exist Figure 4GIn the example, the electronic device 402 detects a user interaction with the second product 436. Specifically, the user of the electronic device 402 lifts (e.g., grabs and / or moves) the second product 436 with the hand 408b. In some examples, the electronic device 402 adds the second product 436 to the digital shopping cart in response to detecting the user interaction (e.g., lifting the second product 436 with the hand 408b). Figure 4G 4, the user of electronic device 402 performs an air gesture using hand 408a. Specifically, hand 408a performs a pointing gesture toward second product 436. In some examples, in response to detecting the air gesture performed by hand 408a, electronic device 402 adds second product 436 to the digital shopping cart discussed above. It should be understood that in some examples, the air gesture may include a gesture with Figure 4G The gestures shown are different gestures, such as those with reference Figure 3C One or more characteristics of the one or more air gestures. Figure 4H As shown, based on the detected interaction and / or air gesture between the user of the electronic device 402 and the second product 436, the electronic device 402 adds the second product 436 to the digital shopping cart. Figure 4H In the example, the electronic device 402 presents a virtual element 432b in the environment 304 corresponding to a visual confirmation that the second product 436 is added to the digital shopping cart. In some examples, the presentation of the virtual element 432b in the environment 304 has a reference Figure 4E One or more characteristics of virtual element 418 c are shown and described as being presented in environment 304. Adding second product 436 to the digital shopping cart in response to detecting the mid-air gesture performed by hand 408 a ensures that the user of electronic device 402 intends to add second product 436 to the digital shopping cart and / or purchase second product 436, thereby reducing errors in the interaction and conserving computing resources associated with correcting errors.

[0093] Figure 4I-1 The electronic device 402 is illustrated as presenting, via the display generation component 430, a virtual element 433 indicating that one or more products of interest to the user of the electronic device 402 are not currently in a digital shopping cart (e.g., or optionally a real-world (e.g., physical) shopping cart). Figure 4I-1 As shown, virtual element 433 indicates a shopping list (e.g., as shown in reference Figure 3G The product (e.g., a nail) associated with the shopping list included in the user interface 328 shown and described has not yet been added to the digital shopping cart (e.g., the electronic device 402 has not detected a user interaction that triggers the electronic device 402 to add the product to the digital shopping cart, such as reference Figure 4G430). In some examples, the electronic device 402 presents the virtual element 433 based on determining that one or more criteria are satisfied. For example, the one or more criteria include a criterion that is satisfied when a product from the shopping list is available for purchase within a corresponding area of ​​the physical environment (e.g., a store) where the electronic device 402 is currently located and the product is not currently included in an active digital shopping cart (e.g., the digital shopping cart includes one or more products that have not yet been purchased). In addition, for example, the one or more criteria include a criterion that is satisfied when the electronic device 402 detects that the user of the electronic device 402 is located at and / or moves toward an exit of the corresponding area of ​​the physical environment (e.g., a checkout counter 438). Additionally or alternatively, for example, the one or more criteria include a criterion that is satisfied when the electronic device 402 detects that the user of the electronic device 402 has left the corresponding area of ​​the physical environment (e.g., the electronic device 402 detects that the current location of the electronic device 402 is outside the corresponding area of ​​the physical environment).

[0094] Figure 4I-2 to Figure 4K The electronic device 402 is illustrated as facilitating the purchase of one or more products in the digital shopping cart discussed above (e.g., including the first product 406 and / or the second product 436). For example, the electronic device 402 facilitates the purchase of one or more products by presenting one or more virtual elements in the environment 404 for guiding the user of the electronic device 402 through the digital purchase process. In some examples, the one or more virtual elements are associated with a digital payment service. For example, a user interface of the digital payment service (e.g., a user interface of a digital shopping cart) can be accessed by the user of the electronic device 402 through one or more user inputs (e.g., as described in reference to FIG. 4 ). Figure 4I-2As described above). In some examples, the user interface of the digital payment service is accessed through a device (e.g., a mobile phone, a tablet computer, and / or a smart watch) that communicates with the electronic device 402. In some examples, the digital payment service includes information associated with one or more payment methods (e.g., the payment method is defined by the user of the electronic device 402 through the user interface of the digital payment service and is saved to the user profile). For example, one or more payment methods include a default payment method (e.g., a payment method preferred by the user based on one or more settings associated with the user profile). In some examples, the electronic device 402 grants the user access to the digital payment service (e.g., initiating a purchase of one or more products in the digital shopping cart) based on satisfying one or more criteria. For example, one or more criteria include criteria that are satisfied when the identity of the user of the electronic device 402 is verified (e.g., using one or more authentication processes, such as biometric authentication (e.g., including voice, retinal, facial and / or fingerprint recognition) and / or two-factor authentication (e.g., including entering a password and / or using a second electronic device different from the electronic device 402)). As an alternative or in addition to verifying the identity of the user of electronic device 402 prior to accessing the digital payment service, electronic device 402 optionally verifies the identity of the user of electronic device 402 after accessing the digital payment service and prior to completing the purchase of one or more products included in the digital shopping cart (e.g., electronic device 402 responds to input provided by the user of electronic device 402 corresponding to a request to complete the purchase of one or more products included in the digital shopping cart (e.g., such as a reference to a user's ID). Figure 4J In some examples, the identity of the user of electronic device 402 is verified by a device (e.g., a mobile phone, a tablet computer, and / or a smart watch) that communicates with electronic device 402.

[0095] Figure 4I-2 The example illustrates a user of electronic device 402 providing input corresponding to a request to initiate a purchase of one or more products included in a digital shopping cart (e.g., input that satisfies one or more criteria for accessing a digital payment service). Figure 4I-2As shown, a user of electronic device 402 provides an audio input 440 (e.g., audio input 440 is a verbal command) corresponding to a request to initiate a purchase of one or more products included in a digital shopping cart. For example, electronic device 402 uses audio input 440 to authenticate the voice of the user of electronic device 402. In some examples, electronic device 402 initiates the purchase of one or more products included in the digital shopping cart through a user input different from a verbal input that satisfies one or more criteria (e.g., the user input includes an air gesture, an input provided through a hardware device (e.g., hardware buttons 460a and 460b), and / or an input provided through a touch-sensitive surface). In some examples, one or more criteria for accessing a digital payment service include a criterion that is satisfied when electronic device 402 is within a threshold distance (e.g., 0.5 meters, 1 meter, 2 meters, 5 meters, 10 meters, 15 meters, 20 meters, 25 meters, or 50 meters) of a location corresponding to an exit of a store. In some examples, the one or more criteria for accessing the digital payment service include criteria that are met when one or more devices (e.g., cameras and / or other sensors) located within the store (e.g., at the checkout counter 438 and / or near the store's exit) identify that one or more products owned by the user of the electronic device 402 are not currently being purchased. The one or more devices are optionally in communication with the electronic device 402. In some examples, the purchase of the one or more products is optionally initiated by a second device (e.g., a mobile phone, tablet computer, and / or smartwatch) in communication with the electronic device 402 (e.g., using a corresponding application associated with the digital payment service).

[0096] In some examples, the electronic device 402 automatically initiates a process for purchasing one or more products in the digital shopping cart (e.g., without direct user input corresponding to a request to initiate a purchase of the one or more products in the digital shopping cart). For example, when the electronic device 402 is within a threshold distance of a location corresponding to an exit of a store, the electronic device 402 automatically initiates a process for purchasing the one or more products (e.g., the electronic device 402 presents a virtual element 442a in the environment 404, as shown in FIG. 4 ). Figure 4J In some examples, when one or more devices located within the store detect and / or determine that the user of the electronic device 402 possesses one or more products and the one or more products have not yet been purchased, the electronic device 402 initiates a process for purchasing the one or more products in the digital shopping cart (e.g., the one or more devices communicate with the electronic device 402, and in response to the one or more devices providing an indication to the electronic device 402, the electronic device 402 presents the virtual element 442a in the environment 404, as shown in reference to FIG. Figure 4J). In some examples, one or more devices detect one or more products via a bar code, label, and / or other marking on the one or more products (e.g., on the packaging of the one or more products). In some examples, the one or more devices optionally communicate with the electronic device 402 and / or an application associated with a digital payment service, and the one or more devices detect the one or more products by retrieving information from a digital shopping cart. In some examples, the electronic device 402 initiates a process for purchasing one or more products in the digital shopping cart in response to detecting one or more objects associated with an exit of a store in the environment 404. For example, the electronic device 402 detects (e.g., using image sensors 414a-414c (e.g., optionally through object or image recognition)) an exit sign and / or a checkout counter 438 (e.g., in response to detecting one or more objects associated with an exit of a store in the environment 404, the electronic device 402 presents a virtual element 442a in the environment 404, as shown and described in detail). Figure 4J shown and described).

[0097] Figure 4J The electronic device 402 is shown in response to the Figure 4I-2 In some examples, virtual element 442a corresponds to a user interface of a digital shopping cart and / or a digital payment service application. Figure 4J As shown, three products are included in the digital shopping cart: screws (e.g., corresponding to the first product 406); tape (e.g., corresponding to the second product 436); and nails (e.g., corresponding to the third product (e.g., the third product is added to the digital shopping cart after the first product 406 and the second product 436 are added)). In some examples, the screws, tape, and nails correspond to products included in the digital shopping list stored in the file of the application of the electronic device 402 (e.g., as shown in FIG. Figure 3G As shown, the shopping list included in the user interface 328 includes Figure 4J In some examples, virtual element 442a includes the cost of each product and the total cost of one or more products in the digital shopping cart. Figure 4J 4, virtual element 442a includes selectable option 444 that can be selected to complete the purchase of one or more products in the digital shopping cart. In some examples, the digital shopping cart is displayed, optionally via a display generation component (e.g., different from display generation component 430) in communication with electronic device 402. For example, electronic device 402 optionally does not include display generation component 430 and / or does not present virtual environment 404.

[0098] like Figure 4J As shown, the electronic device 402 detects that the user of the electronic device 402 provides an input corresponding to a selection of the selectable option 444 (e.g., the input corresponding to the selection of the selectable option 444 includes attention (e.g., gaze 422) to the selectable option 444 and / or an air gesture performed using the hand 408a, as similar to the above discussion). In some examples, one or more user inputs other than the selection of the selectable option are used to complete the purchase of one or more products in the digital shopping cart. For example, the electronic device 402 completes the purchase of one or more products (e.g., through a digital payment service) in response to input provided through a hardware device (such as hardware buttons 460a and 460b) (e.g., the user of the electronic device 402 double-clicks the hardware buttons 460a and / or 460b, or the user of the electronic device 402 performs a press and hold input using the hardware buttons 460a and / or 460b). In some examples, the electronic device 402 alternatively completes the purchase of one or more products (e.g., through a digital payment service) in response to an audio input (e.g., a verbal command). In some examples, the electronic device 402 completes the purchase of one or more products in response to input provided through a touch-sensitive surface (eg, of a second device) in communication with the electronic device 402 .

[0099] In some examples, in response to a user input corresponding to a request to complete a purchase of one or more products in a digital shopping cart (e.g., in response to a request to complete a purchase of one or more products in a digital shopping cart). Figure 4J In some examples, electronic device 402 presents one or more visual and / or audio indications indicating that the purchase of the one or more products is complete. For example, Figure 4K As shown, electronic device 402 presents virtual element 442b in environment 404, which corresponds to a visual indication indicating that the purchase of one or more products included in the digital shopping cart is complete. In some examples, in addition to (e.g., or instead of) presenting virtual element 442b in environment 404, electronic device 402 provides an audio output to the user of electronic device 402 to provide an indication of the completion of the purchase of one or more products. In some examples, presenting virtual element 442b in environment 404 includes providing an audio output as described in reference to FIG. Figure 4EOne or more characteristics of virtual element 418c are shown and described as being presented in environment 404. In some examples, electronic device 402 optionally does not include display generation component 430 and / or optionally does not present environment 404, and confirmation of purchase of one or more products is provided via a device in communication with electronic device 402. For example, a virtual element (e.g., including one or more characteristics of virtual element 442b) is displayed by a display component of a device in communication with electronic device 402, and / or an audio output is provided by a speaker of the device (e.g., display of the virtual element and / or audio output is provided by interaction with an application associated with a digital payment service on a device in communication with electronic device 402).

[0100] Based on the completion of the purchase of one or more products in the digital shopping cart, the electronic device 402 ends the enhanced shopping experience (e.g., Figure 4A In some examples, based on the purchase of one or more products, electronic device 402 may update information associated with one or more products stored in an information repository (e.g., in memory and / or in a file of an application). For example, because electronic device 402 determines that the quantity of one or more products owned by the user of electronic device 402 exceeds a reference quantity, the user of electronic device 402 may leave the store. FIG. 3A to FIG. 3I The threshold amount may therefore be removed from the shopping list stored in the application's file (e.g., from Figure 3G In some examples, based on determining that the location of electronic device 402 is not within an area of ​​the physical environment corresponding to a store and / or business (e.g., the user of electronic device 402 leaves the store), electronic device 402 stops presenting virtual element 442b in environment 404. In some examples, electronic device 402 stops presenting virtual element 442b in environment 404 after 1 second, 2 seconds, 5 seconds, 10 seconds, 15 seconds, 20 seconds, 25 seconds, 30 seconds, 45 seconds, 60 seconds, or 120 seconds of presentation time in environment 404. In some examples, electronic device 402 responds to user input (e.g., with reference to Figure 4E The one or more characteristics of the user input for stopping the presentation of the virtual element 418c in the environment 404) are used to stop the presentation of the virtual element 418c.

[0101] Figure 5 A flowchart illustrating an example process for presenting one or more virtual elements associated with a product having a quantity less than a threshold amount in an environment according to some examples of the present disclosure. In some examples, process 500 begins at an electronic device in communication with one or more displays and one or more input devices. In some examples, the electronic device is optionally similar to or corresponding to Figure 2The head mounted display of the device 201. Figure 5 As shown, in some examples, at 502, the electronic device detects a first product in the physical environment of a user of the electronic device via one or more input devices. Figure 3A As shown, electronic device 302 detects product 306 (eg, a bag of screws) in the physical environment of a user of electronic device 302 .

[0102] In some examples, at 504, based on determining that one or more first criteria are satisfied, the electronic device performs one or more actions, the one or more first criteria comprising a first criterion that is satisfied when the electronic device detects one or more indications of interest by a user of the electronic device in the first product. In some examples, detecting the one or more indications of interest includes detecting attention (e.g., gaze) directed to the first product (e.g., as described in reference to Figure 3A In some examples, detecting one or more indications of interest includes detecting a user interaction with a first product in a physical environment (e.g., as described in reference to Figure 3B In some examples, detecting one or more indications of interest includes detecting a gesture (e.g., an air gesture), such as using Figure 3C In some examples, one or more actions include presenting one or more virtual elements in the environment via one or more displays in communication with the electronic device, such as a reference Figure 3E In some examples, the one or more virtual elements include one or more areas of a physical environment (e.g., a store / business) where the first product can be purchased. In some examples, the one or more actions include adding information associated with the first product to an information repository associated with a corresponding application. For example, Figure 3G As shown, electronic device 302 adds information associated with product 306 to a shopping list stored in a file of a corresponding application (eg, accessible to a user of electronic device 302 via user interface 328).

[0103] It should be understood that process 500 is an example, and more, fewer, or different operations may be performed in the same or different order. In addition, the operations in process 500 described above are optionally performed by running an information processing device such as a general-purpose processor (e.g., as described in relation to Figure 2 or one or more functional modules in a dedicated chip and / or through Figure 2 other components to achieve this.

[0104] Figure 6A flowchart illustrating an exemplary process for presenting one or more virtual elements associated with one or more products in an environment based on the location of an electronic device according to some examples of the present disclosure. In some examples, process 600 begins at an electronic device in communication with one or more displays and one or more input devices. In some examples, the electronic device is optionally similar to or corresponding to Figure 2 The head mounted display of the device 201. Figure 6 As shown, at 602, the electronic device detects the location of the electronic device in the physical environment. Figure 4A As shown and described, electronic device 402 detects that the location of electronic device 402 is within a store.

[0105] In some examples, at 604, based on determining that the location of the electronic device satisfies one or more first criteria, the electronic device presents, via one or more displays, in the environment at 606 one or more first virtual elements (e.g., such as reference elements) indicating a first location of a first product within a first area of ​​the physical environment. Figure 4B The one or more first criteria include a first criterion that is satisfied when the location of the electronic device is within a first area of ​​the physical environment that includes a first location corresponding to a first product, wherein the first product is associated with one or more indications of a user's intent to purchase the first product, and the one or more indications of the user's intent are detected by the electronic device. For example, as described with reference to Figure 4A As shown and described, electronic device 402 detects that the location of electronic device 402 is within a store that includes a first product (e.g., first product 406). For example, first product 406 corresponds to a product associated with one or more indications of a user's intent to purchase first product 406 detected by electronic device 402 (e.g., first product 406 corresponds to reference FIG. 3A to FIG. 3F In some examples, the product is included in an information repository (e.g., previously stored by the electronic device in a memory and / or in a file of an application). In some examples, the product is included in a shopping list included in a file of an application (e.g., as referenced in FIG. 1 ). Figure 3G shown and described).

[0106] It should be understood that process 600 is an example, and more, fewer, or different operations may be performed in the same or different order. In addition, the operations in process 600 described above are optionally performed by running an information processing device such as a general-purpose processor (e.g., as described in relation to Figure 2 or one or more functional modules in a dedicated chip and / or through Figure 2 other components to achieve this.

[0107] Therefore, in accordance with the above, some examples of the present disclosure relate to a method, the method comprising: at an electronic device in communication with one or more input devices: detecting, via the one or more input devices, a first product in a physical environment of a user of the electronic device. In some examples, the method further comprises: performing one or more actions based on determining that one or more first criteria are satisfied, the one or more first criteria comprising a first criterion that is satisfied when the electronic device detects one or more indications of interest of the user of the electronic device in the first product.

[0108] Additionally or alternatively, in some examples, the one or more actions include adding information associated with the first product to an information repository associated with the corresponding application.

[0109] Additionally or alternatively, in some examples, the one or more indications of interest of the user of the electronic device in the first product include an attention of the user of the electronic device to the first product.

[0110] Additionally or alternatively, in some examples, the one or more indications of interest by the user of the electronic device in the first product include a gesture performed by the user of the electronic device with respect to the first product.

[0111] Additionally or alternatively, in some examples, the one or more indications of interest by a user of the electronic device in the first product include user interactions with the first product in a physical environment.

[0112] Additionally or alternatively, in some examples, the electronic device is further in communication with one or more displays, and the one or more actions include presenting, via the one or more displays, a first virtual element associated with the first product in the environment.

[0113] Additionally or alternatively, in some examples, presenting the first virtual element includes presenting a visual indication of a first area in the physical environment where the first product is available for purchase, wherein the first area is a closest area of ​​one or more areas in the physical environment where the first product is available for purchase relative to a location of the electronic device.

[0114] Additionally or alternatively, in some examples, presenting the first virtual element includes presenting one or more visual indications of one or more areas in the physical environment where the first product is available for purchase, wherein the one or more visual indications of the one or more areas are presented based on the one or more areas being within a threshold distance of the location of the electronic device.

[0115] Additionally or alternatively, in some examples, the one or more visual indications of one or more areas of the physical environment include presenting one or more visual indications of one or more distances of the one or more areas of the physical environment relative to the location of the electronic device.

[0116] Additionally or alternatively, in some examples, one or more areas of the physical environment correspond to one or more predetermined areas of the physical environment selected according to one or more preferences of a user of the electronic device.

[0117] Additionally or alternatively, in some examples, one or more areas of the physical environment correspond to one or more areas of the physical environment selected according to a user history stored by the electronic device.

[0118] Additionally or alternatively, in some examples, presenting one or more visual indications of one or more areas of the physical environment includes presenting a visual indication of a first quantity of a first product available for purchase at a first area of ​​the one or more areas of the physical environment.

[0119] Additionally or alternatively, in some examples, the method further includes: when the first virtual element is presented in the environment, detecting, via one or more input devices, a first input corresponding to a user interaction with the first virtual element, wherein the user interaction corresponds to an indication of a user intent to purchase the first product; and in response to receiving the first input, storing information associated with the user intent to purchase the first product.

[0120] Additionally or alternatively, in some examples, the method further includes: when presenting the first virtual element in the environment, detecting, via one or more input devices, a first input that satisfies one or more second criteria; and in response to receiving the first input, ceasing to present the first virtual element in the environment.

[0121] Additionally or alternatively, in some examples, the method further includes: after performing one or more actions, based on determining that the position of the electronic device is within a first area of ​​the physical environment where the first product is available for purchase, presenting in the environment via one or more displays one or more first virtual elements indicating a first position of the first product within the first area of ​​the physical environment.

[0122] Additionally or alternatively, in some examples, the method further includes: after performing one or more actions, based on determining that one or more second criteria are met, presenting via one or more displays one or more first virtual elements indicating that the first product is available for purchase within a first area of ​​the physical environment.

[0123] Additionally or alternatively, in some examples, the one or more first criteria include a second criterion that is satisfied based on determining that the quantity of the first product is less than a threshold amount.

[0124] Additionally or alternatively, in some examples, the one or more input devices include one or more image sensors, and determining that the quantity of the first products is less than the threshold amount includes detecting the quantity of the first products using the one or more image sensors.

[0125] Additionally or alternatively, in some examples, the one or more input devices include one or more audio sensors, and determining that the quantity of the first product is less than the threshold amount includes detecting, using the one or more audio sensors, an audio input corresponding to a sound emitted from the first product.

[0126] Additionally or alternatively, in some examples, the electronic device includes a head-mounted display.

[0127] According to the above, some examples of the present disclosure relate to a method, the method comprising: at an electronic device that communicates with one or more displays and one or more input devices, detecting a position of the electronic device in a physical environment. In some examples, the method further comprises: presenting in the environment via one or more displays one or more first virtual elements indicating a first position of a first product in a first area of ​​the physical environment based on determining that the position of the electronic device satisfies one or more first criteria, the one or more first criteria comprising a first criterion satisfied when the position of the electronic device is within a first area of ​​the physical environment that includes a first position corresponding to the first product, wherein the first product is associated with one or more indications of a user's intent to purchase the first product, and the one or more indications of the user's intent are detected by the electronic device.

[0128] Additionally or alternatively, in some examples, the method also includes: presenting one or more second virtual elements different from the one or more first virtual elements in an environment associated with the first product via one or more displays based on determining that the position of the electronic device satisfies one or more first criteria and one or more second criteria, the one or more second criteria including a second criteria that is satisfied when the first product is in the field of view of a user of the electronic device.

[0129] Additionally or alternatively, in some examples, the method further includes: detecting, via one or more input devices, a first input corresponding to a user interaction with the one or more second virtual elements while the one or more second virtual elements are displayed; and in response to detecting the first input, adding the first product to the digital shopping cart.

[0130] Additionally or alternatively, in some examples, the method further includes: after adding the first product to the digital shopping cart, detecting, via one or more input devices, a second input that satisfies one or more criteria for initiating a purchase of one or more products included in the digital shopping cart including the first product; and in response to detecting the second input, facilitating the purchase of the one or more products.

[0131] Additionally or alternatively, in some examples, the method further includes: after adding the first product to the digital shopping cart, based on determining that one or more second criteria are met, presenting in the environment via one or more displays one or more second virtual elements associated with a second product different from the first product, the second product being available for purchase in the first area of ​​the physical environment.

[0132] Additionally or alternatively, in some examples, one or more indications of user intent are detected by the electronic device at a second location outside of the first area of ​​the physical environment.

[0133] Additionally or alternatively, in some examples, the one or more indications of user intent include a first indication corresponding to information associated with a first product stored in a file of the corresponding application.

[0134] Additionally or alternatively, in some examples, the file of the corresponding application includes information associated with multiple products including the first product. In some examples, the method also includes: based on determining that the first area of ​​the physical environment includes a second location corresponding to a second product of the multiple products that is different from the first product, presenting in the environment via one or more displays one or more second virtual elements indicating a second location of the second product within the first area of ​​the physical environment.

[0135] Additionally or alternatively, in some examples, the method further includes: detecting, via one or more input devices, a first input satisfying one or more second criteria while presenting one or more first virtual elements in the environment; and in response to receiving the first input, stopping presenting the one or more first virtual elements in the environment.

[0136] Additionally or alternatively, in some examples, the electronic device includes a head-mounted display.

[0137] Some examples of the present disclosure relate to an electronic device comprising: one or more processors; a memory; and one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for executing any of the above methods.

[0138] Some examples of the present disclosure relate to a non-transitory computer-readable storage medium that stores one or more programs that include instructions that, when executed by one or more processors of an electronic device, cause the electronic device to perform any of the above methods.

[0139] Some examples of the present disclosure relate to an electronic device comprising one or more processors, a memory, and a component for performing any of the above methods.

[0140] Some examples of the present disclosure relate to an information processing apparatus used in an electronic device, the information processing apparatus including a component for executing any one of the above methods.

[0141] The present disclosure contemplates that, in some instances, the data utilized may include personal information data that uniquely identifies or can be used to contact or locate a specific person. Such personal information data may include demographic data, content consumption activity, location-based data, phone numbers, email addresses, social media IDs, home addresses, data or records related to a user's health or fitness level (e.g., vital sign measurements, medication information, exercise information), date of birth, or any other identifying or personal information.

[0142] The present disclosure recognizes that the use of such personal information data in the present technology can be used to benefit users. For example, content consumption activities can be used to suggest content to users. Therefore, the use of such personal information data enables users to use electronic devices in a coordinated manner. In addition, the present disclosure also anticipates other uses of personal information data that benefit users. For example, health and fitness data can be used to provide insights into the user's overall health status, or can be used as positive feedback to individuals who use technology to pursue health goals.

[0143] The disclosure anticipates that entities responsible for the collection, analysis, disclosure, transmission, storage or other purposes of such personal information data will comply with sound privacy policies and / or privacy measures. Specifically, such entities should implement and adhere to the use of privacy policies and practices that are recognized as meeting or exceeding the industry or government requirements for maintaining the privacy and security of personal information data. Such policies should be easy for users to access and should be updated as changes in the collection and / or use of data occur. Personal information from users should be collected for legal and reasonable entity purposes and should not be shared or sold outside these legal purposes. In addition, such collection / sharing should be carried out after receiving the informed consent of the user. In addition, such entities should consider taking any necessary steps to protect and safeguard access to such personal information data, and ensure that other entities with access to personal information data comply with their privacy policies and procedures. In addition, such entities can subject themselves to third-party assessments to prove that they comply with widely accepted privacy policies and practices. In addition, policies and practices should be adapted to specific types of personal information data collected and / or accessed, and to applicable laws and standards including considerations of special jurisdictions. For example, in the United States, the collection or acquisition of certain health data may be governed by federal and / or state laws, such as the Health Insurance Portability and Accountability Act (HIPAA); while health data in other countries may be subject to other regulations and policies and should be handled accordingly. Therefore, different privacy measures should be advocated for different types of personal data in each country.

[0144] Regardless of the foregoing, the present disclosure also contemplates implementation schemes in which users selectively block the use or access of personal information data. That is, the present disclosure contemplates hardware elements and / or software elements to prevent or block access to such personal information data. For example, with respect to network services, the present technology may be configured to allow a user to choose to "opt in" or "opt out" to participate in the collection of personal information data at any time during or after registration for the service. In another example, a user may choose not to enable content consumption activity tracking. In addition to providing "opt-in" and "opt-out" options, the present disclosure also contemplates providing notifications related to access or use of personal information. For example, a user may be notified that their personal information data will be accessed when content playback is initiated, and then the user may be reminded again just before the personal information data is accessed by the device.

[0145] In addition, it is the intention of the present disclosure that personal information data should be managed and processed in a manner that minimizes the risk of unintentional or unauthorized access or use. Once the data is no longer needed, the risk can be minimized by limiting data collection and data deletion. In addition, and when applicable, including in certain health-related applications, data de-identification can be used to protect the privacy of users. De-identification can be facilitated by removing specific identifiers (e.g., date of birth, etc.), controlling the amount or specificity of stored data (e.g., collecting location data at the city level rather than at the address level), controlling how the data is stored (e.g., aggregating data between users), and / or other methods when appropriate.

[0146] Thus, while the present disclosure broadly covers the use of personal information data to implement one or more of the various disclosed embodiments, the present disclosure also contemplates that various embodiments may also be implemented without access to such personal information data. That is, various embodiments of the present technology will not fail to function properly due to the lack of all or a portion of such personal information data. For example, synchronous media and communication session interactions may be performed in accordance with embodiments of the present disclosure without tracking a user's content consumption activity.

[0147] For the purpose of explanation, the foregoing description is described by reference to specific examples. However, the above illustrative discussion is not intended to be exhaustive or to limit the present disclosure to the precise form disclosed. Based on the above teachings, many modifications and variations are possible. The examples are selected and described in order to best illustrate the principles of the present disclosure and its practical application, so as to enable other technical personnel in the field to best use the present disclosure and the various described examples with various modifications suitable for the specific purpose contemplated.

Claims

1. A method, comprising: At an electronic device in communication with one or more input devices: detecting, via the one or more input devices, a first product in a physical environment of a user of the electronic device; as well as Based on determining that one or more first criteria are satisfied, one or more actions are performed, the one or more first criteria including a first criterion that is satisfied when the electronic device detects one or more indications of interest by the user of the electronic device in the first product. 2 . The method of claim 1 , wherein the one or more actions include adding information associated with the first product to an information repository associated with a corresponding application. 3 . The method of claim 1 , wherein the one or more indications of interest of the user of the electronic device in the first product include attention of the user of the electronic device to the first product. 4 . The method of claim 1 , wherein the one or more indications of interest of the user of the electronic device in the first product comprise gestures performed by the user of the electronic device with respect to the first product. 5 . The method of claim 1 , wherein the one or more indications of interest of the user of the electronic device in the first product include user interactions with the first product in the physical environment. 6 . The method of claim 1 , wherein the electronic device is further in communication with one or more displays, and wherein the one or more actions include presenting a first virtual element associated with the first product in an environment via the one or more displays.

7. A method according to claim 6, wherein presenting the first virtual element includes presenting a visual indication of a first area in the physical environment where the first product is available for purchase, wherein the first area is a closest area of ​​one or more areas in the physical environment where the first product is available for purchase relative to the location of the electronic device.

8. A method according to claim 6, wherein presenting the first virtual element includes presenting one or more visual indications of one or more areas in the physical environment where the first product is available for purchase, wherein the one or more visual indications of the one or more areas are presented based on the one or more areas being within a threshold distance of the location of the electronic device.

9. The method of claim 8, wherein presenting the one or more visual indications of the one or more areas of the physical environment comprises presenting one or more visual indications of one or more distances of the one or more areas of the physical environment relative to the location of the electronic device.

10. The method of claim 8, wherein the one or more areas of the physical environment correspond to one or more predetermined areas of the physical environment selected according to one or more preferences of the user of the electronic device.

11. The method of claim 8, wherein the one or more regions of the physical environment correspond to one or more regions of the physical environment selected according to a user history stored by the electronic device.

12. The method of claim 8, wherein presenting the one or more visual indications of the one or more areas of the physical environment comprises presenting a visual indication of a first quantity of the first product available for purchase at a first area of ​​the one or more areas of the physical environment.

13. The method according to claim 6, further comprising: detecting, via the one or more input devices, a first input corresponding to a user interaction with the first virtual element while the first virtual element is presented in the environment, wherein the user interaction corresponds to an indication of a user intent to purchase the first product; and In response to receiving the first input, information associated with the user intent to purchase the first product is stored.

14. The method according to claim 6, further comprising: detecting, via the one or more input devices, a first input satisfying one or more second criteria while presenting the first virtual element in the environment; as well as In response to receiving the first input, presentation of the first virtual element in the environment is stopped.

15. The method according to claim 1, further comprising: After performing the one or more actions, based on determining that the position of the electronic device is within a first area of ​​the physical environment where the first product is available for purchase, one or more first virtual elements indicating the first position of the first product within the first area of ​​the physical environment are presented via one or more displays.

16. The method according to claim 1, further comprising: After performing the one or more actions, based on determining that one or more second criteria are satisfied, presenting, via one or more displays, one or more first virtual elements indicating that the first product is available for purchase within a first area of ​​the physical environment.

17. The method of claim 1, wherein the one or more first criteria include a second criterion that is satisfied based on determining that the quantity of the first product is less than a threshold amount.

18. The method of claim 17, wherein the one or more input devices include one or more image sensors, and determining that the quantity of the first product is less than the threshold amount includes detecting the quantity of the first product using the one or more image sensors.

19. The method of claim 17, wherein the one or more input devices include one or more audio sensors, and determining that the quantity of the first product is less than the threshold amount includes detecting, using the one or more audio sensors, an audio input corresponding to a sound emitted from the first product.

20. The method of claim 1, wherein the electronic device comprises a head mounted display.

21. An electronic device, comprising: one or more processors; Memory; as well as One or more programs, the one or more programs are stored in the memory and are configured to be executed by the one or more processors, the one or more programs comprising instructions for executing the method according to any one of claims 1 to 20.

22. A non-transitory computer-readable storage medium storing one or more programs, wherein the one or more programs include instructions that, when executed by one or more processors of an electronic device, cause the electronic device to perform a method according to any one of claims 1 to 20.

Images