Dynamic metadata generation and processing

Abstract

The present disclosure relates to dynamic metadata generation and processing. In some embodiments, an electronic device obtains and displays a file container that defines a virtual object corresponding to a physical object. The file container also defines multiple other allowed configurations of the virtual object. At a later time, the electronic device displays the virtual object using an alternative process. In response to user input corresponding to a request to update the configuration of the object, the electronic device contacts the source of the object to obtain the availability of the updated configuration of the object. In some embodiments, in response to an indication that a physical object with an updated configuration is available, the electronic device presents the virtual object with the updated configuration.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Application No. 63 / 168,121, filed on March 30, 2021, and U.S. Patent Application No. 17 / 654,391, filed on March 10, 2022, the contents of which are incorporated herein by reference in their entirety for all purposes. Technical Field

[0003] The present invention relates generally to presenting objects in a computerized environment, and more particularly to presenting objects in such an environment using dynamic metadata. Background Art

[0004] Online marketplaces enable manufacturers and retailers to market goods to potential customers. Online marketplaces may include information about goods for sale, including one or more images of the goods and one or more available configurations of the goods. Some online marketplaces may enable users to view virtual representations of the available goods in augmented reality (AR), for example, to show the user the actual size and / or scale of the available goods. Summary of the Invention

[0005] Some embodiments described in the present disclosure relate to realistic virtual objects. For example, a virtual object can correspond to a physical object that can be used for purchase. In some embodiments, an electronic device can display a virtual object based on a file of dynamic metadata that defines an object. For example, multiple variants of the physical object can be available for use. In some embodiments, in response to a request to display a virtual object with a configuration different from the current configuration of the virtual object, the electronic device can contact the source of the virtual object (for example, a second electronic device associated with the manufacturer or retailer of the object, such as a server) to obtain the availability (for example, for purchase) of the physical object with the requested configuration. In some embodiments, when the availability of various configurations changes, the electronic device requests and / or receives frequent updates from the source of the virtual object. A comprehensive description of the embodiments is provided in the accompanying drawings and the specific embodiments, and it should be understood that this disclosure does not limit the scope of this disclosure in any way. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] For a better understanding of the various described embodiments, reference should be made to the following detailed description taken in conjunction with the following drawings, in which like reference numerals tend to designate corresponding parts throughout the several views.

[0007] Figure 1 An electronic device displaying XR content according to some embodiments of the present disclosure is shown.

[0008] Figure 2A block diagram illustrating an exemplary architecture for an electronic device according to some embodiments of the present disclosure is shown.

[0009] Figures 3A-3D An exemplary manner in which an electronic device presents options associated with a virtual object according to some embodiments is shown.

[0010] Figure 4A-4B An exemplary manner in which an electronic device presents options associated with a virtual object according to some embodiments is shown.

[0011] Figure 5A-5B An exemplary manner in which an electronic device presents configuration information associated with real objects in the physical environment of the electronic device according to some embodiments is shown.

[0012] Figure 6A Exemplary notifications related to virtual objects are shown in accordance with some embodiments.

[0013] Figure 6B An electronic device presenting a web browsing user interface and a virtual collection user interface in accordance with some embodiments is shown.

[0014] Figure 7 An exemplary file defining a virtual object corresponding to a real object is shown in accordance with some embodiments.

[0015] Figure 8 An example process for presenting a virtual object according to some embodiments is shown. DETAILED DESCRIPTION

[0016] In the following description of the embodiment, the accompanying drawings will be cited, which form a part of the following description and show, by way of example, specific embodiments of optional implementation in the accompanying drawings. It should be understood that, without departing from the scope of the disclosed embodiment, other embodiments may be optionally used and structural changes may be optionally made. In addition, although the following description uses the terms "first", "second", etc. to describe various elements, these elements should not be limited by the terms. These terms are simply used to distinguish one element from another. For example, without departing from the scope of the various described embodiments, a first object or a first file may be named as a second object or a second file, and similarly, a second object or a second file may be named as a first object or a first file. Both the first object and the second object are objects, but they are not the same object. Similarly, both the first file and the second file are files, but they are not the same file.

[0017] Some embodiments described in the present disclosure relate to realistic virtual objects. For example, a virtual object can correspond to a physical object that can be used for purchase. In some embodiments, an electronic device can display a virtual object based on a file of dynamic metadata that defines an object. For example, multiple variants of the physical object can be available for use. In some embodiments, in response to a request to display a virtual object with a configuration different from the current configuration of the virtual object, the electronic device can contact the source of the virtual object (for example, a second electronic device associated with the manufacturer or retailer of the object, such as a server) to obtain the availability (for example, for purchase) of the physical object with the requested configuration. In some embodiments, when the availability of various configurations changes, the electronic device requests and / or receives frequent updates from the source of the virtual object. A comprehensive description of the embodiments is provided in the accompanying drawings and the specific embodiments, and it should be understood that this disclosure does not limit the scope of this disclosure in any way.

[0018] The terms used in the description of the various embodiments described herein are only for the purpose of describing specific embodiments and are not intended to be limiting. As used in the description of the various embodiments described and in the appended claims, the singular forms "a" and "the" are intended to also include plural forms unless the context clearly indicates otherwise. It will also be understood that the terms "and / or" used herein refer to and encompass any and all possible combinations of one or more items in the associated listed items. It will also be understood that the terms "comprise" and / or "comprising" when used in this manual specify the presence of stated features, integers, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts and / or their groupings.

[0019] The term "if" is optionally interpreted to mean "when," "upon," or "in response to determining," or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined that," or "if [stated condition or event] is detected" are optionally interpreted to mean "upon determining," or "in response to determining," or "upon detecting [stated condition or event]," or "in response to detecting [stated condition or event]," depending on the context.

[0020] The physical environment refers to the physical world that people can sense and / or interact with without the help of electronic systems. A physical environment, such as a physical park, includes physical objects, such as physical trees, physical buildings, and physical people. People can directly sense and / or interact with the physical environment, such as through sight, touch, hearing, taste, and smell.

[0021] In contrast, an extended reality (XR) environment refers to a fully or partially simulated environment that people perceive and / or interact with via electronic systems. In some embodiments, in XR, a subset of a person's physical motion, or a representation thereof, is tracked, and in response, one or more features of one or more virtual objects simulated in the XR environment are adjusted in a manner that conforms to at least one law of physics. For example, an XR system may detect device rotation and / or movement, and in response, adjust the graphical content and sound field presented to the person in a manner similar to the way such views and sounds change in a physical environment. In some cases (e.g., for accessibility reasons), adjustments to features of virtual objects in the XR environment may be made in response to representations of physical motion (e.g., voice commands). Examples of XR include virtual reality, mixed reality, and augmented reality.

[0022] A person can sense XR objects and / or interact with CGR objects using any of their senses, including vision, hearing, touch, taste, and smell. For example, a person can sense and / or interact with an audio object that creates a 3D or spatial audio environment that provides the perception of a point audio source in 3D space. As another example, an audio object can enable audio transparency that selectively introduces ambient sounds from the physical environment with or without computer-generated audio. In some XR environments, a person can sense and / or interact only with audio objects. Examples of XR include virtual reality and mixed reality.

[0023] There are many different types of electronic systems that enable people to sense and / or interact with various XR environments. Examples include head-mounted systems, projection-based systems, heads-up displays (HUDs), vehicle windshields with integrated display capabilities, windows with integrated display capabilities, displays formed as lenses designed to be placed on a person's eyes (e.g., similar to contact lenses), headphones / earpieces, speaker arrays, input systems (e.g., wearable or handheld controllers with or without tactile feedback), smartphones, tablets, and desktop / laptop computers. A head-mounted system can have one or more speakers and an integrated opaque display. Alternatively, a head-mounted system can be configured to accept an external opaque display (e.g., a smartphone). A head-mounted system can incorporate one or more imaging sensors for capturing images or video of the physical environment, and / or one or more microphones for capturing audio of the physical environment. Instead of an opaque display, a head-mounted system can have a transparent or translucent display. The transparent or translucent display can have a medium through which light representing the image is directed to the person's eyes. The display can utilize digital light projection, OLED, LED, uLED, liquid crystal on silicon, laser scanning light source, or any combination of these technologies. The medium can be an optical waveguide, a holographic medium, an optical combiner, an optical reflector, or any combination thereof. In one embodiment, a transparent or translucent display can be configured to selectively become opaque. Projection-based systems can employ retinal projection technology that projects graphic images onto a person's retina. Projection systems can also be configured to project virtual objects into a physical environment, for example, as a hologram or onto a physical surface.

[0024] Figure 1 An electronic device 100 is shown that displays XR content according to some embodiments of the present disclosure. In some embodiments, the electronic device 100 is a handheld or mobile device, such as a tablet computer, a laptop computer, or a smartphone. Figure 2 An example of an electronic device 100 is described. Figure 1 As shown, the electronic device 100 and the table 120 are located in the physical environment 110. In some embodiments, the electronic device 100 can be configured to capture an area of ​​the physical environment 110 that includes the table 120 (shown in the field of view of the electronic device 100). In some embodiments, in response to a trigger, the electronic device 100 can be configured to display a 3D XR object 130 (e.g., Figure 1), which is located on top of the XR representation 120' of the real-world table 120. For example, an object 130 can be displayed on the surface of the table 120' in the XR environment displayed on the device 100 in response to detecting the flat surface of the table 120 in the physical environment 110. In some embodiments, the electronic device 100 is able to display virtual objects on the virtual surfaces of other virtual objects. For example, if the table 120 is not in the physical environment 110 of the electronic device 100 and the electronic device 100 does not display the virtual table 120' based on the physical table 120, the electronic device 100 can still display the cube 130 on the surface of the virtual table 120'. In some embodiments, a user may want to view a virtual object corresponding to a physical object for purchase in the XR environment. The electronic device can display the virtual object based on a file that includes one or more properties of the object with adjustable values. In some embodiments, the file defines the source of the physical object corresponding to the virtual object, and the electronic device is able to contact the source of the physical object corresponding to the virtual object to obtain real-time availability of the object with various configurations (e.g., for purchase). The examples described herein describe systems and methods for dynamically updating one or more properties of a virtual object corresponding to a real object.

[0025] Figure 2 1 is a block diagram illustrating an exemplary architecture for a system or device 200 according to some embodiments. In some embodiments, the device 200 is a mobile device, such as a mobile phone (e.g., a smartphone), a tablet computer, a laptop computer, a desktop computer, an auxiliary device that communicates with another device, etc. In some embodiments, as Figure 2 As shown, device 200 includes various components such as communication circuitry 202, processor 204, memory 206, image sensor 210, position sensor 214, orientation sensor 216, microphone 218, touch-sensitive surface 220, speaker 222, and / or display generation component 224. These components optionally communicate via a communication bus 208 of device 200.

[0026] Device 200 includes communication circuitry 202. Communication circuitry 202 optionally includes circuitry for communicating with electronic devices, networks such as the Internet, an intranet, a wired network and / or a wireless network, a cellular network, and a wireless local area network (LAN). Communication circuitry 202 optionally includes circuitry for communicating using near field communication and / or short range communication such as ) circuit for communication.

[0027] The processor 204 includes one or more general purpose processors, one or more graphics processors, and / or one or more digital signal processors. In some embodiments, the memory 206 is a non-transitory computer readable storage medium (e.g., flash memory, random access memory) that stores computer readable instructions that are configured to be executed by the processor 204 to perform the techniques, processes, and / or methods described below (e.g., reference to Figure 3A-Figure 8 ). In some embodiments, the memory 206 may include more than one non-transitory computer-readable storage medium. A non-transitory computer-readable storage medium may be any medium (e.g., excluding signals) that can tangibly contain or store computer-executable instructions for use by or in conjunction with an instruction execution system, apparatus, and device. In some embodiments, the storage medium is a transient computer-readable storage medium. In some embodiments, the storage medium is a non-transitory computer-readable storage medium. Non-transitory computer-readable storage media may include, but are not limited to, magnetic storage devices, optical storage devices, and / or semiconductor storage devices. Examples of such storage devices include magnetic disks, optical disks based on CD, DVD, or Blu-ray technology, and persistent solid-state memory such as flash memory, solid-state drives, and the like.

[0028] The device 200 includes a display generation component 224. In some embodiments, the display generation component 224 may include a single display, such as an LED or LCD display, and in other embodiments, the display generation component may include a projector, a display with touch capabilities, a retinal projector, etc. In some embodiments, the display generation component 224 includes multiple displays. In some embodiments, the device 200 includes one or more touch-sensitive surfaces 220 for receiving user input, such as tap input and swipe input or other gestures. In some embodiments, the display generation component 224 and the touch-sensitive surface 220 form a touch-sensitive display, for example, a touch screen integrated with the device 200 or a touch screen external to the device 200 that communicates with the device 200.

[0029] Device 200 optionally includes an image sensor 210. Image sensor 210 optionally includes one or more visible light image sensors (such as a charge-coupled device (CCD) sensor) and / or a complementary metal oxide semiconductor (CMOS) sensor operable to obtain images of physical objects from a real environment. Image sensor 210 also optionally includes one or more infrared (IR) sensors, such as passive IR sensors or active IR sensors, for detecting infrared light from the real environment. For example, an active IR sensor includes an IR emitter, such as an IR spot emitter, for emitting infrared light into the real environment. Image sensor 210 also optionally includes one or more event cameras configured to capture the movement of physical objects in the real environment. Image sensor 210 also optionally includes one or more depth sensors configured to detect the distance of physical objects from device 200. In some embodiments, information from the one or more depth sensors can allow the device to identify and distinguish objects in the real environment from other objects in the real environment. In some embodiments, the one or more depth sensors can allow the device to determine the texture and / or topography of objects in the real environment.

[0030] In some embodiments, the device 200 uses a combination of a CCD sensor, an event camera, and a depth sensor to detect the physical environment around the device 200. In some embodiments, the image sensor 220 includes a first image sensor and a second image sensor. The first image sensor and the second image sensor work in tandem and are optionally configured to capture different information about physical objects in the real environment. In some embodiments, the first image sensor is a visible light image sensor and the second image sensor is a depth sensor. In some embodiments, the device 200 uses the image sensor 210 to detect the position and orientation of the device 200 and / or the display generation component 224 in the real environment. For example, the device 200 uses the image sensor 210 to track the position and orientation of the display generation component 224 relative to one or more fixed objects in the real environment.

[0031] In some embodiments, the device 200 includes a microphone 218. The device 200 uses the microphone 218 to detect sounds from the user and / or the user's real environment. In some embodiments, the microphone 218 includes a microphone array (including multiple microphones) that optionally operate in conjunction to identify ambient noise or locate sound sources in the space of the real environment.

[0032] Device 200 includes a location sensor 214 for detecting the location of device 200 and / or display generating component 224. For example, location sensor 214 may include a GPS receiver that receives data from one or more satellites and allows device 200 to determine the absolute location of the device in the physical world.

[0033] Device 200 includes an orientation sensor 216 for detecting the orientation and / or movement of device 200 and / or display generation component 224. For example, device 200 uses orientation sensor 216 to track changes in the position and / or orientation of device 200 and / or display generation component 224, such as relative to physical objects in a real environment. Orientation sensor 216 optionally includes one or more gyroscopes and / or one or more accelerometers. Device 200 is not limited to Figure 2 Rather, the present invention does not include any specific components and configurations, but may include other components or additional components in various configurations.

[0034] Attention is now directed to examples of a graphical user interface ("UI") and associated processes implemented on an electronic device, such as electronic device 100 or device 200. The GUI may include the display of one or more configurable virtual objects corresponding to physical objects.

[0035] Figures 3A-3D FIG. 1 shows an exemplary manner in which the electronic device 100 presents options associated with a virtual object according to some embodiments. Figure 3A As shown, in some embodiments, the electronic device 100 can present a representation of the physical environment 110, including an XR representation 302' of a carpet 302 in the physical environment 110 and a virtual sofa 304. In some embodiments, the virtual sofa 304 can correspond to a physical sofa that can be purchased through a retailer and / or manufacturer.

[0036] In some embodiments, the electronic device 100 is based on a device having the same Figure 7A file in the same or similar format as the file 700 described above presents the virtual sofa 304. The file can be provided by the source of the virtual sofa 304. In some embodiments, the source is a second electronic device (e.g., a server) associated with a retailer and / or manufacturer of a physical sofa corresponding to the virtual sofa 304. The file corresponding to the virtual sofa 304 may include a configuration of one or more characteristics currently displayed with the virtual sofa 304 and an indication of the source of the virtual sofa 304. For example, the one or more characteristics may include options available to customers who wish to purchase a physical sofa corresponding to the virtual sofa 304, such as an upholstery style, a base style, and a size. Other characteristics are possible. In some embodiments, the virtual sofa 304 is displayed in an XR environment that is not uniquely associated with the source of the virtual sofa 304. For example, instead of displaying the virtual sofa 304 in an XR environment associated with an online store of the source of the virtual sofa 304, the virtual sofa 304 may be displayed in an XR environment in which virtual objects from multiple sources can be displayed. In some embodiments, the other virtual objects in the XR environment may be other virtual objects corresponding to physical objects available for purchase, one or more items of content, one or more application windows, or other virtual content accessible to the electronic device 100 .

[0037] In some embodiments, in response to detecting a request from a user to change one or more characteristics of the virtual couch 304, the electronic device 100 sends a query for the availability of the updated characteristics to the source of the virtual couch 304. In response, in some embodiments, the electronic device 100 receives an indication of the availability of the updated characteristics from the virtual couch source. Figure 3A As shown, for example, the electronic device 100 detects user input selecting a virtual sofa 304. In some embodiments, the user input is the user touching the virtual sofa 304 with a hand 306. For example, the electronic device 100 includes a touch screen, and the electronic device 100 detects the location where the hand 306 touches the touch screen, at which the virtual sofa 304 is displayed. In some embodiments, the electronic device 100 can communicate with other types of input devices. Therefore, the user input selecting the sofa can be an input other than a touch input.

[0038] exist Figure 3B In response to Figure 3A, the electronic device 100 presents a menu 308 including variable properties of the virtual sofa 304. For example, the menu 308 may include an option menu 310 for the style of the footing of the virtual sofa 304 and an option menu 312 for the upholstery of the sofa. In some implementations, the menu 308 further includes: an option 314, which, when selected, causes the electronic device 100 to initiate a process for purchasing a physical sofa having the same configuration as the virtual sofa 304; and an option 314, which, when selected, causes the electronic device 100 to stop displaying the menu 308.

[0039] In some embodiments, the characteristics included in the menu 308 may be defined by characteristics included in a file that the electronic device 100 uses to generate the virtual sofa 304. In some embodiments, before displaying the menu 308, the electronic device 100 responds to the selection of Figure 3A In some embodiments, the electronic device 100 receives a virtual sofa 304 and sends a request for the currently available features of the virtual sofa 304 to the source of the virtual sofa 304. For example, over time, the value of each feature and / or feature that the user can modify may change according to the configuration of the physical sofa currently available from the source. In some embodiments, the source of the virtual sofa 304 sends an updated configuration to the electronic device 100 on demand or automatically (for example, at fixed time intervals, whenever the available configuration changes). In some embodiments, the configuration of the virtual sofa 304 is stored away from the electronic device 100 (for example, stored on a server associated with the source of the virtual sofa 304), and the electronic device 100 contacts the remote storage location to obtain the available configurations of the virtual sofa 304 in response to the input corresponding to the request to present the virtual sofa 304.

[0040] In some embodiments, the electronic device can also present other virtual objects corresponding to the physical objects that can be used for sale, but does not include purchase information in the file defining the virtual object, such as the logic for completing the purchase by interacting with the virtual object. In these cases, for example, the electronic device can present the virtual object and one or more menus for configuring the virtual object (e.g., similar to menus 310 and 312 associated with the virtual sofa 304), but may not present the option for purchasing the object (e.g., similar to option 314). In some embodiments, the virtual object that does not include purchase logic can still be user-configurable based on the configuration settings embedded in the file defining the object. In some embodiments, when presenting a virtual object provided by a source that is not authorized by the manufacturer of the physical object corresponding to the virtual object, the electronic device does not contact the source of the virtual object to dynamically update the available configurations of the virtual object.

[0041] In some embodiments, in response to detecting that a different value for any characteristic of the virtual sofa 304 is selected via user interaction with the menu 310 or the menu 312, the electronic device 100 may send a query to the source of the virtual sofa 304 regarding the availability of the selected configuration. In some embodiments, in response to receiving an indication from the source that the selected configuration is available (e.g., for purchase), the electronic device 100 updates the virtual sofa 304 that will be displayed as having the selected configuration. In some embodiments, the source of the virtual sofa 304 checks the availability of each characteristic in the configuration separately (e.g., checking for a sofa without legs and checking for a sofa with the selected upholstery). In some embodiments, the source of the virtual sofa 304 checks the availability of the characteristics of the configuration in combination (e.g., checking for a sofa without corners but with the selected upholstery).

[0042] For example, in Figure 3C In the embodiment, the electronic device 100 is presented with Figure 3B In some embodiments, the electronic device 100 updates the virtual sofa 304 in response to detecting a sequence of one or more user inputs. Figure 3C The one or more user inputs include selecting an updated interior style in the menu 312. In some embodiments, the electronic device 100 displays the virtual sofa 304 in response to receiving an indication from the source of the virtual sofa 304 that the selected interior is available. Figure 3C The updated virtual sofa 304 in .

[0043] In some embodiments, if the price of the virtual sofa 304 with the updated configuration is different from the price of the virtual sofa 304 with the previous indication, the electronic device 100 can update the indication of the sofa price included in the option 314 to purchase the physical sofa corresponding to the virtual sofa 304. In some embodiments, the price of the sofa can change over time due to sales and promotions. In some embodiments, by querying the source of the sofa for its current price, the electronic device 101 can present current and accurate pricing information that takes into account any relevant sales and / or promotions. In some embodiments, factors such as other items in the user's shopping cart can affect the pricing information presented by the electronic device. For example, a seller can run a buy-one-get-one-free promotion or a free shipping promotion based on the total purchase price of the user's shopping cart, which causes the pricing information to change based on the available promotions and whether the user's shopping cart meets the promotion conditions.

[0044] In some cases, the user may request a configuration of the virtual couch 304 that is not currently available. In some embodiments, in response to the user's request, the electronic device 100 sends a query to the source of the virtual couch 304 for the availability of the configuration requested by the user, and in response, receives an indication from the source that the requested configuration is not available (e.g., rather than receiving an indication of availability and price, if the configuration is available). In some embodiments, such as Figure 3D As shown, in response to the indication that the requested configuration is not available, the electronic device 100 displays an indication 318 that the requested configuration is not available within the menu 308. For example, in Figure 3D , the user has selected from menu 312 Figure 3A-3C In some embodiments, the interior style shown in Figure 3D As shown, even if the configuration is not available, the electronic device 100 updates the display of the virtual sofa 304 to be displayed with the configuration requested by the user. However, in some embodiments, the electronic device 100 stops updating the virtual sofa 304 instead of updating the virtual sofa 304 to be displayed with the unavailable configuration. In contrast, in some examples, such as Figure 3C , the electronic device 100 continues to display the virtual couch 304, as shown. In some examples, the electronic device 100 stops displaying the virtual couch 304 in the XR environment. In some embodiments, the requested object may be unavailable because it is not available for purchase (e.g., a physical object with the requested configuration has been sold) or because the source of the object cannot verify the digital rights management information associated with the requested configuration. For example, the digital rights management information may not be verified because (e.g., the manufacturer of the physical object) does not allow the source of the virtual object to sell a physical item with the requested configuration.

[0045] It should be understood that the electronic device 100 can adjust the characteristics of the virtual sofa 304, and more generally, adjust the characteristics of other virtual objects corresponding to physical objects available for purchase, in addition to the interior style. For example, the electronic device 100 can present a virtual sofa 304 with different configurations of the sofa's footing style in a manner similar to how the electronic device 100 displays virtual sofas 304 with various interior styles. For example, in response to detecting that the user selects a different footing style from the menu 310, the electronic device 100 sends a query to the sofa source to obtain the availability of sofas with configurations that include the user-selected footing style. In some embodiments, the virtual sofa 304 includes additional characteristics that can be similarly modified by the user.

[0046] As mentioned above Figures 3A-3DAs described, in some embodiments, a virtual object can have multiple properties that can be modified by a user of the electronic device 100. In some embodiments, the electronic device presents a virtual object that includes one property that can be modified by the user.

[0047] Figure 4A-4B FIG. 1 shows an exemplary manner in which the electronic device 100 presents options associated with a virtual object according to some embodiments. Figure 4A As shown, in some embodiments, the electronic device 100 can present a representation of the physical environment 110, including an XR representation 302' of the carpet 302 in the physical environment 110 and a virtual seat 404. In some embodiments, the virtual seat 404 can correspond to a physical seat that can be purchased through a retailer and / or manufacturer.

[0048] In some embodiments, the electronic device 100 is based on a device having the same Figure 7 The same or similar format of the file 700 described above presents the virtual seat 404. In some embodiments, the definition Figures 3A-3D The file for the virtual sofa 304 shown in FIG and the file defining the virtual seat 404 can have the same file format. The file defining the virtual seat 404 can be provided by the source of the virtual seat 404. In some embodiments, the source is a second electronic device (e.g., a server) associated with the retailer and / or manufacturer of the physical seat corresponding to the virtual seat 404. The file corresponding to the virtual seat 404 can include a configuration of one or more characteristics currently displayed with the virtual seat 404 and an indication of the source of the virtual seat 404. For example, the one or more characteristics can include options available to customers who wish to purchase the physical seat corresponding to the virtual seat 404, such as interior style, base style, and size. Other characteristics are possible. In some embodiments, the virtual seat 404 is displayed in an XR environment that is not uniquely associated with the source of the virtual seat 404. For example, instead of displaying the virtual seat 404 in an XR environment associated with the online store of the source of the virtual seat 404, the virtual seat 404 can be displayed in an XR environment in which virtual objects from multiple sources can be displayed. In some embodiments, the other virtual objects in the XR environment may be other virtual objects corresponding to physical objects available for purchase, one or more items of content, one or more application windows, or other virtual content accessible to the electronic device 100 .

[0049] exist Figure 4AFor example, virtual seat 404 is displayed simultaneously with menu 408 including interior menu 412, option 416 for purchasing a physical seat corresponding to virtual seat 404, and option 418 for closing menu 408. In some embodiments, virtual seat 404 is displayed with the interior option selected in menu 412, and the pricing information included in option 416 may be pricing information associated with a physical seat having the interior option selected in menu 412. In some embodiments, in response to a sequence of one or more inputs including selections of different interior options, electronic device 100 sends a request to the source of virtual seat 404 for an indication of the availability of physical seats having the user-selected options. In response, electronic device 100 receives an indication of the availability of virtual seat 404 having the user-selected configuration.

[0050] For example, in Figure 4B In the embodiment, the electronic device 100 is presented with Figure 4A In some embodiments, the selected interior option is not currently available for purchase, and the electronic device 100 updates the menu 408 to include an indication 420 that the selected configuration of the virtual seat 404 is not available. Figure 4B As shown, even if the configuration is not available for purchase, the electronic device 100 will display the virtual seat 404 with the updated interior configuration. In some embodiments, based on determining that the selected configuration is not available, the electronic device 100 stops updating the virtual seat 404.

[0051] In some embodiments, in addition to presenting virtual objects corresponding to real objects available for purchase, the electronic device 100 is able to recognize the real objects and present purchasing information, including available options and pricing associated with the real objects. Figure 5A-5B An exemplary manner in which the electronic device 100 presents configuration information associated with real objects in the physical environment of the electronic device 100 according to some embodiments is shown.

[0052] exist Figure 5A In the embodiment, the electronic device 100 presents an XR environment that includes a representation 120' of a real table 120 that exists in the physical environment of the electronic device. In some embodiments, the real table 120 is associated with a catalog of products available for purchase. Figure 5A As shown, the electronic device 100 can detect a selection of the representation 120' of the real table 120. For example, the user can select the representation 120' by touching the representation 120' on a touch screen of the electronic device 100. In some embodiments, other inputs and input devices are possible.

[0053] In some embodiments, in response to Figure 5A, the electronic device 100 presents a menu 504 including options associated with the real table 120. For example, the menu 504 may include menus 506 and 508 indicating the configuration of the real table 120, an option 510 for purchasing another table having the same configuration as the real table 120, and an option 512 for stopping displaying the menu 504. In some embodiments, the menu 504 is presented based on a file associated with the table 120, the file having the same format as the file defining the virtual sofa 304 and the virtual chair 404. In some embodiments, one or more of these files have the same format as described below with reference to FIG. Figure 7 The files described are of similar or identical format.

[0054] In some embodiments, the user can use menus 506 and 508 to select different values ​​for the characteristics of the real table. For example, if the user wishes to purchase a real table with a different color than the real table 120, the user can select a different color from menu 506. In some embodiments, if the user wishes to purchase a real table with a different size than the real table 120, the user can select a different size from menu 508. In some embodiments, in response to detecting that the user has changed one or more characteristics in menu 504, the electronic device 100 can update the pricing information for purchasing the table included in option 510 (for example, after receiving pricing information from the source of the table). In some embodiments, the electronic device 100 can display a representation of the virtual table with the user-selected configuration in response to detecting that the user has changed one or more characteristics of the table in menus 506 and 508. In some embodiments, the electronic device 100 displays the virtual table instead of displaying the representation 120' of the real table 120. In some embodiments, the electronic device 100 displays the virtual table adjacent to the representation 120' of the real table 120.

[0055] Figure 6A An exemplary notification 606 associated with a virtual object according to some embodiments is shown. The notification 606 can indicate a change in metadata associated with the virtual object, such as a change in available configurations, a change in price, the availability of a special offer related to a real object associated with the virtual object, etc.

[0056] In some embodiments, a user can request notifications associated with a particular object and / or specific events related to the object. In some embodiments, the electronic device 100 can present a notification similar to notification 606 that is associated with a virtual object that the user has saved, for example, in an object management system or application. The user can set and modify settings that indicate which types of notifications should be received for each object or for all objects. For example, a user may wish to receive a notification when a sofa of a particular color is available for purchase, when a piece of clothing goes on sale, or when the price of any object saved by the user changes. In some embodiments, the user can disable notifications related to virtual objects one at a time or for all virtual objects for which the user has subscribed to notifications.

[0057] In some embodiments, the source of the virtual object may send an indication of a notification 606 to the electronic device 100. For example, in response to detecting an update to the metadata of the virtual object, the source of the virtual object may send an indication of the change in metadata to the electronic device 100. In some embodiments, in response to receiving the indication of the change in metadata from the source of the object, the electronic device 100 presents the notification 606.

[0058] exist Figure 6A In some embodiments, the electronic device 100 may display the notification 606 while displaying other user interfaces, such as other system interfaces (e.g., a home screen user interface, a settings user interface, a notification user interface), or while displaying user interfaces of one or more applications (including applications associated with the notification 606 and applications not associated with the notification 606). For example, the electronic device 100 may display the notification 606 in a system user interface object that is overlaid on one or more other user interfaces.

[0059] exist Figure 6BIn the embodiment of the present invention, for example, the electronic device 100 presents a web browsing user interface 610 and a virtual collection user interface 630. For example, the web browsing user interface 610 displays an online store that sells a corresponding real object (e.g., a box). In some embodiments, the web browsing user interface 610 includes an indication 612 of the URL of the website currently being displayed, text 614 indicating the name of the real object, a virtual object 616 corresponding to the real object, a menu 618 for changing the characteristics of the real object (e.g., color), and a selectable option that causes the electronic device 100 to initiate a process for purchasing the real object when selected. In some embodiments, the virtual object 616 is based on the virtual object described below with reference to Figure 7 The described file 700 is generated by a file similar to that described in the preceding text. In some embodiments, the virtual collection user interface 630 includes a virtual object 632 that is based on the same file as the virtual object 616 included in the web browsing user interface 610.

[0060] Although virtual object 616 and virtual object 632 correspond to the same file and the same real object, in some embodiments, such as Figure 6B As shown, virtual objects 616 and 632 have different appearances from each other because they are displayed in user interfaces of different applications. For example, virtual object 616 is displayed in web browsing user interface 610, where a two-dimensional image and metadata related to the object (such as the name 614 of the object) and a menu 618 of the object's properties are embedded in a web page presented by web browsing user interface 610, while virtual object 632 is displayed as a three-dimensional image in virtual collection user interface 630. In some embodiments, in response to detecting selection of virtual object 632 in virtual collection user interface 630, electronic device 100 selects virtual object 632 in virtual collection user interface 630. Figure 5B Object metadata is presented in a menu similar to menu 504 shown in FIG.

[0061] In some embodiments, the virtual object file is transferable and retains changes made to the metadata associated with the file by the user or the source of the object. Thus, if the user were to change the color of an object in the web browsing user interface 610, save the object file, and then open the object file in the virtual collection user interface 630, the electronic device 100 would display the virtual object 632 in, for example, the color selected by the user in the web browsing user interface 610. As another example, if the price of the object changes, the web browsing user interface 610 and the virtual collection user interface 630 would update the display of the metadata associated with the virtual object to reflect the price change.

[0062] In some embodiments, the electronic device 100 can facilitate displaying virtual objects in multiple user interfaces. For example, the electronic device 100 can detect files associated with the virtual object 616 when displaying the web browsing user interface 610 and display the virtual object 632 in the virtual collection user interface 630. In some embodiments, the virtual object 616 is based on a file having Figure 7 In some embodiments, the representation of the virtual object 616 displayed in the web browsing user interface 610 is stored in an image file that is different from the virtual object file but included in or associated with the virtual object file, and the virtual object file has the format of the file 700. By identifying virtual objects in multiple applications, in some embodiments, a user can save multiple virtual objects with customized properties from multiple sources, applications, etc. in an application of the user's choice, such as the virtual collection application 630. In some embodiments, if the metadata of any saved object is updated by the source of the saved object, the electronic device 100 can display the updated virtual object in the virtual collection application 630.

[0063] Figure 7 An exemplary file 700 defining virtual objects corresponding to real objects according to some embodiments is shown. In some embodiments, the electronic device 100 can display one or more virtual objects (e.g., virtual sofa 304, virtual chair 404) and / or display options related to one or more real objects (e.g., real table 120) based on the exemplary file 700.

[0064] In some embodiments, the file 700 may be provided by a source of an object corresponding to the file 700. The file 700 may include indications 702a-702c of one or more properties of the object, an indication 706 of the source of the object, an indication 708 of authentication information (e.g., digital rights management) for the object, pricing information 710 associated with the object, and an indication 712 of one or more retailers of the object.

[0065] In some embodiments, the indication 706 of the source of the object may include information for contacting the source of the object to obtain updated information about the object, such as the availability and / or pricing information of various configurations of the object. In some embodiments, the indication 708 of authentication information may include a private signature that can be verified by a third-party server (e.g., an electronic device other than the electronic device presenting the virtual object or the source of the virtual object). In some embodiments, if the signature includes information corresponding to an authentication performed by the manufacturer of the physical object corresponding to the virtual object, the third-party server may verify the signature, which grants the source of the object (e.g., the retailer of the object) permission to market and sell the physical object corresponding to the virtual object. It should be understood that in some embodiments, the corresponding physical object may be sold by multiple manufacturers-authorized retailers, and in these cases, two virtual objects may correspond to the same product but be associated with different sources and, therefore, have different configurations (e.g., attributes 702a-702c and options 704a-704i). In some embodiments, the indication 702a-702c of each corresponding attribute of the object may include multiple options 704a-704i associated with the corresponding attribute. For example, the file associated with the virtual sofa 304 may include instructions for the sofa's upholstery and footing, and these instructions may include an indication of the available options for each attribute. It should be understood that the number of attributes for a corresponding object and the number of options for each attribute may vary widely between objects and attributes. In some embodiments, the indications 704a-704i for each option for each attribute may include information about the availability of the corresponding option.

[0066] In some embodiments, the file 700 can be stored on the electronic device 100 and updated by the source of the object (e.g., in response to a request from the electronic device 100, automatically at predetermined time intervals, automatically whenever the availability of one or more attributes changes). For example, updates to the file 700 can include adding or removing one or more attributes 702a-702c, adding or removing one or more options 704a-704i available for one or more attributes, updating the pricing 710 of the object, updating the retailer 712, and / or updating the authentication data 708 (e.g., expiration and / or renewal of the digital rights management of the object). In some embodiments, the file 700 can be stored by the source of the object, and the electronic device 100 can request information from the file 700 in response to a user request to view and / or update a virtual object defined by the file 700.

[0067] Figure 8An example process 800 for presenting a virtual object according to some embodiments is shown. Process 800 is optionally performed at an electronic device such as device 100 or device 200. Some operations in process 800 may be optionally combined and / or optionally omitted. In some embodiments, the order of some operations in process 800 may be optionally changed.

[0068] In some embodiments, the operations of process 800 are performed at an electronic device in communication with a display generation component and one or more input devices. The computer-generated environment is optionally displayed / presented by the display generation component (e.g., using display generation component 214). In some embodiments, the computer-generated environment includes a representation of a real-world environment (or at least one representation of one or more portions of a real-world environment) and, optionally, one or more virtual objects. In some embodiments, the representation of the real-world environment, or at least one or more portions of the real-world environment, is captured using one or more input devices, such as image sensor 210.

[0069] In some embodiments, an electronic device displays (802) a virtual object via a display, the virtual object being based on a file defining the virtual object, a plurality of configurations of the virtual object, and a source of the virtual object, wherein the virtual object is displayed in a first configuration of the plurality of configurations of the virtual object. In some embodiments, the electronic device receives (804) an input corresponding to a request to display the virtual object in a second configuration of the plurality of configurations of the virtual object, the second configuration being different from the first configuration, via the one or more input devices. In some embodiments, the electronic device sends (806) a query to the source of the virtual object for the availability of the virtual object having the second configuration. In some embodiments, the electronic device receives (808) an indication of the availability of a physical object corresponding to the virtual object having the second configuration from the source of the virtual object. In some embodiments, based on determining that the indication indicates that the physical object corresponding to the virtual object is available in the second configuration, the electronic device displays (810) the virtual object having the second configuration via a display generation component.

[0070] In some embodiments, based on determining that the indication indicates that the virtual object is unavailable in the second configuration, the electronic device displays an indication via the display that the virtual object with the second configuration is unavailable. In some embodiments, the indication of availability is an indication of the availability of a physical object corresponding to the virtual object for purchase. In some embodiments, the indication of availability is the availability of the virtual object displayed by the electronic device. In some embodiments, based on determining that the file defining the virtual object meets one or more criteria, the electronic device sends a query to the source of the virtual object for the availability of the virtual object with the second configuration. In some embodiments, based on determining that the file defining the virtual object does not meet one or more criteria, the electronic device abandons sending the query for the availability of the virtual object with the second configuration. In some embodiments, the one or more criteria require that the file defining the virtual object include valid digital rights management information. In some embodiments, the indication indicates whether multiple features of the second configuration are available individually. In some embodiments, the indication indicates whether multiple features of the second configuration are available in combination. In some embodiments, the virtual object is initially accessed via an environment associated with the source of the object and later accessed via an environment independent of the source of the virtual object. In some embodiments, the electronic device displays the multiple configurations of the second virtual object and the source of the second virtual object via the display. In some embodiments, the second virtual object is defined by a second file, and the second file and the file defining the virtual object have a common file format. In some embodiments, the electronic device determines whether the file defining the object includes valid security credentials. In some embodiments, displaying the virtual object includes displaying the virtual object in response to determining that the file includes valid security credentials. In some embodiments, the electronic device identifies the corresponding virtual object corresponding to the corresponding real object based on an image of the corresponding real object. In some embodiments, the electronic device requests the availability of the corresponding virtual object from a directory. In some embodiments, the electronic device receives a first indication of the availability of the corresponding virtual object from the directory. In some embodiments, the electronic device presents a user interface element indicating the availability of the corresponding virtual object. In some embodiments, the multiple configurations of the virtual object are a first multiple configuration available during a first time period (812). In some embodiments, during a second time period different from the first time period, the multiple configurations of the virtual object are a second multiple configuration available during the second time period, and the second multiple configurations are different from the first multiple configurations (814).

[0071] For purposes of explanation, the foregoing description has been described with reference to specific embodiments. However, the above illustrative discussion is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible in light of the above teachings. The embodiments have been chosen and described in order to best illustrate the principles of the invention and its practical application so as to enable others skilled in the art to best utilize the invention and the various described embodiments with various modifications suitable for the specific purposes contemplated.

Claims

1. A method for updating a virtual object, comprising: At an electronic device including a memory and one or more processors in communication with a display and one or more input devices: storing, using the memory of the electronic device, a file defining a virtual object, a plurality of configurations of the virtual object, and a source of the virtual object; displaying the virtual object based on the file using the display, wherein the virtual object is displayed in a first configuration of the plurality of configurations of the virtual object; receiving, via the one or more input devices, an input corresponding to a request to display the virtual object in a second configuration of the plurality of configurations of the virtual object, the second configuration being different from the first configuration; in response to receiving the input corresponding to the request to display the virtual object in the second configuration, sending a query to the source of the virtual object for availability of the virtual object in the second configuration; receiving, from the source of the virtual object, an indication of the availability of a physical object corresponding to the virtual object having the second configuration, wherein the indication indicates that the physical object corresponding to the virtual object is available with the second configuration; In response to receiving the indication of the availability of the physical object corresponding to the virtual object having the second configuration from the source of the virtual object: updating the file with the indication of the availability of the physical object corresponding to the virtual object having the second configuration; as well as Based on determining that the indication indicates that the physical object corresponding to the virtual object is available in the second configuration, the virtual object is displayed via the display having the second configuration.

2. The method according to claim 1, further comprising: After receiving the indication indicating that the physical object corresponding to the virtual object is available in the second configuration, while the virtual object is displayed in the first configuration, receiving a second indication from the source of the virtual object indicating that the physical object corresponding to the virtual object is not available in the second configuration, and In response to receiving the second indication from the source of the virtual object: updating the file using the second instruction; as well as Based on determining that the indication indicates that the virtual object is unavailable in the second configuration, an indication is displayed via the display that the virtual object is unavailable in the second configuration. 3 . The method of claim 1 , wherein the indication of availability is an indication of availability for purchasing the physical object corresponding to the virtual object. The method of claim 1 , wherein the indication of availability is the availability of the virtual object for display by the electronic device.

5. The method according to claim 1, further comprising: Prior to sending the query for the availability of the virtual object having the second configuration, determining that the file defining the virtual object satisfies one or more criteria, wherein the electronic device sends the query for the availability of the virtual object having the second configuration to the source of the virtual object based on determining that the file defining the virtual object satisfies the one or more criteria, and the method further comprises: storing, using the memory of the electronic device, a second file defining a second virtual object, a plurality of configurations of the second virtual object, and a source of the second virtual object; displaying the second virtual object based on the second file using the display, wherein the second virtual object is displayed in a first configuration of the plurality of configurations of the second virtual object; receiving, via the one or more input devices, an input corresponding to a request to display the second virtual object in a second configuration of the plurality of configurations of the second virtual object, the second configuration being different from the first configuration; determining that the second file defining the second virtual object does not satisfy the one or more criteria; and Based on the determination that the second file defining the second virtual object does not satisfy the one or more criteria, sending the query for availability of the second virtual object having the second configuration is abandoned.

6. The method of claim 5, wherein the one or more standards require that the file defining the virtual object include valid digital rights management information. The method of claim 1 , wherein the indication specifies whether a plurality of characteristics of the second configuration are respectively available. The method of claim 1 , wherein the indication specifies whether a plurality of characteristics of the second configuration are available in combination.

9. The method of claim 1, wherein the virtual object is initially accessed via an environment associated with the source of the object and thereafter accessed via an environment independent of the source of the virtual object.

10. The method according to claim 1, further comprising: displaying, via the display, a plurality of configurations of a second virtual object and a source of the second virtual object, The second virtual object is defined by a second file, and the second file and the file defining the virtual object have a common file format.

11. The method according to claim 1 , further comprising: Prior to displaying the virtual object, a determination is made that the file defining the object includes valid security credentials, wherein displaying the virtual object includes displaying the virtual object in response to determining that the file includes the valid security credentials.

12. The method according to claim 1, further comprising: identifying a corresponding virtual object corresponding to the corresponding real object based on the image of the corresponding real object; requesting availability of the corresponding virtual object from a directory; receiving a first indication of the availability of the corresponding virtual object from the catalog; as well as A user interface element is presented that indicates the availability of the corresponding virtual object.

13. The method of claim 1, wherein: The plurality of configurations of the virtual object is a first plurality of configurations available during a first time period, and During a second time period different from the first time period, the plurality of configurations of the virtual object is a second plurality of configurations available during the second time period, the second plurality of configurations being different from the first plurality of configurations.

14. The method according to claim 1, further comprising: opening the file defining the virtual object using a first application accessible to the electronic device; displaying, via the display, a first representation of the virtual object having a first appearance; opening the file defining the virtual object using a second application accessible to the electronic device and different from the first application; as well as A second representation of the virtual object, different from the first representation, is displayed via the display, the second representation having a second appearance different from the first appearance.

15. The method according to claim 1, further comprising: At the first time: receiving, via the one or more input devices, an input corresponding to a request to display the virtual object when the file defining the virtual object is not open via the electronic device; as well as In response to receiving the input corresponding to the request to display the virtual object: determining that the file defining the virtual object is saved with a third configuration; and displaying the virtual object in the third configuration via the display; as well as At a second time different from the first time: receiving, via the one or more input devices, a second input corresponding to a request to display the virtual object when the file defining the virtual object is not open via the electronic device; as well as In response to receiving the second input corresponding to the request to display the virtual object: determining that the file defining the virtual object is saved with a fourth configuration different than the third configuration; and The virtual object is displayed in the fourth configuration via the display.

16. The method according to claim 1, further comprising: receiving an indication of a change in a characteristic of the virtual object from the source of the virtual object; In response to receiving the indication of a change in the characteristic of the virtual object: Presenting an indication of the change in the characteristic of the virtual object, wherein the electronic device presents the indication of the change in the characteristic of the virtual object regardless of whether the electronic device has access to an application provided by the source of the virtual object.

17. The method according to claim 1, further comprising: displaying, via the display, a representation of the physical object corresponding to the virtual object, wherein the representation of the physical object is based on a second file different than the file defining the virtual object; as well as In response to displaying the representation of the physical object based on the second file, the file defining the virtual object is identified.

18. The method according to claim 17, further comprising: Based on identifying the file defining the virtual object, the virtual object is displayed via the display.

19. An electronic device comprising: one or more processors; Memory; as well as One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions for executing the method according to any one of claims 1 to 18.

20. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions that, when executed by one or more processors of an electronic device, cause the electronic device to perform the method according to any one of claims 1 to 18.

21. A computer program product comprising a program comprising computer executable instructions which, when executed by one or more processors, cause the one or more processors to perform the method according to any one of claims 1 to 18.

Images