Techniques for resizing virtual objects

Abstract

According to some embodiments, example processes for changing (e.g., non-linearly) the size of a virtual object are described.

Description

[0001] Cross-references to related applications

[0002] This patent application claims priority to U.S. Provisional Application No. 63 / 070,089, filed August 25, 2020, entitled “TECHNIQUES FOR RESIZINGVIRTUAL OBJECTS”, the contents of which are incorporated herein by reference in their entirety for all purposes. Technical Field

[0003] This disclosure relates generally to extended reality environments, and more specifically to methods and techniques for managing the display of virtual objects in extended reality environments. Background Technology

[0004] Users frequently use devices to interact with extended reality environments. Device usage is often limited by the device's battery life. Therefore, one or more technologies are needed to manage power usage to extend battery life in devices. Summary of the Invention

[0005] According to some embodiments, a method is performed in a system having one or more processors, memory, one or more image sensors, and a display device. The method includes: displaying a first virtual object having a first size via the display device, wherein a distance between a first position corresponding to a location of the first virtual object and a second position corresponding to a viewpoint of the display device is between a first threshold distance and a second threshold distance less than the first threshold distance; detecting a change in the distance between the first and second positions while the first virtual object of the first size is displayed, and while the distance between the first and second positions is between the first and second threshold distances; and in response to detecting the change in the distance between the first and second positions: maintaining the first size according to determining that a first set of criteria is met, wherein the first set of criteria includes criteria satisfied when the changed distance between the first and second positions is between the first and second threshold distances. The display of a first virtual object, wherein a first virtual object of a first size is displayed when the distance between the first position and the second position changes between a first threshold distance and a second threshold distance; and according to determining that a second set of criteria is met, wherein the second set of criteria includes a criterion that is met when the distance between the first position and the second position changes between a third threshold distance and a fourth threshold distance greater than the third threshold distance: modifying the first virtual object to have a second size different from the first size; and displaying a first virtual object of the second size, wherein the first threshold distance is less than the third threshold distance, wherein the second threshold distance is less than the fourth threshold distance, and wherein a first virtual object of the second size is displayed when the distance between the first position and the second position changes between the third threshold distance and the fourth threshold distance.

[0006] According to some embodiments, a system includes a display device; one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors. The one or more programs include instructions for: displaying a first virtual object having a first size via the display device, wherein a distance between a first position corresponding to the location of the first virtual object and a second position corresponding to a viewpoint of the display device is between a first threshold distance and a second threshold distance less than the first threshold distance; detecting a change in the distance between the first and second positions when the first virtual object of the first size is displayed, and when the distance between the first and second positions is between the first and second threshold distances; and in response to detecting the change in the distance between the first and second positions: maintaining... The first virtual object of a first size is displayed when the distance between a first position and a second position changes between a first threshold distance and a second threshold distance; and the first virtual object of a first size is modified to have a second size different from the first size when a second set of criteria is satisfied, wherein the second set of criteria includes criteria satisfied when the distance between a first position and a second position changes between a third threshold distance and a fourth threshold distance greater than the third threshold distance: and the first virtual object of a second size is displayed when the distance between a first position and a second position changes between a third threshold distance and a fourth threshold distance greater than the third threshold distance; and the first virtual object of a second size is displayed when the distance between a first position and a second position changes between a third threshold distance and a fourth threshold distance.

[0007] According to some embodiments, a non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a system device having a display device. The one or more programs include instructions for: displaying a first virtual object having a first size via the display device, wherein a distance between a first position corresponding to the location of the first virtual object and a second position corresponding to the location of a viewpoint of the display device is between a first threshold distance and a second threshold distance less than the first threshold distance; detecting a change in the distance between the first and second positions when the first virtual object of the first size is displayed, and when the distance between the first and second positions is between the first and second threshold distances; and in response to detecting the change in the distance between the first and second positions: maintaining... The first virtual object of a first size is displayed when the distance between a first position and a second position changes between a first threshold distance and a second threshold distance; and the first virtual object of a first size is modified to have a second size different from the first size when a second set of criteria is satisfied, wherein the second set of criteria includes criteria satisfied when the distance between a first position and a second position changes between a third threshold distance and a fourth threshold distance greater than the third threshold distance: and the first virtual object of a second size is displayed when the distance between a first position and a second position changes between a third threshold distance and a fourth threshold distance greater than the third threshold distance; and the first virtual object of a second size is displayed when the distance between a first position and a second position changes between a third threshold distance and a fourth threshold distance.

[0008] According to some embodiments, a transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a system device having a display device. The one or more programs include instructions for: displaying a first virtual object having a first size via the display device, wherein a distance between a first position corresponding to the location of the first virtual object and a second position corresponding to the location of a viewpoint of the display device is between a first threshold distance and a second threshold distance less than the first threshold distance; detecting a change in the distance between the first and second positions when the first virtual object of the first size is displayed, and when the distance between the first and second positions is between the first and second threshold distances; and in response to detecting a change in the distance between the first and second positions: maintaining [the specified distance] according to determining that a first set of criteria is met, wherein the first set of criteria includes criteria satisfied when the changed distance between the first and second positions is between the first and second threshold distances. The first virtual object of a first size is displayed when the distance between a first position and a second position changes between a first threshold distance and a second threshold distance; and the first virtual object of a first size is modified to have a second size different from the first size when a second set of criteria is satisfied, wherein the second set of criteria includes criteria satisfied when the distance between a first position and a second position changes between a third threshold distance and a fourth threshold distance greater than the third threshold distance: and the first virtual object of a second size is displayed when the distance between a first position and a second position changes between a third threshold distance and a fourth threshold distance greater than the third threshold distance; and the first virtual object of a second size is displayed when the distance between a first position and a second position changes between a third threshold distance and a fourth threshold distance.

[0009] According to some embodiments, a system includes a display device and one or more processors. The system includes: means for displaying a first virtual object having a first size via the display device, wherein a distance between a first position corresponding to the location of the first virtual object and a second position corresponding to a viewpoint of the display device is between a first threshold distance and a second threshold distance less than the first threshold distance; means for detecting a change in the distance between the first and second positions when the first virtual object of the first size is displayed, and when the distance between the first and second positions is between the first and second threshold distances; and means for operating in response to detecting a change in the distance between the first and second positions: according to determining that a first set of criteria is met, wherein the first set of criteria includes criteria satisfied when the changed distance between the first and second positions is between the first and second threshold distances. Maintaining the display of a first virtual object having a first size, wherein the first virtual object having a first size is displayed when the distance between the first position and the second position changes between a first threshold distance and a second threshold distance; and modifying the first virtual object to have a second size different from the first size according to a determination that a second set of criteria is met, wherein the second set of criteria includes a criterion that is met when the distance between the first position and the second position changes between a third threshold distance and a fourth threshold distance greater than the third threshold distance: and displaying a first virtual object having a second size, wherein the first threshold distance is less than the third threshold distance, wherein the second threshold distance is less than the fourth threshold distance, and wherein the first virtual object having a second size is displayed when the distance between the first position and the second position changes between the third threshold distance and the fourth threshold distance.

[0010] According to some embodiments, a method is performed in a system having one or more processors, memory, one or more image sensors, and a display device. The method includes: displaying a first virtual object having a first size via the display device, wherein a distance between a first position corresponding to the location of the first virtual object and a second position corresponding to a viewpoint of the display device is within a first consecutive set of distances; detecting a change in the distance between the first and second positions while the first virtual object of the first size is displayed, and while the distance between the first and second positions is within the first consecutive set of distances; and in response to detecting the change in the distance between the first and second positions: maintaining [the object] according to determining that a first set of criteria is met, wherein the first set of criteria includes criteria met when the changed distance between the first and second positions is within the first consecutive set of distances. The first virtual object with a first size is displayed when the distance between the first position and the second position changes within a first consecutive distance group; and the first virtual object is modified to have a second size different from the first size and displayed when a second set of criteria is determined to be satisfied, wherein the second set of criteria includes criteria satisfied when the distance between the first position and the second position changes within a second consecutive distance group, wherein the second consecutive distance group is different from the first consecutive distance group, and wherein the first virtual object with the second size is displayed when the distance between the first position and the second position changes within a second consecutive distance group.

[0011] According to some embodiments, a system includes a display device; one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors. The one or more programs include instructions for: displaying a first virtual object having a first size via the display device, wherein a distance between a first position corresponding to the location of the first virtual object and a second position corresponding to a viewpoint of the display device is within a first consecutive set of distances; detecting a change in the distance between the first and second positions when the first virtual object of the first size is displayed, and when the distance between the first and second positions is within the first consecutive set of distances; and in response to detecting the change in the distance between the first and second positions: determining that a first set of criteria is met, wherein the first set of criteria includes satisfying a condition when the changed distance between the first and second positions is within the first consecutive set of distances. The criteria are met to maintain the display of a first virtual object having a first size, wherein the first virtual object is displayed when the distance between the first position and the second position changes within a first consecutive distance group; and according to determining that a second set of criteria is met, wherein the second set of criteria includes criteria that are met when the distance between the first position and the second position changes within a second consecutive distance group, the first virtual object is modified to have a second size different from the first size and a first virtual object with a second size is displayed, wherein the second consecutive distance group is different from the first consecutive distance group, and wherein the first virtual object with a second size is displayed when the distance between the first position and the second position changes within a second consecutive distance group.

[0012] According to some embodiments, a non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a system device having a display device. The one or more programs include instructions for: displaying a first virtual object having a first size via the display device, wherein the distance between a first position corresponding to the location of the first virtual object and a second position corresponding to the viewpoint of the display device is within a first consecutive set of distances; detecting a change in the distance between the first and second positions when the first virtual object of the first size is displayed, and when the distance between the first and second positions is within the first consecutive set of distances; and in response to detecting the change in the distance between the first and second positions: determining that a first set of criteria is met, wherein the first set of criteria includes satisfying the condition that the changed distance between the first and second positions is within the first consecutive set of distances. The criteria are met to maintain the display of a first virtual object having a first size, wherein the first virtual object is displayed when the distance between the first position and the second position changes within a first consecutive distance group; and according to determining that a second set of criteria is met, wherein the second set of criteria includes criteria that are met when the distance between the first position and the second position changes within a second consecutive distance group, the first virtual object is modified to have a second size different from the first size and a first virtual object with a second size is displayed, wherein the second consecutive distance group is different from the first consecutive distance group, and wherein the first virtual object with a second size is displayed when the distance between the first position and the second position changes within a second consecutive distance group.

[0013] According to some embodiments, a transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a system device having a display device. The one or more programs include instructions for: displaying a first virtual object having a first size via the display device, wherein the distance between a first position corresponding to the location of the first virtual object and a second position corresponding to the viewpoint of the display device is within a first consecutive set of distances; detecting a change in the distance between the first and second positions when the first virtual object of the first size is displayed, and when the distance between the first and second positions is within the first consecutive set of distances; and in response to detecting the change in the distance between the first and second positions: determining that a first set of criteria is met, wherein the first set of criteria includes satisfying the condition that the changed distance between the first and second positions is within the first consecutive set of distances. The criteria are met to maintain the display of a first virtual object having a first size, wherein the first virtual object is displayed when the distance between the first position and the second position changes within a first consecutive distance group; and according to determining that a second set of criteria is met, wherein the second set of criteria includes criteria that are met when the distance between the first position and the second position changes within a second consecutive distance group, the first virtual object is modified to have a second size different from the first size and a first virtual object with a second size is displayed, wherein the second consecutive distance group is different from the first consecutive distance group, and wherein the first virtual object with a second size is displayed when the distance between the first position and the second position changes within a second consecutive distance group.

[0014] According to some embodiments, a system includes a display device and one or more processors. The system includes: means for displaying a first virtual object having a first size via the display device, wherein a distance between a first position corresponding to the location of the first virtual object and a second position corresponding to a viewpoint of the display device is within a first consecutive set of distances; means for detecting a change in the distance between the first and second positions when the first virtual object of the first size is displayed, and when the distance between the first and second positions is within the first consecutive set of distances; and means for operating, in response to detecting a change in the distance between the first and second positions, to: determine that a first set of criteria is met, wherein the first set of criteria includes the change in distance between the first and second positions being within the first consecutive set of distances. The first virtual object is displayed with a first size when the distance between the first and second positions changes within a first consecutive distance group, according to the criteria of the first set of criteria, wherein the first virtual object is displayed with a second size different from the first size when the distance between the first and second positions changes within a first consecutive distance group; and the second virtual object is modified to have a second size different from the first size and displayed with a second size when the distance between the first and second positions changes within a second consecutive distance group, according to the criteria of the second set of criteria, wherein the second consecutive distance group is different from the first consecutive distance group, and wherein the first virtual object with a second size is displayed when the distance between the first and second positions changes within a second consecutive distance group.

[0015] According to some embodiments, a method is performed in a system having one or more processors, memory, one or more image sensors, and a display device. The method includes: displaying a first virtual object via the display device, wherein the first virtual object has a first size and the first virtual object is located at a first position in an extended reality (XR) environment; and detecting a request to change the first size of the virtual object while the first virtual object with the first size is displayed. In response to detecting the request to change the first size of the virtual object, the method includes: modifying the first virtual object to have a second size different from the first size based on determining that a first set of criteria is met, wherein the first set of criteria includes criteria that are met when a detected line of sight is deviated from the first position; and maintaining the first virtual object with the first size based on determining that the first set of criteria is not met.

[0016] According to some embodiments, a system includes a display device; one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors. The one or more programs include instructions for: displaying a first virtual object via the display device, wherein the first virtual object has a first size and is located at a first position in an extended reality (XR) environment; and detecting a request to change the first size of the virtual object when the first virtual object with the first size is displayed. In response to detecting a request to change the first size of the virtual object, the one or more programs include instructions for: modifying the first virtual object to have a second size different from the first size based on determining that a first set of criteria is met, wherein the first set of criteria includes criteria that are met when a detected line of sight is deviated from the first position; and maintaining the first virtual object with the first size based on determining that the first set of criteria is not met.

[0017] According to some embodiments, a non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a system device having a display device. The one or more programs include instructions for: displaying a first virtual object via the display device, wherein the first virtual object has a first size and is located at a first position in an extended reality (XR) environment; and detecting a request to change the first size of the virtual object when the first virtual object with the first size is displayed. In response to detecting the request to change the first size of the virtual object, the one or more programs include instructions for: modifying the first virtual object to have a second size different from the first size based on determining that a first set of criteria is met, wherein the first set of criteria includes criteria that are met when a detected line of sight is deviated from the first position; and maintaining the first virtual object with the first size based on determining that the first set of criteria is not met.

[0018] According to some embodiments, a transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a system device having a display device. The one or more programs include instructions for: displaying a first virtual object via the display device, wherein the first virtual object has a first size and is located at a first position in an extended reality (XR) environment; and detecting a request to change the first size of the virtual object when the first virtual object with the first size is displayed. In response to detecting the request to change the first size of the virtual object, the one or more programs include instructions for: modifying the first virtual object to have a second size different from the first size based on determining that a first set of criteria is met, wherein the first set of criteria includes criteria that are met when a detected line of sight deviates from the first position; and maintaining the first virtual object with the first size based on determining that the first set of criteria is not met.

[0019] According to some embodiments, a system includes a display device; a memory; and one or more processors. The system includes: means for displaying a first virtual object having a first size via the display device, wherein the first virtual object has a first size and the first virtual object is located at a first position in an extended reality (XR) environment; means for detecting a request to change the first size of the virtual object when the first virtual object with the first size is displayed; and means for operating in response to detecting the request to change the first size of the virtual object: modifying the first virtual object to have a second size different from the first size based on satisfying a first set of criteria, wherein the first set of criteria includes criteria satisfied when a detected line of sight deviates from the first position; and maintaining the first virtual object having the first size based on determining that the first set of criteria is not satisfied.

[0020] Executable instructions for performing these functions are optionally included in a non-transitory computer-readable storage medium or other computer program product configured for execution by one or more processors. Attached Figure Description

[0021] In the following description, reference is made to the accompanying drawings, which form part of this disclosure and illustrate various examples of the disclosure. It should be understood that other examples may also be utilized, and structural and operational changes may be made without departing from the scope of this disclosure. The same reference numerals are used in different drawings to denote similar or identical items.

[0022] Figures 1A to 1B An exemplary system used in various extended reality technologies is shown.

[0023] Figures 2A to 2B Exemplary techniques for non-linearly resizing virtual objects are shown according to some implementation schemes.

[0024] Figures 3A to 3B Exemplary techniques for non-linearly resizing virtual objects are shown according to some implementation schemes.

[0025] Figures 4A to 4B Exemplary techniques for non-linearly resizing virtual objects are shown according to some implementation schemes.

[0026] Figures 5A to 5B Exemplary techniques for non-linearly resizing virtual objects are shown according to some implementation schemes.

[0027] Figures 6A to 6B Exemplary techniques for non-linearly resizing virtual objects are shown according to some implementation schemes.

[0028] Figures 7A to 7B Exemplary techniques for non-linearly resizing virtual objects are shown according to some implementation schemes.

[0029] Figures 8A to 8B Exemplary techniques for non-linearly resizing virtual objects are shown according to some implementation schemes.

[0030] Figures 9A to 9B Exemplary techniques for non-linearly resizing virtual objects are shown according to some implementation schemes.

[0031] Figures 10A to 10B Exemplary techniques for non-linearly resizing virtual objects are shown according to some implementation schemes.

[0032] Figure 11 This is a flowchart illustrating a method for non-linearly resizing a virtual object according to some implementation schemes.

[0033] Figure 12 This is a flowchart illustrating a method for non-linearly resizing a virtual object according to some implementation schemes. Detailed Implementation

[0034] Various examples of electronic systems and technologies for using such systems in relation to various extended reality technologies are described.

[0035] Humans can interact with and / or perceive the physical environment or physical world without the aid of electronic devices. The physical environment can include physical features, such as physical objects or surfaces. An example of a physical environment is a physical forest that includes physical plants and animals. Humans can directly perceive and / or interact with the physical environment through various means, such as hearing, vision, taste, touch, and smell.

[0036] In contrast, humans can use electronic devices to interact with and / or perceive fully or partially simulated extended reality (XR) environments. These XR environments can include mixed reality (MR) content, augmented reality (AR) content, virtual reality (VR) content, and so on. Using an XR system, a person's physical movements, or some of their representations, can be tracked, and in response, the characteristics of virtual objects simulated in the XR environment can be adjusted in a manner consistent with at least one physical law. For example, the XR system can detect movement of the user's head and adjust the graphical and auditory content presented to the user (similar to how such views and sounds change in a physical environment). Similarly, the XR system can detect movement of the electronic device presenting the XR environment (e.g., a mobile phone, tablet, laptop, etc.) and adjust the graphical and auditory content presented to the user (similar to how such views and sounds change in a physical environment). In some cases, the XR system can adjust the characteristics of the graphical content in response to other inputs, such as representations of physical movement (e.g., voice commands).

[0037] Many different types of electronic systems enable users to interact with and / or perceive XR environments. A non-exclusive list of examples includes head-up displays (HUDs), head-mounted systems, projection-based systems, windows or vehicle windshields with integrated display capabilities, displays shaped as lenses to be placed on the user's eyes (e.g., contact lenses), head-mounted headphones / earpieces, input systems with or without haptic feedback (e.g., wearable or handheld controllers), speaker arrays, smartphones, tablets, and desktop / laptop computers. Head-mounted systems may have opaque displays and one or more speakers. Other head-mounted systems may be configured to accept opaque external displays (e.g., smartphones). Head-mounted systems may include one or more image sensors for capturing images or video of the physical environment, and / or one or more microphones for capturing audio of the physical environment. Head-mounted systems may have transparent or semi-transparent displays, rather than opaque displays. Transparent or semi-transparent displays may have a medium through which light is directed to the user's eyes. Displays may utilize various display technologies, such as uLED, OLED, LED, liquid crystal on silicon, laser scanning light sources, digital light projection, or combinations thereof. Optical waveguides, optical reflectors, holographic media, optical combiners, combinations thereof, or other similar techniques can be used as media. In some implementations, transparent or translucent displays can be selectively controlled to become opaque. Projection-based systems can utilize retinal projection techniques that project graphic images onto a user's retina. Projection systems can also project virtual objects into a physical environment (e.g., as holograms or onto a physical surface).

[0038] Figure 1A and Figure 1B An exemplary system 100 for various extended reality technologies is depicted.

[0039] In some examples, such as Figure 1A As shown, system 100 includes device 100a. Device 100a includes various components such as processor 102, RF circuitry 104, memory 106, image sensor 108, orientation sensor 110, microphone 112, position sensor 116, speaker 118, display 120, and touch-sensitive surface 122. These components optionally communicate via communication bus 150 of device 100a.

[0040] In some examples, components of system 100 are implemented in base station equipment (e.g., computing devices such as remote servers, mobile devices, or laptop computers), and other components of system 100 are implemented in head-mounted display (HMD) devices designed to be worn by users, wherein the HMD devices communicate with the base station equipment. In some examples, device 100a is implemented in either base station equipment or an HMD device.

[0041] like Figure 1B As shown, in some examples, system 100 includes two (or more) communicating devices, such as via wired or wireless connections. A first device 100b (e.g., a base station device) includes a processor 102, RF circuitry 104, and a memory 106. These components optionally communicate via a communication bus 150 of device 100b. A second device 100c (e.g., a head-mounted device) includes various components such as processor 102, RF circuitry 104, memory 106, image sensor 108, orientation sensor 110, microphone 112, position sensor 116, speaker 118, display 120, and touch-sensitive surface 122. These components optionally communicate via a communication bus 150 of device 100c.

[0042] In some examples, System 100 is a mobile device. In some examples, System 100 is an HMD device. In some examples, System 100 is a wearable HUD device.

[0043] System 100 includes a processor 102 and a memory 106. Processor 102 includes one or more general-purpose processors, one or more graphics processors, and / or one or more digital signal processors. In some examples, memory 106 is one or more non-transitory computer-readable storage media (e.g., flash memory, random access memory) storing computer-readable instructions configured to be executed by processor 102 to perform the techniques described below.

[0044] System 100 includes RF circuitry 104. RF circuitry 104 optionally includes circuitry for communicating with electronic devices, networks (such as the Internet, intranets), and / or wireless networks (such as cellular networks and wireless local area networks (LANs)). RF circuitry 104 optionally includes circuitry for using near-field communication and / or short-range communication (such as... The circuit used for communication.

[0045] System 100 includes a display 120. In some examples, display 120 includes a first display (e.g., a left-eye display panel) and a second display (e.g., a right-eye display panel), each displaying an image to the corresponding eye of the user. Corresponding images are simultaneously displayed on the first and second displays. Optionally, the corresponding images include representations of the same virtual and / or physical objects from different viewpoints, thereby creating a parallax effect that provides the user with a stereoscopic effect of objects on the display. In some examples, the first display may include multiple other displays (e.g., sub-displays), such as a main display and secondary displays. In some embodiments, the main display has a different resolution than the secondary displays when system 100 is operating. In some examples, display 120 includes a single display (e.g., a first display or a second display). In some examples, this single display includes multiple other displays (e.g., sub-displays), such as a main display and secondary displays. For each eye of the user, a corresponding image is simultaneously displayed on a first and a second area of ​​the single display. Optionally, the corresponding images include representations of the same virtual objects and / or the same physical objects from different viewpoints, thereby producing a parallax effect that provides the user with a stereoscopic effect of objects on a single display.

[0046] In some examples, system 100 includes a touch-sensitive surface 122 for receiving user input, such as tap input and swipe input. In some examples, display 120 and touch-sensitive surface 122 form a touch-sensitive display.

[0047] System 100 includes an image sensor 108. Image sensor 108 may optionally include one or more visible light image sensors (such as charge-coupled device (CCD) sensors) and / or complementary metal-oxide-semiconductor (CMOS) sensors operable to acquire images of physical objects from a real environment. Image sensor may also optionally include one or more infrared (IR) sensors, such as passive or active IR sensors, for detecting infrared light from the real environment. For example, an active IR sensor includes an IR emitter, such as an IR point emitter, for emitting infrared light into the real environment. Image sensor 108 may also optionally include one or more event cameras configured to capture movement of physical objects in the real environment. Image sensor 108 may also optionally include one or more depth sensors configured to detect the distance between the physical object and system 100. In some examples, system 100 uses a combination of CCD sensors, event cameras, and depth sensors to detect the physical environment surrounding system 100. In some examples, image sensor 108 includes a first image sensor and a second image sensor. The first and second image sensors are optionally configured to capture images of physical objects in a real-world environment from two different perspectives. In some examples, system 100 uses image sensor 108 to receive user input, such as gestures. In some examples, system 100 uses image sensor 108 to detect the position and orientation of system 100 and / or display 120 in a real-world environment. For example, system 100 uses image sensor 108 to track the position and orientation of display 120 relative to one or more stationary objects in a real-world environment.

[0048] In some examples, system 100 includes a microphone 112. System 100 uses microphone 112 to detect sound from a user and / or the user's real-world environment. In some examples, microphone 112 includes a microphone array (comprising multiple microphones) that may optionally operate in series to identify ambient noise or locate sound sources in the space of a real-world environment.

[0049] System 100 includes an orientation sensor 110 for detecting the orientation and / or movement of system 100 and / or display 120. For example, system 100 uses orientation sensor 110 to track changes in the position and / or orientation of system 100 and / or display 120, such as with respect to physical objects in a real environment. Orientation sensor 110 may optionally include one or more gyroscopes and / or one or more accelerometers.

[0050] Figures 2A to 2B , Figures 3A to 3B , Figures 4A to 4B , Figures 5A to 5B , Figures 6A to 6B , Figure 11 and Figure 12Exemplary techniques for non-linearly resizing virtual objects are illustrated according to some implementation schemes. It is noteworthy that when resizing virtual objects using a user interface in an XR environment, Figures 2A to 2B , Figures 3A to 3B , Figures 4A to 4B , Figures 5A to 5B , Figures 6A to 6B , Figure 11 and Figure 12 The techniques described herein can reduce visual interference.

[0051] Figure 2A An exemplary scenario is shown where a person 250 is holding a display device 202 and standing at a location in an environment. In this scenario, the person 250 is able to stand at a physical location in the physical environment. In some embodiments, this physical location may have a corresponding virtual location within the XR environment.

[0052] Additionally, in this scenario, the display device 202 is a tablet computer. In some embodiments, the display device 202 can be another type of electronic device, such as a mobile phone, personal computer, television, head-mounted display, etc. In some embodiments, the display device 202 is a wearable device. In some embodiments, the display device 202 may include one or more components of system 100, such as one or more processors (e.g., processor 102), memory (e.g., memory 106), camera sensors (e.g., image sensor 108), or motion sensors (e.g., orientation sensor 110). For example, the display device 202 may be... Figure 1A The embodiment of device 100a of system 100 shown is illustrated. Alternatively, in some embodiments, display device 202 may be... Figure 1B An embodiment of the second device 100c shown in the figure.

[0053] exist Figure 2A In the scenario shown, display device 202 (or another component of system 100) is configured to modify (or initiate modification of) virtual objects, such as virtual object 208, within the XR environment. In some examples, the virtual object can be modified by changing its inherent properties, such as its dimensions (e.g., in 3D space), color, quality, etc.

[0054] like Figure 2A As shown, in an XR environment, the viewpoint of display device 202 is located at a distance 204 from the location of virtual object 208 (virtual object location) (viewpoint location). In some embodiments, the viewpoint is located at a position corresponding to the field of view of one or more cameras that communicate with or are part of display device 202.

[0055] In some implementations, the viewpoint location and the virtual object location are virtual locations within the XR environment, and similarly, distance 204 is a virtual distance (e.g., the distance between two virtual locations).

[0056] In some embodiments, the viewpoint position and / or virtual object position may correspond to one or more physical locations in the physical environment, and distance 204 may correspond to the physical distance between physical locations. In some embodiments, physical locations are tracked by one or more components of the display device 202 and used to calculate the viewpoint position, virtual object position, and / or distance 204. In some embodiments, distance 204 may be a multiple of the corresponding physical distance, or vice versa.

[0057] exist Figure 2A In this context, object 208 is a virtual whiteboard. In some implementations, one or more words or symbols are displayed on the virtual whiteboard. As discussed in more detail below, in some examples, the size of object 208 can be changed, and the size of one or more words or symbols can similarly change by an amount proportional to the change in the size of object 208. For example, when the size of object 208 increases, the size of the words or symbols on object 208 increases to a size proportional to the increase in the size of object 208, or vice versa.

[0058] In some implementations, a virtual whiteboard may correspond to a physical object in the physical environment. For example, a virtual whiteboard may be a virtual object that corresponds to a physical whiteboard.

[0059] It should be recognized that object 208 can be any type of object. Object 208 can be any type of virtual structure or virtual artifact, and in some embodiments, it can correspond to any type of physical structure or physical artifact within a physical environment.

[0060] Figure 2A The distances 236a to 236f are further shown in order of maximum (e.g., distance 236a) to minimum (e.g., distance 236f). Similar to distance 204, in some embodiments, distances 236a to 236f may correspond to (or be) physical distances.

[0061] Each of distances 236a to 236f is a corresponding distance from the location of the virtual object. In some implementations, when the viewpoint is located in the environment aligned with distance 236f, the viewpoint is at the location of the virtual object; in other words, distance 236f is zero.

[0062] like Figure 2AAs shown, distances 236a to 236f are grouped into consecutive distance groups 238a to 238c. In some embodiments, consecutive distance groups 238a to 238c correspond to one or more physically consecutive distance groups similar to those described above with respect to distances 236a to 236f. ​​In some embodiments, these consecutive distance groups are alternatively adjacent distance groups. In some embodiments, a consecutive distance group is a distance group defined (e.g., programmatically and / or theoretically) by not having discrete boundaries between each distance in the distance group. In some embodiments, an adjacent distance group is a distance group defined (e.g., programmatically and / or theoretically) by having discrete boundaries between each distance in the distance group (e.g., including multiple consecutive distance groups, where at least one consecutive distance group shares a boundary). In some embodiments, a consecutive distance group covering a 10-meter range may include distances between 1 meter and 10 meters, while an adjacent distance group covering a 10-meter range may include distances between 1 meter and 3 meters, 3 meters and 6 meters, 7 meters and 8 meters and 10 meters.

[0063] Continuous distance group 238a includes all distances between the viewpoint position and the virtual object position that are less than or equal to distance 236a (e.g., maximum distance value, non-zero threshold distance, and / or end value) and greater than distance 236c (e.g., minimum distance value, threshold distance, and / or end value). Continuous distance group 238b includes all distances between the viewpoint position and the virtual object position that are less than or equal to distance 236c and greater than distance 236e (e.g., maximum distance value, non-zero threshold distance, and / or end value). Continuous distance group 238c includes all distances between the viewpoint position and the virtual object position that are less than or equal to distance 236e and greater than distance 236f (e.g., maximum distance value, non-zero threshold distance, and / or end value).

[0064] In some implementations, two or more consecutive distance groups are adjacent consecutive distance groups. For example, consecutive distance groups 238a and 238b are adjacent consecutive distance groups, but consecutive distance groups 238a and 238c are not adjacent consecutive distance groups because they are separated by consecutive distance group 238b. In some implementations, one or more other consecutive distance groups exist between the viewpoint location and the virtual object location. In some implementations, different consecutive distance groups exist between two or more consecutive distance groups 238a, 238b, and 238c.

[0065] like Figure 2A As shown, the distance between the viewpoint position and the virtual object position (as indicated by dashed line 210) is within the continuous distance group 238a. In other words, distance 204 is within the continuous distance group 238a, while the distance between the viewpoint position and the virtual object position is 204.

[0066] like Figure 2A As shown, the virtual object 208 has a first size. In some embodiments, the size of the virtual object may include the length, width, height, perimeter, area, volume, etc.

[0067] In some implementations, one or more virtual objects (other virtual objects) are adjacent to virtual object 208 or at any other location in the environment. In some implementations, consecutive distance groups 238a to 238c are associated with a set of other virtual objects (e.g., configured to change based on a set of other virtual objects), while another consecutive distance group is associated with yet another set of other virtual objects.

[0068] In some implementations, other virtual objects are located at virtual positions such that the viewpoint position is also a distance 204 from the virtual positions of other virtual objects. In some implementations, the size of the other virtual objects is different from or the same as the size of virtual object 208.

[0069] In some implementations, at least one of the other virtual objects is within a continuous distance group, including continuous distance groups 238a to 238c.

[0070] Figure 2B It shows the corresponding Figure 2A In the scenario shown, the exemplary display screen 230 of the display device 202. Figure 2B As shown, the XR environment is displayed on the screen 230.

[0071] In some implementations, the XR environment includes a representation 218 of a virtual object 208. For example... Figure 2B As shown, 218 is Figure 2A The representation of virtual object 208 in the XR environment. In some implementations, representation 218 appears in the XR environment corresponding to... Figure 2A At a depth of 204 at a mid-distance.

[0072] like Figure 2B As shown, the display device 202 displays representation 218 on the display screen 230 at a size corresponding to the size of the virtual object 208 when the distance between the viewpoint position and the virtual object position is 204.

[0073] In some implementation schemes, Figure 2B In the XR environment, the display device 202 displays a representation of one or more virtual objects based on the size of one or more virtual objects and / or the distance between the viewpoint position and the position of each corresponding virtual object.

[0074] Figure 3AThis illustrates an exemplary scenario where person 250 has moved to different (physical or virtual) locations within the environment. For example... Figure 3A As shown, the viewpoint position is 304 units away from the virtual object position. Although Figure 3A The distance 304 in the middle is more than Figure 2A The distance 204 is a smaller distance, but the distance 304 still remains within the continuous distance group 238a (e.g., as shown by dashed line 310).

[0075] Figure 3B It shows the corresponding Figure 3A The display screen is 230 in the scenario shown. Assuming that in... Figure 3A In this process, display device 202 (or another component of system 100) detects the distance between the viewpoint position and the virtual object position (e.g., from...). Figure 2A The distance in the middle is 204 to Figure 3A The display device 202 determines that the size of the virtual object 208 should not be reset in response to the detected change in distance 304. Based on this determination, the display device 202 maintains the size of the virtual object 208. In some embodiments, the display device 202 determines that the size of the virtual object 208 should not be reset because distance 304 is within (or has been maintained within) a continuous distance group 238a.

[0076] It is worth noting that, reference Figure 2A and Figure 3A Although the distance between the viewpoint and the virtual object's position has decreased, the size of the virtual object 208 remains unchanged. This is at least because the display device 202 does not linearly or continuously change the size of some virtual objects based on a specific distance (or exact distance) between the viewpoint and the virtual object's position.

[0077] exist Figure 3B In this context, the display device 202 is enlarged to represent the virtual object 208 from the viewpoint of the display device 202. Here, because the person 250 is standing closer (in... Figure 3A China and Belgium in Figure 2A (The middle) is a virtual object that has not yet changed size, so it represents a size of 218 relative to... Figure 2B The value of 218 increases.

[0078] Figure 4A This illustrates an exemplary scenario where person 250 has moved to different locations in the environment. (Example:) Figure 4A As shown, the distance between the viewpoint and the virtual object is 404. (Reference) Figure 3A and Figure 4A Because the distance to 404 is greater than Figure 3A The distance of 304 is smaller, so the viewpoint is closer. Figure 4AThe location of virtual objects within. Additionally, such as... Figure 4A As shown, the distance between the viewpoint position and the virtual object position is within a new group of consecutive distances. For example, in Figure 4A In the middle, distance 404 is within the continuous distance group 238b, while... Figure 3A The medium distance 304 is within the continuous distance group 238a.

[0079] Assuming in Figure 4A In this process, display device 202 (or another component of system 100) detects a change in the distance between the viewpoint position and the virtual object position (e.g., from...). Figure 3A The distance from 304 to Figure 4A (Distance 404). In response to detecting a change in distance, the display device 202 determines that the size of the virtual object 208 should be reset, and resets the size of the virtual object 208 based on this determination. For example, in Figure 4A In the middle, virtual object 208 is Figure 4A China and Belgium in Figure 3A Smaller in the middle.

[0080] In some implementations, because distance 404 is within a new consecutive distance group (e.g., consecutive distance group 238b instead of consecutive distance group 238a), display device 202 determines that the size of virtual object 208 should be reset.

[0081] As mentioned above Figure 2A and Figure 3A As discussed, the display device 202 does not linearly or continuously change the size of some virtual objects based on a specific distance (or exact distance) between the viewpoint position and the virtual object position. However, as Figure 2A , Figure 3A and Figure 4A As shown, display device 202 non-linearly changes the size of the virtual object 208 based on whether the distance between the viewpoint and the virtual object 208 falls within a specific set of consecutive distances. Therefore, in some embodiments, the change in the size of the virtual object is related to a set of consecutive distances rather than a specific distance between the viewpoint position and the virtual object position. In some embodiments, non-linearly changing the size of the virtual object reduces the number of operations (e.g., changing / updating the display) that display device 202 must perform to update its display, thus saving battery life of display device 202. Additionally, the size of the virtual object (e.g., virtual object 208) can be changed to keep the size of its representation on the display device (e.g., representation 218) within a desired range. For example, reducing... Figure 4A The size of the virtual object 208 in the middle can prevent it from being in Figure 4BThe representation 218 occupies an undesirable amount of display space. Similarly, increasing the size of the virtual object 208 in response to an increase in the distance between the display device 202 and the virtual object 208 can prevent its representation 218 from being displayed at an inappropriate size (e.g., text too small to read).

[0082] In some implementations, the second virtual object is adjacent to virtual object 208. In some implementations, the display device 202 does not (e.g., abandons) resizing the second virtual object even when the distance between the second virtual object and the viewpoint is the same as the distance between virtual object 208 and the viewpoint.

[0083] In some implementations, the display device 202 does not resize the second virtual object because it is not an object of a type configured to be resized. For example, certain types of objects are not resizable, such as objects configured not to be resizable by one or more settings, parameters, processes, applications, etc. In some implementations, certain types of virtual objects are not resizable because they have one or more characteristics or features suitable for these types of virtual objects to maintain the same size. In some implementations, one or more applications and / or processes communicating with the display device 202 can determine that because the object has one or more characteristics, such as text, user interface element type (e.g., icon, input box, field selector), color, and / or minimum display resolution, resizing the object in this case would distort the object and / or potentially result in less user interaction with the object, making it unsuitable for resizing.

[0084] In some implementations, the display device 202 does not reset the size of the second virtual object because the virtual object is not associated with one or more of the consecutive distance groups 238a to 238b.

[0085] In some implementations, the display device 202 does not resize the second virtual object because the virtual object has already been resized a predetermined number of times (e.g., once, twice, etc.). In some implementations, the display device 202 does not resize the second virtual object because, during a session displaying an XR environment or detecting user activity associated with that XR environment (e.g., interaction with a representation of the second virtual object in the XR environment), the representation of the second virtual object has already been resized a predetermined number of times.

[0086] In some implementations, the display device 202 does not resize the second virtual object because the object would consume more battery power to resize or update. For example, the second virtual object will consume more battery power to resize or update when it is an object that requires a higher resolution to be displayed at a smaller size, an object that exceeds its maximum size and requires a resolution to be resized, an object that needs to be displayed at a higher resolution than the objects displayed simultaneously, and / or requires running one or more processes before resizing or updating the information displayed on the object.

[0087] Figure 4B It shows the corresponding Figure 4A Display screen 230 in the scene shown. Figure 4B In this context, the display device 202 reduces the size of representation 218 to represent the virtual object 208 from the viewpoint of the display device 202. Here, even if the person 250 stands closer (in... Figure 4A China and Belgium in Figure 3A In the case of virtual object 208, since the size of virtual object 208 has been reduced, it means that the size of 218 has been reduced. However, in other examples, based on the reduction in the amount of change in the size of virtual object 208 relative to the distance between display device 202 and virtual object 208, it means that 218 can be... Figure 3A The same or larger size appears as shown. In some implementations, the size of the virtual object 208 can be changed so that it appears in... Figure 4B The representation of 218 in [the text] and its [in the text] Figure 2B The sizes are the same.

[0088] Figure 5A This illustrates an exemplary scenario where person 250 has moved to different locations in the environment. (Reference) Figure 4A and Figure 5A Because the distance to 504 is greater than Figure 4A The distance in the middle is 404, which is a smaller distance, so the viewpoint position is closer. Figure 5A The location of the virtual object in the [location]. Although distance 504 is a smaller distance than distance 404, distance 504 still remains within the continuous distance group 238b (e.g., as shown by dashed line 510), just as distance 404 is within [the range]. Figure 4A Between consecutive distance groups 238b.

[0089] Assuming in Figure 5A In this process, display device 202 (or another component of system 100) detects a change in the distance between the viewpoint position and the virtual object position (e.g., from...). Figure 4A The distance from 404 to Figure 5AIn response to detecting a change in distance, display device 202 determines that the size of virtual object 208 should not be reset, and maintains the size of virtual object 208 based on this determination. Display device 202 maintains the size of virtual object 208 while distance 504 remains within the continuous distance group 238b. In some embodiments, display device 202 in Figure 5A The above text is used in conjunction with the previous text. Figure 3A The technology discussed is similar to that of other technologies.

[0090] Figure 5B It shows the corresponding Figure 5A Display screen 230 in the scene shown. Figure 5B In this context, the display device 202 is enlarged to represent the virtual object 208 from the viewpoint of the display device 202. Here, because the person 250 is standing closer (in... Figure 5A China and Belgium in Figure 4A middle) Figure 5A The virtual object in the display device 202 has not yet changed size, so this indicates that the size of 218 has increased. In some implementations, the display device 202 is in... Figure 5B The above text is used in conjunction with the previous text. Figure 3B The technology discussed is similar to that of other technologies.

[0091] refer to Figure 2B , Figure 3B , Figure 4B and Figure 5B By changing the size of the virtual object 208 as described herein, the display device 202 is able to simulate the size of the representation 218 changing as the display device 202 moves within a continuous range of distances, regardless of the distance between the viewpoint position and the virtual object position, while also maintaining the representation 218 on the display. In some embodiments, the size of the virtual object 208 is adjusted such that when the entire representation 218 is displayed, the representation 218 is maintained on the display and / or when a representation of all portions of the virtual object 208 is displayed, no portion of the virtual object 208 is truncated by the edge of the display screen 230 (or visually represented as extending beyond one or more edges of the display screen 230 (or being transversely interrupted by the edge of the display screen)).

[0092] In some implementations, non-linearly changing the size of virtual objects reduces visual distractions (or the number of moving objects) on display device 202. In some implementations, non-linearly changing the size of virtual objects mitigates the effect of objects appearing to be moving away from the user, an effect that occurs when the virtual object is continuously resized. In some implementations, non-linearly changing the size of virtual objects mitigates the effect of objects appearing to be moving away, an effect that occurs when the virtual object is continuously resized.

[0093] Figure 6A This illustrates an exemplary scenario where person 250 has moved to different locations in the environment. (Example:) Figure 6A As shown, the distance between the viewpoint and the virtual object is 604. (Reference) Figure 5A and Figure 6A Because the distance to 604 is greater than Figure 5A The distance in the middle is 504, which is a smaller distance, so the viewpoint position is closer. Figure 6A The location of virtual objects within. Additionally, such as... Figure 6A As shown, the distance between the viewpoint position and the virtual object position is within a new group of consecutive distances. For example, in Figure 6A In the middle, distance 604 is within the continuous distance group 238c, while... Figure 5A The mid-range 504 is within the continuous distance group 238b.

[0094] Assuming in Figure 6A In this process, display device 202 (or another component of system 100) detects a change in the distance between the viewpoint position and the virtual object position (e.g., from...). Figure 5A The distance in the middle is 504 to Figure 6A (Distance 604 in the distance). In response to detecting a change in distance, the display device 202 determines that the size of the virtual object 208 should be reset, and resets the size of the virtual object 208 based on this determination. For example, the virtual object 208 in Figure 6A China and Belgium in Figure 5A Smaller in some implementations. The display device 202 is in... Figure 6A The above text is used in conjunction with the previous text. Figure 4A The technology discussed is similar to that of other technologies.

[0095] Figure 6B It shows the corresponding Figure 6A Display screen 230 in the scene shown. Figure 6B In this context, the display device 202 reduces the size of representation 218 to represent the virtual object 208 from the viewpoint of the display device 202. Here, even if the person 250 stands closer (in... Figure 6A China and Belgium in Figure 5A In the case of virtual object 208, since the size of virtual object 208 has been reduced, it means that the size of 218 has been reduced. However, in other examples, based on the reduction in the amount of change in the size of virtual object 208 relative to the distance between display device 202 and virtual object 208, it means that 218 can be... Figure 5A The same or larger size appears as shown. In some implementations, the size of the virtual object 208 can be changed so that it appears in... Figure 6B The representation of 218 in the middle is... Figure 2B and 4BThe size of 218 is the same as that of the display device 202. In some embodiments, the display device 202 is in Figure 6B The above text is used in conjunction with the previous text. Figure 4B The technology discussed is similar to that of other technologies.

[0096] Figures 7A to 7B , Figures 8A to 8B , Figures 9A to 9B , Figures 10A to 10B , Figure 11 and Figure 12 Exemplary techniques for non-linearly resizing virtual objects are shown according to some implementation schemes. Figure 7A , Figure 8A , Figure 9A and Figure 10A One or more components are shown (e.g., virtual object position, object 208, distances 236a to 236f, consecutive distance groups 238a to 238c), as described above regarding Figure 2A As stated above.

[0097] It is worth noting that, in Figures 7A to 7B , Figures 8A to 8B , Figures 9A to 9B , Figures 10A to 10B , Figure 11 and Figure 12 In this context, the technology can reduce interference from modifying virtual objects in an XR environment. For simplicity, Figures 7A to 7B , Figures 8A to 8B , Figures 9A to 9B and Figures 10A to 10B One or more techniques are described in which display device 202 (or another component of system 100) changes (or does not change) the size of a virtual object based on whether display device 202 detects that a person 250 is looking at the virtual object and the distance between the viewpoint and the position corresponding to the virtual object's location. However, in some embodiments, techniques may be deployed in which display device 202 changes (or does not change) the size of the virtual object based on whether a user is looking at it, regardless of the distance between the viewpoint and the virtual object.

[0098] Figure 7A An exemplary scenario is shown where a person 250 is holding a display device 202 at a location in the environment. Figure 7A As shown, the distance between the viewpoint position and the virtual object position is 704.

[0099] like Figure 7A As shown, although the viewpoint position is 704 units away from the virtual object position, the distance between the viewpoint position and the virtual object position is still within the continuous distance group 238a (for example, as shown by the dashed line 710). In other words, the distance 704 is within the continuous distance group 238a.

[0100] Figure 7B It shows the corresponding Figure 7A In the scenario shown, the display screen 230 of the display device 202. In some implementations, such as Figure 7B As shown, display device 202 uses the same method as described above. Figure 2B The technology described above is similar to the technology used to display an XR environment on a display screen 230.

[0101] exist Figure 7B In this context, the detected gaze position 750 (or the detected gaze position 750 of person 250) is positioned on the display screen 230 at a location corresponding to the position in the XR environment. The detected gaze position 750 is located at... Figure 7B The images shown on display screen 230 are for illustrative purposes only and may or may not be actually displayed on display screen 230.

[0102] Figure 8A The illustration shows a person 250 in an exemplary scene, where the person 250 has moved to different locations in the environment. Figure 8A As shown, the distance between the viewpoint and the virtual object is 804. (Reference) Figure 7A and Figure 8A Because the distance to 804 is greater than Figure 7A The distance in the middle is 704, which is a smaller distance, so the viewpoint position is closer. Figure 8A The location of the virtual object in the [location]. In some implementations, a distance of 804 is equal to a distance of 404.

[0103] In some implementation schemes, such as Figure 8A As shown, the distance between the viewpoint position and the virtual object position is within a new group of consecutive distances. For example, in Figure 8A In the middle, distance 804 is within the continuous distance group 238b, while... Figure 7A The medium distance 704 is within the continuous distance group 238a.

[0104] Figure 8B It shows the corresponding Figure 8A Display screen 230 in the scene shown.

[0105] refer to Figures 8A to 8B Assume that display device 202 receives information corresponding to a detected gaze position 750. In response to receiving information corresponding to the detected gaze position 750, display device 202 (or any other component of system 100) determines that the detected gaze position 750 is within a predetermined threshold distance from the virtual object position or within a predetermined threshold distance from the representation 218 in the screen space on display screen 230. In some embodiments, display device 202 maintains the size of the virtual object 208 due to this determination, such as... Figure 8A As shown.

[0106] In some implementations, in response to detecting a change in the distance between the viewpoint position and the virtual object position, the display device 202 receives information corresponding to the detected line-of-sight position and / or determines whether the detected line-of-sight position is within a predetermined threshold distance of the virtual object 208.

[0107] In some implementations, although the viewpoint position and the virtual object position are within the same continuous distance group, the operation of the display device 202 (see reference) Figures 4A to 4B The operation is different from that of display device 202 (see reference). Figures 8A to 8B ).review Figure 4A When the distance 404 is within the continuous distance group 238b, the display device 202 magnifies the virtual object 208. However, in Figure 8A In this process, because the detected gaze position is within a predetermined threshold distance of the virtual object 208, the size of the virtual object 208 remains unchanged. In some implementations, when the gaze position is within the predetermined threshold distance of the virtual object 208, the display device 202 determines that a person 250 is looking at the virtual object 208.

[0108] In some implementations, display device 202 changes the size of other virtual objects while maintaining the size of virtual object 208. In some implementations, display device 202 changes the size of other virtual objects based on determining that the detected gaze position 750 is not within a predetermined threshold distance corresponding to the position of at least one of the other virtual objects. In some implementations, when the gaze position is not within the predetermined threshold distance corresponding to the position of a virtual object, display device 202 determines that person 250 is moving their gaze away from at least one of the other virtual objects.

[0109] Return to Figure 8B The increased size of display device 202 indicates the size of 218. In Figure 8B In this context, the display device 202 is enlarged to represent the virtual object 208 from the viewpoint of the display device 202. Here, because the person 250 is standing closer (in... Figure 8A China and Belgium in Figure 7A (The virtual object in the middle) has not yet changed size, so it means that the size of 218 has increased.

[0110] Figure 9A This illustrates a scenario where person 250 does not move to different locations. (For example...) Figure 9A As shown, the viewpoint position is kept at a distance of 804 / 904 from the virtual object position.

[0111] Figure 9B It shows the corresponding Figure 9A The display screen 230 is shown in the scenario depicted. (As shown) Figure 9BAs shown, the detected line-of-sight position 750 is in relation to... Figure 8B The different positions in the middle.

[0112] Assuming in Figures 9A to 9B In this process, display device 202 receives information corresponding to the detected gaze position 750. In response to receiving the information corresponding to the detected gaze position 750, display device 202 determines that the distance between the detected gaze position 750 and the position corresponding to the virtual object 208 (or representation 218) is not within a predetermined threshold distance. For example... Figure 9A As shown, the display device 202 changes the size of the virtual object 208 based on this determination.

[0113] In some implementations, the display device 202 changes the size of the virtual object 208 because when the detected gaze position 750 is not within a predetermined threshold distance from the position corresponding to the virtual object 208, the display device 202 has determined that the person is not looking directly at the virtual object 208. In some implementations, changing the size of the virtual object when the person is looking at it creates a user interface that is more likely to be visually distracting or misleading.

[0114] In some implementations, the size of the virtual object 208 is reduced to the size corresponding to the continuous distance group 238b.

[0115] In some implementations, when the display device 202 changes the size of the virtual object 208, it maintains the size of other virtual objects based on determining that the detected gaze position 750 is within a predetermined threshold distance of other virtual objects (e.g., detecting that a person is moving their gaze away from the virtual object). In some implementations, because the virtual object is not an object configured to resize (or resize when the display device 202 is within a specific consecutive group of distances), as described above regarding... Figures 4A to 4B Therefore, the display device 202 maintains the size of other virtual objects.

[0116] Return to Figure 9B Although 250 people stood Figure 8A and Figure 9A At the same position in the text, but because the size of virtual object 208 has been reduced, it indicates that the size of 218 has been reduced.

[0117] Figure 10A This illustrates a scenario where person 250 does not move to different locations. (For example...) Figure 10A As shown, the distance between the viewpoint position and the virtual object position is maintained at 804 / 904 / 1004.

[0118] Figure 10B It shows the corresponding Figure 10ADisplay screen 230 in the scene shown. Regarding... Figures 10A to 10B Because the detected gaze position 750 is currently within a predetermined threshold distance from the virtual object 208 (e.g., the user (or person) is looking at the virtual object 208), the display device 202 uses the above-described... Figure 8A The technology described above, similar to other technologies, maintains (or does not change) the size of the virtual object 208. Furthermore, because the person 250 remains within the same continuous distance group 238b, the display device 202 maintains (or does not change) the size of the virtual object 208. Moreover, because the virtual object 208 does not change size and the person 250 stands in the same position, the device 202 does not change the size of the representation 218.

[0119] Although Figures 7A to 7B , Figures 8A to 8B , Figures 9A to 9B and Figures 10A to 10B The illustration depicts a scenario where display device 202 (or another component of system 100) changes (or does not change) the size of a virtual object based on detected movement of a user's gaze within display screen 230, but one or more other embodiments may also be implemented. For example, display device 202 (or another component of system 100) may change the size of the virtual object as display device 202 moves, such that the virtual object does not appear in the viewer's field of vision and / or display screen 230. Similarly, when display device 202 is positioned such that the virtual object does appear in the viewer's field of vision and / or display screen 230, display device 202 (or another component of system 100) may refrain from changing the size of the virtual object. Additionally or alternatively, when the virtual object moves in the environment while the viewer's gaze or field of vision / display screen 230 remains stationary, display device 202 (or another component of system 100) may change the size of the virtual object such that, for example, the detected viewer's gaze is not focused on or directed at the object, or the virtual object does not appear in the viewer's field of vision and / or display screen 230. Similarly, when a virtual object moves in the environment while a person's line of sight or field of view / display 230 remains stationary, display device 202 (or another component of system 100) can refrain from changing the size of the virtual object, such that (for example) the detected person's line of sight is focused on or looking at the object, or the virtual object appears in the person's field of view and / or display 230.

[0120] In addition, regarding Figures 2A to 2B , Figures 3A to 3B , Figures 4A to 4B , Figures 5A to 5B , Figures 6A to 6B , Figures 7A to 7B , Figures 8A to 8B , Figures 9A to 9B and Figures 10A to 10BIn the example shown, the distance (e.g., distance 204, distance 304, distance 404, etc.) changes based on a change in viewpoint position. However, one or more other implementations can also be implemented. For example, additionally or alternatively, the distance may change based on a change in the position of the virtual object.

[0121] In addition, regarding Figures 2A to 2B , Figures 3A to 3B , Figures 4A to 4B , Figures 5A to 5B , Figures 6A to 6B , Figures 7A to 7B , Figures 8A to 8B , Figures 9A to 9B and Figures 10A to 10B In the example shown, display device 202 is illustrated as having a single two-dimensional display (and a corresponding single detected gaze 750). It should be understood that in some embodiments, display device 202 may include a pair of stereoscopic displays 230 and a detected gaze 750 for each eye.

[0122] Figure 11 This is a flowchart illustrating a method for non-linearly resizing a virtual object according to some embodiments. In some embodiments, system 100 ( Figure 1A and Figure 1B The method is executed by the display device 202 in some embodiments. In some embodiments, the method is executed by a third device or system other than the display device 202 or system 100. In some embodiments, the method is executed by a combination of one or more of the system 100, the display device 202, and the third device or system.

[0123] At box 1102, a first virtual object is displayed in the XR environment. The first virtual object has a first position corresponding to the location of the first virtual object. In some embodiments, the first position is within a first consecutive group of distances from a second position corresponding to the viewpoint of the display device (e.g., viewpoint position). For example, as described above regarding... Figures 2A to 2B The virtual object 208 is displayed on the display screen 230 of the display device 202, and the virtual object 208 has a first size. Furthermore, in Figures 2A to 2B In some implementations, the virtual object 208 is located at a position corresponding to the virtual object location in the XR environment.

[0124] In some implementations, the first virtual object corresponds to a physical object or artifact in a physical environment that is at least a portion of an XR environment. In some implementations, the size of the virtual object can include one or more of the following: length, width, area, volume, etc.

[0125] In some implementations, the distance between the first position and the second position is within a first consecutive distance group. For example, as mentioned above... Figures 2A to 2B The virtual object 208 is located at a position corresponding to a virtual object position (e.g., a first position) that is a distance 204 from the viewpoint position. Furthermore, in Figures 2A to 2B In some implementations, distance 204 is within a continuous distance group 238a.

[0126] In some implementations, continuous distance groups can be defined in different ways. In some implementations, continuous distance groups can be defined as sequential distance groups. For example, a sequential distance group could correspond to all locations corresponding to virtual object positions that are 3 to 5 meters apart from locations corresponding to viewpoint positions.

[0127] In some implementations, a continuous distance group can be defined by a point in space. For example, a viewpoint located at any coordinate position from (0,3) to (0,5), while the position corresponding to the virtual object's location is at position (0,0).

[0128] In some implementations, one or more line segments define a continuous group of distances. For example, one end of a line segment (e.g., the minimum threshold distance) might be at 3 meters, and the other end of the line segment (e.g., the maximum threshold distance) might be at 5 meters. In some implementations, a line segment defines the distance between a location corresponding to a virtual object and a location corresponding to a viewpoint location.

[0129] refer to Figure 11 At box 1104, when displaying a first virtual object of a first size, and when the distance between the first and second positions is within a first consecutive distance group, a change in the distance between the first and second positions is detected. For example, refer to... Figures 2A to 2B , Figures 3A to 3B and / or Figures 4A to 4B When the viewpoint moves away from the virtual object from a distance of 204 to a distance of 304, or when the viewpoint moves away from the virtual object from a distance of 304 to a distance of 404, the change in distance is detected.

[0130] In some implementations, a change is detected when one or more of the viewpoint position and the virtual object position change. In some implementations, the change in distance is an incremental change in distance. In some implementations, a change in distance is not recognized until it meets a certain threshold. For example, in some implementations, a change in distance is detected when the display device has moved a first threshold distance (such as one meter) from its initial position. Furthermore, in some implementations, a change in distance is not detected when the display device has not moved a first threshold distance from its original position.

[0131] In some implementations, the term "first position" refers to a different position when the virtual object's position changes and / or is updated to refer to the new position of the virtual object when the virtual object moves. In some implementations, the term "second position" refers to a different position when the viewpoint's position changes and / or is updated to refer to the new position of the viewpoint when the viewpoint moves. In some implementations, a change is detected when the distance between the first position and the second position has decreased or increased by an increment, and no change is detected when the distance between the first position and the second position has increased by a distance less than that increment.

[0132] In some implementations, the first position and the second position are positions in three-dimensional space. In some implementations, the first position and the second position are positions in two-dimensional space.

[0133] refer to Figure 11 At boxes 1106 and 1108, in response to detecting a change in the distance between the first and second positions and based on determining that a first set of criteria is met, the display of the first virtual object is maintained. In some embodiments, the first set of criteria includes criteria that are met when the distance between the first and second positions is within (or remains within) a first consecutive distance group. For example, refer to... Figures 2A to 2B and Figures 3A to 3B Because when Figures 2A to 2B and Figures 3A to 3B During the comparison, the distance between the viewpoint position and the virtual object position is kept within the continuous distance group 238a, thus maintaining the size of the virtual object 208.

[0134] In some embodiments, displaying a first virtual object includes displaying a representation of the first virtual object. In some embodiments, modifying the first virtual object to have a second size different from a first size includes modifying the size of the representation of the first virtual object based on the second size. In some embodiments, when displaying a representation of the first virtual object, and based on determining that a first set of criteria is met, the electronic device modifies the size of the representation of the first virtual object when displaying a first virtual object with a second size different from the first size. In some embodiments, the representation of the first virtual object includes a two-dimensional object, and the first virtual object is a three-dimensional object.

[0135] In some implementations, the first virtual object is maintained to have a first size, while the viewpoint position (e.g., the second position) is at an arbitrary distance from the virtual object position (e.g., the first position) within a first contiguous distance group (e.g., 238a).

[0136] refer to Figure 11At boxes 1106 and 1110, in response to detecting a change in the distance between the first and second positions and based on determining that a second set of criteria is met, the first virtual object is modified to have a second size different from the first size, and the first virtual object is displayed with the second size. In some embodiments, the second set of criteria includes criteria that are met when the distance between the first and second positions is not within (or does not remain within) a first consecutive distance group. In some embodiments, the second set of criteria includes criteria that are met when the distance between the first and second positions is within a second consecutive distance group. For example, refer to... Figures 3A to 3B and Figures 4A to 4B Because Figure 4A The distance between the mid-viewpoint position and the virtual object position is 404 compared to the distance between the mid-viewpoint position and the virtual object position. Figure 3A The mid-distance 304 (e.g., within the continuous distance group 238a) is within the new continuous distance group (e.g., 238b), thus changing the size of the virtual object 208.

[0137] In some implementations, the second set of criteria includes a criterion that is met when the distance between the first and second positions is within a second consecutive distance group and the first position is above (or below) a specific threshold distance within the second consecutive distance group. In other words, in some implementations, the size of the virtual object is determined based on whether the distance between the virtual object and the viewpoint is above (or below) a threshold distance included in the new consecutive distance group. For example, when the new consecutive distance group is 1 to 2 meters between the viewpoint and the virtual object, the virtual object is not resized until the distance between the virtual object and the viewpoint is above a threshold distance (e.g., 1.2 meters, 1.33 meters, 1.41 meters) when the viewpoint is moving away from the virtual object, or until the distance between the virtual object and the viewpoint is below a threshold distance (e.g., 1.8 meters, 1.7 meters, 1.74 meters) when the viewpoint is moving towards the virtual object. In other words, the virtual object is not resized until it is determined that the distance between the viewpoint and the virtual object is exactly within the new consecutive distance group. In some implementations, resizing the virtual object is abandoned until the distance between the virtual object and the viewpoint is higher than a threshold distance for eliminating jitter, and / or the size of the virtual object is continuously increased and decreased as the distance between the virtual object and the viewpoint approaches the end of a new set of consecutive distances.

[0138] In some implementations, once the distance between a virtual object and the viewpoint is above a threshold distance (e.g., 1.2 meters) (or below a threshold distance) and the object has been resized, the virtual object is not resized again when the distance between it and the viewpoint falls within a new consecutive distance group (1 to 2 meters), even if the distance between the virtual object and the virtual location becomes below (or above) the threshold distance (e.g., 1.2 meters). Therefore, in some implementations, when the virtual object is within a new consecutive distance group, it is only resized once and is not resized again until the distance between the viewpoint and the virtual object falls within a different consecutive distance group.

[0139] In some implementations, displaying the first virtual object includes displaying a representation of the first virtual object. In some implementations, while displaying a representation of the first virtual object having a first size, and based on determining that a second set of criteria is met, the electronic device modifies the size of the representation of the first virtual object while maintaining the display of the first virtual object having the first size.

[0140] In some embodiments, a first virtual object of a second size is displayed when the first virtual object is within a second consecutive distance group. In some embodiments, a first virtual object of a second size is displayed because the distance between the first and second positions is within a first consecutive distance group (e.g., 238b). In some embodiments, a first virtual object of a second size is displayed because the first and second positions are at any distance within the first consecutive distance group (e.g., 238b). In some embodiments, a first virtual object of a second size is displayed when the distance between the first and second positions is within a second consecutive distance group (or at any position or distance within the second consecutive distance group) from the virtual object.

[0141] In some implementations, the size of the virtual object does not depend on a specific distance between the first and second positions. Instead, the size of the virtual object depends on a consecutive group of distances between the first and second positions. In some implementations, the size of the virtual object is independent of its position relative to the viewpoint of the display device in the XR environment.

[0142] In some implementations, the size of the virtual object depends on a specific group of consecutive distances. For example, the virtual object may be small when the distance between the first and second positions is within a first group of consecutive distances, medium-sized when the distance is within a second group of consecutive distances, and large when the distance is within a third group of consecutive distances. In some implementations, the first size is a predetermined size based on a first group of consecutive distances. In some implementations, the second size is a predetermined size based on a second group of consecutive distances.

[0143] In some implementations, the change in the distance between the first and second positions results in a decrease in the distance between the first and second positions, with the second size being smaller than the first size. In some implementations, the change in the distance between the first and second positions results in an increase in the distance between the first and second positions, with the second size being larger than the first size.

[0144] In some implementations, a first virtual object is displayed as part of a second size different from the first size, while a third virtual object maintains a sixth size. In some implementations, when one or more other objects change size, the size of the other one or more objects remains the same. In some implementations, while changing the size of one object, the size of another object is maintained, and the size of one object is continuously adjusted so that the other object appears to be moving relative to that object. In some of these implementations, the other object behaves as if it is moving closer to / away from the viewpoint.

[0145] In some implementations, the second set of criteria includes criteria that are met when the first virtual object has not changed before being displayed with a first size. In some implementations, some virtual objects undergo only a predetermined number of size changes during the session. In some implementations, the size changes of different virtual objects are based on different determinations involving different groups of consecutive distances, different predetermined maximum number of scalings, different time limits related to when the virtual object first appears, etc.

[0146] In some implementations, the second set of criteria includes criteria that are met when it is determined that the field of view of the electronic device is not within a predetermined distance of the location corresponding to the first position.

[0147] In some implementations, in response to detecting a change in the distance between the first and second locations and based on determining that a third set of criteria is met (where the third set of criteria includes criteria met when the first location is within a third consecutive distance group), a first virtual object with a third size different from the first and second sizes is displayed. In some implementations, the third consecutive distance group is different from the first and second consecutive distance groups. In some implementations, the first virtual object is modified to have a third size different from the first and second sizes, and is displayed with the third size when the first and second locations are at any distance from each other within the third consecutive distance group.

[0148] In some implementations, a first consecutive distance group and a second consecutive distance group corresponding to the viewpoint location are adjacent consecutive distance groups. In some implementations, adjacent consecutive distance groups are groups that share each other's minimum and / or maximum thresholds. For example, if the first consecutive distance group is greater than or equal to 3 meters and less than 5 meters, and the second consecutive distance group is greater than or equal to 5 meters and less than 10 meters, then the first and second consecutive distance groups are adjacent consecutive distance groups because they share their respective minimum and maximum thresholds of 5 meters. In some implementations, consecutive distance groups are adjacent consecutive distance groups because they are adjacent distance groups.

[0149] In some implementations, the third consecutive distance group is adjacent to the first consecutive distance group, or to the second consecutive distance group, or to both. In some implementations, all consecutive distance groups or no consecutive distance groups are adjacent.

[0150] In some embodiments, the first and second consecutive distance groups do not overlap. In some embodiments, the third consecutive distance group overlaps with the first, second, or both (e.g., boundary locations, such as virtual locations 236c and / or 236e). In some embodiments, no consecutive distance groups overlap.

[0151] In some implementations, each distance in the first consecutive distance group is outside the second consecutive distance group. In other words, each distance in the first consecutive distance group is not within the second consecutive distance group (and vice versa).

[0152] In some implementations, a second virtual object is displayed. In some implementations, a second virtual object of a first size is displayed based on the determination that a first set of criteria is met. In some implementations, a second virtual object of a first size is also displayed based on the determination that a second set of criteria is met.

[0153] In some implementations, virtual objects only change size when they first appear.

[0154] In some implementations, a second virtual object with a fourth size is displayed. In some implementations, the display of the second virtual object with the fourth size is maintained based on the determination that a first set of criteria is met. In some implementations, based on the determination that a second set of criteria is met, the second virtual object is modified to have a fifth size different from the fourth size, and the second virtual object with the fifth size is displayed.

[0155] Figure 12 This is a flowchart illustrating a method for non-linearly resizing a virtual object according to some embodiments. In some embodiments, system 100 ( Figure 1A and Figure 1B The method is executed by the display device 202 in some embodiments. In some embodiments, the method is executed by a third device or system other than the display device 202 or system 100. In some embodiments, the method is executed by a combination of one or more of the system 100, the display device 202, and the third device or system.

[0156] At box 1202, a first virtual object (e.g., 208) is displayed. In some embodiments, the first virtual object has a first size and is located at a first position in the XR environment. In some embodiments, the first virtual object may have the characteristics described above. Figure 11 One or more characteristics of the first virtual object.

[0157] refer to Figure 12 At box 1204, while displaying a first virtual object with a first size (e.g., 208), a request to change the first size of the virtual object is detected. In some embodiments, displaying the first virtual object includes displaying a representation of the first virtual object.

[0158] refer to Figure 12 At boxes 1206 and 1208, in response to detecting a request to change the first size of a virtual object and based on determining that a first set of criteria is met, the first virtual object is modified such that the first virtual object has a second size different from the first size. In some embodiments, the first set of criteria includes criteria that are met when it is determined that a detected line of sight is away from a first position. For example, in Figure 9A The size ratio of the virtual object 208 in the middle is Figure 8A The size of the virtual object 208 is smaller because the distance between the detected user's line of sight 750 and the position corresponding to the virtual object 208 is not within the predetermined distance.

[0159] In some implementation schemes, such as those mentioned above Figure 11The request to change the size of a virtual object is detected by detecting a change in the distance between the location corresponding to the virtual object's position and the viewpoint position. In some embodiments, the request to change the size of the virtual object is detected when a request is received from one or more internal or external applications, processes, sensors, etc. In some embodiments, the request to change the size of the virtual object is detected based on one or more parameters (e.g., application parameters).

[0160] In some implementations, when it is determined that the detected user gaze is within a predetermined distance from a third virtual object different from the first virtual object, it is determined that the detected user gaze has deviated from the first position. In some implementations, the detected user gaze is located at a position within the XR environment.

[0161] In some implementations, it is determined that the user is looking at a virtual object when the position of the detected user gaze (750) is within a threshold distance from the position of the third virtual object (or from (e.g., away from) the first virtual object).

[0162] In some implementations, when it is determined that the detected line of sight is not within a predetermined distance from a portion of the XR environment, it is determined that the detected line of sight is away from a first virtual object. In some implementations, the portion of the XR environment includes two or more virtual objects. In some implementations, the portion of the XR environment includes a first virtual object.

[0163] In some implementations, it is determined that the user is looking at the virtual object when the detected location of the user's gaze is within a threshold distance from one or more locations corresponding to a portion of the XR environment that includes the first virtual object.

[0164] In some implementations, the first set of criteria includes criteria that are no longer satisfied when the virtual object (e.g., 208) is not changed before it is displayed with a first size.

[0165] In some implementations, some virtual objects undergo only a predetermined number of resizing events (e.g., once or twice) during a session. In some implementations, the resizing of different virtual objects is based on different determinations of different time limits involving different consecutive distance groups, different predetermined maximum number of scaling events, and / or when the virtual object first appears.

[0166] In some implementation schemes, the above-mentioned... Figure 11 The first set of standards discussed includes standards that are satisfied when the distance between the electronic device and the location corresponding to the virtual object's position is within a second consecutive set of distances.

[0167] In some implementation schemes, the above-mentioned... Figure 11 One or more techniques discussed herein are used to modify a first virtual object so that the first virtual object has a second size that is different from the first size, thereby abandoning the modification of the second virtual object.

[0168] In some implementations, after modifying the first virtual object to have a second size different from the first size, and when the first virtual object has a second size, and based on determining that the first set of criteria is met, the first virtual object is kept to have a second size.

[0169] In some implementations, the size of the virtual object is maintained when the detected gaze returns to the object after the virtual object has been modified to the size of the object based on the determination that it does not meet the first set of criteria.

[0170] In some implementations, modifying the first virtual object to have a second size different from the first size includes modifying the size of the representation of the first virtual object. In some implementations, maintaining the first virtual object having a first size includes modifying the size of the representation of the first virtual object. In some implementations, the representation of the first virtual object is a two-dimensional object, and the first virtual object is a three-dimensional object.

[0171] In some embodiments, maintaining the display of a first virtual object having a first size includes modifying the size of the representation of the first virtual object based on the changing distance between the first and second positions and the first size. In some embodiments, displaying a first virtual object having a second size includes modifying the size of the representation of the first virtual object based on the changing distance between the first and second positions and the second size.

[0172] refer to Figure 12 At boxes 1206 and 1210, in response to detecting a request to change the first size of a virtual object and based on determining that a first set of criteria is not met, the first virtual object maintains its first size. In some implementations, the first virtual object with the first size is displayed when the user's current line of sight is outside a first threshold distance. For example, with... Figure 7A Compared to virtual object 208 in the middle, in Figure 8A The size of the virtual object 208 is maintained because the detected distance between the user's gaze 750 and the position corresponding to the virtual object 208 (e.g., the first position) is within a predetermined distance. Additionally, the display of the first virtual object having a first size is maintained, which includes, for example, in... Figure 7B and Figure 8B The change in the size represented in the middle.

[0173] In some implementations, this set of criteria is not met when it is determined that the user's current gaze is directed toward the first virtual object. In some implementations, the user's current gaze is determined to be directed toward the first virtual object when the detected user gaze is directed toward a second location within the XR environment, wherein the distance between the second location and the first location of the first virtual object is greater than or equal to a threshold distance.

[0174] In some implementations, after maintaining the first virtual object at a first size and based on determining that a first set of criteria is met, the first virtual object is modified to have a second size different from the first size. In some implementations, modifying the first virtual object includes displaying a virtual object different from the first virtual object.

[0175] In some implementations, after maintaining the first virtual object such that the first visible object has a second size and based on determining that a first set of criteria is met, the first virtual object is modified such that the first visible object has a third size different from the first and second sizes.

[0176] In some implementations, after maintaining the size of an object based on determining that a first set of criteria is met, the size of the virtual object is changed when a detected line of sight moves away from the object (e.g., when the user's gaze leaves the object).

[0177] In some implementations, a first virtual object of a second size is displayed at a second location in the XR environment. In some implementations, the above-mentioned... Figure 11 One or more techniques are discussed, in which a second position is located within a first contiguous distance group in an XR environment, and a virtual object is displayed at the second position.

[0178] In some implementations, the size of the virtual object is a predetermined size corresponding to a first consecutive group of distances in the XR environment. In some implementations, the size of the virtual object does not depend on a specific distance between the virtual object and the location corresponding to the viewpoint, but rather on a consecutive group of distances between the virtual object and the location corresponding to the viewpoint. In some implementations, the size of the virtual object is independent of its position relative to the viewpoint at the display device in the XR environment. In some implementations, the size of the virtual object depends on a consecutive group of distances.

[0179] As described above, one aspect of this technology is the collection and use of data available from various sources to provide dedicated resource management for devices, thereby saving battery life for users and providing dedicated content to device users. This disclosure contemplates that, in some instances, such collected data may include personal information data that uniquely identifies or can be used to contact or locate specific individuals. Such personal information data may include demographic data, location-based data, telephone numbers, email addresses, Twitter IDs, home addresses, data or records related to a user's health or fitness level (e.g., vital sign measurements, medication information, exercise information), date of birth, or any other identifying or personal information.

[0180] This disclosure recognizes that the use of such personal information data in the techniques of this invention can benefit users. For example, personal information data can be used to extend the battery life of a user's device. Thus, for example, the use of such personal information data helps a system properly manage resources to extend the battery life of a device. Furthermore, this disclosure also anticipates other uses of personal information data that benefit users. For example, health and fitness data can be used to provide insights into a user's overall health status or can be used as positive feedback for individuals using technology to pursue health goals.

[0181] This disclosure assumes that entities responsible for collecting, analyzing, disclosing, transmitting, storing, or otherwise using such personal information data will comply with established privacy policies and / or privacy practices. Specifically, such entities should implement and adhere to privacy policies and practices that are recognized as meeting or exceeding industry or governmental requirements for maintaining the privacy and security of personal information data. These policies should be readily accessible to users and should be updated as data collection and / or use change. Personal information from users should be collected for the entity's lawful and reasonable purposes and not shared or sold outside of these lawful uses. Furthermore, such collection / sharing should be conducted only after obtaining informed consent from users. Additionally, such entities should consider taking any necessary steps to protect and safeguard access to such personal information data and ensure that others with access to such personal information data comply with their privacy policies and processes. Furthermore, such entities may be subject to third-party evaluations to demonstrate their compliance with widely accepted privacy policies and practices. Additionally, policies and practices should be adapted to the specific types of personal information data collected and / or accessed, and to applicable laws and standards, including specific considerations regarding jurisdiction. For example, in the United States, the collection or acquisition of certain health data may be governed by federal and / or state laws, such as the Health Insurance Portability and Accountability Act (HIPAA); while in other countries, health data may be subject to other regulations and policies and should be handled accordingly. Therefore, different privacy practices should be maintained for different types of personal data in each country.

[0182] Regardless of the foregoing, this disclosure also envisions examples of users selectively blocking the use or access to personal information data. That is, this disclosure anticipates providing hardware and / or software components to prevent or block access to such personal information data. For example, in relation to managing resources for low-power devices, this technology can be configured to allow users to opt-in or opt-out at any time during or after service registration to participate in the collection of personal information data. In another example, users may choose not to provide eye-tracking data, such as pupil position, pupil dilation, and / or blink rate for dedicated resource management. In yet another example, users may choose to limit the duration of eye-tracking data retention or completely prohibit the development of baseline eye-tracking profiles. In addition to providing "opt-in" and "opt-out" options, this disclosure also envisions providing notifications related to access to or use of personal information. For example, users may be notified when downloading an application that their personal information data will be accessed, and then reminded again just before the application accesses the personal information data.

[0183] Furthermore, the purpose of this disclosure is to manage and process personal information data to minimize the risk of unintentional or unauthorized access or use. Once data is no longer needed, this risk can be minimized by limiting data collection and deleting data. Additionally, and where applicable, including in certain health-related applications, data deidentification can be used to protect user privacy. Where appropriate, deidentification can be facilitated by removing specific identifiers (e.g., date of birth, etc.), controlling the amount or characteristics of stored data (e.g., collecting location data at the city level rather than address level), controlling how data is stored (e.g., aggregating data among users), and / or other methods.

[0184] Therefore, while this disclosure broadly covers the use of personal information data to implement one or more of the various disclosed examples, it also contemplates that various examples can be implemented without access to such personal information data. That is, various examples of the inventive technology will not be unable to function properly due to the lack of all or part of such personal information data. For example, preferences can be inferred based on non-personal information data or an absolute minimum of personal information (such as content requested by the device associated with the user, other non-personal information or publicly available information available to the system controlling the device), thereby managing the resources of low-power devices and selecting content (e.g., status updates and / or objects) and delivering them to the user.

Claims

1. A method comprising: In a system having memory, a display device, and one or more processors: A first virtual object of a first size is displayed via the display device, wherein the distance between a first position corresponding to the location of the first virtual object and a second position corresponding to the viewpoint of the display device is between a first threshold distance and a second threshold distance less than the first threshold distance; When the first virtual object with the first size is displayed, and when the distance between the first position and the second position is between the first threshold distance and the second threshold distance, a change in the distance between the first position and the second position is detected; as well as In response to detecting a change in the distance between the first position and the second position: Based on the determination that a first set of criteria is met, wherein the first set of criteria includes criteria that are met when the changed distance between the first position and the second position is between the first threshold distance and the second threshold distance, the display of the first virtual object having the first size is maintained, wherein the first virtual object having the first size is displayed when the changed distance between the first position and the second position is between the first threshold distance and the second threshold distance; as well as Based on the determination that a second set of criteria is met, wherein the second set of criteria includes criteria that are met when the changed distance between the first position and the second position is between a third threshold distance and a fourth threshold distance greater than the third threshold distance: Modify the first virtual object to have a second size different from the first size; and Display the first virtual object having the second size, wherein the first threshold distance is less than the third threshold distance, wherein the second threshold distance is less than the fourth threshold distance, and wherein the first virtual object having the second size is displayed when the changed distance between the first position and the second position is between the third threshold distance and the fourth threshold distance.

2. The method according to claim 1, wherein the second threshold distance is less than the third threshold distance.

3. The method according to claim 1, wherein the distance between the first threshold distance and the second threshold distance is not equal to the distance between the third threshold distance and the fourth threshold distance.

4. The method according to any one of claims 1 to 3, wherein displaying the first virtual object includes displaying a representation of the first virtual object.

5. The method of claim 4, wherein maintaining the display of the first virtual object having the first size includes modifying the size of the representation of the first virtual object based on the changed distance between the first position and the second position and the first size.

6. The method of claim 4, wherein displaying the first virtual object having the second size includes modifying the size of the representation of the first virtual object based on the changed distance between the first position and the second position and the second size.

7. The method of claim 4, wherein the representation of the first virtual object comprises a two-dimensional object, and the first virtual object is a three-dimensional object.

8. The method according to any one of claims 1 to 3, further comprising: In response to detecting a change in the distance between the first position and the second position: Based on the determination that a third set of criteria is met, wherein the third set of criteria includes criteria that are met when the changed distance between the first position and the second position is between a fifth threshold distance and a sixth threshold distance: Modify the first virtual object to have a third size different from the first and second sizes; and The first virtual object with the third size is displayed, wherein the first threshold distance and the third threshold distance are less than the fifth threshold distance, and wherein the second threshold distance and the fourth threshold distance are less than the sixth threshold distance, and wherein the first virtual object with the third size is displayed when the changed distance between the first position and the second position is between the fifth threshold distance and the sixth threshold distance.

9. The method according to claim 8, wherein the fifth threshold distance is less than the sixth threshold distance.

10. The method of claim 9, wherein The fourth threshold distance is equal to the fifth threshold distance.

11. The method of claim 9, wherein one or more of the following: The second threshold distance is less than the third threshold distance, and The fourth threshold distance is less than the fifth threshold distance.

12. The method according to any one of claims 1 to 3, wherein one or more of the first threshold distance and the second threshold distance are adjacent to one or more of the third threshold distance and the fourth threshold distance.

13. The method according to any one of claims 1 to 3, wherein one or more of the first threshold distance and the second threshold distance are not between one or more of the third threshold distance and the fourth threshold distance.

14. The method according to any one of claims 1 to 3, wherein each distance between the first threshold distance and the second threshold distance is outside the distance between the third threshold distance and the fourth threshold distance.

15. The method according to any one of claims 1 to 3, wherein the second set of criteria includes criteria satisfied when a determination is made that the detected user line of sight is not within a predetermined distance of the position corresponding to the first position.

16. The method according to any one of claims 1 to 3, wherein the first position and the second position are positions in three-dimensional space.

17. The method according to any one of claims 1 to 3, wherein the first position and the second position are positions in two-dimensional space.

18. The method according to any one of claims 1 to 3, wherein the second set of criteria includes criteria satisfied when a determination is made that the system's field of view is not within a predetermined distance of the position corresponding to the first position.

19. The method according to any one of claims 1 to 3, further comprising: A second virtual object with a fourth size is displayed via the display device, wherein: Based on the determination that the first set of criteria is met, the display of the second virtual object having the fourth size is maintained; and Based on the determination that the second set of criteria is met: Modify the second virtual object to have a fifth size that is different from the fourth size; and Display the second virtual object having the fifth size.

20. The method according to any one of claims 1 to 3, wherein the second set of criteria includes criteria that are satisfied when the first virtual object has not changed before being displayed with the first size.

21. The method according to any one of claims 1 to 3, wherein: The change in the distance between the first position and the second position resulted in a decrease in the distance between the first position and the second position, and the second size was smaller than the first size; and The change in the distance between the first position and the second position resulted in an increase in the distance between the first position and the second position, with the second size being larger than the first size.

22. The method according to any one of claims 1 to 3, wherein: The first size is a predetermined size based on one or more distances between the first threshold distance and the second threshold distance; and The second size is a predetermined size based on one or more distances between the third threshold distance and the fourth threshold distance.

23. The method according to any one of claims 1 to 3, wherein displaying the first virtual object having the second size includes maintaining a third virtual object having a sixth size.

24. The method according to any one of claims 1 to 3, wherein the second set of criteria includes criteria satisfied when the distance between the first position and the second position is above a threshold distance, wherein the threshold distance is the distance between the third threshold distance and the fourth threshold distance.

25. A computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a system having a display device, the one or more programs including instructions for: A first virtual object of a first size is displayed via the display device, wherein the distance between a first position corresponding to the location of the first virtual object and a second position corresponding to the viewpoint of the display device is between a first threshold distance and a second threshold distance less than the first threshold distance; When the first virtual object with the first size is displayed, and when the distance between the first position and the second position is between the first threshold distance and the second threshold distance, a change in the distance between the first position and the second position is detected; as well as In response to detecting a change in the distance between the first position and the second position: Based on the determination that a first set of criteria is met, wherein the first set of criteria includes criteria that are met when the changed distance between the first position and the second position is between the first threshold distance and the second threshold distance, the display of the first virtual object having the first size is maintained, wherein the first virtual object having the first size is displayed when the changed distance between the first position and the second position is between the first threshold distance and the second threshold distance; as well as Based on the determination that a second set of criteria is met, wherein the second set of criteria includes criteria that are met when the changed distance between the first position and the second position is between a third threshold distance and a fourth threshold distance greater than the third threshold distance: Modify the first virtual object to have a second size different from the first size; and Display the first virtual object having the second size, wherein the first threshold distance is less than the third threshold distance, wherein the second threshold distance is less than the fourth threshold distance, and wherein the first virtual object having the second size is displayed when the changed distance between the first position and the second position is between the third threshold distance and the fourth threshold distance.

26. The computer-readable storage medium of claim 25, wherein the second threshold distance is less than the third threshold distance.

27. The computer-readable storage medium of claim 25, wherein the distance between the first threshold distance and the second threshold distance is not equal to the distance between the third threshold distance and the fourth threshold distance.

28. The computer-readable storage medium according to any one of claims 25 to 27, wherein displaying the first virtual object includes displaying a representation of the first virtual object.

29. The computer-readable storage medium of claim 28, wherein maintaining the display of the first virtual object having the first size includes modifying the size of the representation of the first virtual object based on the changed distance between the first position and the second position and the first size.

30. The computer-readable storage medium of claim 28, wherein displaying the first virtual object having the second size includes modifying the size of the representation of the first virtual object based on the changed distance between the first position and the second position and the second size.

31. The computer-readable storage medium of claim 28, wherein the representation of the first virtual object comprises a two-dimensional object, and the first virtual object is a three-dimensional object.

32. The computer-readable storage medium according to any one of claims 25 to 27, wherein the one or more programs further include instructions for performing the following operations: In response to detecting a change in the distance between the first position and the second position: Based on the determination that a third set of criteria is met, wherein the third set of criteria includes criteria that are met when the changed distance between the first position and the second position is between a fifth threshold distance and a sixth threshold distance: Modify the first virtual object to have a third size different from the first and second sizes; and The first virtual object with the third size is displayed, wherein the first threshold distance and the third threshold distance are less than the fifth threshold distance, and wherein the second threshold distance and the fourth threshold distance are less than the sixth threshold distance, and wherein the first virtual object with the third size is displayed when the changed distance between the first position and the second position is between the fifth threshold distance and the sixth threshold distance.

33. The computer-readable storage medium of claim 32, wherein the fifth threshold distance is less than the sixth threshold distance.

34. The computer-readable storage medium of claim 33, wherein... The fourth threshold distance is equal to the fifth threshold distance.

35. The computer-readable storage medium of claim 33, wherein one or more of the following: The second threshold distance is less than the third threshold distance, and The fourth threshold distance is less than the fifth threshold distance.

36. The computer-readable storage medium according to any one of claims 25 to 27, wherein one or more of the first threshold distance and the second threshold distance are adjacent to one or more of the third threshold distance and the fourth threshold distance.

37. The computer-readable storage medium according to any one of claims 25 to 27, wherein one or more of the first threshold distance and the second threshold distance are not between one or more of the third threshold distance and the fourth threshold distance.

38. The computer-readable storage medium according to any one of claims 25 to 27, wherein each distance between the first threshold distance and the second threshold distance is outside the distance between the third threshold distance and the fourth threshold distance.

39. The computer-readable storage medium according to any one of claims 25 to 27, wherein the second set of criteria includes criteria satisfied when a determination is made that the detected user line of sight is not within a predetermined distance of the position corresponding to the first position.

40. The computer-readable storage medium according to any one of claims 25 to 27, wherein the first location and the second location are locations in three-dimensional space.

41. The computer-readable storage medium according to any one of claims 25 to 27, wherein the first location and the second location are locations in two-dimensional space.

42. The computer-readable storage medium according to any one of claims 25 to 27, wherein the second set of criteria includes criteria satisfied when a determination is made that the system's field of view is not within a predetermined distance of the position corresponding to the first position.

43. The computer-readable storage medium according to any one of claims 25 to 27, wherein the one or more programs further include instructions for performing the following operations: A second virtual object with a fourth size is displayed via the display device, wherein: Based on the determination that the first set of criteria is met, the display of the second virtual object having the fourth size is maintained; and Based on the determination that the second set of criteria is met: The second virtual object is modified to have a fifth size, which is different from the fourth size; as well as Display the second virtual object having the fifth size.

44. The computer-readable storage medium according to any one of claims 25 to 27, wherein the second set of criteria includes criteria that are satisfied when the first virtual object has not changed before being displayed with the first size.

45. The computer-readable storage medium according to any one of claims 25 to 27, wherein: The change in the distance between the first position and the second position resulted in a decrease in the distance between the first position and the second position, and the second size was smaller than the first size; as well as The change in the distance between the first position and the second position resulted in an increase in the distance between the first position and the second position, with the second size being larger than the first size.

46. ​​The computer-readable storage medium according to any one of claims 25 to 27, wherein: The first size is a predetermined size based on one or more distances between the first threshold distance and the second threshold distance; and The second size is a predetermined size based on one or more distances between the third threshold distance and the fourth threshold distance.

47. The computer-readable storage medium according to any one of claims 25 to 27, wherein displaying the first virtual object having the second size includes maintaining a third virtual object having a sixth size.

48. The computer-readable storage medium according to any one of claims 25 to 27, wherein the second set of criteria includes criteria satisfied when the distance between the first location and the second location is above a threshold distance, wherein the threshold distance is the distance between the third threshold distance and the fourth threshold distance.

49. A system comprising: Display devices; One or more processors; and The memory stores one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for performing the following operations: A first virtual object of a first size is displayed via the display device, wherein the distance between a first position corresponding to the location of the first virtual object and a second position corresponding to the viewpoint of the display device is between a first threshold distance and a second threshold distance less than the first threshold distance; When the first virtual object with the first size is displayed, and when the distance between the first position and the second position is between the first threshold distance and the second threshold distance, a change in the distance between the first position and the second position is detected; as well as In response to detecting a change in the distance between the first position and the second position: Based on the determination that a first set of criteria is met, wherein the first set of criteria includes criteria that are met when the changed distance between the first position and the second position is between the first threshold distance and the second threshold distance, the display of the first virtual object having the first size is maintained, wherein the first virtual object having the first size is displayed when the changed distance between the first position and the second position is between the first threshold distance and the second threshold distance; as well as Based on the determination that a second set of criteria is met, wherein the second set of criteria includes criteria that are met when the changed distance between the first position and the second position is between a third threshold distance and a fourth threshold distance greater than the third threshold distance: Modify the first virtual object to have a second size different from the first size; and Display the first virtual object having the second size, wherein the first threshold distance is less than the third threshold distance, wherein the second threshold distance is less than the fourth threshold distance, and wherein the first virtual object having the second size is displayed when the changed distance between the first position and the second position is between the third threshold distance and the fourth threshold distance.

50. The system of claim 49, wherein the second threshold distance is less than the third threshold distance.

51. The system of claim 49, wherein the distance between the first threshold distance and the second threshold distance is not equal to the distance between the third threshold distance and the fourth threshold distance.

52. The system according to any one of claims 49 to 51, wherein displaying the first virtual object includes displaying a representation of the first virtual object.

53. The system of claim 52, wherein maintaining the display of the first virtual object having the first size includes modifying the size of the representation of the first virtual object based on the changed distance between the first position and the second position and the first size.

54. The system of claim 52, wherein displaying the first virtual object having the second size includes modifying the size of the representation of the first virtual object based on the changed distance between the first position and the second position and the second size.

55. The system of claim 52, wherein the representation of the first virtual object comprises a two-dimensional object, and the first virtual object is a three-dimensional object.

56. The system according to any one of claims 49 to 51, wherein the one or more programs further include instructions for the following operations: In response to detecting a change in the distance between the first position and the second position: Based on the determination that a third set of criteria is met, wherein the third set of criteria includes criteria that are met when the changed distance between the first position and the second position is between a fifth threshold distance and a sixth threshold distance: Modify the first virtual object to have a third size different from the first and second sizes; and The first virtual object with the third size is displayed, wherein the first threshold distance and the third threshold distance are less than the fifth threshold distance, and wherein the second threshold distance and the fourth threshold distance are less than the sixth threshold distance, and wherein the first virtual object with the third size is displayed when the changed distance between the first position and the second position is between the fifth threshold distance and the sixth threshold distance.

57. The system of claim 56, wherein the fifth threshold distance is less than the sixth threshold distance.

58. The system according to claim 57, wherein The fourth threshold distance is equal to the fifth threshold distance.

59. The system of claim 57, wherein one or more of the following: The second threshold distance is less than the third threshold distance, and The fourth threshold distance is less than the fifth threshold distance.

60. The system according to any one of claims 49 to 51, wherein one or more of the first threshold distance and the second threshold distance are adjacent to one or more of the third threshold distance and the fourth threshold distance.

61. The system according to any one of claims 49 to 51, wherein one or more of the first threshold distance and the second threshold distance are not between one or more of the third threshold distance and the fourth threshold distance.

62. The system according to any one of claims 49 to 51, wherein each distance between the first threshold distance and the second threshold distance is outside the distance between the third threshold distance and the fourth threshold distance.

63. The system according to any one of claims 49 to 51, wherein the second set of criteria includes criteria satisfied when a determination is made that the detected user line of sight is not within a predetermined distance of the position corresponding to the first position.

64. The system according to any one of claims 49 to 51, wherein the first position and the second position are positions in three-dimensional space.

65. The system according to any one of claims 49 to 51, wherein the first position and the second position are positions in two-dimensional space.

66. The system according to any one of claims 49 to 51, wherein the second set of criteria includes criteria satisfied when a determination is made that the system's field of view is not within a predetermined distance of the position corresponding to the first position.

67. The system according to any one of claims 49 to 51, wherein the one or more programs further include instructions for the following operations: A second virtual object with a fourth size is displayed via the display device, wherein: Based on the determination that the first set of criteria is met, the display of the second virtual object having the fourth size is maintained; and Based on the determination that the second set of criteria is met: The second virtual object is modified to have a fifth size, which is different from the fourth size; as well as Display the second virtual object having the fifth size.

68. The system according to any one of claims 49 to 51, wherein the second set of criteria includes criteria that are no longer satisfied when the first virtual object is not changed before being displayed with the first size.

69. The system according to any one of claims 49 to 51, wherein: The change in the distance between the first position and the second position resulted in a decrease in the distance between the first position and the second position, and the second size was smaller than the first size; and The change in the distance between the first position and the second position resulted in an increase in the distance between the first position and the second position, with the second size being larger than the first size.

70. The system according to any one of claims 49 to 51, wherein: The first size is a predetermined size based on one or more distances between the first threshold distance and the second threshold distance; and The second size is a predetermined size based on one or more distances between the third threshold distance and the fourth threshold distance.

71. The system according to any one of claims 49 to 51, wherein displaying the first virtual object having the second size includes maintaining a third virtual object having a sixth size.

72. The system according to any one of claims 49 to 51, wherein the second set of criteria includes criteria satisfied when the distance between the first position and the second position is above a threshold distance, wherein the threshold distance is the distance between the third threshold distance and the fourth threshold distance.

73. A system comprising: Display devices; Memory; as well as Apparatus for performing the method according to any one of claims 1 to 3.

74. A computer program product comprising one or more programs configured to be executed by one or more processors of a system communicating with a display device, the one or more programs comprising instructions for performing the method according to any one of claims 1 to 3.

Images