Techniques for manipulating computer graphics objects

Abstract

This disclosure relates to techniques for manipulating computer graphics objects. The invention discloses a computer-generated virtual object manipulator having one or more power representations for manipulating computer-generated virtual objects. Selection of a virtual object causes the object manipulator to be displayed above the virtual object. The object manipulator may include a conical single-axis translation power representation for each of the one or more object axes, a disk-shaped single-axis scaling power representation for each of the one or more object axes, an arc-shaped rotation power representation for rotation about each of the one or more object axes, and an object center power representation for free-space movement of the virtual object. The object manipulator may also include a slice-shaped dual-axis translation power representation, which may be displayed after hovering over an area in a specific plane.

Description

[0001] Cross-references to related applications

[0002] This application claims the benefit of U.S. Provisional Application No. 63 / 216,397, filed June 29, 2021, and U.S. Patent Application No. 17 / 807,226, filed June 16, 2022, the contents of which are incorporated herein by reference in their entirety for all purposes. Technical Field

[0003] This disclosure relates in its entirety to computer graphics editors. Background Technology

[0004] Some computer graphics environments provide two-dimensional and / or three-dimensional environments in which at least some of the objects displayed for user observation are virtual and computer-generated. In some use cases, users can create or modify computer graphics environments, such as by using content-generating environments like graphics editors or graphics editing interfaces to edit, generate, or otherwise manipulate computer graphics virtual objects. Editors that allow for intuitive editing of computer-generated virtual objects are desired. Summary of the Invention

[0005] Some examples of this disclosure relate to a computer-generated virtual object manipulator having one or more power representations for manipulating a computer-generated virtual object. In some examples, selection of a virtual object can cause the object manipulator to appear above the virtual object. The object manipulator may include a conical single-axis translation power representation for each of the one or more object axes, a disk-shaped single-axis scaling power representation for each of the one or more object axes, an arc-shaped rotation power representation for rotation about each of the one or more object axes, and an object center power representation for free-space movement of the virtual object. The object manipulator may also include a slice-shaped dual-axis translation power representation that can be displayed after selection.

[0006] Clicking a specific single-axis translation indicator will cause some or all other indicators to disappear, and dragging the single-axis translation indicator along its associated object axis will translate the virtual object along that object axis. Clicking a specific single-axis scaling indicator will cause some or all other indicators to disappear, and dragging the single-axis scaling indicator along its associated object axis will scale the virtual object non-uniformly along that object axis, or scale the virtual object uniformly in all directions. Clicking a specific arc rotation indicator will display a complete ring on the plane of the specific arc rotation indicator, and will cause some or all other indicators to disappear, and dragging the selected rotation indicator along its ring will rotate the virtual object around its associated object axis. Hovering over an area in the plane defined by a rotation indicator will cause a slice-shaped dual-axis translation indicator to appear, selecting the slice-shaped dual-axis translation indicator will cause some or all other indicators to disappear, and dragging the selected dual-axis translation indicator will translate the virtual object in two dimensions. Clicking and dragging the object center indicator will cause the virtual object to be repositioned in free space in multiple dimensions. The accompanying drawings and detailed descriptions provide a comprehensive description of these examples, and it should be understood that the scope of this disclosure is not limited in any way. Attached Figure Description

[0007] To better understand the various embodiments described, reference should be made to the following detailed description in conjunction with the accompanying drawings, in which similar reference numerals often indicate corresponding parts throughout the drawings.

[0008] Figure 1 An electronic device is shown that displays an extended reality (XR) environment (e.g., a computer-generated environment) according to an example of this disclosure.

[0009] Figure 2 A block diagram illustrating an exemplary architecture of a system or device according to examples of this disclosure is shown.

[0010] Figure 3A An example of an editing environment graphical user interface (GUI) including representative virtual objects is shown according to this disclosure.

[0011] Figure 3B The selected virtual object and its associated object manipulator are shown as examples according to this disclosure.

[0012] Figure 3C Examples of the present disclosure are shown. Figure 3B Compared to the selected virtual object with a redirecting object manipulator viewed from a different perspective.

[0013] Figure 4AThe example shown illustrates the selection of a single-axis translational energy representation for a virtual object according to this disclosure.

[0014] Figure 4B The single-axis translation of a virtual object, represented by a single-axis translation energy, is shown as an example according to this disclosure.

[0015] Figure 5A The appearance and selection of a dual-axis translation energy representation of a virtual object according to an example of this disclosure are shown.

[0016] Figure 5B The dual-axis translation of a virtual object using dual-axis translation energy representation is shown as an example according to this disclosure.

[0017] Figure 6A The example shown illustrates the selection of a single-axis scaling energy representation of a virtual object according to this disclosure.

[0018] Figure 6B Uniform scaling of a virtual object represented using scaling energy is shown as an example according to this disclosure.

[0019] Figure 6C The non-uniform scaling of a virtual object represented by scaling energy is shown as an example according to this disclosure.

[0020] Figure 7A The highlights and selections of the rotational energy representation of a virtual object according to an example of this disclosure are shown.

[0021] Figure 7B The selection of a rotational energy representation of a virtual object is shown according to an example of this disclosure.

[0022] Figure 7C The rotation of a virtual object represented by rotational energy is shown in the example according to this disclosure.

[0023] Figure 8A The example shown illustrates the selection of the object-centered energy representation of a virtual object according to this disclosure.

[0024] Figure 8B An example of omnidirectional translation (i.e., screen space movement) of a virtual object using object-centered energy representation according to this disclosure is shown.

[0025] Figure 9 A flowchart illustrating a process for manipulating virtual objects, according to an example of this disclosure, is shown. Detailed Implementation

[0026] Computer graphics environments such as XR environments may include XR content. In some embodiments, XR content may be presented to a user via an XR file, which includes data representing the XR content and / or data describing how the XR content is presented. In some embodiments, the XR file includes data representing one or more XR scenes and one or more triggers for presenting the one or more XR scenes. For example, an XR scene may be anchored to a horizontal, flat surface such that the XR scene can be presented when the horizontal, flat surface is detected (e.g., within the field of view of one or more cameras). The XR file may also include data about one or more virtual objects associated with the XR scene, and / or triggers and actions associated with the XR virtual objects.

[0027] Typically, to simplify the generation of XR files and / or the editing of computer-generated graphics, a computer graphics editor that includes a content generation environment (e.g., an editing environment GUI) can be used. In some implementations, the content generation environment itself is an XR environment (e.g., a two-dimensional and / or three-dimensional environment). For example, the content generation environment may include one or more virtual objects and one or more representations of real-world objects. In some implementations, virtual objects are overlaid on top of the physical environment or its representation.

[0028] A physical environment refers to the physical world that people can sense and / or interact with without the aid of electronic devices. A physical environment can include physical features such as physical surfaces or physical objects. For example, a physical environment corresponds to a physical park that includes physical trees, physical buildings, and physical people. People can directly sense and / or interact with a physical environment through senses such as sight, touch, hearing, taste, and smell. In contrast, an XR environment refers to a fully or partially simulated environment that people sense and / or interact with via electronic devices. For example, an XR environment can include augmented reality (AR) content, mixed reality (MR) content, virtual reality (VR) content, etc. In the case of an XR system, a subset of a person's physical motion or a representation thereof is tracked, and in response, one or more features of one or more virtual objects simulated in the XR system are adjusted in a manner consistent with at least one physical law. For example, an XR system can detect head movement and, in response, adjust the graphical content and sound field presented to the person in a manner similar to how such views and sounds change in a physical environment. For example, an XR system can detect movement of electronic devices (e.g., mobile phones, tablets, laptops, etc.) that present the XR environment, and in response, adjust the graphical content and sound field presented to the user in a manner similar to how such views and sounds would change in a physical environment. In some cases (e.g., for accessibility reasons), an XR system may adjust the characteristics of the graphical content in the XR environment in response to representations of physical motion (e.g., voice commands).

[0029] In some implementations, the physical environment is captured by one or more cameras of an electronic device and actively displayed in an XR environment (e.g., via a display generating component). In other implementations, the physical environment is provided (e.g., passively) by the electronic device, for example, if the display generating component includes a translucent or transparent element through which a user can see the physical environment.

[0030] In such a content-generating environment, users can create virtual objects from scratch (including the appearance of the virtual objects, their behavior / actions, and / or the triggering of their behavior / actions). Additionally or alternatively, virtual objects can be created by other content creators and imported into the content-generating environment, where they can be placed within an XR environment or scene. In some implementations, virtual objects generated in or throughout the content-generating environment (e.g., via generating XR files and importing or opening them in a computer graphics editor application or XR viewer application) can be exported to other environments or XR scenes.

[0031] In some implementations, the editing environment GUI may include one or more graphical user interface elements to enable one or more transformations of virtual objects. The graphical user interface elements used to transform virtual objects may be referred to herein as “manipulators” or “manipulator elements.” Manipulators can be used to perform movement, rotation, or scaling actions on virtual objects. In some implementations, manipulators may provide multiple elements to enable multiple transformation actions. In some implementations, manipulators may provide the ability to perform movement, rotation, and scaling actions on virtual objects (e.g., as described herein with respect to manipulators). As used herein, the term “enabled representation” refers to a user-interactive graphical user interface manipulator optionally displayed on a display generation component.

[0032] Some examples of this disclosure relate to a computer-generated virtual object manipulator having one or more power representations for manipulating a computer-generated virtual object. In some examples, selection of a virtual object can cause the object manipulator to appear above the virtual object. The object manipulator may include a conical single-axis translation power representation for each of the one or more object axes, a disk-shaped single-axis scaling power representation for each of the one or more object axes, an arc-shaped rotation power representation for rotation about each of the one or more object axes, and an object center power representation for free-space movement of the virtual object. The object manipulator may also include a slice-shaped dual-axis translation power representation that can be displayed after selection.

[0033] Clicking a specific single-axis translation indicator will cause some or all other indicators to disappear, and dragging the single-axis translation indicator along its associated object axis will translate the virtual object along that object axis. Clicking a specific single-axis scaling indicator will cause some or all other indicators to disappear, and dragging the single-axis scaling indicator along its associated object axis will scale the virtual object non-uniformly along that object axis, or scale the virtual object uniformly in all directions. Clicking a specific arc rotation indicator will display a complete ring on the plane of the specific arc rotation indicator, and will cause some or all other indicators to disappear, and dragging the selected rotation indicator along its ring will rotate the virtual object around its associated object axis. Hovering over an area in the plane defined by a rotation indicator will cause a slice-shaped dual-axis translation indicator to appear, selecting the slice-shaped dual-axis translation indicator will cause some or all other indicators to disappear, and dragging the selected dual-axis translation indicator will translate the virtual object in two dimensions. Clicking and dragging the object center indicator will cause the virtual object to be repositioned in free space in multiple dimensions. The accompanying drawings and detailed descriptions provide a comprehensive description of these examples, and it should be understood that the scope of this disclosure is not limited in any way.

[0034] Many different types of electronic systems enable people to sense and / or interact with various XR environments. Implementation schemes of electronic devices and user interfaces for such systems are described. In some implementations, the device is a portable communication device, such as a laptop or tablet. In some implementations, the device is a mobile phone that also includes other functions such as a personal digital assistant (PDA) and / or music player functionality. In some implementations, the device is a wearable device, such as a watch, head-mounted display, etc.

[0035] Examples include head-mounted systems, projection-based systems, head-up displays (HUDs), vehicle windshields with integrated display capabilities, windows with integrated display capabilities, displays shaped as lenses designed for placement on a person's eyes (e.g., similar to contact lenses), headphones / earpieces, speaker arrays, input systems (e.g., wearable or handheld controllers with or without haptic feedback), smartphones, tablets, and desktop / laptop computers. Head-mounted systems may have an integrated opaque display and one or more speakers. Alternatively, head-mounted systems may be configured to receive an external opaque display (e.g., a smartphone). Head-mounted systems may incorporate one or more imaging sensors for capturing images or video of the physical environment, and / or one or more microphones for capturing audio of the physical environment. Head-mounted systems may have transparent or semi-transparent displays instead of opaque displays. Transparent or semi-transparent displays may have a medium through which light representing the image is directed to a person's eyes. The display may utilize digital light projection, OLED, LED, uLED, liquid crystal on silicon, laser scanning light sources, or any combination of these technologies. The medium can be an optical waveguide, a holographic medium, an optical combiner, an optical reflector, or any combination thereof. In some implementations, transparent or translucent displays can be configured to selectively become opaque. Projection-based systems can employ retinal projection techniques that project graphic images onto the human retina. Projection systems can also be configured to project virtual objects onto a physical environment, such as as holograms or on a physical surface.

[0036] It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer or television set. In some embodiments, portable and non-portable electronic devices may optionally include touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). In some embodiments, the device does not include touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads), but is instead capable of outputting display information (such as the user interface of this disclosure) for display on an integrated or external display device, and is capable of receiving input information from an integrated or external input device having one or more input mechanisms (such as one or more buttons, mice, touchscreen displays, styluses, and / or touchpads). In some embodiments, the device has a display but is capable of receiving input information from a separate input device having one or more input mechanisms (such as one or more buttons, mice, touchscreen displays, and / or touchpads).

[0037] In the following discussion, an electronic device communicating with a display generating component and one or more input devices is described. It should be understood that the electronic device optionally communicates with one or more other physical user interface devices, such as a touch-sensitive surface, physical keyboard, mouse, joystick, hand-tracking device, eye-tracking device, stylus, etc. Furthermore, as mentioned above, it should be understood that the described electronic device, display, and touch-sensitive surface are optionally distributed among two or more devices. Therefore, as used in this disclosure, information on or displayed by the electronic device is optionally used to describe information output by the electronic device for display on a separate display device (touch-sensitive or non-touch-sensitive). Similarly, as used in this disclosure, input received on the electronic device (e.g., touch input received on a touch-sensitive surface of the electronic device, or touch input received on the surface of a stylus) is optionally used to describe input received on a separate input device from which the electronic device receives input information.

[0038] The device typically supports a variety of applications, such as one or more of the following: drawing applications, rendering applications, word processing applications, website creation applications, disk editing applications, spreadsheet applications, game applications, telephone applications, video conferencing applications, email applications, instant messaging applications, fitness support applications, photo management applications, digital camera applications, digital video camera applications, web browsing applications, digital music player applications, TV channel browsing applications, and / or digital video player applications. Additionally, the device may support applications for generating or editing computer-generated graphics and / or XR environment content (e.g., applications with content generation environments).

[0039] Various applications running on the device optionally use a common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and the corresponding information displayed on the device are optionally adjusted and / or varied for different applications, and / or adjusted and / or varied within the respective applications. In this way, the device's common physical architecture (such as the touch-sensitive surface) optionally utilizes a user interface that is intuitive and clear to the user to support various applications.

[0040] Figure 1 An electronic device 100 displaying an XR environment (e.g., a computer-generated environment) according to an example of this disclosure is shown. In some embodiments, the electronic device 100 is a handheld or mobile device, such as a tablet computer, laptop, smartphone, or head-mounted display. Reference is made below. Figure 2 An example of describing device 100 is a structural block diagram. Figure 1As shown, electronic device 100 and table 120 are located in physical environment 110. In some embodiments, electronic device 100 may be configured to capture an area of ​​physical environment 110 including table 120 (shown in the field of view of electronic device 100). In some embodiments, in response to a trigger, electronic device 100 may be configured to be in a computer-generated environment (e.g., by...) Figure 1 A virtual object 130 is displayed in a cube representation (as shown). This virtual object does not exist in the physical environment 110 but is displayed on top of a computer-generated representation 120' of a real-world table 120 in a computer-generated environment. For example, the virtual object 130 may be displayed on the surface of table 120' in a computer-generated environment displayed via device 100 in response to detecting a flat surface of table 120 in the physical environment 110. It should be understood that the virtual object 130 is a representative virtual object and one or more different virtual objects (e.g., virtual objects with various dimensions, such as two-dimensional or three-dimensional virtual objects) may be included and rendered in a three-dimensional computer-generated environment. For example, the virtual object may represent an application or user interface displayed in a computer-generated environment. In some examples, the application or user interface may include the display of content items of a content application (e.g., photos, videos, etc.). Additionally, it should be understood that the 3D environment (or 3D virtual object) described herein may be a representation of a 3D environment (or three-dimensional virtual object) displayed in a two-dimensional (2D) context (e.g., displayed on a 2D screen).

[0041] Figure 2 A block diagram illustrating an exemplary architecture of a system or device 200 according to an example of this disclosure is shown. In some embodiments, device 200 is a mobile device, such as a mobile phone (e.g., a smartphone), tablet computer, laptop computer, desktop computer, head-mounted display, assistive device for communicating with another device, etc. In some embodiments, such as Figure 2 As shown, device 200 includes various components such as communication circuitry 202, processor 204, memory 206, image sensor 210, position sensor 214, orientation sensor 216, microphone 218, touch-sensitive surface 220, speaker 222, display generation component 224, hand tracking sensor 230, and / or eye tracking sensor 232. These components may optionally communicate via communication bus 208 of device 200.

[0042] Device 200 includes communication circuitry 202. Communication circuitry 202 optionally includes circuitry for communicating with electronic devices, networks (such as the Internet, intranets, wired and / or wireless networks, cellular networks, and wireless local area networks (LANs)). Communication circuitry 202 optionally includes circuitry for using near-field communication (NFC) and / or short-range communication such as... The circuit used for communication.

[0043] Processor 204 includes one or more general-purpose processors, one or more graphics processors, and / or one or more digital signal processors. In some embodiments, memory 206 is a non-transitory computer-readable storage medium (e.g., flash memory, random access memory, or other volatile or non-volatile memory or storage device) storing computer-readable instructions configured to be executed by processor 204 to perform the techniques, processes, and / or methods described below. In some embodiments, memory 206 may include more than one non-transitory computer-readable storage medium. A non-transitory computer-readable storage medium can be any medium (e.g., excluding signals) that can tangibly contain or store computer-executable instructions for use by or in connection with instruction execution systems, apparatuses, and devices. In some embodiments, the storage medium is a transient computer-readable storage medium. In some embodiments, the storage medium is a non-transitory computer-readable storage medium. Non-transitory computer-readable storage media may include, but are not limited to, magnetic storage devices, optical storage devices, and / or semiconductor storage devices. Examples of such storage devices include magnetic disks, optical discs based on CD, DVD, or Blu-ray technology, and persistent solid-state memories such as flash memory, solid-state drives, etc.

[0044] Device 200 includes a display generating component 224. In some embodiments, the display generating component 224 includes a single display (e.g., a liquid crystal display (LCD), an organic light-emitting diode (OLED), or other type of display). In some embodiments, the display generating component 224 includes multiple displays. In some embodiments, the display generating component 224 may include a display with touch capability (e.g., a touchscreen), a projector, a holographic projector, a retinal projector, etc. In some embodiments, device 200 includes a touch-sensitive surface 220 for receiving user input such as tap input and swipe input or other gestures. In some embodiments, the display generating component 224 and the touch-sensitive surface 220 form a touch-sensitive display (e.g., a touchscreen integrated with device 200 or a touchscreen external to device 200 and communicating with device 200).

[0045] Device 200 optionally includes an image sensor 210. Image sensor 210 optionally includes one or more visible light image sensors (such as charge-coupled device (CCD) sensors) and / or complementary metal-oxide-semiconductor (CMOS) sensors operable to obtain images of physical objects from a real-world environment. Image sensor 210 also optionally includes one or more infrared (IR) sensors, such as passive or active IR sensors, for detecting infrared light from the real-world environment. For example, an active IR sensor includes an IR emitter for emitting infrared light into the real-world environment. Image sensor 210 also optionally includes one or more cameras configured to capture movement of physical objects in the real-world environment. Image sensor 210 also optionally includes one or more depth sensors configured to detect the distance between the physical object and device 200. In some embodiments, information from one or more depth sensors may allow the device to identify objects in the real-world environment and distinguish them from other objects in the real-world environment. In some embodiments, one or more depth sensors may allow the device to determine the texture and / or shape of objects in the real-world environment.

[0046] In some embodiments, device 200 uses a combination of a CCD sensor, an event camera, and a depth sensor to detect the physical environment surrounding device 200. In some embodiments, image sensor 220 includes a first image sensor and a second image sensor. The first and second image sensors work cooperatively and are optionally configured to capture different information about physical objects in the real-world environment. In some embodiments, the first image sensor is a visible light image sensor, and the second image sensor is a depth sensor. In some embodiments, device 200 uses image sensor 210 to detect the position and orientation of device 200 and / or display generation component 224 in the real-world environment. For example, device 200 uses image sensor 210 to track the position and orientation of display generation component 224 relative to one or more stationary objects in the real-world environment.

[0047] In some embodiments, device 200 includes microphone 218 or other audio sensors. Device 200 uses microphone 218 to detect sound from a user and / or the user's real-world environment. In some embodiments, microphone 218 includes an array of microphones (multiple microphones) that optionally operate in cooperation to identify ambient noise or locate sound sources in the space of the real-world environment.

[0048] Device 200 includes a position sensor 214 for detecting the position of device 200 and / or displaying the position of generating component 224. For example, position sensor 214 may include a GPS receiver that receives data from one or more satellites and allows device 200 to determine the absolute position of the device in the physical world.

[0049] Device 200 includes an orientation sensor 216 for detecting the orientation and / or movement of the device 200 and / or display generating component 224. For example, device 200 uses orientation sensor 216 to track changes in the position and / or orientation of the device 200 and / or display generating component 224, such as relative to a physical object in a real-world environment. Orientation sensor 216 may optionally include one or more gyroscopes and / or one or more accelerometers.

[0050] In some embodiments, device 200 includes a hand-tracking sensor 230 and / or an eye-tracking sensor 232. The hand-tracking sensor 230 is configured to track the position / location of one or more portions of a user's hand, and / or the movement of one or more portions of the user's hand relative to the extended reality environment, relative to the display generation component 224, and / or relative to another defined coordinate system. The eye-tracking sensor 232 is configured to track the position and movement of the user's gaze (more generally, the eyes, face, or head) relative to the real world or extended reality environment and / or relative to the display generation component 224. In some embodiments, the hand-tracking sensor 230 and / or the eye-tracking sensor 232 are implemented together with the display generation component 224. In some embodiments, the hand-tracking sensor 230 and / or the eye-tracking sensor 232 are implemented separately from the display generation component 224.

[0051] In some implementations, the hand-tracking sensor 230 may use an image sensor 210 (e.g., one or more IR cameras, 3D cameras, depth cameras, etc.) that captures three-dimensional information from the real world, including one or more hands (e.g., one or more hands of a human user). In some examples, the hand can be distinguished with sufficient resolution to differentiate the fingers and their corresponding positions. In some implementations, one or more image sensors 210 are positioned relative to the user to define the image sensor's field of view and interaction space, in which the finger / hand positions, orientations, and / or movements captured by the image sensors are used as input (e.g., to differentiate from the user's resting hand or other hands of other people in the real-world environment). Tracking the fingers / hands used for input (e.g., gestures) can be advantageous because it does not require the user to touch, hold, or wear any type of beacon, sensor, or other marker.

[0052] In some embodiments, the eye-tracking sensor 232 includes at least one eye-tracking camera (e.g., an infrared (IR) camera) and / or an illumination source (e.g., an IR light source, such as an LED) that emits light toward the user's eyes. The eye-tracking camera may be pointed at the user's eyes to receive reflected IR light from the light source directly or indirectly from the eyes. In some embodiments, both eyes are tracked separately by the respective eye-tracking camera and illumination source, and focus / gaze can be determined by tracking both eyes. In some embodiments, one eye (e.g., the dominant eye) is tracked by the respective eye-tracking camera / illumination source.

[0053] Equipment 200 is not limited to Figure 2 The components and configurations of the device 200 may include fewer components, other components, or additional components in various configurations. The person using device 200 is optionally referred to herein as a user of the device. Attention will now turn to examples of user interfaces (“UIs”) implemented on electronic devices such as device 100 and device 200, and associated processes. A UI may be part of a computer graphics editor, which may include a display of a computer graphics editing environment.

[0054] Figure 3A An editing environment GUI, including a representative virtual object 330, is shown as some examples according to this disclosure. The editing environment GUI may be displayed on an electronic device (e.g., similar to device 100 or 200), including but not limited to portable or non-portable computing devices such as tablet computing devices, laptop computing devices, or desktop computing devices. Figure 3A A 3D environment defined by the X, Y, and Z axes and including virtual objects 330 is shown in a first operating mode (e.g., scene editing mode). Figure 3A In the example, virtual object 330 is a cube, but it should be understood that the cube is merely representative and one or more different virtual objects (e.g., one-dimensional (1D), 2D or 3D objects) can be imported or selected from the content library (which includes many shapes, objects, symbols, text, numbers, etc.) and included in the 3D environment.

[0055] Additionally, it should be understood that the 3D environment (or 3D virtual object) described herein can be a representation of a 3D environment (or 3D virtual object) displayed in a two-dimensional (2D) context (e.g., displayed on a 2D screen). In some examples, the 3D environment may display grid lines or other indicators to help content creators place and / or determine the size of virtual objects within the 3D environment. Figure 3AIn the example, a position indicator 332 is shown above the virtual object 330. The position indicator 332 is purely symbolic and may represent the displayed cursor or other current position indicator, or simply indicate the presence of a finger or stylus touch, or hovering over the virtual object 330 without displaying any visible indicator. Selection of the virtual object 330 represented by the position indicator 332, such as a finger / stylus touch or a tap above the virtual object, or a mouse click, selects the virtual object for editing and brings up the object manipulator.

[0056] Figure 3B A selected virtual object 330 and its associated object manipulator are shown according to an example of this disclosure. When virtual object 330 is selected as described above, the object manipulator can appear above the virtual object to enable transformation of the virtual object. Figure 3B In the examples, the object manipulator is shown in its default state and may include tapered single-axis translation power representations 334-X, 334-Y, and 334-Z oriented along the X', Y', and Z' object axes in the X, Y, and Z directions (i.e., parallel to the X, Y, and Z axes), respectively, where the object axes have an origin at the object center indicator 336. In some examples, the single-axis translation power representations may be displayed using a unique color associated with each of the X, Y, and Z axes. The object manipulator may also include disk-scaled power representations 338-X, 338-Y, and 338-Z, located "behind" the single-axis translation power representations 334-X, 334-Y, and 334-Z (i.e., between the single-axis translation power representations and the object center indicator 336), and aligned with the single-axis translation power representations on the same object axis. The object manipulator may also include arc-shaped rotational energy representations 340-X, 340-Y, and 340-Z for rotation about the object axes X', Y', and Z', respectively. Each arc-shaped rotational energy representation may be displayed in a different plane defined by the multiple object axes. In some examples, the arc-shaped rotational energy representations may be displayed using a unique color associated with each of the X, Y, and Z axes. Figure 3B In the example, the rotational energy representation 340-X lies in the X=0 plane, the rotational energy representation 340-Y lies in the Y=0 plane, and the rotational energy representation 340-Z lies in the non-zero Z-plane (i.e., all planes intersecting with the object center indicator 336). To define the view of the virtual object, Figure 3B The 3D environment (and other 3D environments mentioned throughout this disclosure) can be divided into eight regions or eight sections in a 3D space defined by the object axis, with the relative origin at the object center indicator 336. In some examples of this disclosure, the power representation of the object manipulator can appear within the eight viewing sections, such as... Figure 3BAs shown, this ensures that the energy representation is always displayed in front of the virtual object for unobstructed viewing and easy access.

[0057] In some examples, the object manipulator retains its default size even when the 3D environment and any virtual objects within it are scaled up or down. Maintaining the object manipulator at its default size keeps it easy to use, even when virtual objects are very small. However, in other examples, the object manipulator can grow or shrink as the 3D environment is scaled up or down. In some examples, the appearance (e.g., color, thickness, shadow, shape, position) of one or more of the following—object center indicator 336, single-axis translation power indicators 334-X, 334-Y, and 334-Z, disk scaling power indicators 338-X, 338-Y, and 338-Z, and arc rotation power indicators 340-X, 340-Y, and 340-Z—can be changed in the object manipulator properties pane, which may appear as an overlay within the 3D environment or may be displayed in a window outside the 3D environment.

[0058] Figure 3C Examples of the present disclosure are shown. Figure 3B Compared to the selected virtual object 330 with a reoriented object manipulator viewed from a different perspective. Figure 3B This represents the viewing perspective of a specific eight sections within 3D space. In other words, if a user is viewing a virtual object 330 from a specific eight sections within 3D space defined by the object axis, then... Figure 3B Display the content that the user will see. Figure 3C In the example, with Figure 3B In contrast, the perspective has changed to a different eight-part division (as evidenced by the changed positions of the X and Y axes). Figure 3C In the middle, the single-axis translation energy representation 334-X and the scaling energy representation 338-X have reversed their orientation in the X direction, and are consistent with... Figure 3B In comparison, the rotational energy representation of 340-Y and 340-Z has also been repositioned. In some examples, the object manipulator's reorientation can change automatically when the user's viewpoint switches to different octaves in 3D space. In some examples, once the viewpoint switches to a different octave, the object manipulator can quickly move to a new, discrete orientation (e.g., from...). Figure 3B orientation to Figure 3C In some examples, the object manipulator can move dynamically and gradually change its orientation. For example, when the 3D environment rotates clockwise around the Z-axis (i.e., in...), the orientation changes. Figure 3B and Figure 3C Looking down from the center, the rotating indicator shows a 340-Z rotation. Figure 3BThe orientation gradually and continuously rotated to Figure 3C Orientation. In some examples, the selection of the redirection type for an object manipulator (e.g., quick or continuous) can be set in the object manipulator properties pane, which can appear as an overlay in the 3D environment or can be displayed in a window outside the 3D environment. This redirection provides the advantage of always displaying the object manipulator in the foreground of the manipulated virtual object, ensuring that all parts of the object manipulator remain visible and accessible as the viewpoint of the 3D environment changes. This improved accessibility of the object manipulator provides easier and more precise object manipulation.

[0059] Figure 4A The single-axis translation performance representation 434-X of a virtual object 430 according to an example of this disclosure is shown. When displaying an object manipulator (such as...), Figure 3B As shown in the example), (e.g., by moving the cursor over the indicator and clicking and holding the mouse button, by continuous touch on the translation indicator, etc.) selecting the translation indicator 434-X indicated by the position indicator 432 can cause the selected translation indicator to remain displayed when some or all other parts of the object manipulator disappear, such as... Figure 4A As shown in the example. Although Figure 4A The choice of X-direction translational energy representation 434-X is shown for illustrative purposes only, but it should be understood that in other alternative examples, the Y-direction translational energy representation or the Z-direction translational energy representation may be alternatively chosen (see, for example...). Figure 3B The translation energy in the figure represents 334-Y or 334-Z).

[0060] Figure 4B The single-axis translation of a virtual object 430 of 434-X is shown in an example according to this disclosure, using single-axis translation energy to represent the single-axis translation of the object. Figure 4B In the example (the example is) Figure 4A(Continuing from the example), the selected translation power representation 434-X is dragged by a specific amount in the +X direction along its associated object axis from position A (e.g., by clicking and holding the mouse button and moving the mouse, by sliding a touch finger, etc.) to position B, as indicated by arrow 442. While dragging the power representation 434-X, the virtual object 430 and the object center indicator 436 can be translated in the same direction as the power representation. In some examples, the object center indicator 436 can be displayed in a different manner than the original center 448 of the object position (e.g., a different color, shape, shadow, opacity, etc.). During translation, a line 446 extending from the original center 448 of the object position to the power representation 434-X can be displayed. In some examples, the line 446 can be displayed using a visual gradient, such as a darker shadow near the power representation 434-X and a lighter shadow near the original center 448 of the object position, or vice versa. In some examples, the appearance (e.g., color, thickness, shadow, shape, position) of one or more of the object center indicator 436, the original center 448 of the object position, and line 446 can be changed in the object manipulator properties pane, which can appear as an overlay in the 3D environment or can be displayed in a window outside the 3D environment. Although Figure 4B The translation of the virtual object 430 in the X direction is shown for illustrative purposes only, but it should be understood that in other alternative examples, the energy indication of translation in the Y direction or the energy indication of translation in the Z direction (see, for example) Figure 3B The translation energy representation (334-Y or 334-Z) can be optionally selected and used to translate virtual objects in the Y or Z direction, respectively.

[0061] When the power indicator 434-X is dragged by a specific amount in the +X direction, the virtual object 430 can be translated along with the power indicator in the +X direction by the same amount (i.e., linearly) from its original position (indicated by the dashed line) to a new position (indicated by the solid line). In other examples, the translation of the power indicator 434-X and the virtual object 430 can occur proportionally. In one specific example, the translation of the power indicator 434-X and the virtual object 430 can occur at a ratio of 1:2, such that the virtual object will be translated by twice the amount of the power indicator. This type of translation ratio can be advantageous when moving the virtual object a large distance. In another specific example, the translation of the power indicator 434-X and the virtual object 430 can occur at a ratio of 5:1, such that the virtual object will be translated by only one-fifth of the amount of the power indicator. This can be advantageous when fine distance control is required to move the virtual object a short distance. It should be noted that the above ratios are for illustrative purposes only, and other ratios are envisioned. In some examples, the translation scale represented by the single-axis translation power can be changed in the object manipulator properties pane, which can appear as an overlay in the 3D environment or can be displayed in a window outside the 3D environment.

[0062] In some examples, the ball 444 may appear at an intermediate position between the energy representation 434-X and the original center 448 of the object position (or at some other scale). The ball 444 can indicate the amount by which the virtual object 430 has moved and / or the amount by which the energy representation 434-X has moved, which is advantageous when a specific amount of movement of the virtual object and / or the energy representation is required. In some examples, the line 446 and the ball 444 may have different appearances (e.g., solid lines, dashed lines, dotted lines, outlines, wireframes, or different shadings) depending on whether they are in front of, behind, or inside the previous or current volume of the virtual object 430. Although Figure 4A and Figure 4B An exemplary translation in the +X direction is shown, but in other examples, translation can be performed in the -X direction. In some examples, the position and appearance of the ball 444 (including the information it provides) and the line 446 can be changed in the object manipulator properties pane, which may appear as an overlay in the 3D environment or may be displayed in a window outside the 3D environment.

[0063] Figure 5A The dual-axis translation capability of a virtual object 530 according to an example of this disclosure is shown, representing the appearance and selection of 550-XY. When displaying object manipulators (such as...), Figure 3B (As shown in the example), hovering over the Figure 5AA sliced ​​dual-axis translation power representation 550-XY can appear above an area in the plane defined by the rotation power representation 540-Z and within the arc of that rotation power representation indicated by the position indicator 532 (e.g., by moving a finger or cursor over that area). Selecting the dual-axis translation power representation 550-XY (e.g., by clicking and holding a mouse button, a sustained touch over that area, etc.) can make some or all other parts of the object manipulator disappear. In some examples, the sliced ​​dual-axis translation power representation 550-XY can be shaded or otherwise appear different from the surrounding area in the 3D environment. In one example, each sliced ​​dual-axis translation power representation can be displayed with the same color as the adjacent rotation power representation on the same plane. In some examples, the appearance of the dual-axis translation power representation 550-XY (e.g., color, thickness, shadow, shape, position) can be changed in the object manipulator properties pane, which can appear as an overlay in the 3D environment or can be displayed in a window outside the 3D environment. Although Figure 5A The choice of a dual-axis translational energy representation 550-XY is shown for illustrative purposes only, but it should be understood that in other alternative examples, a dual-axis YZ direction translational energy representation or a dual-axis XZ direction translational energy representation may be alternatively selected.

[0064] Figure 5B The example shown illustrates the dual-axis translation of a virtual object 530 representing 550-XY using dual-axis translation energy according to this disclosure. Figure 5B In the example (the example is) Figure 5A (a continuation of the example above), as mentioned above. Figure 5AAs described above, a dual-axis translation power representation 550-XY has been selected, and thus appears at position A, while all other components of the object manipulator have disappeared. The dual-axis translation power representation 550-XY is dragged in the XY direction (two-dimensional translation) from position A to position B (e.g., by clicking and holding a mouse button and moving the mouse, by sliding a touch finger, etc.) by a specific amount, as shown by arrow 542. When the power representation 550-XY is dragged, the virtual object 530 and the object center indicator 536 can be translated in the same direction along with the power representation. In some examples, the object center indicator 536 can be displayed in a different way than the original center 548 of the object position (e.g., different color, shape, shadow, opacity, etc.). During translation, a line 546 extending from the original center 548 of the object position to the current object center indicator 536 can be displayed. In some examples, line 546 can be displayed using visual gradients, such as a darker shadow near the object center indicator and a lighter shadow near the original center 548 of the object's location, or vice versa. In some examples, the appearance of the object center indicator 536, the original center 548 of the object's location, and line 546 can be changed in the object manipulator properties pane, which can appear as an overlay in the 3D environment or can be displayed in a window outside the 3D environment. Although Figure 5B The translation of the virtual object 530 in the +X+Y direction is shown for illustrative purposes only, but it should be understood that in other alternative examples, a translation energy representation in the YZ direction or a translation energy representation in the XZ direction may be alternatively selected and used to translate the virtual object in the YZ direction or the XZ direction (negative or positive), respectively.

[0065] exist Figure 5BIn the example, when the power representation 550-XY is dragged by a specific amount in the +X+Y direction, the virtual object 530 can be translated together with the power representation in the +X+Y direction from its original position (indicated by the dashed line) to a new position (indicated by the solid line) by the same amount (i.e., in a linear relationship) as its original position (indicated by the dashed line). In other examples, the translation of the power representation 550-XY and the virtual object 530 can occur proportionally (i.e., with different translation amounts). In one specific example, the translation of the power representation 550-XY and the virtual object 530 can occur at a 1:2 ratio, such that the virtual object will be translated twice by the amount of the power representation in each of the X and Y directions. This type of translation ratio can be advantageous when moving the virtual object a large distance. In another specific example, the translation of the power representation 550-XY and the virtual object 530 can occur at a 5:1 ratio, such that the virtual object will be translated only one-fifth of the amount of the power representation in both the X and Y directions. This can be advantageous when fine-grained distance control is required to move virtual objects short distances. Note that the scales described above are for illustrative purposes only, and other scales are envisioned. In some examples, the translation scale represented by the dual-axis translation power can be changed in the object manipulator properties pane, which may appear as an overlay within the 3D environment or may be displayed in a window outside the 3D environment.

[0066] In some examples, the ball 544 may appear at an intermediate position between the object center indicator and the original center 536 of the object position (or at some other scale). The ball 544 may indicate the amount that the virtual object 530 has moved (e.g., ΔX, ΔY) and / or the amount that the energy representation 550-XY has moved, which is advantageous when a specific amount of movement of the virtual object and / or the energy representation is required. In some examples, the line 546 and the ball 544 may have different appearances (e.g., solid line, dashed line, dotted line, outline, wireframe, or different shading) depending on whether they are in front of, behind, or inside the previous or current volume of the virtual object 530. Although Figure 5A and Figure 5BAn exemplary dual-axis translation energy representation 550-XY and object translation in the +X+Y direction are shown, but in other examples, translation can be performed in other XY directions (e.g., an XY translation energy representation in another quadrant can be selected, displayed, and used to perform translation in other XY directions (e.g., in the -X+Y, +XY, or -XY directions)). Furthermore, in other examples, different dual-axis translation energy representations can be selected, displayed, and used to perform other dual-axis translations (e.g., in the XZ or YZ direction). In some examples, the position and appearance of the ball 544 (including the information it provides) and the line 546 can be changed in the object manipulator properties pane, which can appear as an overlay in the 3D environment or can be displayed in a window outside the 3D environment.

[0067] Figure 6A The selection of a single-axis scaling power display 638-X for a virtual object 630 is shown according to an example of this disclosure. This is when displaying an object manipulator (such as...). Figure 3B As shown in the example), (e.g., by moving the cursor over the scaling indicator and clicking and holding the mouse button, by continuous touch on the scaling indicator, etc.) selecting the scaling indicator line 638-X indicated by the position indicator 632 can cause the selected scaling indicator to remain displayed even when some or all other parts of the object manipulator disappear, such as... Figure 6A As shown in the example. Although Figure 6A The choice of X-direction scaled energy representation 638-X is shown for illustrative purposes only, but it should be understood that in other alternative examples, either the Y-direction scaled energy representation or the Z-direction scaled energy representation can be optionally selected (see, for example...). Figure 3B The translation energy in the figure represents 338-Y or 338-Z).

[0068] In some examples, the single-axis scaling energy representation 638-X can scale uniformly, while in other examples, the scaling energy representation can scale non-uniformly. For example, to select non-uniform scaling, another input can be generated when a specific scaling energy representation is selected (e.g., pressing an option key), while if no other input is generated when a specific scaling energy representation is selected, uniform scaling is selected. In some examples, the scaling energy representations can present different appearances depending on whether they are configured for uniform or non-uniform scaling. For example, all scaling energy representations can be the same color (e.g., gray) when they are configured for uniform scaling, while each scaling energy representation can have a color assigned to a specific dimension when they are configured for non-uniform scaling.

[0069] Figure 6BThe uniform scaling of a virtual object 630 representing 638-X using scaling energy representation, according to an example of this disclosure, is shown. Figure 6B Example (the example is) Figure 6A In the example continuation), the scaling power representation 638-X has been selected for uniform scaling (e.g., selected without receiving further input invoking non-uniform scaling), and (e.g., by clicking and holding the mouse button and moving the mouse, by sliding a touch finger, etc.) is dragged a certain amount along its associated object axis in the increasing X direction from position A to position B, as indicated by arrow 642. While dragging the power representation 638-X along its associated object axis in the increasing X direction, the virtual object 630 can be scaled outwards (i.e., the virtual object 630 expands outwards from the object center indicator 636) and uniformly in all directions, from its original volume (indicated by the dashed line) to a new volume (indicated by the solid line), as shown... Figure 6B As shown. However, it should be understood that the power indicator 638-X can also be dragged along its object axis in a decreasing X direction (i.e., from point A toward the center 636 of the object location) to scale the virtual object 630 inward and uniformly in all directions, making the virtual object smaller than its original size (i.e., the virtual object shrinks inward toward the object center indicator). During scaling, a line 646 extending from the object center indicator 636 to the power indicator 638-X can be displayed. In some examples, the line 646 can be displayed using visual gradients, such as a darker shadow near the power indicator 638-X and a lighter shadow near the object center indicator 636, or vice versa. In some examples, the appearance (e.g., color, thickness, shadow, shape, position) of one or more of the object center indicator 636 and the line 646 can be changed in the object manipulator properties pane, which may appear as an overlay in the 3D environment or may be displayed in a window outside the 3D environment.

[0070] When the power representation 638-X is dragged by a specific amount along its associated object axis in the X direction, the virtual object 630 can scale uniformly by the same amount (i.e., linearly) in the X, Y, and Z directions. In other examples, the dragging of the power representation 638-X and the uniform scaling of the virtual object 630 can occur proportionally. In a specific example for illustrative purposes only, dragging the power representation 638-X along its associated object axis in an increasing X direction and uniformly scaling the virtual object 630 can occur at a 1:2 ratio, such that the virtual object will scale uniformly and outward in all directions to twice the extent that the power representation is dragged along its associated object axis in the increasing X direction. This can be advantageous when scaling the virtual object uniformly in large quantities. In another example, for illustrative purposes only, dragging the power representation 638-X along its associated object axis in the increasing X direction and uniformly scaling the virtual object 630 can occur at a 5:1 ratio, such that the virtual object will scale outwards uniformly in the X, Y, and Z directions to only one-fifth the extent of dragging the power representation along its associated object axis in the increasing X direction. This can be advantageous when fine scaling control is required to scale the virtual object uniformly in small increments. In some examples, the scaling ratio and the appearance of the single-axis scaling power representation can be changed in the object manipulator properties pane, which may appear as an overlay in the 3D environment or may be displayed in a window outside the 3D environment.

[0071] In some examples, the ball 644 may appear at an intermediate position between the energy representation 638-X and the object center indicator 636 (or at some other scale). The ball 644 may indicate the amount by which the virtual object 630 has been uniformly scaled (e.g., ΔX, ΔY, ΔZ, percentage size increase / decrease, percentage volume increase / decrease, etc.) and / or the amount by which the energy representation 638-X has been dragged (e.g., ΔX, percentage size increase / decrease, etc.), which may be advantageous when uniform scaling of a specific amount or percentage of the virtual object is required. In some examples, the line 646 and the ball 644 may have different appearances (e.g., solid lines, dashed lines, dotted lines, outlines, wireframes, or different shadings) depending on whether they are outside or inside the previous or current volume of the virtual object 630. In some examples, the position and appearance of the ball 644 (including the information it provides) and the appearance of the line 646 can be changed in the object manipulator properties pane, which may appear as an overlay in the 3D environment or may be displayed in a window outside the 3D environment. Although Figure 6A For illustrative purposes only, the use of scaling energy representation 638-X to uniformly scale virtual object 630 is shown; however, it should be understood that in other alternative examples, scaling energy representation in the Y direction or scaling energy representation in the Z direction (see, for example) may be used. Figure 3BThe scaling capability (338-Y or 338-Z) can be optionally selected and used to uniformly scale virtual objects as described above.

[0072] Figure 6C The non-uniform scaling of a virtual object 630 representing 638-X using scaling energy representation, as shown in the example of this disclosure, is illustrated. Figure 6C In the example (the example is) Figure 6A (Continuing from the example), the scaling power representation 638-X has been selected for non-uniform scaling (e.g., selected upon receiving further input invoking non-uniform scaling), and (e.g., by clicking and holding the mouse button and moving the mouse, by sliding a touch finger, etc.) is dragged a certain amount along its associated object axis in the increasing +X direction from position A to position B, as shown by arrow 642. When the power representation 638-X is dragged along its associated object axis in the increasing +X direction, the virtual object 630 can be non-uniformly scaled only in the increasing +X direction, as... Figure 6C As shown. In other words, the size of the virtual object 630 remains constant in the Y and Z directions and the -X direction. However, in other examples, the virtual object 630 may scale uniformly in the increasing and decreasing X direction (the decreasing X direction indicated by arrow 664), but non-uniformly relative to the Y and Z directions (i.e., no scaling occurs in the Y and Z directions). However, it should be understood that the energy representation 638-X can also be dragged in the decreasing X direction (i.e., from point A towards the original center 648 of the object location) to non-uniformly scale the virtual object 630 in the X direction, such that the virtual object becomes smaller than its original size in the X dimension (i.e., the virtual object shrinks inward toward the object center indicator in the X dimension). During scaling, a line 646 extending from the original center 648 of the object location to the energy representation 638-X may be displayed. In some examples, the line 646 may be displayed using visual gradients, such as a darker shadow near the energy representation 638-X and a lighter shadow near the original center 648 of the object location, or vice versa. In some examples, the appearance (e.g., color, thickness, shadow, shape, position) of one or more of the original center 648 and line 646 of the object's location can be changed in the object manipulator properties pane, which may appear as an overlay in the 3D environment or may be displayed in a window outside the 3D environment.

[0073] When the power representation 638-X is dragged by a specific amount in the X direction, the virtual object 630 can scale non-uniformly only in the X direction by the same amount (i.e., in a linear relationship). In other examples, the dragging of the power representation 638-X and the non-uniform scaling of the virtual object 630 can occur proportionally (but not linearly). In one particular example, for illustrative purposes only, dragging the power representation 638-X in the increasing X direction and non-uniformly scaling the virtual object 630 can occur at a 1:2 ratio, such that the virtual object will scale non-uniformly only in the increasing X dimension to twice the extent that the power representation is dragged in the increasing X dimension. This can be advantageous when scaling the virtual object non-uniformly in large quantities. In another example, for illustrative purposes only, dragging the power representation 638-X in the increasing X direction and non-uniformly scaling the virtual object 630 can occur at a 5:1 ratio, such that the virtual object will scale non-uniformly only in the increasing X dimension to only one-fifth the extent that the power representation is dragged in the increasing X direction. This can be advantageous when fine scaling control is required to scale virtual objects with a small amount of non-uniformity.

[0074] In some examples, the ball 644 may appear at an intermediate position (or at some other scale) between the energy representation 638-X and the original center 648 of the object position. The ball 644 may indicate the amount by which the virtual object 630 has been non-uniformly scaled (e.g., ΔX, percentage size increase, percentage volume increase, etc.) and / or the amount by which the energy representation 638-X has been dragged (e.g., ΔX, percentage size increase, etc.), which can be advantageous when a specific amount or percentage of non-uniform scaling of the virtual object is required. In some examples, the line 646 and the ball 644 may have different appearances depending on whether they are outside or inside the previous or current volume of the virtual object 630. In some examples, the position and appearance of the ball 644 (including the information it provides) and the appearance of the line 646 can be changed in the object manipulator properties pane, which may appear as an overlay in the 3D environment or may be displayed in a window outside the 3D environment. Although Figure 6C For illustrative purposes only, a non-uniform scaling of the virtual object 630 using a scaling energy representation 638-X is shown; however, it should be understood that in other alternative examples, the scaling energy representation in the Y direction or the scaling energy representation in the Z direction (see, for example) may be used. Figure 3B The scaling capability (338-Y or 338-Z) can be optionally selected and used to non-uniformly scale virtual objects as described above.

[0075] Figure 7A The illustration shows the highlighting and selection of the rotational power representation 740-Z of a virtual object 730 according to an example of this disclosure. When displaying the object manipulator (e.g.) Figure 3BAs shown in the example, such as that indicated by position indicator 732. Figure 7A The highlighting of the 740-Z rotation indicator (e.g., by hovering over the area, by moving the cursor over the area, etc.) can thicken and / or brighten the 740-Z rotation indicator to make selection and subsequent manipulation easier. The 740-Z rotation indicator can then be selected (e.g., by clicking a mouse button, tapping, releasing a sustained touch, applying additional force to a sustained touch, etc.). Although Figure 7A The example only shows the selected rotary energy indicator 740-Z, but in other examples, either of the other two rotary energy indicators can also be highlighted, visually thickened, or selected.

[0076] Figure 7B The selection of a rotational energy representation 740-Z for a virtual object 730 according to an example of this disclosure is shown. Figure 7B In the example (the example is) Figure 7A (Continuing from the example), when the rotational energy representation 740-Z is selected, some or all other parts of the object manipulator can disappear, and Figure 7A The arc-shaped rotational energy representation can be transformed into a full-toroidal rotational energy representation. In some examples, the full-toroidal rotational energy representation 740-Z can be displayed using a color associated with the Z-axis. In some examples, the full-toroidal rotational energy representation 740-Z can maintain its default size even when the 3D environment and any virtual objects in the environment are zoomed in or out. Keeping the full-toroidal rotational energy representation 740-Z at its default size allows the rotational energy representation to remain easy to use, even when the virtual objects are very small. However, in other examples, the full-toroidal rotational energy representation 740-Z can grow or shrink when the 3D environment is zoomed in or out. (For example, by dragging) Rotating the selected full-toroidal rotational energy representation 740-Z can cause the virtual object 730 to rotate around the virtual object axis Z', which is parallel to the Z-axis of the environment's 3D coordinate system (and in...). Figure 7B (The examples overlap with it). Although Figure 7B The example only shows the full-ring rotation energy representation 740-Z that appears after selection, but in other examples, different full-ring rotation energy representations, such as a full-ring X-axis rotation energy representation or a full-ring Y-axis rotation energy representation, can be displayed after selection, and the different full-ring rotation energy representation can be dragged to rotate the virtual object around different virtual object axes (e.g., X' or Y').

[0077] Figure 7C The example shown illustrates the rotation of a virtual object 730 representing 740-Z using rotational energy representation according to this disclosure. Figure 7C In the example (this example is) Figure 7B(Continuing from the previous example), using the position indicator 732, dragging from point A to point B along the selected rotational power representation 740-Z (as shown by arrow 742) causes the virtual object 730 to rotate about its virtual object axis, as shown by arrow 752. In some examples, as the rotational power representation 740-Z is rotated, a sliced ​​area 754 may appear, which may be defined by a line 756 and be shaded or otherwise appear different from the surrounding area in the 3D environment to provide a visual indication of the amount of rotation. In some examples, a portion 762 of the rotational power representation 740-Z may be shaded, darkened, or otherwise appear different from the rest of the power representation to provide a visual indication of the amount of rotation. In some examples, a ball 758 may appear at the center 730 of the virtual object and indicate the amount of rotation (e.g., degrees). In some examples, an arc 760 may appear outside the rotational power representation 740-Z and may include points, hashes, or tic markers and optionally include the amount of rotation. The ball 758 and / or the arc 760 can advantageously provide a precise indication of the amount of rotation, which is useful when a specific amount of rotation is desired.

[0078] Figure 8A The selection of the object center indicator (power indicator) 836 for a virtual object 830 is shown according to an example of this disclosure. This is when the object manipulator is displayed (e.g., ...). Figure 3B As shown in the example), (e.g., by moving the cursor over the indicator and clicking and holding the mouse button, by continuously touching the indicator while panning, etc.) selecting the object center indicator 836 indicated by the position indicator 832 can cause the selected object center indicator to remain displayed even when some or all other parts of the object manipulator disappear, such as... Figure 8A As shown in the example.

[0079] Figure 8B This illustrates an example of omnidirectional translation (i.e., screen-space movement) of a virtual object 830 using an object-centered display representation 836 according to this disclosure. Figure 8B In the example (the example is) Figure 8A(Continuing from the example), the selected object center energy representation 836 is dragged from position A to position B by a certain ΔXΔYΔZ amount, as indicated by arrow 842. When the object center energy representation 836 is dragged, the virtual object 830 can be translated along with the energy representation from its original position (indicated by the dashed line) by the same ΔXΔYΔZ amount (i.e., linearly) to the new position (indicated by the solid line). In some examples, the object center energy representation 836 can be displayed in a different way than the original center 848 of the object position (e.g., different color, shape, shadow, opacity, etc.). During translation, a line 846 extending from the original center 848 of the object position to the object center energy representation 836 can be displayed. In some examples, the line 846 can be displayed using visual gradients, such as a darker shadow near the object center energy representation 836 and a lighter shadow near the original center 848 of the object position, or vice versa. In some examples, the appearance of the object center power representation 836, line 846, and the original center 848 of the object position can be changed in the object manipulator properties pane, which can appear as an overlay in the 3D environment or can be displayed in a window outside the 3D environment.

[0080] In some examples, the ball 844 may appear at an intermediate position between the object's center power representation 836 and the object's original center 848 (or at some other scale). The ball 844 can indicate the amount of movement that the virtual object 830 has made, which can be advantageous when a specific amount of movement of the virtual object is required. In some examples, the line 846 and the ball 844 may have different appearances (e.g., solid lines, dashed lines, dotted lines, outlines, wireframes, or different shades) depending on whether they are in front of, behind, or inside the previous or current volume of the virtual object 830. In some examples, the position and appearance of the ball 844, as well as the information it displays, can be changed in the object manipulator properties pane, which may appear as an overlay in the 3D environment or may be displayed in a window outside the 3D environment.

[0081] Figure 9 A flowchart illustrating process 966 for manipulating virtual objects is shown according to an example of this disclosure. Process 966 begins at 968 with the selection of a virtual object in the 3D environment, which causes the object manipulator to be displayed. In some examples, the view of the virtual object can be changed at 970, which can cause the object manipulator to be redirected. A specific object manipulator enablement representation can be selected at 972, which can cause some or all other object manipulator enablement representations to disappear.

[0082] If a single-axis translation indicator is selected at 974, then at 976, the virtual object can be translated in the direction associated with the selected single-axis translation indicator. If a dual-axis translation indicator is selected at 978, then at 980, the virtual object can be translated in the direction indicated by the drag indicator. If a single-axis scaling indicator is selected at 982, and uniform scaling is selected at 984, then at 986, the virtual object can be scaled uniformly in all dimensions. If non-uniform scaling is selected at 988, then at 990, the virtual object can be scaled non-uniformly in the direction associated with the selected scaling indicator. If a rotation indicator is selected at 992, then at 994, the virtual object can be rotated around the axis associated with the selected rotation indicator. If a screen-space movement indicator is selected at 996, then at 998, the virtual object can be moved in the direction indicated by the drag indicator.

[0083] It should be understood that process 966 is an example, and more, fewer, or different operations can be performed in the same or different order. Furthermore, the operations in process 966 described above can optionally be performed by running an information processing device such as a general-purpose processor (e.g., as per [reference to...]). Figure 2 One or more functional modules in the aforementioned (or dedicated chip) and / or by Figure 2 Other components are used to implement it.

[0084] Therefore, based on the foregoing, some examples of this disclosure relate to a method comprising, at an electronic device communicating with a display and one or more input devices, presenting a graphical environment including a virtual object having multiple object axes using a display; upon presenting the virtual object, receiving input indicating selection of the virtual object; after receiving input indicating selection of the virtual object, presenting an object manipulator together with the virtual object, the object manipulator having multiple power representations including multiple rotational power representations for rotating the virtual object, each rotational power representation for rotating the virtual object about a different object axis; upon presenting the object manipulator, receiving input indicating selection of a particular rotational power representation; after receiving input indicating selection of a particular rotational power representation, enlarging the selected particular rotational power representation into a ring and stopping the display of other rotational power representations; upon presenting the selected ring rotational power representation, receiving input indicating rotation of the selected ring rotational power representation; and after receiving input indicating rotation of the selected ring rotational power representation, rotating the selected virtual object about an object axis associated with the selected ring rotational power representation. As a supplement or alternative to one or more of the examples above, in some examples, the method further includes presenting the graphical environment from a viewpoint of a specific eight-partition in 3D space, and repositioning one or more power representations of an object manipulator when the viewpoint changes, such that the power representation displayed by the object manipulator is within the eight-partition of the current viewpoint. As a supplement or alternative to one or more of the examples above, in some examples, the method further includes presenting each rotational power representation as an arc in a different plane defined by two of a plurality of object axes. As a supplement or alternative to one or more of the examples above, in some examples, the method further includes hovering over an area in a specific plane defined by a specific rotational power representation and within the arc of the specific rotational power representation; after hovering over the area, presenting a dual-axis translation power representation within the area in the specific plane; receiving input indicating selection and movement of the dual-axis translation power representation; and, upon receiving input indicating movement of the dual-axis translation power representation, translating the selected virtual object in a two-dimensional translation along the specific plane. As a supplement or alternative to one or more of the examples above, in some examples, the amount of two-dimensional translation of the selected virtual object is the same as the amount of movement in the dual-axis translation energy representation. As a supplement or alternative to one or more of the examples above, in some examples, the amount of two-dimensional translation of the selected virtual object is different from the amount of movement in the dual-axis translation energy representation. As a supplement or alternative to one or more of the examples above, in some examples, the object manipulator includes presenting multiple scaling energy representations different from multiple rotational energy representations, each scaling energy representation used to scale the virtual object.As a supplement or alternative to one or more of the examples above, in some examples, each scaling power representation is associated with a different object axis, and the method further includes, when presenting multiple scaling power representations, receiving input indicating a selection of a particular scaling power representation; after receiving input indicating a selection of a particular scaling power representation, stopping the display of other scaling power representations; when presenting the selected scaling power representation, receiving input indicating a translation of the selected scaling power representation along the object axis associated with the selected scaling power representation; and after receiving input indicating a translation of the selected scaling power representation along the object axis associated with the selected scaling power representation, scaling the selected virtual object. As a supplement or alternative to one or more of the examples above, in some examples, the method further includes scaling the selected virtual object uniformly in all directions associated with each object axis. As a supplement or alternative to one or more of the examples above, in some examples, the method further includes receiving a modifier input upon receiving an input indicating selection of a particular scaling power representation, and, after receiving the modifier input and an input indicating translation of the selected scaling power representation along an object axis associated with the selected scaling power representation, non-uniformly scaling the selected virtual object in a first direction associated with the object axis of the selected scaling power representation, while maintaining the size of the selected virtual object in other directions associated with the object axis of unselected scaling power representations. As a supplement or alternative to one or more of the examples above, in some examples, the method further includes scaling the selected virtual object in a second direction opposite to the first direction associated with the object axis of the selected scaling power representation. As a supplement or alternative to one or more of the examples above, in some examples, the object rendering manipulator includes rendering a plurality of single-axis translation power representations, each single-axis translation power representation used to translate a virtual object. As a supplement or alternative to one or more of the examples above, in some examples, each single-axis translation energy representation is associated with a different object axis, and the method further includes, when presenting multiple single-axis translation energy representations, receiving input indicating a selection of a particular single-axis translation energy representation; after receiving input indicating a selection of a particular single-axis translation energy representation, stopping the display of other single-axis translation energy representations; when presenting the selected single-axis translation energy representation, receiving input indicating a first one-dimensional translation of the selected single-axis translation energy representation along the object axis associated with the selected single-axis translation energy representation; and after receiving input indicating a translation of the selected single-axis translation energy representation along the object axis associated with the selected single-axis translation energy representation, translating the selected virtual object along the object axis associated with the selected single-axis translation energy representation in a second one-dimensional translation. As a supplement or alternative to one or more of the examples above, in some examples, the amount of the second one-dimensional translation is the same as the amount of the first one-dimensional translation.As a supplement or alternative to one or more of the examples above, in some examples, the amount of the second one-dimensional translation differs from the amount of the first one-dimensional translation. As a supplement or alternative to one or more of the examples above, in some examples, the object manipulator includes presenting an object-centered energy representation for omnidirectional translation of a virtual object. As a supplement or alternative to one or more of the examples above, in some examples, the method further includes, when presenting the object-centered energy representation, receiving input indicating a selection of the object-centered energy representation; after receiving input indicating a selection of the object-centered energy representation, receiving input indicating translation of the selected object-centered energy representation in one or more directions; and after receiving input indicating translation of the selected object-centered energy representation in one or more directions, translating the selected virtual object in one or more directions. As a supplement or alternative to one or more of the examples above, in some examples, the method further includes, while presenting the object manipulator but before receiving input indicating a selection of a particular rotational display, receiving input indicating highlighting of a particular rotational display, and after receiving input indicating highlighting of a particular rotational display, causing the particular rotational display to modify its appearance by one or more of thickening and highlighting. Additionally or alternatively, in some examples, a non-transitory computer-readable storage medium stores instructions that, when executed by one or more processors, cause one or more processors to perform the method according to one or more of the examples above. Additionally or alternatively, in some examples, the electronic device includes one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors, the one or more programs including instructions for performing the method according to one or more of the examples above.

[0085] For purposes of explanation, the foregoing description has been given by reference to specific embodiments. However, the illustrative discussion above is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible based on the teachings above. The embodiments were chosen and described to best elucidate the principles of the invention and its practical application, thereby enabling others skilled in the art to best utilize the invention with various modifications suitable for the contemplated particular purpose, as well as the various described embodiments.

Claims

1. A method for manipulating computer graphics objects, comprising: At an electronic device that communicates with a display and one or more input devices: The display is used to present a graphical environment including virtual objects with multiple object axes; When presenting the virtual object, receive input indicating a selection of the virtual object; Upon receiving the input indicating a selection of the virtual object, an object manipulator is presented along with the virtual object. The object manipulator has multiple power representations, including multiple rotational power representations. Each rotational power representation is selectable to rotate the virtual object around a different object axis of the virtual object, and each rotational power representation is presented along a different plane defined by two different object axes of the multiple object axes. When presenting the object manipulator, input indicating a selection of a specific rotational energy representation is received, the specific rotational energy representation being presented along a specific plane defined by two specific object axes of the plurality of object axes of the virtual object; Upon receiving the input indicating a selection of the specific rotational energy representation, the selected specific rotational energy representation is magnified along the specific plane to surround the virtual object, and other rotational energy representations are stopped from being displayed; When presenting a selected specific rotational energy representation, receive input representing the rotation of the selected specific rotational energy representation; as well as After receiving the input representing the rotation of the selected specific rotational energy representation, the selected virtual object is rotated about a specific object axis orthogonal to the specific plane associated with the selected specific rotational energy representation.

2. The method according to claim 1, further comprising: The graphical environment is presented from the perspective of a specific eight-part area in 3D space; as well as When the viewing angle changes, one or more power representations of the object manipulator are repositioned such that the plurality of power representations of the object manipulator are in eight zones of the current viewing angle.

3. The method of claim 1, wherein presenting the object manipulator includes presenting a plurality of scaling power representations different from the plurality of rotation power representations, each scaling power representation being selectable for scaling the virtual object.

4. The method of claim 3, wherein each scaling indication is associated with a different object axis, the method further comprising: When presenting the plurality of scaled display representations, receive input indicating a selection of a particular scaled display representation; After receiving the input indicating a selection of the specific scaling display, the display of other scaling displays is stopped; When presenting the selected scaling display representation, receive input indicating the translation of the selected scaling display representation along the object axis associated with the selected scaling display representation; as well as After receiving the input indicating a translation of the selected scaling representation along the object axis associated with the selected scaling representation, the selected virtual object is scaled.

5. The method of claim 4, further comprising uniformly scaling the selected virtual object in all directions associated with each object axis.

6. The method according to claim 4, further comprising: Upon receiving the input indicating a selection of a specific scaling performance representation, receive modifier input; as well as After receiving the modifier input and the input indicating a translation of the selected scaling power representation along the object axis associated with the selected scaling power representation, the selected virtual object is non-uniformly scaled in a first direction associated with the object axis of the selected scaling power representation, while maintaining the size of the selected virtual object in other directions associated with the object axis of the unselected scaling power representation.

7. The method of claim 6, further comprising scaling the selected virtual object in a second direction opposite to the first direction associated with the object axis of the selected scaling representation.

8. The method according to claim 1, further comprising: When the object manipulator is presented, but before receiving the input indicating a selection of a specific rotational energy representation, input indicating a highlighted specific rotational energy representation is received; as well as Upon receiving the input indicating a highlight of the specific rotational indicator, the appearance of the specific rotational indicator is modified by one or more of thickening and brightening.

9. An electronic device, comprising: One or more processors; Memory; as well as One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions that, when executed by the one or more processors, cause the electronic device to perform a method including the following operations: Use the monitor to present a graphical environment that includes virtual objects with multiple object axes; When presenting the virtual object, receive input indicating a selection of the virtual object; Upon receiving the input indicating a selection of the virtual object, an object manipulator is presented along with the virtual object. The object manipulator has multiple power representations, including multiple rotational power representations. Each rotational power representation is selectable to rotate the virtual object around a different object axis of the virtual object, and each rotational power representation is presented along a different plane defined by two different object axes of the multiple object axes. When presenting the object manipulator, input indicating a selection of a specific rotational energy representation is received, the specific rotational energy representation being presented along a specific plane defined by two specific object axes of the plurality of object axes of the virtual object; Upon receiving the input indicating a selection of the specific rotational energy representation, the selected specific rotational energy representation is magnified along the specific plane to surround the virtual object, and other rotational energy representations are stopped from being displayed; When presenting a selected specific rotational energy representation, receive input representing the rotation of the selected specific rotational energy representation; as well as After receiving the input representing the rotation of the selected specific rotational energy representation, the selected virtual object is rotated about a specific object axis orthogonal to the specific plane associated with the selected specific rotational energy representation.

10. The electronic device of claim 9, wherein the one or more programs further include instructions for performing the method, the method further including presenting each rotational energy representation as an arc.

11. The electronic device of claim 10, wherein the one or more programs further include instructions for performing the method, the method further comprising: Hovering over a specific plane defined by a specific rotational energy representation and above the region within the arc represented by the specific rotational energy representation; After hovering over the region, a dual-axis translational energy indication is presented within the region in the specific plane; Receive input representing the selection and movement of the dual-axis translation indication; as well as Upon receiving the input representing the movement indicated by the dual-axis translation capability, the selected virtual object is translated along the specific plane in two-dimensional translation.

12. The electronic device of claim 11, wherein the amount of the two-dimensional translation of the selected virtual object is the same as the amount of movement represented by the dual-axis translation indication.

13. The electronic device of claim 11, wherein the amount of the two-dimensional translation of the selected virtual object is different from the amount of movement represented by the dual-axis translation indication.

14. The electronic device of claim 10, wherein presenting the object manipulator includes presenting a plurality of single-axis translation indications, each single-axis translation indication being selectable for translating the virtual object.

15. The electronic device of claim 14, wherein each single-axis translation indicator is associated with a different object axis, the method further comprising: When presenting the plurality of single-axis translation energy representations, receive input indicating a selection of a specific single-axis translation energy representation; After receiving the input indicating a selection of the specific single-axis translation indication, the display of other single-axis translation indications is stopped; When presenting the selected single-axis translation energy representation, receive input representing a first one-dimensional translation of the selected single-axis translation energy representation along the object axis associated with the selected single-axis translation energy representation; and After receiving the input representing the translation of the selected single-axis translation energy representation along the object axis associated with the selected single-axis translation energy representation, the selected virtual object is translated along the object axis associated with the selected single-axis translation energy representation in a second one-dimensional translation.

16. The electronic device of claim 15, wherein the amount of the second one-dimensional translation is the same as the amount of the first one-dimensional translation.

17. The electronic device of claim 15, wherein the amount of the second one-dimensional translation is different from the amount of the first one-dimensional translation.

18. A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to: At an electronic device that communicates with a display and one or more input devices: The display is used to present a graphical environment including virtual objects with multiple object axes; When presenting the virtual object, receive input indicating a selection of the virtual object; Upon receiving the input indicating a selection of the virtual object, an object manipulator is presented along with the virtual object. The object manipulator has multiple power representations, including multiple rotational power representations. Each rotational power representation is selectable to rotate the virtual object around a different object axis of the virtual object, and each rotational power representation is presented along a different plane defined by two different object axes of the multiple object axes. When presenting the object manipulator, input indicating a selection of a specific rotational energy representation is received, the specific rotational energy representation being presented along a specific plane defined by two specific object axes of the plurality of object axes of the virtual object; Upon receiving the input indicating a selection of the specific rotational energy representation, the selected specific rotational energy representation is magnified along the specific plane to surround the virtual object, and other rotational energy representations are stopped from being displayed; When presenting a selected specific rotational energy representation, receive input representing the rotation of the selected specific rotational energy representation; as well as After receiving the input representing the rotation of the selected specific rotational energy representation, the selected virtual object is rotated about a specific object axis orthogonal to the specific plane associated with the selected specific rotational energy representation.

19. The non-transitory computer-readable storage medium of claim 18, wherein presenting the object manipulator includes presenting an object-centered power representation selectable for translating the virtual object in one or more directions.

20. The non-transitory computer-readable storage medium of claim 19, further storing instructions that, when executed by the one or more processors, further cause the one or more processors to: When presenting the object center visibility representation, receive input indicating a selection of the object center visibility representation; After receiving the input indicating a selection of the object center energy representation, the system receives an input indicating a translation of the selected object center energy representation in one or more directions; and After receiving the input indicating that the center of the selected object can be translated in one or more directions, the selected virtual object is translated in one or more directions.

Images