Sampling and remixing techniques in immersive environment

Abstract

The invention discloses a sampling and remixing technique in an immersive environment. During a sampling phase, the system enables a user to capture samples of a 3D digital component within an immersive environment. The 3D digital component may include 3D objects rendered and displayed within an immersive environment. The 3D digital component may also include an object attribute component for rendering the 3D object, such as a texture, a color scheme, an animation, an action path, or a physical parameter. A sample of the 3D digital component is stored into a sample palette data structure (SPDS) of the tissue sample. During the remix phase, the system enables a user to apply samples stored in the SPDS to modify the 3D object and / or the immersive environment. The user may add the sampled object to the immersive environment to modify the immersive environment. A user may apply one or more object-based samples to the 3D object to modify one or more object attributes of the 3D object.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Application Serial No. 63 / 509,503, entitled “TECHNIQUES FOR SAMPLING AND REMIXING IN IMMERSIVE ENVIRONMENTS,” filed on June 21, 2023. This related application (including any appendices or attachments thereto) is incorporated herein by reference in its entirety. Background of the Invention Technical Field

[0004] Various embodiments relate generally to computer science and immersive environments, and more particularly to sampling and remixing techniques in immersive environments. Background Art

[0005] Generally speaking, a two-dimensional (2D) computing environment is provided by a computing device that executes a 2D application implemented via a 2D interface, such as a desktop environment implemented via a desktop interface, and a three-dimensional (3D) immersive environment (IE) is provided by a computing device that executes a 3D application implemented via an IE interface, such as a VR or AR environment implemented via a virtual reality (VR) or augmented reality (AR) interface, respectively. Due to the unique advantages provided by the immersive environment, playing video games or performing productive work in 3D applications via an IE interface is becoming increasingly popular. For example, when playing video games or performing productive work in an immersive environment, the IE interface places the user in a more engaging environment, which allows for a better sense of space and scale compared to traditional 2D environments and 2D interfaces.

[0006] Sampling digital materials in a 2D environment via a 2D interface is a technique commonly performed by designers as a way to collect 2D digital materials for inspiration, to help find solutions to current design problems, and / or to create a library of digital samples for subsequent use in 2D environments and 2D applications. Materials commonly sampled from 2D environments include text, images, videos, and other similar 2D digital components. Similarly, conventional techniques for sampling digital materials in immersive environments also involve these types of 2D digital components.

[0007] One disadvantage of conventional techniques for sampling digital materials in an immersive environment is that conventional techniques are structured to sample only 2D digital components (such as text, images, and videos). It is noteworthy that there is currently no technology available for sampling or collecting 3D digital components in an immersive environment. In addition, because 3D digital components cannot be sampled in an immersive environment at present, 3D digital components in one immersive environment or 3D application cannot be reused or applied to another immersive environment or 3D application. Therefore, it is usually necessary to design and generate all 3D objects of a specific immersive environment or 3D application, including all related 3D models and various properties of 3D objects. Designing and generating each 3D object for a given immersive environment or 3D application consumes a lot of computing resources and requires designers to invest a lot of time and effort. These problems are even more serious for immersive environments and 3D applications that include a large number of 3D objects.

[0008] As indicated above, there is a need in the art for more efficient techniques for sampling and reusing digital components in immersive environments. Summary of the invention

[0009] Various embodiments include a computer-implemented method for capturing one or more samples within a three-dimensional (3D) immersive environment. The computer-implemented method includes rendering and displaying a first 3D object within a first 3D immersive environment, the first 3D object including at least a first component for rendering and displaying the first 3D object. The computer-implemented method also includes capturing the at least first component as a first sample, the first sample being stored in a sample data structure.

[0010] Various embodiments include a computer-implemented method for applying one or more samples within a three-dimensional (3D) immersive environment. The computer-implemented method includes displaying a first 3D immersive environment including a first 3D object. The computer-implemented method also includes applying a first sample to the first 3D object to modify a first property of the first 3D object, wherein the first sample is captured from a second 3D object.

[0011] At least one technical advantage of the disclosed technology over the prior art is that the disclosed technology enables sampling and acquisition of 3D digital components from an immersive environment, which is not possible using prior art methods. In addition, the disclosed technology enables a designer / user to navigate an immersive environment, select 3D objects or other 3D digital components to be sampled and stored in a sample palette data structure (SPDS), which can then be accessed via a sample palette user interface (SPUI). Once accessed, the sampled 3D objects or 3D digital components can be reused or modified and applied in different immersive environments or 3D applications. In this way, the disclosed technology does not require a designer to design and generate each 3D object of an immersive environment or 3D application, as is the case with the prior art. Therefore, the disclosed technology can significantly reduce the computer resources required to design and generate 3D objects included in an immersive environment or 3D application, and can also significantly reduce the overall designer time and effort required to create an immersive environment or 3D application. These technical advantages provide one or more technical improvements over prior art methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The manner in which the above-described features of the various embodiments can be understood in detail in the more particular description of the inventive concept briefly described above, by reference to various embodiments, some of which are illustrated in the accompanying drawings. It should be noted, however, that the drawings illustrate only typical embodiments of the inventive concept and are therefore not to be considered limiting of the scope in any way, and that there are other equally effective embodiments.

[0013] Figure 1 An IE system configured to implement one or more aspects of various embodiments is shown.

[0014] Figure 2 According to various implementation schemes Figure 1 Conceptual diagram of IE database;

[0015] FIG. 3A to FIG. 3B According to various implementation schemes Figure 1 Conceptual image of SPDS;

[0016] Figure 4 According to various implementation schemes Figure 1 Screenshots of sampled immersive environments;

[0017] Figure 5 According to various embodiments Figure 1 Screenshot of the sample UI for the fox object displayed in the sample immersive environment of ;

[0018] Figure 6 According to various embodiments Figure 1Screenshot of the sample UI of a chair object displayed in the sample immersive environment of ;

[0019] FIG. 7A to FIG. 7B According to various embodiments Figure 1 Screen capture of the segmentation operation of a chair object displayed in a sample immersive environment;

[0020] Figure 8 According to various embodiments Figure 1 Screenshot of a monochrome UI displayed in a sample immersive environment;

[0021] Fig. 9 According to various embodiments Figure 1 Screenshot of the palette UI displayed in the sample immersive environment;

[0022] Fig.10 According to various embodiments Figure 1 Screenshot of the suggestion window displayed in the sample immersive environment;

[0023] Fig.11 Shown according to various embodiments Figure 1 The SPUI's sample collection menu;

[0024] Fig.12 Shown according to various embodiments Figure 1 The object collection UI of SPUI;

[0025] Fig.13 Shown according to various embodiments Figure 1 SPUI's texture collection UI;

[0026] Fig.14 Shown according to various embodiments Figure 1 Color scheme collection UI for SPUI;

[0027] Fig.15 Shown according to various embodiments Figure 1 Animation collection UI of SPUI;

[0028] Fig.16 Shown according to various embodiments Figure 1 Action collection UI of SPUI;

[0029] Fig.17 Shown according to various embodiments Figure 1 The physical parameter set UI of SPUI;

[0030] Fig.18 Shown according to various embodiments Figure 1 SPUI's monochrome and palette collection UI;

[0031] FIG. 19A to FIG. 19B A flow chart setting forth method steps for capturing a sample within a 3D immersive environment and viewing the sample in a SPUI according to various embodiments;

[0032] Fig. 20 According to various embodiments Figure 1 Screenshot of the Remix IE selection menu displayed in the sampled immersive environment;

[0033] Fig.21 According to various implementation schemes Figure 1 Screenshots of the remixed immersive environment;

[0034] Fig. 22 is added to according to various embodiments Figure 1 Screenshot of the sampled objects for the remixed immersive environment;

[0035] Fig.23 is added to according to various embodiments Figure 1 Screenshot of the additional sampled objects for the remixed immersive environment;

[0036] Fig.24 According to various embodiments Figure 1 a screenshot of a first object selected for remixing within the remix immersive environment of FIG.

[0037] FIG. 25A to FIG. 25B According to various implementation schemes Figure 1 A close-up screenshot of a first object of the remix within the remix immersive environment;

[0038] Fig.26 is selected for use with Figure 1 A screenshot of a first object remixed with a second object within the remixed immersive environment;

[0039] Fig. 27 is transferred to Fig.26 A screenshot of the animation properties of a first object within the remixed immersive environment;

[0040] Fig.28 is transferred to Fig. 27 A screenshot of the fully animated properties of the first object within the remixed immersive environment;

[0041] Fig.29 According to various implementation schemes Figure 1 Screenshot of palette remixing of multiple objects within the remixed immersive environment;

[0042] Fig.30 According to various implementation schemes Fig.29 Screenshot of a full palette remix of multiple objects within the immersive environment;

[0043] Fig.31 is added to according to various embodiments Figure 1 Screenshot of a sampled chair object from the remixed immersive environment;

[0044] Fig.32 According to various embodiments, the Fig.31 Screen shot of the "peek" revisit feature for a sampled chair object;

[0045] Fig.33 According to various embodiments, the Fig.31 Screenshot of the "Fully Immersive" revisit feature for a sample of chair objects;

[0046] Fig.34 A flow chart illustrating method steps for reusing samples within a 3D immersive environment according to various embodiments. DETAILED DESCRIPTION

[0047] In the following description, numerous specific details are set forth to provide a more thorough understanding of various embodiments. However, one skilled in the art will appreciate that the inventive concept may be practiced without one or more of these specific details.

[0048] As used herein, an "IE interface" includes 3D specific hardware and software components for interacting with a 3D immersive environment (IE). For example, 3D hardware may include a 3D display, one or more 3D controllers operating in 3D, one or more tracking devices, and one or more cameras. For example, 3D software may include an IE engine that generates a 3D immersive environment and displays a 3D immersive scene on a 3D display. A 3D immersive scene includes a specific view of a 3D immersive environment. Examples of IE interfaces include virtual reality (VR) interfaces and augmented reality (AR) interfaces.

[0049] As used herein, an "immersive environment" (IE) includes a computer-generated 3D environment that includes one or more selectable 3D objects. A 3D display may display a 3D immersive scene (such as a VR scene or an AR scene) that includes a specific view of the immersive environment, depending on the position (position / location) of the user's viewpoint within the immersive environment. The immersive environment includes one or more IE scenes, each IE scene including a specific sub-portion of the immersive environment currently displayed and viewed in the 3D display. Examples of 3D immersive environments include virtual environments generated by VR interfaces, augmented environments generated by AR interfaces, augmented spaces with projections or displays (such as an immersive Van Gogh experience), and the like.

[0050] As used herein, a "3D object" includes a computer-generated 3D digital component within an immersive environment. A 3D object includes a 3D model and one or more object properties. Object properties of a 3D object include, but are not limited to, textures, color schemes, animations, motions, and physical parameters. Object properties of a 3D object include 3D digital components used to render the 3D object.

[0051] As used herein, "samples of 3D digital components" include capturing a 3D digital component from within an immersive environment, recording and / or documenting metadata associated therewith. 3D digital components that may be sampled include object-based samples and color-based samples.

[0052] As used herein, "object-based samples" include object samples and object attribute samples. Object samples include metadata for the entire 3D object, including but not limited to the 3D model, texture, color scheme, animation, motion, and physical parameters of the 3D object. Object attribute samples include metadata for specific attributes of a specific 3D object, including but not limited to texture metadata, color scheme metadata, animation metadata, motion path metadata, or physical parameter metadata for a specific 3D object. Object attribute samples for specific attributes of a specific 3D object are separate and different from object samples for the specific 3D object. The metadata of object-based samples can be used to render the entire object (in the case of object samples), or to render specific attributes of an object (in the case of object attribute samples).

[0053] As used herein, "color-based samples" include monochrome samples and palette samples. Monochrome samples include metadata for a single color associated with a specific point / position within an immersive environment. Palette samples include metadata for multiple colors associated with multiple 3D objects.

[0054] As used herein, a "sample palette data structure" (SPDS) includes a data structure for storing, collecting and organizing samples of 3D digital components. As used herein, a "sample palette user interface" (SPUI) includes a user interface for accessing and viewing samples collected and organized in the SPDS. The collected samples can be accessed via the SPUI to reuse / apply the samples to generate new 3D objects, new immersive environments and / or new 3D applications. Reusing samples includes using object samples of 3D objects in a first immersive environment to add the 3D objects to a second immersive environment to generate a modified / new immersive environment. Reusing samples also includes using samples to modify properties of 3D objects to generate modified / new 3D objects, which is referred to as "remixing".

[0055] Advantageously, the disclosed technology enables sampling and collection of 3D digital components from an immersive environment, rather than sampling of only 2D digital components of an immersive environment as provided by conventional techniques. As a designer / user navigates the immersive environment, the designer may select 3D objects and other 3D digital components to sample and store them in a sample palette data structure (SPDS) that collects and organizes the sampled 3D digital components. The sampled 3D digital components may then be accessed via a sample palette user interface (SPUI) that enables the user to view and reuse / apply the sampled 3D digital components to modify 3D objects, immersive environments, and / or 3D applications in order to generate and render new 3D objects, new immersive environments, and / or new 3D applications. In this manner, the disclosed technology does not require each 3D object of an immersive environment or 3D application to be initially designed and generated from scratch as in the prior art. Instead, in the disclosed technology, the sampled 3D digital components may be reused to reduce or eliminate one or more design and / or generation steps required in the prior art. Therefore, the disclosed technology improves the efficiency with which 3D objects, immersive environments, and / or 3D applications can be designed and generated relative to the prior art in terms of computer resource consumption and designer time and effort. Specifically, the disclosed technology can significantly reduce the amount of computer processing time and processing resources required to generate 3D objects, immersive environments, and / or 3D applications relative to the prior art.

[0056] System Overview

[0057] Figure 1An IE system 100 configured to implement one or more aspects of various embodiments is shown. As shown, the IE system 100 includes, but is not limited to, a computer system 106, an IE database 180, and a server 190 interconnected via 192, which may be a wide area network (WAN) such as the Internet, a local area network (LAN), or any other suitable network. The server 190 may include an image server including computer hardware (such as a processor, memory, and storage) for storing a collection of stock images / pictures and searchable metadata associated with the stock images / pictures to enable search functionality across the collection of stock images / pictures.

[0058] The computer system 106 may include at least one processor 102, an input / output (I / O) device 108, and a memory unit 104 coupled together. The computer system 106 may include a server, a personal computer, a laptop or tablet computer, a mobile computer system, or any other device suitable for practicing the various embodiments described herein. In general, the processor 102 may be any technically feasible processing device or hardware unit capable of processing data and executing software applications and program codes. Each processor 102 executes software and performs the functions and operations set forth in the embodiments described herein. For example, the processor 102 may include a general-purpose processor (such as a central processing unit), a special-purpose processor (such as a graphics processing unit), a specific application processor, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component, or any combination of different processing units.

[0059] The memory unit 104 may include a hard disk, a random access memory (RAM) module, a flash memory unit, or any other type of memory unit or combination thereof. The processor 102 and the I / O device 108 read data from the memory 104 and write data to the memory. The memory unit 104 stores software applications and data. Instructions from the software constructs within the memory unit 104 are executed by the processor 102 to implement the inventive operations and functions described herein.

[0060] I / O devices 108 are also coupled to memory 104 and may include devices capable of receiving input and devices capable of providing output. I / O devices 108 may include input and output devices not specifically listed in IE hardware 170, such as a network card for connecting to network 192, speakers, manufacturing devices (such as 3D printers), etc. In addition, I / O devices may include devices capable of receiving input and providing output, such as a touch screen, a universal serial bus (USB) port, etc.

[0061] As shown, the computer system 106 is also connected to various IE hardware 170, including but not limited to an IE headset 172, one or more IE controllers 176, and one or more tracking devices 178. Each IE controller 176 includes an IE tracking device tracked by the tracking device 178, which determines the 3D position information of the IE controller 176. For example, the IE controller 176 may include a 6-degree-of-freedom (6DOF) controller that operates in 3D. The IE headset 172 can display images in a 3D stereoscopic image, such as an IE scene 174 and various sampling / remixing UIs (SRUIs) 160. The IE headset 172 includes an IE tracking device tracked by the tracking device 178, which can determine the 3D position information of the IE headset 172. In some embodiments, the tracking device 178 tracks the 3D position of the user's viewpoint by tracking the 3D position of the IE headset 172. In some embodiments, IE hardware 170 includes VR hardware 170, including but not limited to VR headset 172, one or more VR controllers 176, and one or more VR tracking devices 178. In other embodiments, IE hardware 170 includes AR hardware 170, including but not limited to AR headset 172, one or more AR controllers 176, and one or more AR tracking devices 178. In other embodiments, IE hardware 170 includes other types of IE hardware for displaying and interacting with other types of 3D immersive environments.

[0062] The memory unit 104 stores the IE engine 110, the sampling / remixing (SR) engine 140, the user application 120, the immersive environment 130, the sampling immersive environment 132, the remixing immersive environment 134, the sampling palette data structure (SPDS) 150, and the sampling suggestion 152. Although shown as separate software components, the IE engine 110 and the SR engine 140 can be integrated into a single software component. For example, in other embodiments, the SR engine 140 can be integrated with the IE engine 110. In other embodiments, the user application 120 and / or the SR engine 140 can be stored on the IE headset 172 and executed thereon.

[0063] User application 120 (stored in memory unit 104 and by Figure 1The user applications 120 may include, for example, 3D design applications for creating, modifying, and interacting with the immersive environment 130. In other embodiments, the user applications 120 may include any other type of 3D-based application that implements the immersive environment 130, such as a 3D video game, a 3D data analysis application, etc. The immersive environment 130 may include a 3D virtual environment, which is stored as, for example, data describing the current scene (such as the 3D position, orientation, and details of virtual 3D objects), data describing the user's viewpoint in the virtual environment (3D position and orientation), data related to the rendering of the virtual scene (such as materials, lighting, and virtual camera position), etc.

[0064] Each immersive environment 130 is associated with a plurality of virtual 3D objects, each of which has associated metadata for rendering and displaying the virtual 3D object. The IE database 180 stores metadata of virtual 3D objects of a plurality of different immersive environments 130. In order to render and display a particular immersive environment 130, the IE engine 110 retrieves metadata of each virtual 3D object associated with the particular immersive environment 130, and uses the retrieved metadata to render and display each associated virtual 3D object within the particular immersive environment 130.

[0065] The immersive environment 130 includes a plurality of IE scenes 174, each IE scene 174 including a sub-portion of the immersive environment 130 currently displayed in the IE headset 172. The IE engine 110 renders the IE scene 174 including a 3D representation of the immersive environment 130. The IE scene 174 is then displayed on the IE headset 172. The user can interact with the immersive environment 130 via the IE scene 174 and the IE hardware 170 to perform sampling and reuse / remixing of 3D digital components within the immersive environment 130. For example, the user can navigate within the immersive environment 130 using the IE controller 176, and use a cursor ray displayed in the IE scene 174 and controlled by the IE controller 176 to interact with and select specific 3D objects within the immersive environment 130.

[0066] In some embodiments, the SR engine 140 (stored in the memory unit 104 and operated by Figure 1The SR engine 140 may be configured to perform a sampling and reuse of 3D digital components within the immersive environment 130 (executable by the processor 102). Sampling the 3D digital components includes capturing the 3D digital components from within the immersive environment 130, and recording and / or documenting metadata associated therewith. The 3D digital components that may be sampled within the immersive environment 130 include object-based samples and color-based samples. Object-based samples include object samples and object attribute samples. Color-based samples include monochrome samples and palette samples. The SR engine 140 stores samples of the 3D digital components in the SPDS 150, which is used to collect and organize samples of the 3D digital components. The SR engine 140 also generates and displays a plurality of sampling / remixing user interfaces (SRUIs) 160 on the IE headset 172 to implement sampling and reuse / remixing of the 3D digital components. SRUI 160 includes a Sample Palette User Interface (SPUI) 166 for accessing and viewing samples collected and organized in SPDS 150 to reuse / remix samples to generate new 3D objects, new immersive environments, and / or new 3D applications.

[0067] Figure 2 According to various implementation schemes Figure 1 Conceptual diagram of an IE database 180 of FIG. IE database 180 stores metadata of multiple different immersive environments 130 (such as IE_A, IE_B, IE_C). IE database 180 includes multiple IE entries 230 (such as 230a, 230b, 230c), each IE entry 230 represents a specific immersive environment 130 and stores metadata of the represented immersive environment 130. Each IE entry 230 includes an identifier field 210 and an associated object field 220. The IE identifier field 210 specifies an IE identifier that uniquely identifies the IE entry 230 and the represented immersive environment 130 (such as IE_A, IE_B, IE_C). The associated object field 220 specifies one or more 3D objects associated with the represented immersive environment 130. The associated object field 220 includes a unique object identifier for each of the one or more associated 3D objects, such as objectA1, objectA2, objectA3, etc. The immersive environments 130 may be defined by the 3D objects associated with the immersive environments 130. Thus, a first immersive environment 130 having a different set of associated 3D objects than a second immersive environment 130 may be considered a separate and different immersive environment 130 even if the set of associated 3D objects differs by only a single 3D object.

[0068] The associated object field 220 also includes metadata (such as objectA1_meta, objectA2_meta, objectA3_meta, etc.) for each 3D object associated with the represented immersive environment 130. The metadata for the 3D objects stored in the associated object field 220 includes metadata that defines the 3D objects and is used to render and display the 3D objects. The metadata (object_meta) stored for each 3D object includes metadata for the 3D model of the 3D object and metadata for one or more object properties (textures, color schemes, animations, motions, and / or physical parameters).

[0069] The metadata (model_meta) of the 3D model describes the 3D model of the 3D object. The 3D model can be any technically feasible 3D structure or a mathematical model representing a 3D object, such as a mesh, a point cloud, a wireframe model, or a manifold. In some embodiments, the 3D model 110 includes a polygonal mesh (triangle mesh) composed of interconnected triangles. The 3D model can represent a real-world object or can represent a virtual object, such as a video game character. The texture metadata (texture_meta) of the 3D object describes the texture of the 3D object. The texture of the 3D object can include a set of images that define the appearance (such as color and surface properties) of the 3D object, and the appearance wraps (covers) the mesh of the 3D object. For example, the mesh of a head object can have the texture of eyes, eyebrows, hair, and some shadows to distinguish features such as the nose and mouth of the head object.

[0070] The color scheme of a 3D object includes one or more colors specified for the 3D object, wherein the 3D object generally includes different colors associated with different parts of the 3D object. The color scheme metadata (colorsch_meta) of the 3D object specifies one or more of the most prominent / significant colors of the 3D object, such as the nine most prominent / significant colors. The animation of a 3D object represents the way (dynamic movement characteristics) of the 3D object to perform a specific action, such as the way the 3D object walks, jumps or swings a sword. Animation can provide a set of basic animation cycles, each of which can be regarded as an independent animation, such as swinging a sword. The animation metadata (anim_meta) of a 3D object can describe a data structure with skeleton points across multiple time points. The action of a 3D object includes a predetermined action path of the 3D object, such as a predetermined path for the 3D object to walk. The motion metadata (motion_meta) of a 3D object specifies an action path and an action speed (such as a walking speed). The set of physical parameters of the 3D object may include material parameters and object parameters specified for the 3D object. The set of physical metadata (physical_meta) of the 3D object may specify physical parameters, including friction, bounce, mass, drag, buoyancy, material type (such as wood, ceramic, metal, etc.), mass, friction, drag, bounce, etc. The object properties of the 3D object may also include a 3D position associated with the 3D object. The metadata of the 3D position specifies 3D coordinates (such as x, y, z coordinates) that indicate where the 3D object is placed within the corresponding immersive environment 130. In other embodiments, the object properties of the 3D object may also include a model piece (such as a chair part, or a fox model separate from a sword model).

[0071] When the user selects a particular immersive environment 130, the IE engine 110 (stored in the memory unit 104 and used by Figure 1The processor 102 executes) identifies the selected immersive environment 130 in the IE database 180 via the unique IE identifier 210, and retrieves metadata for each virtual 3D object associated with the selected immersive environment 130 from the associated object field 220. The IE engine 110 can then use the retrieved metadata to render and display each associated 3D object within the selected immersive environment 130. Specifically, for each associated 3D object, the IE engine 110 can use metadata of 3D models, textures, color schemes, animations, actions, and physical parameters to render the 3D object. Each associated 3D object 220 can be rendered and displayed at a location specified by 3D position metadata associated with the 3D object within the selected immersive environment 130. During the sampling phase described below, the user selects a specific sampled immersive environment 132 stored in the IE database 180 from which the 3D digital component is sampled. During the remixing phase described below, the user selects a specific remixed immersive environment 134 stored in the IE database 180 from which the sample is reused / applied.

[0072] FIG. 3A to FIG. 3B According to various implementation schemes Figure 1 1. Conceptual diagram of SPDS150 of FIG. SPDS150 provides underlying storage for collected samples, which are accessed and viewed via SRUI 160 (such as SPUI 166). SPDS150 provides storage for collected samples, which is separate and different from the IE database 180 used to store various immersive environments 130 and 3D objects associated with the immersive environment 130. SPDS150 includes different types of entries representing different types of samples. Different types of entries can be grouped into different sections according to the entry type. As shown, SPDS150 is subdivided into multiple sections, including but not limited to object section 302, texture section 306, color scheme section 310, animation section 314, action section 318, physical parameter section 322, monochrome section 326 and palette section 330. Each section of SPDS150 can store a specific type of entry corresponding to a specific type of sample.

[0073] In this regard, the object section 302 includes zero or more object entries 304 (such as 304a and 304b), each object entry 304 representing an entire object sample sampled / captured from within the immersive environment 130. The texture section 306 includes zero or more texture entries 308 (such as 308a and 308b), each texture entry 308 representing a texture sample sampled / captured from within the immersive environment 130. The color scheme section 310 includes zero or more color scheme entries 312 (such as 312a and 312b), each color scheme entry 312 representing a color scheme sample sampled / captured from within the immersive environment 130. The animation section 314 includes zero or more animation entries 316 (such as 316a and 316b), each animation entry 316 representing an animation sample sampled / captured from within the immersive environment 130. The action section 318 includes zero or more action entries 320 (such as 320a and 320b), each action entry 320 represents an action sample sampled / captured from within the immersive environment 130. The physical parameter section 322 includes zero or more physical parameter entries 324 (such as 324a and 324b), each physical parameter entry 324 represents a physical parameter sample sampled / captured from within the immersive environment 130. The monochrome section 326 includes zero or more monochrome entries 328 (such as 328a and 328b), each monochrome entry 328 represents a monochrome sample sampled / captured from within the immersive environment 130. The palette section 330 includes zero or more palette entries 332 (such as 332a and 332b), each palette entry 332 represents a palette sample sampled / captured from within the immersive environment 130.

[0074] Each entry of the SPDS 150 corresponds to a particular sample and includes a plurality of data fields associated with and describing the sample, such as data fields for a sample identifier 340, an associated object 350, sample metadata 360, a context 370, and a sample icon 380. The sample identifier field 340 of an entry includes a unique identifier that uniquely identifies the entry and the corresponding sample. In some embodiments, the identifier field 340 of the entry may also identify the type of the corresponding sample, such as an object, texture, animation, etc. The associated object field 350 specifies the 3D object from which the sample was derived or captured. The sample metadata field 360 of the entry includes metadata captured for the sample. The sample metadata field 360 includes metadata that can be used to render the entire 3D object or to render specific properties of the 3D object. The type of entry and the corresponding sample determine the type of metadata stored in the sample metadata field 360. For example, a texture entry for a texture sample stores texture metadata in the sample metadata field 360. The context field 370 of the entry specifies context information for the location where the corresponding sample was captured. The sample icon field 380 of the entry includes text and / or graphic data for rendering and displaying (in a simplified manner) a 2D or 3D icon that visually represents the corresponding sample. As discussed below, the data fields included in a particular entry of SPDS150 may vary depending on the type of entry and the type of corresponding sample.

[0075] The object entry 304 representing an object sample capturing metadata of the entire 3D object may include an identifier field 340, a sample metadata field 360, a context field 370, and a sample icon field 380. The identifier field 340 may correspond to an identifier of the 3D object in the IE database 180. In some embodiments, the identifier field 340 of the object entry 304 may also indicate the IE identifier of the specific immersive environment 130 from which the object sample was captured. The sample metadata field 360 includes all metadata associated with the entire 3D object, including metadata describing the 3D model, texture, color scheme, animation, motion, and physical parameters of the 3D object. The context field 370 specifies the sampling location of the 3D object and includes the IE identifier 210 of the specific immersive environment 130 from which the 3D object was sampled, the 3D position coordinates of the 3D object within the specific immersive environment 130 when the 3D object was sampled, and the 3D position coordinates of the user's viewpoint within the specific immersive environment 130 when the 3D object was sampled. The sample icon field 380 may include text and / or graphical metadata for rendering a 2D or 3D object sample icon that visually represents (in a simplified manner) the object sample and the 3D object.

[0076] A texture entry 308 representing a texture sample that captures only the texture properties of a 3D object (but not other properties of the 3D object) may include a sample identifier field 340, an associated object field 350, a sample metadata field 360, and a sample icon field 380. The identifier field 340 uniquely identifies the texture entry 308 and the texture sample, and may further specify the sample type (texture). The associated object field 350 specifies the object identifier of the 3D object from which the texture sample was originally derived and captured. The sample metadata field 360 includes texture metadata for the 3D object. The sample icon field 380 may include text and / or graphical metadata for rendering a 2D or 3D texture sample icon that visually represents (in a simplified manner) the texture sample of the 3D object.

[0077] A color scheme entry 312 representing a color scheme sample that captures only the color scheme properties of a 3D object (but not other properties of the 3D object) may include a sample identifier field 340, an associated object field 350, a sample metadata field 360, and a sample icon field 380. The identifier field 340 uniquely identifies the color scheme entry 312 and the color scheme sample, and may further specify the sample type (color scheme). The associated object field 350 specifies the object identifier of the 3D object from which the color scheme sample was originally derived and captured. The sample metadata field 360 includes the color scheme metadata of the 3D object. The sample icon field 380 may include text and / or graphic metadata for rendering a 2D or 3D color scheme sample icon that visually represents (in a simplified manner) the color scheme sample of the 3D object.

[0078] An animation entry 316 representing an animation sample that captures only the animation properties of a 3D object (but not other properties of the 3D object) may include a sample identifier field 340, an associated object field 350, a sample metadata field 360, and a sample icon field 380. The identifier field 340 uniquely identifies the animation entry 316 and the animation sample, and may further specify the sample type (animation). The associated object field 350 specifies the object identifier of the 3D object from which the animation sample was originally derived and captured. The sample metadata field 360 includes the animation metadata of the 3D object. The sample icon field 380 may include text and / or graphical metadata for rendering a 2D or 3D animation sample icon that visually represents (in a simplified manner) the animation sample of the 3D object.

[0079] An action entry 320 representing an action sample that captures only the action attributes of a 3D object (but not other attributes of the 3D object) may include a sample identifier field 340, an associated object field 350, a sample metadata field 360, and a sample icon field 380. The identifier field 340 uniquely identifies the action entry 320 and the action sample, and may further specify the sample type (action). The associated object field 350 specifies the object identifier of the 3D object from which the action sample was originally derived and captured. The sample metadata field 360 includes the action metadata of the 3D object. The sample icon field 380 may include text and / or graphic metadata for rendering a 2D or 3D action sample icon that visually represents the action sample of the 3D object (in a simplified manner).

[0080] A physical parameter entry 324 representing a physical parameter sample that captures only the physical parameter properties of a 3D object (but not other properties of the 3D object) may include a sample identifier field 340, an associated object field 350, a sample metadata field 360, and a sample icon field 380. The identifier field 340 uniquely identifies the physical parameter entry 324 and the physical parameter sample, and may further specify the sample type (physical parameter). The associated object field 350 specifies the object identifier of the 3D object from which the physical parameter sample was originally derived and captured. The sample metadata field 360 includes physical parameter metadata for the 3D object. The sample icon field 380 may include text and / or graphical metadata for rendering a 2D or 3D physical parameter sample icon that visually represents (in a simplified manner) the physical parameter sample of the 3D object.

[0081] Object-based samples include the above-mentioned whole object samples of the 3D object and object property samples of the properties of the 3D object (texture, color scheme, animation, motion, and physical parameters of the 3D object). It should be noted that each object property sample of the 3D object is separate and different from the object sample of the 3D object. Similarly, each object property entry of the object property sample of the 3D object is separate and different from the object entry of the object sample of the 3D object. Thus, each object property sample of the 3D object can be accessed, viewed, and reused / applied separately and independently of the object sample of the 3D object. In addition, the user may only want to capture specific object property samples of the 3D object without capturing the entire object sample of the 3D object. For example, the user may choose to capture only texture samples and animation samples of the 3D object without capturing other object property samples or object samples of the entire 3D object. In this way, the described technology provides the user with full control over which 3D digital components are sampled within the immersive environment 130.

[0082] A monochrome entry 328 representing a monochrome sample that captures a monochrome color associated with a particular point in the immersive environment 130 may include a sample identifier field 340, a sample metadata field 360, and a sample icon field 380. The sample identifier field 340 uniquely identifies the monochrome entry 328 and the monochrome sample, and may further specify the sample type (monochrome). The sample metadata field 360 includes metadata describing the single color captured. The sample icon field 380 may include text and / or graphical metadata for rendering a 2D or 3D monochrome sample icon that visually represents the monochrome sample (in a simplified manner).

[0083] A palette entry 332 representing a palette sample capturing a plurality of colors associated with a plurality of 3D objects may include a sample identifier field 340, a sample metadata field 360, and a sample icon field 380. The sample identifier field 340 uniquely identifies the palette entry 332 and the palette sample, and may further specify the sample type (palette). The sample metadata field 360 includes metadata describing the plurality of colors captured. The sample icon field 380 may include text and / or graphical metadata for rendering a 2D or 3D palette sample icon that visually represents (in a simplified manner) the palette sample.

[0084] Sampling in an immersive environment

[0085] During the sampling phase, the user selects a particular immersive environment stored in the IE database 180 (referred to herein as the sampled immersive environment 132) from which to sample the 3D digital component. When the user selects the sampled immersive environment 132, the IE engine 110 (stored in the memory unit 104 and used by Figure 1 The IE engine 110 then uses the retrieved metadata to render and display each 3D object associated with the sampled immersive environment 132 on the IE headset 172. In addition, during the sampling phase, the SR engine 140 (stored in the memory unit 104 and by the SR engine 140) identifies the sampled immersive environment 132 in the IE database 180 via the IE identifier 210 and retrieves metadata of the associated virtual 3D objects from the associated object field 220. The IE engine 110 then uses the retrieved metadata to render and display each 3D object associated with the sampled immersive environment 132 on the IE headset 172. Figure 1 The processor 102 of the IE headset 172 generates and displays various sampling / remixing user interfaces (SRUI) 160 to enable sampling and reuse of 3D digital components. The user interacts with the various SRUIs 160 using the IE controller 176 to select a specific 3D digital component to be sampled from the sampling immersive environment 132. In some embodiments, a commonly used specific SRUI 160 can be mapped to a specific button on the IE controller 176 to allow the user to easily access the commonly used SRUI 160, such as the SPUI 166.

[0086] Figure 4 According to various implementation schemes Figure 1 132. As shown, the sample immersive environment 132 includes a simulated 3D medieval village, which includes multiple 3D objects, including a fox character, barrels, buildings, trees, plants, sky, ground, etc. Figure 4 The example of 174 shows a current IE scene 174, which includes a sub-portion of the sampled immersive environment 132 currently displayed in the IE headset 172. The user can control the user's viewpoint within the sampled immersive environment 132 via the IE controller 176 to explore and navigate the sampled immersive environment 132. The user can also control a cursor ray displayed in the IE scene 174 via the IE controller 176 to interact with the SRUI 160 and select a specific 3D object within the sampled immersive environment 132.

[0087] In addition, the SR engine 140 (stored in the memory unit 104 and by Figure 1 102) generates and displays a mode UI 400 in the IE scene 174. The mode UI 400 displays three selectable mode options, including a sampling mode 410, a sample palette mode 420, and a remix mode 430. Selecting the sampling mode 410 starts the sampling phase and enables the user to sample various 3D digital components within the sampling immersive environment 132 via the SRUI 160. Selecting the sample palette mode 420 causes the SR engine 140 to generate and display a sample palette user interface (SPUI) 166, which allows the user to view, access, and reuse / apply the samples currently stored in the SPDS 150. Selecting the remix mode 430 starts the remix phase and enables the user to reuse / apply the current sample within the remix immersive environment 134. In some embodiments, to allow easy access to the mode UI 400, the mode UI 400 may be mapped to a specific button on the IE controller 176, or continuously displayed in a portion of the IE scene 174.

[0088] Figure 5 According to various embodiments Figure 1 A screenshot of a sample UI for a fox object displayed in a sample immersive environment 132 of FIG. Figure 5 In the example of , the user selects sampling mode 410, which causes SR engine 140 to generate and display sampling UI 500. Sampling UI 500 displays selectable options, including segmentation operation 510, whole object samples 520, texture samples 530, color scheme samples 540, animation samples 550, motion samples 560, physical parameter samples 570, monochrome samples 580, and palette samples 590.

[0089] In some embodiments, object-based sampling (object samples 520, texture samples 530, color scheme samples 540, animation samples 550, motion samples 560, physical parameter samples 570) may be initiated by selecting a particular 3D object within the IE scene 174. In response, the SR engine 140 retrieves all object metadata (object_meta) associated with the selected 3D object from the IE database 180 using the IE identifier 210 and the object identifier of the particular sampled immersive environment 132. The object metadata (object_meta) includes 3D model metadata (model_meta), texture metadata (texture_meta), color scheme metadata (colorsch_meta), animation metadata (anim_meta), motion metadata (motion_meta), and physical parameter metadata (physical_meta). The SR engine 140 then parses the object metadata (object_meta) into different types of metadata (texture_meta, colorsch_meta, anim_meta, motion_meta, and physical_meta) for different types of object properties to determine which types of metadata and object properties are available for sampling the selected 3D object. Generally speaking, not all 3D objects include all different types of metadata for all different types of object properties. For example, a tree object or a building object typically does not include animation metadata (anim_meta) and motion metadata (motion_meta) for animation properties and motion properties. The SR engine 140 can then highlight (or indicate in another visual manner) the types of samples that can be captured in the sampling UI 500 based on the metadata types and properties available for the selected 3D object.

[0090] like Figure 5As shown in the example of , the user has selected to sample a fox object 594 within the immersive environment 132. In response, the SR engine 140 retrieves all object metadata (object_meta) associated with the selected fox object 594 from the IE database 180 and parses the object metadata into different types of metadata for different types of object attributes associated with the selected fox object 594. The SR engine 140 determines that the object metadata for the fox object 594 includes all different types of metadata (texture_meta, colorsch_meta, anim_meta, motion_meta, and physical_meta) for all different types of object attributes. The SR engine 140 then highlights (or indicates in another visual manner) the types of sampling options that can be captured for the selected fox object 594 in the sampling UI 500, including whole object samples 520, texture samples 530, color scheme samples 540, animation samples 550, motion samples 560, and physical parameter samples 570.

[0091] Figure 6 According to various embodiments Figure 1 A screenshot of a sample UI for a chair object displayed in a sample immersive environment 132 of FIG. Figure 6 In the example of FIG. 6 , the user selects the chair object 694. In response, the SR engine 140 (stored in the memory unit 104 and by Figure 1 The SR engine 140 (executable by the processor 102) retrieves all object metadata (object_meta) associated with the selected chair object 694 from the IE database 180, and parses the object metadata into different types of metadata for different types of object properties available for the selected chair object 694. The SR engine 140 determines that the object metadata for the chair object 694 includes all different types of metadata except for the animation and motion object properties. The SR engine 140 then highlights (or indicates in another visual manner) the types of sampling options that can be captured for the selected chair object 694 in the sampling UI 500, including whole object samples 520, texture samples 530, color scheme samples 540, and physical parameter samples 570. It should be noted that the sampling UI 500 does not highlight object-based sampling options (animation samples 550 and motion samples 560) that are not available for the selected chair object 694. In other embodiments, the SR engine 140 may delete unavailable sampling options (such as animation samples 550 and motion samples 560) from the sampling UI 500.

[0092] Reference again Figure 5, the user can select among the object-based samples available for the selected fox object 594. As indicated in the sampling UI 500, the available samples include whole object samples 520, texture samples 530, color scheme samples 540, animation samples 550, motion samples 560, and physical parameter samples 570.

[0093] If the user selects the object sample 520, then in response, the SR engine 140 captures / samples the object metadata of the entire fox object 594 by generating a new object entry 304 representing the new object sample in the object section 302 of the SPDS 150. The SR engine 140 then populates the various data fields of the new object entry 304, including the sample identifier field 340, the sample metadata field 360, the context field 370, and the sample icon field 380. Specifically, the SR engine 140 stores all the object metadata (object_meta) retrieved for the fox object 594 into the sample metadata field 360, including the 3D model metadata (model_meta), texture metadata (texture_meta), color scheme metadata (colorsch_meta), animation metadata (anim_meta), motion metadata (motion_meta), and physical parameter metadata (physical_meta). The SR engine 140 may also generate text and / or graphic metadata for rendering (in a simplified manner) a 2D or 3D object icon that visually represents the new object sample and the fox object 594 based on the object metadata (object_meta). The text and / or graphic metadata is then stored in the sample icon field 380 in the new object entry 304.

[0094] For the object sample, the SR engine 140 populates the context field 370 in the new object entry 304, for example, by interacting with the IE engine 110 to determine the context information of the selected fox object 594, the context information including the IE identifier 210 of the sampled immersive environment 132 from which the fox object 594 was sampled, the 3D position coordinates of the fox object 594 within the sampled immersive environment 132 when the fox object 594 was sampled (which can be determined by the current 3D position coordinates of the fox object 594 displayed within the current IE scene 174), and the 3D position coordinates of the user's viewpoint within the sampled immersive environment 132 when the fox object 594 was sampled (which can be determined by the current 3D position coordinates of the user's viewpoint displayed within the current IE scene 174). In some embodiments, the above-mentioned context information is stored in the context field 370 in the form of a reference pointer or link to the 3D position in the specific sampled immersive environment 132 from which the fox object 594 was sampled.

[0095] If the user selects texture sample 530, then in response, the SR engine 140 captures / samples the texture metadata of the fox object 594 by generating a new texture entry 308 in the texture section 306 of the SPDS 150 representing the new texture sample. The SR engine 140 then populates the various data fields of the new texture entry 308, including the sample identifier field 340, the associated object field 350, the sample metadata field 360, and the sample icon field 380. Specifically, the SR engine 140 stores the texture metadata (texture_meta) retrieved and parsed for the fox object 594 in the sample metadata field 360. The associated object field 350 specifies the object identifier of the fox object 594 from which the new texture sample was derived or captured, which object identifier can be retrieved from the IE database 180. The SR engine 140 also generates text and / or graphic metadata for rendering (in a simplified manner) a 2D or 3D texture icon that visually represents the new texture sample based on the texture metadata (texture_meta). The text and / or graphics metadata is then stored in the sample icon field 380 in the new texture entry 308 .

[0096] If the user selects the color scheme sample 540, then in response, the SR engine 140 captures / samples the color scheme metadata of the fox object 594 by generating a new color scheme entry 312 in the color scheme section 310 of the SPDS 150 representing the new color scheme sample. The SR engine 140 then populates the various data fields of the new color scheme entry 312, including the sample identifier field 340, the associated object field 350, the sample metadata field 360, and the sample icon field 380. Specifically, the SR engine 140 stores the color scheme metadata (colorsch_meta) retrieved and parsed for the fox object 594 in the sample metadata field 360. The associated object field 350 specifies the object identifier of the fox object 594 from which the new color scheme sample was derived or captured, which object identifier can be retrieved from the IE database 180. The SR engine 140 also generates text and / or graphic metadata for rendering (in a simplified manner) a 2D or 3D color scheme icon that visually represents the new color scheme sample based on the color scheme metadata (colorsch_meta). The text and / or graphic metadata is then stored in the sample icon field 380 in the new color scheme entry 312 .

[0097] If the user selects the animation sample 550, then in response, the SR engine 140 captures / samples the animation metadata of the fox object 594 by generating a new animation entry 316 in the animation section 314 of the SPDS 150 representing the new animation sample. The SR engine 140 then populates the various data fields of the new animation entry 316, including the sample identifier field 340, the associated object field 350, the sample metadata field 360, and the sample icon field 380. Specifically, the SR engine 140 stores the animation metadata (anim_meta) retrieved and parsed for the fox object 594 in the sample metadata field 360. The associated object field 350 specifies the object identifier of the fox object 594 from which the new animation sample is derived or captured, which object identifier can be retrieved from the IE database 180. The SR engine 140 also generates text and / or graphic metadata for rendering (in a simplified manner) a 2D or 3D animation icon that visually represents the new animation sample based on the animation metadata (anim_meta). For example, the animation icon may display a neutral human model in situ to represent the avatar animation. The text and / or graphic metadata is then stored in the sample icon field 380 in the new animation entry 316.

[0098] If the user selects the motion sample 560, in response, the SR engine 140 captures / samples the motion metadata of the fox object 594 by generating a new motion entry 320 representing the new motion sample in the motion section 318 of the SPDS 150. The SR engine 140 then fills in the various data fields of the new motion entry 320, including the sample identifier field 340, the associated object field 350, the sample metadata field 360, and the sample icon field 380. Specifically, the SR engine 140 stores the motion metadata (motion_meta) retrieved and parsed for the fox object 594 in the sample metadata field 360. The associated object field 350 specifies the object identifier of the fox object 594 from which the new motion sample is derived or captured, which object identifier can be retrieved from the IE database 180. The SR engine 140 also generates text and / or graphic metadata for rendering (in a simplified manner) a 2D or 3D motion icon that visually represents the new motion sample based on the motion metadata (motion_meta). For example, the motion icon can display the outline of a walking path. The text and / or graphic metadata is then stored in the sample icon field 380 in the new action entry 320 .

[0099] If the user selects the physical parameter sample 570, then in response, the SR engine 140 captures / samples the physical parameter metadata of the fox object 594 by generating a new physical parameter entry 324 representing the new physical parameter sample in the physical parameter section 322 of the SPDS 150. The SR engine 140 then populates the various data fields of the new physical parameter entry 324, including the sample identifier field 340, the associated object field 350, the sample metadata field 360, and the sample icon field 380. Specifically, the SR engine 140 stores the retrieved and parsed physical parameter metadata (physical_meta) for the fox object 594 in the sample metadata field 360. The associated object field 350 specifies the object identifier of the fox object 594 from which the new physical parameter sample is derived or captured, which object identifier can be retrieved from the IE database 180. The SR engine 140 also generates text and / or graphical metadata for rendering (in a simplified manner) a 2D or 3D physical parameter icon that visually represents the new physical parameter sample based on the physical parameter metadata (physical_meta). The text and / or graphical metadata is then stored in the sample icon field 380 in the new physical parameter entry 324 .

[0100] It should be noted that each of the new object attribute samples (texture samples, color scheme samples, animation samples, motion samples, and physical parameter samples) of fox object 594 is separate and distinct from the object sample of fox object 594. Likewise, each of the new object attribute entries of the corresponding new object attribute samples is separate and distinct from the object entry of the object sample of fox object 594. Thus, each of the object attribute samples of fox object 594 can be accessed, viewed, and applied separately and independently of the object sample of fox object 594.

[0101] In some embodiments, the SR engine 140 also enables a segmentation function to segment / deconstruct the selected 3D object into two or more sub-parts. If the user selects a particular 3D object in the sampling immersive environment 132 and selects the segmentation operation 510 from the sampling UI 500, the SR engine 140 executes a deconstruction algorithm / tool ​​to separate the selected 3D object into two or more sub-parts. The sub-parts of the 3D object are predefined in the metadata, wherein the deconstruction algorithm / tool ​​identifies these sub-parts via the metadata and displays the separate sub-parts.

[0102] The SR engine 140 then adds each sub-portion of the selected 3D object to the IE entry 230 representing the sampled immersive environment 132 in the IE database 180 as a separate and independent 3D object associated with the sampled immersive environment 132. Each sub-portion of the selected 3D object may then be selected as a 3D object in the same manner as any other 3D object within the sampled immersive environment 132. The above-described object-based sampling operations may then be performed on the selected sub-portion of the selected 3D object in the same manner as any other 3D object within the sampled immersive environment 132.

[0103] FIG. 7A to FIG. 7B According to various embodiments Figure 1 A screenshot of a segmentation operation of a chair object shown in the sample immersive environment 132. Fig. 7A In the example of FIG. 5 , the user selects the chair object 794 and selects the segmentation operation 510 from the sample UI 500. In response, the SR engine 140 executes a deconstruction algorithm / tool ​​to separate / deconstruct the selected chair object 794 into a plurality of sub-parts 710, such as Figure 7B The SR engine 140 then adds each sub-portion of the chair object 794 to the IE entry 230 representing the sampled immersive environment 132 in the IE database 180 as a separate and independent 3D object associated with the sampled immersive environment 132. Each of the plurality of sub-portions 710 can then be individually selected as an independent chair sub-portion object for sampling purposes.

[0104] To capture a color-based sample, the user may select a monochrome sample 580 or a palette sample 590 from the sampling UI 500. If the user selects the monochrome sample 580, then in response, the SR engine 140 generates and displays a monochrome UI to enable the user to capture a monochrome sample of a specific point / location within the sampling immersive environment 132. Figure 8 According to various embodiments Figure 1 800 is a screen shot of a monochrome UI 800 displayed in the sampling immersive environment 132 of FIG. The monochrome UI 800 displays a selectable "Get Sample" button 810 and a monochrome suggestion window 820. The user then selects a specific point / location within the sampling immersive environment 132 via a cursor ray controlled using the IE controller 176.

[0105] exist Figure 8In the example of , the user selects a point / location on the barrel object 894. Once the sample point / location is selected, the user selects the "Get Sample" button 810. In response, the SR engine 140 captures / samples the monochrome sample by generating a new monochrome entry 328 representing the new monochrome sample in the monochrome section 326 of the SPDS 150. The SR engine 140 then populates the various data fields of the new monochrome entry 328, including the sample identifier field 340, the sample metadata field 360, and the sample icon field 380. Specifically, the SR engine 140 stores the monochrome metadata (scolor_meta) determined for the selected sample point in the sample metadata field 360. The SR engine 140 also generates text and / or graphic metadata for rendering (in a simplified manner) a 2D or 3D icon that visually represents the new monochrome sample based on the monochrome metadata (scolor_meta). The text and / or graphic metadata is then stored in the sample icon field 380 in the new monochrome entry 328.

[0106] Various techniques may be used to determine the monochrome metadata (scolor_meta). In some embodiments, the SR engine 140 implements a point sampling technique that projects a ray from a virtual IE controller 176 displayed in the IE scene 174 to a selected sampling point. The projected ray will intersect with the object containing the selected sampling point. The SR engine 140 may then access the color / texture coordinates of the object at the ray intersection point, retrieve the color metadata of the object associated with the color / texture coordinates, and insert the most prominent / most noticeable color based on the retrieved color metadata. The SR engine 140 then generates monochrome metadata (scolor_meta) based on the most prominent / noticeable color returned. In other embodiments, the SR engine 140 implements a region sampling technique by capturing an image of a predetermined area around the selected sampling point, quantizing / sampling the image to determine, for example, the five most dominant / most common color clusters, and returning a single most dominant / most common cluster. The SR engine 140 then generates monochrome metadata (scolor_meta) based on the most dominant / most common cluster returned. Compared to the point sampling technique, the region sampling technique may take into account the effect of illumination on color. In other embodiments, the SR engine 140 implements another type of technique to determine the monochrome metadata (scolor_meta).

[0107] In addition, the SR engine 140 may provide additional suggestions for monochrome samples based on the current monochrome sample via the monochrome suggestion window 820. The monochrome suggestion window 820 may include a plurality of suggested images 830 (such as 830a, 830b, 830c, etc.) for additional monochrome sampling. Specifically, the SR engine 140 may initiate or perform an image search on an image server (such as server 190) that stores a collection of stock images / photos. For example, a search may be performed to identify stock images having colors similar to the current monochrome sample. For each identified image, the SR engine 140 may then retrieve the image from the image server 190, store the image as a sampling suggestion 152 in the memory unit 104, and display the image as a suggested image 830 in the monochrome suggestion window 820. If the user then wishes to sample the suggested image 830, the user may select a particular point / location within the suggested image 830 and select the "Get Sample" button 810. In response, the SR engine 140 captures / samples the monochrome component associated with the sample point / location in the proposed image 830 by generating a new monochrome entry 328 in the monochrome section 326 of the SPDS 150 that represents the new monochrome sample.

[0108] If the user selects the palette sample 590 from the sampling UI 500, then in response, the SR engine 140 generates and displays a palette UI to enable the user to capture a palette sample of a plurality of 3D objects within the sampling immersive environment 132. Generally speaking, a palette includes a group of two or more colors associated with two or more objects. The palette can capture the color scheme that exists across the IE scene and capture more abstract aspects of the IE scene, such as hue, hue, saturation, brightness, contrast, lighting, mood, atmosphere, etc. Fig. 9 According to various embodiments Figure 1 900 . The palette UI 900 displays an optional "global selection" button 902, an optional "get sample" button 910, and a palette suggestion window 920. The user then selects multiple 3D objects within the sampling immersive environment 132 via a cursor ray controlled using the IE controller 176. Alternatively, the user can select the "global selection" button 902 to automatically select all 3D objects displayed within the current IE scene 174.

[0109] exist Fig. 9, the user selects a chair object 950, a bucket object 960, and a tree object 970. Once the 3D objects are selected, the user selects a "Get Sample" button 910. In response, the SR engine 140 captures the palette sample by generating a new palette entry 332 representing the new palette sample in the palette section 330 of the SPDS 150. The SR engine 140 then populates the various data fields of the new palette entry 332, including the sample identifier field 340, the sample metadata field 360, and the sample icon field 380. Specifically, the SR engine 140 stores the palette metadata (cpalette_meta) determined for the selected 3D object in the sample metadata field 360. The SR engine 140 also generates text and / or graphics metadata for rendering (in a simplified manner) a 2D or 3D icon representing the new palette sample based on the palette metadata (cpalette_meta). The text and / or graphic metadata is then stored in the sample icon field 380 in the new palette entry 332 .

[0110] The palette samples include two or more of the most prominent / most significant colors associated with the multiple selected 3D objects. Various techniques may be used to determine the palette metadata (cpalette_meta). In some embodiments, the SR engine 140 implements a regional sampling technique by capturing an image including the multiple selected 3D objects, quantizing / sampling the image to determine, for example, the five most dominant / most common color clusters, and returning the five most dominant / most common color clusters as multiple colors representing the multiple selected 3D objects. The SR engine 140 then generates the palette metadata (cpalette_meta) based on the returned five most dominant / most common color clusters. In other embodiments, the SR engine 140 returns different numbers of the most dominant / most common color clusters as multiple colors representing the multiple selected 3D objects. In other embodiments, the SR engine 140 implements another type of technique to determine the palette metadata (cpalette_meta).

[0111] In addition, the SR engine 140 may provide additional suggestions for palette samples based on the current palette sample via the palette suggestion window 920. The palette suggestion window 920 may include a plurality of suggested images 930 (such as 930a, 930b, 930c, etc.) for additional palette sampling. Specifically, the SR engine 140 may initiate or perform an image search on an image server (such as server 190) that stores a collection of stock images / photos. For example, a search may be performed to identify stock images having a plurality of colors similar to the current palette sample. For each identified image, the SR engine 140 may then retrieve the image from the image server 190, store the image as a sampling suggestion 152 in the memory unit 104, and display the image as a suggested image 930 in the palette suggestion window 920. If the user then wishes to sample a particular suggested image 930, the user may select the particular suggested image 930 and select the "Get Sample" button 910. In response, the SR engine 140 captures / samples the palette components associated with the proposed image 930 by generating a new palette entry 332 in the palette section 330 of the SPDS 150 that represents the new palette sample.

[0112] In some embodiments, the SR engine 140 also provides system-initiated suggestions for additional object-based sampling of 3D digital components within the current IE scene 174 of the sampling immersive environment 132. The SR engine 140 may generate and display various suggestion pop-ups within the sampling immersive environment 132 containing various suggestions for additional samples based on the current sample being acquired and / or the user's interaction with a particular 3D object within the sampling immersive environment 132. For example, the SR engine 140 may generate suggestions for additional samples based on the currently selected 3D object.

[0113] Fig.10 According to various embodiments Figure 1 1020 are screen shots of suggestion windows 1010 and 1020 displayed in the sampling immersive environment 132 of FIG. As shown, the first suggestion window 1010 displays suggestions for sampling additional 3D objects, textures, and color schemes by displaying visual icons of the 3D objects, textures, and color schemes. In other embodiments, the first suggestion window 1010 also displays suggestions for sampling additional animations, actions, and physical parameters (not shown). Fig.10 In the example of FIG. 1 , the user selects a fox object, and in response, the SR engine 140 generates sample suggestions in the first suggestion window 1010 based on the selected fox object.

[0114] For example, the SR engine 140 may initiate or perform a text-based object identifier search in the IE database 180 based on the search term "fox" to identify the 3D objects that are most relevant to the selected fox object (such as, the three most relevant 3D objects). For each identified 3D object, the SR engine 140 retrieves the entire object metadata of the 3D object, stores the object metadata of the 3D object as a sampling suggestion 152 in the memory unit 104, and also parses the object metadata into separate object property metadata for different object properties of the 3D object, including metadata for textures, color schemes, animations, motions, and physical parameters. For each identified 3D object, the SR engine 140 may then generate an object icon for the 3D object and an object property icon for each of the object properties based on the retrieved metadata, and display the icons in the first suggestion window 1010. The user may then select a specific icon in the first suggestion window 1010 for sampling the 3D object or object property corresponding to the selected icon. In a manner similar to generating information about Figure 5 In the manner described above of a new sample of a 3D object or an object property of a 3D object, a new sample entry representing the new object-based sample may be generated and stored in SPDS 150 to capture the new object-based sample. The object property sampled in this manner may also be associated with the currently selected 3D object (such as a fox object), as indicated in the associated object field 350 of the new sample entry.

[0115] In other embodiments, the SR engine 140 may also provide standard default sampling suggestions for specific object attributes based on the currently selected 3D object (such as a fox object). The standard default sampling suggestions may be stored in the memory unit 104 as sampling suggestions 152. As shown, the second suggestion window 1020 displays suggestions for sampling standard default animations, actions, and physical parameters by displaying visualization icons of standard default animations, actions, and physical parameters. In other embodiments, the second suggestion window 1020 also displays suggestions for sampling standard default textures and color schemes by displaying visualization icons (not shown) of standard default textures and color schemes. It should be noted that if the currently selected 3D object is a non-moving object, such as a chair or a bucket, standard default animations and actions will not be suggested. Standard default animations and actions include animations and actions that are typically applied to movable characters (such as fox objects). Standard default physical parameters may include exaggerated / extreme physical parameters that characterize specific objects, such as anvils (high quality), balloons (buoyancy, lightness), footballs (high elasticity), and ice cubes (low friction). The user can then select a specific icon in the second suggestion window 1020 for sampling the standard default object properties corresponding to the selected icon. Figure 5In the manner described above, a new sample entry representing a new object-based sample may be generated and stored in SPDS 150 to capture the new object-based sample. The standard default object properties sampled in this manner may also be associated with the currently selected 3D object (such as a fox object), as indicated in the associated object field 350 of the new sample entry.

[0116] Sampling Palette User Interface

[0117] At any time during the sampling phase or the remixing phase, the user can select the sample palette mode 420 from the mode UI 400 to view the samples currently stored in the SPDS 150. In response to the user's selection of the sample palette mode 420, the SR engine 140 generates and displays a sample palette user interface (SPUI) 166 for viewing and accessing the samples collected and organized in the SPDS 150. The SPUI 166 includes a sample collection menu UI and a plurality of collection UIs, each collection UI corresponding to a different type of sample. The SPUI 166 can be displayed directly in the current immersive environment (such as the sampling immersive environment 132 or the remixing immersive environment (discussed in detail below)).

[0118] Fig.11 Shown according to various embodiments Figure 1 160. As shown, the SPUI 166 initially displays the sample collection menu 1100, which includes an icon window 1110 and a plurality of selectable collections, including an object collection 1120, a texture collection 1130, a color scheme collection 1140, an animation collection 1150, an action collection 1160, a physical parameter collection 1170, and a monochrome and palette collection 1180. The icon window 1110 is initially empty and is populated with sample icons once a sample collection is selected.

[0119] If the user selects the object set 1120, the SR engine 140 generates and displays the object set UI. Fig.12 Shown according to various embodiments Figure 1166 of the SPUI 166. As shown, the object collection UI 1200 continues to display the icon window 1110 and the multiple selectable collections 1120 to 1180 displayed in the sample collection menu 1100. In addition, the SR engine 140 populates the icon window 1110 with zero or more object sample icons 1250 (such as 1250a, 1250b, 1250c, etc.) representing zero or more current object samples stored in the SPDS 150. Specifically, the SR engine 140 can access the object section 302 in the SPDS 150, which stores zero or more object entries 304, each object entry 304 representing an object sample. For each object entry 304, the SR engine 140 can access the sample icon field 380 to retrieve the graphical metadata used to render the object sample icon to visually represent the object sample. The SR engine 140 may then render the object sample icons based on the graphic metadata and display the object sample icons for each object sample stored in the SPDS 150 in the icon window 1110. The SR engine 140 may also display relevant text near each object sample icon, such as an object identifier and / or an IE identifier of the particular immersive environment 130 from which the object sample was captured (as specified in the sample identifier field 340).

[0120] If the user selects the texture set 1130, the SR engine 140 generates and displays a texture set UI. Fig.13 Shown according to various embodiments Figure 1 13. The SR engine 140 may further include a texture collection UI 1300 of the SPUI 166 of the SPDS 150. As shown, the texture collection UI 1300 continues to display the icon window 1110 and the plurality of selectable collections 1120 to 1180 displayed in the sample collection menu 1100. In addition, the SR engine 140 populates the icon window 1110 with zero or more texture sample icons 1350 (such as 1350a, 1350b, 1350c, etc.) representing zero or more current texture samples stored in the SPDS 150. Specifically, the SR engine 140 may access a texture section 306 in the SPDS 150, which stores zero or more texture entries 308, each texture entry 308 representing a texture sample. For each texture entry 308, the SR engine 140 may access a sample icon field 380 to retrieve graphical metadata for rendering a texture sample icon that visually represents the texture sample. SR engine 140 may then render texture sample icons based on the graphics metadata and display the texture sample icons for each texture sample stored in SPDS 150 in icon window 1110. SR engine 140 may also display relevant text, such as a texture sample identifier and / or an associated object (as specified in sample identifier field 340 and associated object field 350), adjacent to each texture sample icon.

[0121] If the user selects the color scheme set 1140 , the SR engine 140 generates and displays the color scheme set UI. Fig.14 Shown according to various embodiments Figure 1 14. The SR engine 140 displays a color scheme collection UI 1400 of the SPUI 166 of the SPDS 150. As shown, the color scheme collection UI 1400 continues to display the icon window 1110 and the plurality of selectable collections 1120 to 1180 displayed in the sample collection menu 1100. In addition, the SR engine 140 populates the icon window 1110 with zero or more color scheme sample icons 1450 (such as 1450a, 1450b, 1450c, etc.) representing zero or more current color scheme samples stored in the SPDS 150. Specifically, the SR engine 140 may access the color scheme section 310 in the SPDS 150, which stores zero or more color scheme entries 312, each color scheme entry 312 representing a color scheme sample. For each color scheme entry 312, the SR engine 140 may access the sample icon field 380 to retrieve graphical metadata for rendering a color scheme sample icon that visually represents the color scheme sample. The SR engine 140 may then render the color scheme sample icons based on the graphic metadata and display the color scheme sample icons for each color scheme sample stored in the SPDS 150 in the icon window 1110. The SR engine 140 may also display relevant text, such as a color scheme sample identifier and / or an associated object (as specified in the sample identifier field 340 and the associated object field 350), adjacent to each color scheme sample icon.

[0122] If the user selects the animation set 1150 , the SR engine 140 generates and displays the animation set UI. Fig.15 Shown according to various embodiments Figure 11500 of the SPUI 166 of the embodiment of the present invention. As shown, the animation set UI 1500 continues to display the icon window 1110 and the multiple selectable collections 1120 to 1180 displayed in the sample set menu 1100. In addition, the SR engine 140 fills the icon window 1110 with zero or more animation sample icons 1550 (such as 1550a, 1550b, etc.) representing zero or more current animation samples stored in the SPDS 150. Specifically, the SR engine 140 can access the animation section 314 in the SPDS 150, which stores zero or more animation entries 316, each animation entry 316 representing an animation sample. For each animation entry 316, the SR engine 140 can access the sample icon field 380 to retrieve the graphic metadata for rendering the animation sample icon that visually represents the animation sample. Then, the SR engine 140 can render the animation sample icon based on the graphic metadata and display the animation sample icon for each animation sample stored in the SPDS 150 in the icon window 1110. The SR engine 140 may also display relevant text adjacent to each animation sample icon, such as an animation sample identifier and / or an associated object (as specified in the sample identifier field 340 and the associated object field 350 ).

[0123] If the user selects the action set 1160, the SR engine 140 generates and displays an action set UI. Fig.16 Shown according to various embodiments Figure 1 166 of the SPUI 1600. As shown, the action set UI 1600 continues to display the icon window 1110 and multiple optional collections 1120 to 1180 displayed in the sample set menu 1100. In addition, the SR engine 140 fills the icon window 1110 with zero or more action sample icons 1650 (such as 1650a, 1650b, etc.) representing zero or more current action samples stored in the SPDS 150. Specifically, the SR engine 140 can access the action section 318 in the SPDS 150, which stores zero or more action entries 320, each of which represents an action sample. For each action entry 320, the SR engine 140 can access the sample icon field 380 to retrieve the graphic metadata for rendering the action sample icon that visually represents the action sample. Then, the SR engine 140 can render the action sample icon based on the graphic metadata and display the action sample icon for each action sample stored in the SPDS 150 in the icon window 1110. The SR engine 140 may also display relevant text adjacent to each action sample icon, such as an action sample identifier and / or an associated object (as specified in the sample identifier field 340 and the associated object field 350 ).

[0124] If the user selects the physical parameter set 1170 , the SR engine 140 generates and displays a physical parameter set UI. Fig.17 Shown according to various embodiments Figure 1 1700 of the SPUI 166 of the embodiment of the present invention. As shown, the physical parameter collection UI 1700 continues to display the icon window 1110 and the multiple selectable collections 1120 to 1180 displayed in the sample collection menu 1100. In addition, the SR engine 140 populates the icon window 1110 with zero or more physical parameter sample icons 1750 (such as 1750a, 1750b, etc.) representing zero or more current physical parameter samples stored in the SPDS 150. Specifically, the SR engine 140 can access the physical parameter section 322 in the SPDS 150, which stores zero or more physical parameter entries 324, each physical parameter entry 324 representing a physical parameter sample. For each physical parameter entry 324, the SR engine 140 can access the sample icon field 380 to retrieve text and / or graphical metadata for rendering a physical parameter sample icon that visually represents the physical parameter sample. The SR engine 140 may then render a physical parameter sample icon based on the text and / or graphical metadata and display the physical parameter sample icon for each physical parameter sample stored in the SPDS 150 in the icon window 1110. In some embodiments, the physical parameter sample icon 1750 includes text data that specifies the values ​​of the various physical parameters captured in the corresponding physical parameter sample (such as Par1_A1, Par2_A1, Par3_A1, Par4_A1, etc.). The SR engine 140 may also display related text near each physical parameter sample icon, such as a physical parameter sample identifier and / or an associated object (as specified in the sample identifier field 340 and the associated object field 350).

[0125] If the user selects the monochrome and palette set 1180, the SR engine 140 generates and displays the monochrome and palette set UI. Fig.18 Shown according to various embodiments Figure 11800 of the SPUI 166 of the embodiment of the present invention. As shown, the monochrome and palette collection UI 1800 continues to display the icon window 1110 and the multiple selectable collections 1120 to 1180 displayed in the sample collection menu 1100. In addition, the SR engine 140 populates the icon window 1110 with zero or more monochrome sample icons 1850 (such as 1850a, 1850b, etc.) representing zero or more current monochrome samples stored in the SPDS 150 and zero or more palette sample icons 1890 (such as 1890a, 1890b, etc.) representing zero or more current palette samples stored in the SPDS 150. The icon window 1110 also includes an optional "global selection" button 1892 for use during the remix stage.

[0126] Specifically, the SR engine 140 may access a monochrome section 326 in the SPDS 150, which stores zero or more monochrome entries 328, each monochrome entry 328 representing a monochrome sample. For each monochrome entry 328, the SR engine 140 may access a sample icon field 380 to retrieve graphical metadata for rendering a monochrome sample icon that visually represents the monochrome sample. The SR engine 140 may then render a monochrome sample icon based on the graphical metadata and display the monochrome sample icon for each monochrome sample stored in the SPDS 150 in the icon window 1110. The SR engine 140 may also display associated text, such as a monochrome sample identifier (as specified in the sample identifier field 340), near each monochrome sample icon.

[0127] Specifically, the SR engine 140 may access a palette section 330 in the SPDS 150 that stores zero or more palette entries 332, each of which represents a palette sample. For each palette entry 332, the SR engine 140 may access a sample icon field 380 to retrieve graphical metadata for rendering a palette sample icon that visually represents the palette sample. The SR engine 140 may then render the palette sample icon based on the graphical metadata and display the palette sample icon for each palette sample stored in the SPDS 150 in the icon window 1110. The SR engine 140 may also display relevant text, such as a palette sample identifier (as specified in the sample identifier field 340), near each palette sample icon.

[0128] In addition, each of the collection UIs 1200, 1300, 1400, 1500, 1600, 1700, and 1800 may provide functionality for managing and organizing samples stored in the SPDS 150. In some embodiments, each of the collection UIs allows a user to rename a sample, such as by clicking on a sample icon to rename the sample icon (providing the sample icon with a new sample identifier), which automatically modifies the sample identifier 340 in the corresponding sample entry stored in the SPDS 150. In other embodiments, each of the collection UIs allows a user to delete a sample, such as by selecting a sample icon to delete the sample icon, which automatically deletes the corresponding sample entry stored in the SPDS 150.

[0129] FIG. 19A to FIG. 19B A flow chart illustrating method steps for capturing a sample within a 3D immersive environment and viewing the sample in a SPUI according to various embodiments is provided. Figures 1 to 18 The method steps are described in detail with reference to a system of FIG. 1900 , but those skilled in the art will appreciate that the method steps may be performed in any order by any system. In some embodiments, method 1900 may be performed by Figure 1 The IE system 100 is executed by the computer system 106 (via the memory unit 104 and the processor 102).

[0130] The method 1900 begins with the SR engine 140 configuring (at step 1910) a sampling mode 410 for a particular sampled immersive environment 132 based on user input. The SR engine 140 receives user input to enter a sampling mode 410 for a particular immersive environment, referred to as the sampled immersive environment 132. In response, the SR engine 140 retrieves the selected sampled immersive environment 132 from the IE database 180 and launches the IE engine 110 to render and display the selected sampled immersive environment 132 on the IE headset 172. Specifically, the user input may include selecting the sampling mode 410 from the mode UI 400 and specifying a particular IE identifier 210 for the selected sampled immersive environment 132. The SR engine 140 may then identify an IE entry 230 corresponding to the IE identifier 210 and retrieve metadata from an associated object field 220 in the corresponding IE entry 230, the metadata including metadata for one or more 3D objects associated with the selected sampled immersive environment 130. The IE engine 110 then renders and displays the sampled immersive environment 132 including one or more associated 3D objects in the IE headset 172 based on the retrieved metadata. The SR engine 140 also generates and displays a sampling UI 500 within the sampled immersive environment 132 on the IE headset 172, the sampling UI 500 displaying selectable options for segmentation operations 510, whole object samples 520, texture samples 530, color scheme samples 540, animation samples 550, motion samples 560, physical parameter samples 570, monochrome samples 580, and palette samples 590. Using the IE controller 176, the user can explore the sampled immersive environment 132 and select a 3D digital component to be sampled / captured via the sampling UI 500.

[0131] At step 1920, the SR engine 140 segments a particular 3D object within the sampled immersive environment 132 based on the user input. The user input received by the SR engine 140 includes selecting a particular 3D object and selecting a segmentation operation 510 from the sampling UI 500. In response, the SR engine 140 performs a segmentation algorithm on the selected 3D object to separate the 3D object into two or more sub-parts. The SR engine 140 then adds each sub-part of the selected 3D object to the IE entry 230 corresponding to the sampled immersive environment 132 in the IE database 180 as a separate and independent 3D object associated with the sampled immersive environment 132. Each sub-part of the selected 3D object can now be selected as a 3D object in the same manner as any other 3D object within the sampled immersive environment 132.

[0132] At step 1930, the SR engine 140 captures at least one object-based sample of a 3D digital component within the sampling immersive environment 132 based on the user input. The user input received by the SR engine 140 may include a selection of a particular 3D object, and in response, the SR engine 140 retrieves object metadata (object_meta) associated with the selected 3D object from the IE database 18. The SR engine 140 then parses the object metadata into different types of object property metadata, including texture metadata (texture_meta), color scheme metadata (colorsch_meta), animation metadata (anim_meta), motion metadata (motion_meta), and physical parameter metadata (physical_meta). The SR engine 140 then highlights / indicates selectable options available to the selected 3D object in the sampling UI 500 based on the parsed metadata being found to be available to the selected 3D object. The user input received by the SR engine 140 may further specify at least one sample selection from the sampling UI 500, such as object sample 520, texture sample 530, color scheme sample 540, animation sample 550, motion sample 560, and / or physical parameter sample 570. The SR engine 140 then captures the at least one new sample by generating at least one new entry in the SPDS 150 for representing the at least one new sample and populating one or more data fields 340, 350, 360, 370, and / or 380 of the at least one new entry, such as with respect to Figure 5 In this manner, one or more new object-based samples of the selected 3D object may be generated and stored in SPDS 150, including any combination of object samples, texture samples, color scheme samples, animation samples, motion samples, and / or physical parameter samples.

[0133] At step 1940, the SR engine 140 identifies and displays one or more object-based sampling suggestions within the sampling immersive environment 132, for example, based on the 3D object currently selected in step 1930. The sample suggestions may include 3D objects identified in the IE database 180 as being related to the currently selected 3D object, object property components of the identified 3D objects, and / or standard default object property components related to the currently selected 3D object. The SR engine 140 may generate a visualization icon for each sample suggestion and display the visualization icon in a suggestion window (such as 1010 or 1020) within the sampling immersive environment 132, such as the icon for the sample suggestion. Fig.10 discussed.

[0134] At step 1950, the SR engine 140 captures an object-based sample of a sample suggestion within the sampling immersive environment 132 based on the user input. The user input received by the SR engine 140 may include a selection of a particular visualization icon displayed in the suggestion window. In response, the SR engine 140 may generate a new sample entry representing the new object-based sample and store it in the SPDS 150. The object properties sampled in this manner may also be associated with the currently selected 3D object, as indicated in the associated object field 350 of the new sample entry.

[0135] At step 1960, the SR engine 140 captures at least one color-based sample of the 3D digital component within the sampling immersive environment 132 based on the user input. The user input received by the SR engine 140 may include a selection of a monochrome sample 580 in the sampling UI 500 and a selection of a specific location / point within the sampling immersive environment 132. In response, the SR engine 140 captures the new monochrome sample by generating a new entry in the SPDS 150 to represent the new monochrome sample and populating one or more data fields 340, 350, 360, 370, and / or 380 of the new entry, such as with respect to the color sample. Figure 8 Alternatively or in addition, the user input received by the SR engine 140 may include a selection of a palette sample 590 in the sampling UI 500 and a selection of a plurality of 3D objects within the sampling immersive environment 132. In response, the SR engine 140 captures the new palette sample by generating a new entry in the SPDS 150 to represent the new palette sample and populating one or more data fields 340, 350, 360, 370, and / or 380 of the new entry, as described with respect to Fig. 9 described.

[0136] At step 1970, the SR engine 140 identifies and displays one or more color-based sample suggestions within the sampling immersive environment 132 based on the current color-based sample captured in step 1960. The sample suggestions may include images from the image server 190 identified as relevant to the current color-based sample.

[0137] At step 1972, the SR engine 140 captures a color-based sample from the sample suggestion based on the user input. The user input received by the SR engine 140 may include a selection of a monochrome sample 580 in the sampling UI 500 and a selection of a specific location / point within the suggested image. In response, the SR engine 140 captures a new monochrome sample based on the selected location / point by generating a new entry in the SPDS 150 to represent the new monochrome sample and populating one or more data fields 340, 350, 360, 370, and / or 380 of the new entry, such as with respect to Figure 8Alternatively, the user input received by the SR engine 140 may include a selection of a palette sample 590 in the sampling UI 500 and a selection of a suggested image. In response, the SR engine 140 captures a new palette sample based on the selected suggested image by generating a new entry in the SPDS 150 to represent the new palette sample and populating one or more data fields 340, 350, 360, 370, and / or 380 of the new entry, as described with respect to Fig. 9 described.

[0138] At step 1976, the SR engine 140 receives from the mode UI 400 a user selection of the sample palette mode 420 to view samples currently stored to the SPDS 150. The user may select the sample palette mode 420 from the mode UI 400 at any time during the sampling phase within the sampling immersive environment 132 or the remixing phase within the remixing immersive environment 134. In response, the SR engine 140 generates and displays a sample collection menu 1100, which includes an icon window 1110 and a plurality of selectable collections for viewing, including an object collection 1120, a texture collection 1130, a color scheme collection 1140, an animation collection 1150, an action collection 1160, a physical parameter collection 1170, and a monochrome and palette collection 1180. The icon window 1110 is initially empty and is populated with sample icons once a sample collection is selected.

[0139] At step 1978, the SR engine 140 receives a user selection of the object collection 1120 from the sample collection menu 1100. In response, the SR engine 140 generates and displays an object collection UI 1200 that displays zero or more object sample icons 1250 representing zero or more object samples stored in the SPDS 150 in the icon window 1110, as shown in FIG. Fig.12 To this end, the SR engine 140 may access zero or more object entries 304 stored in the object section 302 in the SPDS 150 and, for each object entry 304 , access the sample icon field 380 to retrieve graphical metadata for rendering and displaying the object sample icon in the icon window 1110 .

[0140] At step 1980, SR engine 140 receives a user selection of texture set 1130 from sample set menu 1100. In response, SR engine 140 generates and displays texture set UI 1300, which displays zero or more texture sample icons 1350 representing zero or more texture samples stored in SPDS 150 in icon window 1110, such as Fig.13To do this, the SR engine 140 may access zero or more texture entries 308 stored in the texture section 306 in the SPDS 150 and, for each texture entry 308 , access the sample icon field 380 to retrieve graphics metadata for rendering and displaying a texture sample icon in the icon window 1110 .

[0141] At step 1982, the SR engine 140 receives a user selection of a color scheme set 1140 from the sample set menu 1100. In response, the SR engine 140 generates and displays a color scheme set UI 1400 that displays zero or more color scheme sample icons 1450 representing zero or more color scheme samples stored in the SPDS 150 in the icon window 1110, such as Fig.14 To this end, the SR engine 140 may access zero or more color scheme entries 312 stored in the color scheme section 310 in the SPDS 150 and, for each color scheme entry 312 , access the sample icon field 380 to retrieve graphical metadata for rendering and displaying the color scheme sample icon in the icon window 1110 .

[0142] At step 1984, the SR engine 140 receives a user selection of an animation set 1150 from the sample set menu 1100. In response, the SR engine 140 generates and displays an animation set UI 1500 that displays zero or more animation sample icons 1550 representing zero or more animation samples stored in the SPDS 150 in the icon window 1110, such as Fig.15 To this end, the SR engine 140 may access zero or more animation entries 316 stored in the animation section 314 in the SPDS 150 , and for each animation entry 312 , access the sample icon field 380 to retrieve the graphic metadata for rendering and displaying the animation sample icon in the icon window 1110 .

[0143] At step 1986, the SR engine 140 receives a user selection of an action set 1160 from the sample set menu 1100. In response, the SR engine 140 generates and displays an action set UI 1600 that displays zero or more action sample icons 1650 representing zero or more action samples stored in the SPDS 150 in the icon window 1110, such as Fig.16 To this end, the SR engine 140 may access zero or more action entries 320 stored in the action section 318 in the SPDS 150 and, for each action entry 320 , access the sample icon field 380 to retrieve graphical metadata for rendering and displaying the action sample icon in the icon window 1110 .

[0144] At step 1988, the SR engine 140 receives a user selection of a physical parameter set 1170 from the sample set menu 1100. In response, the SR engine 140 generates and displays a physical parameter set UI 1700 that displays zero or more physical parameter sample icons 1750 representing zero or more physical parameter samples stored in the SPDS 150 in the icon window 1110, such as Fig.17 To this end, the SR engine 140 may access zero or more physical parameter entries 324 stored in the physical parameter section 322 in the SPDS 150 and, for each physical parameter entry 324 , access the sample icon field 380 to retrieve text and / or graphic metadata for rendering and displaying the physical parameter sample icon in the icon window 1110 .

[0145] At step 1990, the SR engine 140 receives a user selection of a monochrome and palette set 1180 from the sample set menu 1100. In response, the SR engine 140 generates and displays a monochrome and palette set UI 1800 that displays zero or more monochrome icons 1850 representing zero or more monochrome samples and zero or more palette sample icons 1890 representing zero or more palette samples stored in the SPDS 150 in the icon window 1110, as shown in FIG. Fig.18 To this end, the SR engine 140 may access zero or more monochrome entries 328 in the monochrome section 326 stored in the SPDS 150 and zero or more palette entries 332 in the palette section 330 stored in the SPDS, and for each entry 328 or 332, access the sample icon field 380 to retrieve the graphic metadata for rendering and displaying the monochrome sample icon 1850 or the palette sample icon 1890 in the icon window 1110.

[0146] At step 1992, the SR engine 140 reuses the sampled 3D digital components (stored as samples in the SPDS 150) within the remixed immersive environment 134 based on the user input. As discussed below, reusing the sampled 3D digital components may include, among other things, associating the sampled 3D objects with the remixed immersive environment 134 to modify the remixed immersive environment 134 to generate a new immersive environment, or replacing object properties of a 3D object with the sampled 3D digital components to generate a new / modified 3D object.

[0147] Remix: Sample reuse and remixing within the immersive environment

[0148] In some embodiments, the SPUI 166 may be used to access and view samples stored in the SPDS 150 to reuse / apply the samples in the remixed immersive environment 134. For example, reusing a sample may include adding an object sample of a 3D object from a first immersive environment to a second immersive environment (the remixed immersive environment 134) to generate a new third immersive environment (as described below with respect to Figure 21 to Figure 23 As another example, reusing samples may include using samples from SPDS 150 to modify object properties of a 3D object to generate a new / modified 3D object, which is referred to as "remixing" (as discussed below with respect to Figure 24 to Figure 28 As another example, reusing samples may include using palette samples from SPDS 150 to modify color scheme properties of multiple 3D objects of the IE scene to generate new / modified multiple 3D objects of the new IE scene, which is also referred to as "remixing" (as discussed below with respect to Figure 29 to Figure 30 discussed).

[0149] At any time, to enter the reuse / remix phase, the user may select the remix mode 430 from the mode UI 400 displayed in the IE scene 174. In response to the user's selection of the remix mode 430, the SR engine 140 (stored in the memory unit 104 and by Figure 1 The processor 102 executes) generates and displays a remix IE selection menu to select the remix immersive environment 134. Fig. 20 According to various embodiments Figure 1 10 is a screen shot of a remix IE selection menu 2000 displayed in a sampled immersive environment 132 of FIG. As shown, the remix IE selection menu 2000 displays options for selecting a default remix IE 2010 or a specific user-selected remix IE 2020. In response to the selection of a remix immersive environment 134 (either a default or a user-selected remix immersive environment 134), the SR engine 140 retrieves the selected remix immersive environment 134 from the IE database 180 and starts the IE engine 110 to render and display the remix immersive environment 134 on the IE headset 172. In some embodiments, the remix immersive environment 134 is different from the sampled immersive environment 132. In other embodiments, the remix immersive environment 134 may be the same as the sampled immersive environment 132.

[0150] The default remix immersive environment 134 may include an immersive environment stored in the IE database 180, which includes a normal / simple immersive environment with minimal 3D objects. For example, if the user wishes to modify a specific 3D object using a sample from the SPDS 150 within the normal / simple remix immersive environment 134, the user may select the default remix IE 2010. The user-selected remix IE 2020 includes a specific immersive environment stored in the IE database 180, which the user wishes to select as the remix immersive environment 134. For example, if the user wishes to modify the selected remix immersive environment 134 to generate a new immersive environment that can be stored in the IE database 180, the user may select the user-selected remix IE 2020. It should be noted that the user may also modify a specific 3D object using a sample from the SPDS 150 within the user-selected remix immersive environment 134.

[0151] As follows about Figures 21 to 23 As described above, the user may add the sampled 3D object to the initial remixed immersive environment 134 to modify the remixed immersive environment 134 , thereby generating a new remixed immersive environment 134 . Fig.21 According to various implementation schemes Figure 1 Screenshot of the remixed immersive environment 134. Fig.21 In the example of , the remixed immersive environment 134 includes a user-selected remixed immersive environment 134 that includes a simulated lighthouse environment. As shown, the remixed immersive environment 134 may include one or more native 3D objects 2110 (such as 2110a, 2110b, 2110c, etc.), which include objects such as building / structure objects and landscape objects (boulders, trees, vegetation, etc.). As used herein, the native 3D objects of the remixed immersive environment 134 include 3D objects included in the original / initial remixed immersive environment 134 before the remixed immersive environment 134 is modified via samples from SPDS 150. In some embodiments, the native 3D objects are not sampled objects and are not derived from sampled samples.

[0152] As shown, the SR engine 140 also generates and displays a sample set menu 1100 of the SPUI 166 within the remix immersive environment 134. Fig.11As discussed, the sample collection menu 1100 includes an icon window 1110 and a plurality of selectable collections for viewing, the plurality of selectable collections including an object collection 1120, a texture collection 1130, a color scheme collection 1140, an animation collection 1150, an action collection 1160, a physical parameter collection 1170, and a monochrome and palette collection 1180 (not shown). To add a 3D object to the initial remixed immersive environment 134, the user may select an object collection 1120 from the sample collection menu 1100. In response, the SR engine 140 generates and displays the object collection UI 1200 and populates the icon window 1110 with zero or more object sample icons 1250 (such as 1250a, 1250b, 1250c, etc.) representing zero or more current object samples stored in the SPDS 150, as shown in FIG. Fig.12 The user may then interact with the object collection UI 1200 to add the sampled 3D objects to the remixed immersive environment 134 .

[0153] Fig. 22 is added to according to various embodiments Figure 1 Screen shot of a sampled object of the remixed immersive environment 134. Fig. 22 In the example of , the user selected the fox icon 1250a corresponding to the fox object sample of the fox object 2210 stored in the SPDS 150 from the object collection UI 1200. The user has dragged the fox icon 1250a into the remix immersive environment 134 (as indicated by the dashed arrow), indicating that the user wishes to add the sampled fox object to the remix immersive environment 134. In response, the SR engine 140 retrieves the object entry 304 corresponding to the selected fox icon 1250a in the SPDS 150, and renders and displays the fox object 2210 within the remix immersive environment 134 based on the sample metadata 360 stored in the retrieved object entry 304.

[0154] In response to the user dragging the fox icon 1250a into the remixed immersive environment 134, the SR engine 140 may also store the fox object 2210 as a new object associated with the remixed immersive environment 134 stored in the IE database 180. The SR engine 140 may do this by accessing the IE entry 230 corresponding to the remixed immersive environment 134 in the IE database 180 and adding the object identifier and fox object metadata (object_meta) of the fox object to the associated object field 220 in the corresponding IE entry 230. By doing so, the initial remixed immersive environment 134 is modified to generate a new remixed immersive environment 134 including the new associated fox object 2210. In this way, the 3D object sampled from within the first immersive environment may be reused (added to the second immersive environment) in the second immersive environment to modify the second immersive environment, thereby generating a new third immersive environment. It should be noted that the second immersive environment is different from the first immersive environment.

[0155] Fig.23 is added to according to various embodiments Figure 1 Screen shot of an additional sampled object in the remixed immersive environment 134. Fig.23 In the example of , the user includes additional 3D objects 2310 (such as 2310a, 2310b, 2310c) into the remixed immersive environment 134 by selecting the corresponding object icon 1250 from the object collection UI 1200 and dragging it into the remixed immersive environment 134. In response, the SR engine 140 retrieves the object entry 304 corresponding to the selected icon 1250 from the SPDS 150, and renders and displays the additional object 2310 within the remixed immersive environment 134 based on the sample metadata 360 in the corresponding object entry 304. The SR engine 140 may also store the additional object 2310 as a new object associated with the remixed immersive environment 134 in the corresponding IE entry 230 stored in the IE database 180 to further modify the remixed immersive environment 134.

[0156] In some embodiments, samples from SPDS 150 may be used to modify object properties of 3D objects displayed within remix immersive environment 134 to generate new 3D objects, which is referred to as "remixing." For example, to modify a specific object property of a 3D object, a user may select a collection UI corresponding to the specific object property from sample collection menu 1100 to view a sample icon of samples of the specific object property currently stored in SPDS 150. The user may then select the specific sample icon and drag it onto the 3D object to replace the current object property of the 3D object with the sampled object property corresponding to the selected sample icon.

[0157] Fig.24According to various embodiments Figure 1 A screenshot of a first object selected for remixing within the remix immersive environment 134 of FIG. Fig.24 In the example of FIG. 1 , the remixed immersive environment 134 includes Fig. 22 The sampled fox object 2210. In other embodiments, the fox object 2210 may include a native 3D object that does not include the sampled object and is not derived from the sampled object. Fig.24 In the example of FIG. 1 , a user has selected a texture collection 1130 from the collection menu 1100, and in response, the SR engine 140 has generated and displayed a texture collection UI 1300 showing zero or more texture sample icons 1350 representing zero or more texture samples / entries 308 stored in the SPDS 150, as shown in FIG. Fig.13 The user may then interact with the texture collection UI 1300 to modify the current texture properties associated with the selected fox object 2210 within the remixed immersive environment 134. As shown, the user has selected a first texture icon 1350a corresponding to a first texture sample of a first texture stored in the SPDS 150 from the texture collection UI 1300. The user has also dragged the first texture icon 1350a onto the fox object 2210 in the remixed immersive environment 134 (as indicated by the dashed arrow), indicating that the user wishes to transfer the first texture to the fox object 2210 (replacing the current texture of the fox object 2210 with the first texture).

[0158] In response, the SR engine 140 retrieves the texture entry 308 corresponding to the first texture icon 1350a in the SPDS 150, and retrieves the texture metadata (texture_meta) in the sample metadata field 360 of the texture entry 308. The SR engine 140 then replaces the current texture metadata associated with the fox object 2210 (stored in the associated object field 220 of the IE entry 230 corresponding to the remixed immersive environment 134 in the IE database 180) with the retrieved texture metadata (texture_meta) of the first texture. FIG. 25A to FIG. 25B According to various implementation schemes Figure 1 A close-up screenshot of a remixed first object within the remixed immersive environment 134 . Fig.25A Shows Fig. 22 An initial fox object 2210 with initial texture attributes. Fig.25A A modified fox object 2510 is shown having a first texture corresponding to the first texture icon 1350a after applying the texture sample.

[0159] Similarly, the user may interact with other types of collection UIs to change other types of object properties of 3D objects within the remixed immersive environment 134. To modify the color scheme properties of the 3D objects, the user may select a color scheme collection 1140 from the sample collection menu 1100. In response, the SR engine 140 generates and displays the color scheme collection UI 1400, which shows zero or more color scheme sample icons 1450 representing zero or more color scheme samples stored in the SPDS 150, such as the color scheme sample icons 1450 for the 3D objects. Fig.14 The user may then select the first color scheme icon 1450 corresponding to the first color scheme sample of the first color scheme stored in the SPDS 150 and drag the first color scheme icon 1450 onto the selected 3D object within the remixed immersive environment 134. In response, the SR engine 140 retrieves the first color scheme entry 328 corresponding to the first color scheme icon 1450 in the SPDS 150 and retrieves the color scheme metadata (colorsch_meta) in the sample metadata field 360 of the first color scheme entry 328. The SR engine 140 then replaces the current color scheme metadata associated with the selected 3D object (stored in the associated object field 220 of the IE entry 230 corresponding to the remixed immersive environment 134 in the IE database 180) with the retrieved color scheme metadata (colorsch_meta) of the first color scheme. In this manner, the color scheme attributes of the selected 3D object will be replaced by the first color scheme sample to generate a new / modified 3D object.

[0160] Likewise, to modify the animation properties of the 3D object, the user may select an animation set 1150 from the sample set menu 1100. In response, the SR engine 140 generates and displays an animation set UI 1500 showing zero or more animation sample icons 1550 representing zero or more animation samples stored in the SPDS 150, such as the animation sample icons 1550 and the animation sample icons 1550. Fig.15As described. The user may then select the first animation icon 1550 corresponding to the first animation sample of the first animation stored in the SPDS 150, and drag the first animation icon 1550 onto the selected 3D object within the remixed immersive environment 134. In response, the SR engine 140 retrieves the first animation entry 316 corresponding to the first animation icon 1550 in the SPDS 150, and retrieves the animation metadata (anim_meta) in the sample metadata field 360 of the first animation entry 316. The SR engine 140 then replaces the current animation metadata associated with the selected 3D object (stored in the associated object field 220 of the IE entry 230 corresponding to the remixed immersive environment 134 in the IE database 180) with the retrieved animation metadata (anim_meta) of the first animation. In this way, the animation properties of the selected 3D object will be replaced by the first animation sample to generate a new / modified 3D object.

[0161] Likewise, to modify the action properties of the 3D object, the user may select an action set 1160 from the sample set menu 1100. In response, the SR engine 140 generates and displays an action set UI 1600 showing zero or more action sample icons 1650 representing zero or more action samples stored in the SPDS 150, such as the action sample icons 1650 and the action sample icons 1650. Fig.16 As described. The user may then select the first action icon 1650 corresponding to the first action sample of the first action stored in the SPDS 150, and drag the first action icon 1650 onto the selected 3D object within the remixed immersive environment 134. In response, the SR engine 140 retrieves the first action entry 320 corresponding to the first action icon 1650 in the SPDS 150, and retrieves the action metadata (motion_meta) in the sample metadata field 360 of the first action entry 320. The SR engine 140 then replaces the current action metadata associated with the selected 3D object (stored in the associated object field 220 of the IE entry 230 corresponding to the remixed immersive environment 134 in the IE database 180) with the retrieved action metadata (motion_meta) of the first action. In this way, the action attributes of the selected 3D object will be replaced by the first action sample to generate a new / modified 3D object.

[0162] Likewise, to modify the physical parameter properties of the 3D object, the user may select a physical parameter set 1170 from the sample set menu 1100. In response, the SR engine 140 generates and displays a physical parameter set UI 1700 showing zero or more physical parameter sample icons 1750 representing zero or more physical parameter samples stored in the SPDS 150, as shown in FIG. Fig.17As described. The user may then select the first physical parameter icon 1750 corresponding to the first physical parameter sample of the first physical parameter stored in the SPDS 150, and drag the first physical parameter icon 1750 onto the selected 3D object within the remixed immersive environment 134. In response, the SR engine 140 retrieves the first physical parameter entry 324 corresponding to the first physical parameter icon 1750 in the SPDS 150, and retrieves the physical parameter metadata (physical_meta) in the sample metadata field 360 of the first physical parameter entry 324. The SR engine 140 then replaces the current physical parameter metadata associated with the selected 3D object (stored in the associated object field 220 of the IE entry 230 corresponding to the remixed immersive environment 134 in the IE database 180) with the retrieved physical parameter metadata (physical_meta) of the first physical parameter. In this manner, the physical parameter attributes of the selected 3D object will be replaced by the first physical parameter sample to generate a new / modified 3D object.

[0163] In addition, the color scheme properties of the 3D object may also be modified using the single color samples stored in the SPDS 150. In these embodiments, the user may select a single color and palette set 1180 from the sample set menu 1100. In response, the SR engine 140 generates and displays a single color and palette set UI 1800 showing zero or more single color sample icons 1850 representing zero or more single color samples stored in the SPDS 150, such as the single color sample icons 1850 and the single color sample icons 1850. Fig.18 The user may then select the first monochrome icon 1850 corresponding to the first monochrome sample of the first monochrome stored in the SPDS 150 and drag the first monochrome icon 1850 onto the selected 3D object within the remixed immersive environment 134. In response, the SR engine 140 retrieves the first monochrome entry 328 corresponding to the first monochrome icon 1850 in the SPDS 150 and retrieves the monochrome metadata (scolor_meta) in the sample metadata field 360 of the first monochrome entry 328. The SR engine 140 then replaces the current color scheme metadata associated with the selected 3D object (stored in the associated object field 220 of the IE entry 230 corresponding to the remixed immersive environment 134 in the IE database 180) with the retrieved monochrome metadata (scolor_meta) of the first monochrome. In this manner, the color scheme attributes of the selected 3D object will be replaced by the first monochrome sample to generate a new / modified 3D object.

[0164] In some embodiments, one or more object properties of a first 3D object may be modified using a plurality of sampled object properties of a second 3D object. For example, the remixed immersive environment 134 may include a first 3D object and a second 3D object. A user may select the second 3D object and drag it onto the first 3D object, indicating that the user wishes to transfer one or more object properties of the second 3D object to the first 3D object. In response, the SR engine 140 may generate and display a transferable property UI that displays one or more object property samples of the second 3D object that are currently available and stored in the SPDS 150. The user may then interact with the transferable property UI to select one or more object properties of the second 3D object and transfer them to the first 3D object.

[0165] Fig.26 is selected for use with Figure 1 A screenshot of a first object remixed with a second object within the remixed immersive environment 134. Fig.26 In the example of , the remixed immersive environment 134 includes a first object 2610 (a robot object) and a second object 2620 (a fox object). The first object 2160 may include a native object of the remixed immersive environment 134 or a sampled object added to the remixed immersive environment 134 via the SPUI 166. The second object 2260 includes a sampled object stored in the SPDS 150, which is added to the remixed immersive environment 134 via the SPUI 166, and the second object 2260 has one or more associated sampled object properties also stored in the SPDS 150.

[0166] As shown, the user has selected and dragged the second object 2620 onto the first object 2610 (as indicated by the dashed arrow), indicating that the user wishes to transfer one or more object properties of the second object 2620 to the first object 2610. In response, the SR engine 140 may generate and display a transferable properties UI 2600 that displays one or more object property samples of the second object 2620 that are currently available and stored in the SPDS 150. To do so, the SR engine 140 may retrieve all object property entries of the object property samples associated with the second object 2620 in the SPDS 150, for example, by searching for the object identifier of the second object 2620 in the associated object field 350 of the entry in the SPDS 150. In some embodiments, the transferable properties UI 2600 displays only the object property samples of the second object 2620 that were captured separately and stored in the SPDS 150 (separate from the object samples of the second object 2620).

[0167] exist Fig.26In the example of , it is determined that all different types of object property samples of the second object 2620 are individually captured and stored in the SPDS 150, and therefore the transferable property UI 2600 displays texture samples 2630, color scheme samples 2640, animation samples 2650, motion samples 2660, and physical parameter samples 2670 as available for use. In other embodiments, only a subset of the different types of object property samples of the second object 2620 are individually captured and stored in the SPDS 150, and therefore the transferable property UI 2600 will only display a subset of the different types of samples as available for use. The transferable property UI 2600 also displays an optional "Apply Transfer" button 2690.

[0168] The user may then select one or more object properties of the second object 2620 to transfer to the first object 2610 by selecting one or more available samples from the transferable properties UI 2600. As shown, the user selects to transfer the animation and motion object properties of the second object 2620 by selecting animation sample 2650 and motion sample 2660 (as indicated by bold text). In other embodiments, the user selects a different set of available samples from the transferable properties UI 2600. The user may then select an "Apply Transfer" button 2690 to initiate the transfer process. In response, the SR engine 140 uses the information described above with respect to the transfer process. Fig.24 , Fig.25A and Fig.25B The remix and transfer operation discussed transfers one or more object properties of the second object 2620 to the first object 2610 .

[0169] Fig. 27 is transferred to Fig.26 A screenshot of the animation properties of the first object within the remixed immersive environment 134. In some embodiments, the SR engine 140 may display a visualization 2710 of the transfer of object properties between the second object 2620 and the first object 2610. Fig. 27 In the example of , visualization 2710 of an animated property transfer between first object 2610 and second object 2620 is displayed. In some embodiments, visualization 2710 may include sample icons corresponding to samples of the transferred object properties.

[0170] Fig.28 is transferred to Fig. 27 2610. In some embodiments, the SR engine 140 displays an indication within the remixed immersive environment 134 when the transfer of properties to the first object 2610 is complete. Fig.28 In the example of , the first object 2610 is displayed in a dimmed or highlighted form to indicate that the animation property transfer is completed.

[0171] It should be noted that the 3D objects in the remixed immersive environment 134 modified using the samples in the SPDS 150 may include new 3D objects associated with the remixed immersive environment 134 (in the associated object field 220 of the IE entry 230 corresponding to the remixed immersive environment 134 in the IE database 180). For example, the modified fox object 2510 includes a new fox object 2510, and the modified robot object 2810 includes a new robot object 2810, which are associated with the remixed immersive environment 134. Therefore, generating a new fox object 2510 associated with the remixed immersive environment 134 and / or generating a new robot object 2810 associated with the remixed immersive environment further generates a new remixed immersive environment 134 with new associated objects. Thus, using the samples stored in the SPDS 150 to modify the object properties of the 3D objects can be used to generate new / modified 3D objects and a new / modified remixed immersive environment 134. In addition, any new / modified 3D objects generated using samples in SPDS 150 may also be sampled and added as whole object samples to SPDS 150. For example, a new fox object 2510 and / or a new robot object 2810 may be sampled to generate a new fox object sample and / or a new robot object sample in SPDS 150.

[0172] In some embodiments, the color scheme properties of multiple 3D objects of the IE scene 174 can be modified using the palette samples stored in the SPDS 150. In these embodiments, the user can select a palette and a palette set 1180 from the sample set menu 1100. In response, the SR engine 140 generates and displays a palette and palette set UI 1800, which shows a selectable "global selection" button 1892 and zero or more palette sample icons 1890 representing zero or more palette samples stored in the SPDS 150, such as the color scheme properties of the IE scene 174. Fig.18 The user may then select two or more 3D objects within the remixed immersive environment 134 , or select a “global selection” button 1892 to select all 3D objects displayed in the current IE scene 174 of the remixed immersive environment 134 .

[0173] The user may then select the first palette icon 1890 corresponding to the first palette sample of the first palette stored in the SPDS 150 and drag the first palette icon 1890 onto any selected 3D object within the remixed immersive environment 134. In response, the SR engine 140 retrieves the first palette entry 332 corresponding to the first palette icon 1890 in the SPDS 150 and retrieves the palette metadata (cpalette_meta) in the sample metadata field 360 of the first palette entry 332. It should be noted that the palette metadata specifies two or more different colors defining the first palette. The SR engine 140 then replaces the current palette metadata (stored in the associated object field 220 of the IE entry 230 corresponding to the remixed immersive environment 134 in the IE database 180) associated with the two or more selected 3D objects based on the retrieved palette metadata (cpalette_meta) of the first palette. For example, the SR engine 140 may achieve this by randomly replacing the color schemes of the two or more selected 3D objects with two or more different colors defining the first color palette. In this way, the color scheme properties of the two or more selected 3D objects may be replaced based on multiple colors of the first color palette samples to generate two or more new / modified 3D objects and a new / modified remixed immersive environment 134.

[0174] Fig.29 According to various implementation schemes Figure 1 Screen shot of palette remixing of multiple objects initiated within the remix immersive environment 134. Fig.29 In the example of , the remixed immersive environment 134 includes a user-selected remixed immersive environment 134, which includes a simulated office park environment. As shown, the remixed immersive environment 134 includes a plurality of 3D objects 2910 (such as 2910a, 2910b, 2910c, etc.), which include objects such as building / structure objects, landscape objects (boulders, trees, vegetation, etc.) and furniture objects. The SR engine 140 has generated and displayed the palette and palette set UI 1800 and at least the first palette sample icon 1890a representing the first palette sample stored in the SPDS 150.

[0175] exist Fig.29 In the example of FIG. 1 , the user has selected the “global selection” button 1892 to select all 3D objects 2910 displayed in the current IE scene 174 of the remixed immersive environment 134. The user has also selected the first palette icon 1890a and dragged it onto the selected 3D object within the remixed immersive environment 134, which initiates the transfer of the first palette to all objects in the current IE scene 174 of the remixed immersive environment 134. Fig.30 According to various implementation schemes Fig.29 14. A screenshot of a full palette remix of multiple objects within the remixed immersive environment 134. As shown, in response to the above user input, the SR engine 140 replaces the color scheme associated with all 3D objects displayed in the current IE scene 174 based on the multiple colors defined in the first palette sample (such as by randomly replacing the color scheme of the 3D objects with two or more different colors of the first palette). As shown, the remixed immersive environment 134 includes multiple new / modified 3D objects 3010 (such as 3010a, 3010b, 3010c, etc.).

[0176] In some embodiments, the SR engine 140 also provides a "revisit" function during the remix stage. When a selection is made for a particular sampled 3D object displayed within the remix immersive environment 134, the revisit function allows the user to view the sampled immersive environment 132 from which the selected 3D object was originally sampled. In some embodiments, the revisit function can be mapped to a specific button on the IE controller 176 to allow the user to easily access the revisit function at any time during the remix stage.

[0177] Figure 31 to Figure 33 are screen shots showing the operation of the revisit function. Fig.31 is added to according to various embodiments Figure 1 A screenshot of a sampled chair object 3110 of the remixed immersive environment 134. Fig.31 In the example of , the chair object 3110 was previously sampled from a particular sampling immersive environment 134. As discussed above, when an object sample of a 3D object is captured within the sampling immersive environment 134, an object entry 304 of the object sample is generated and stored in the SPDS 150. The object entry 304 includes a context field 370 for storing context information specifying the sampling position of the 3D object. Specifically, the context field 370 includes the IE identifier 210 of the particular sampling immersive environment 132 from which the 3D object was sampled, the 3D position coordinates of the 3D object within the particular sampling immersive environment 132 when the 3D object was sampled, and the 3D position coordinates of the user's viewpoint within the particular sampling immersive environment 132 when the 3D object was sampled.

[0178] exist Fig.31 In the example of FIG. 1 , the user adds the chair object 3110 to the remixed immersive environment 134 via interaction with the object collection UI 1200, as shown in FIG. Fig. 22The user also selects a revisit function for the chair object 3110 (e.g., by selecting the chair object 3110 and selecting a specific button on the IE controller 176 that is mapped to the revisit function). In response, the SR engine 140 retrieves the object entry 304 corresponding to the chair object 3110 stored in the SPDS 150, and retrieves the context information from the context field 370 of the object entry 304. The context field 370 specifies the IE identifier 210 of the specific sampled immersive environment 132 from which the chair object 3110 was sampled, and the 3D position coordinates of the chair object 3110 and the user's viewpoint within the specific sampled immersive environment 132 when the chair object 3110 was sampled.

[0179] The SR engine 140 then starts the IE engine 110 to render and display at least a portion of the identified sampled immersive environment 132 within the current IE scene 174 based on the retrieved context information in the context field 370. To this end, the IE engine 110 may retrieve the IE entry 230 corresponding to the identified sampled immersive environment 132 (based on the IE identifier field 210), and render and display the identified sampled immersive environment 132 based on the metadata in the associated object field 220. The IE engine 110 may also render and display the chair object 3110 at a specific user viewpoint within the identified sampled immersive environment 132 based on the 3D position coordinates of the chair object 3110 and the user viewpoint when the chair object 3110 was sampled, as further specified in the context field 370.

[0180] In some embodiments, the revisit functionality provides a “preview” of the identified sampled immersive environment 132 , where only a sub-portion of the remixed immersive environment 134 of the current IE scene 174 is overlaid by a small portion of the identified sampled immersive environment 132 . Fig.32 According to various embodiments, the Fig.31 31. A screenshot of the "preview" revisit function of the sampled chair object 3110 of FIG. 31. As shown, the sub-portion of the identified sampled immersive environment 132 is rendered and displayed within the sub-portion 3120 of the remixed immersive environment 134 of the current IE scene 174. Therefore, in the IE scene 174 currently displayed in the IE headset 172, both the remixed immersive environment 134 and the identified sampled immersive environment 132 are displayed simultaneously, wherein only the sub-portion 3120 of the remixed immersive environment 134 is covered by the sub-portion of the identified sampled immersive environment 132.

[0181] In other embodiments, the revisit functionality provides a “full immersion” of the identified sampled immersive environment 132 , wherein the entire remixed immersive environment 134 of the current IE scene 174 is replaced by the identified sampled immersive environment 132 . Fig.33 According to various embodiments, the Fig.31 31 shows a screenshot of the “full immersion” revisit function of the sampled chair object 3110. As shown, the identified sampled immersive environment 132 completely replaces the remixed immersive environment 134 in the current IE scene 174. Therefore, in the IE scene 174 currently displayed in the IE headset 172, the remixed immersive environment 134 is no longer displayed, and only the sampled immersive environment 132 is displayed.

[0182] Fig.34 A flowchart of method steps for reusing samples within a 3D immersive environment according to various embodiments is shown. Figures 1 to 18 and Figures 20 to 33 The method steps are described in detail with reference to a system of FIG. 3400 , but one skilled in the art will appreciate that the method steps may be performed in any order by any system. In some embodiments, method 3400 may be performed by Figure 1 The IE system 100 is executed by the computer system 106 (via the memory unit 104 and the processor 102).

[0183] The method 3400 begins with the SR engine 140 configuring (at step 3410) a remix immersive environment 134 for the remix stage based on user input. The SR engine 140 may receive user input for a default or user-selected remix immersive environment 134. In response, the SR engine 140 retrieves the selected remix immersive environment 134 from the IE database 180 and launches the IE engine 110 to render and display the remix immersive environment 134 on the IE headset 172. The remix immersive environment 134 may include one or more native 3D objects 2110. The SR engine 140 also generates and displays a sample set menu 1100 of the SPUI 166 within the remix immersive environment 134 for selection from an object set 1120, a texture set 1130, a color scheme set 1140, an animation set 1150, an action set 1160, a physical parameter set 1170, or a monochrome and palette set 1180.

[0184] At step 3420, the SR engine 140 adds at least one sampled 3D object to the remixed immersive environment 134 based on the user input. The user input may include the user selecting an object collection 1120 from the sample collection menu 1100. In response, the SR engine 140 generates and displays the object collection UI 1200 and populates the icon window 1110 with object sample icons 1250 representing the object samples stored in the SPDS 150, as described with respect to FIG. Fig.12 As stated in Fig. 22As described, the user input may further include selecting at least one icon 1250 corresponding to the at least one sampled object from the object collection UI 1200. In response, the SR engine 140 retrieves the object entry 304 corresponding to the selected icon 1250 from the SPDS 150, and renders and displays the at least one sampled object within the remix immersive environment 134 based on the sample metadata 360 stored in the retrieved object entry 304. Adding the at least one sampled object to the remix immersive environment 134 in this manner generates a new / modified remix immersive environment 134.

[0185] At step 3430, the SR engine 140 stores the at least one added sampled object as a new object associated with the remixed immersive environment 134. The SR engine 140 may do this by accessing the IE entry 230 corresponding to the remixed immersive environment 134 in the IE database 180 and adding the object identifier of the at least one added object and its corresponding object metadata (object_meta) to the associated object field 220 in the IE entry 230. By doing so, the initial remixed immersive environment 134 is modified to generate a new remixed immersive environment 134 including the at least one newly added object.

[0186] At step 3440, the SR engine 140 modifies at least one object property of at least one 3D object in the remixed immersive environment 134 using at least one selected sample stored in the SPDS 150 based on the user input. Fig.24 , Fig.25A and Fig.25B As described above, the user input may include selecting a specific set UI corresponding to a specific object property from the sample set menu 1100 to view sample icons of samples of the specific object property stored in the SPDS 150. Then, the user may select the specific sample icon and drag it onto the 3D object to replace the current corresponding object property of the 3D object with the sampled object property corresponding to the selected sample icon. Figure 26 to Figure 28 As described, the user input may include dragging the second 3D object onto the first 3D object within the remixed immersive environment 134 to transfer one or more object properties of the second 3D object to the first 3D object. The user input also includes selecting one or more object property samples of the second 3D object stored in the SPDS 150 via the transferable property UI 2600, thereby causing the one or more selected object properties of the second 3D object to be transferred to the first 3D object. The SR engine 140 then replaces the current object property metadata associated with the modified object in the IE database 180 (stored in the associated object field 220 of the IE entry 230 corresponding to the remixed immersive environment 134) with the object property metadata of the selected sample.

[0187] At step 3450, SR engine 140 captures the modified object based on the user input as a new object sample stored in SPDS 150. SR engine 140 may achieve this by generating a new entry in SPDS 150 representing the new object sample and populating one or more data fields 340, 350, 360, 370, and / or 380 of the new entry, as described in relation to Figure 5 described.

[0188] At step 3460, the SR engine 140 uses the palette samples stored in the SPDS 150 to modify the color scheme properties of the plurality of 3D objects in the remixed immersive environment 134 based on the user input. Figure 29 to Figure 30 As described, the user input may include selecting multiple 3D objects within the remixed immersive environment 134 (or selecting the “global selection” button 1892 to select all objects in the IE scene). The user input may also include selecting a specific palette icon 1890 corresponding to a specific palette sample stored in the SPDS 150 from the palette collection UI 1800. In response, the SR engine 140 modifies / replaces the current palette metadata associated with the two or more selected 3D objects (stored in the associated object field 220 of the IE entry 230 corresponding to the remixed immersive environment 134 in the IE database 180) based on the palette metadata of the selected palette sample.

[0189] At step 3470, the SR engine 140 applies a "revisit" function to the sampled 3D objects within the remixed immersive environment 134 based on the user input. Figure 31 to Figure 33 As described, the user input may include a selection of a "revisit" function and a specific object previously sampled from a specific sampled immersive environment 134. In response, the SR engine 140 retrieves the object entry 304 corresponding to the selected object stored in the SPDS 150, and retrieves context information from the context field 370 of the object entry 304. The context field 370 specifies the IE identifier 210 of the specific sampled immersive environment 132 from which the selected object was sampled, and the 3D position coordinates of the object and the user's viewpoint within the specific sampled immersive environment 132 when the object was sampled. The SR engine 140 then launches the IE engine 110 to render and display at least a portion of the specific sampled immersive environment 132 within the current IE scene 174 based on the retrieved context information in the context field 370. In some embodiments, the revisit function provides a "preview" of the identified sampled immersive environment 132. In other embodiments, the revisit function provides a "full immersion" of the identified sampled immersive environment 132.

[0190] In summary, during the sampling phase, the user can explore the sampling immersive environment to capture samples of 3D digital components within the sampling immersive environment. The 3D digital components may include 3D objects rendered and displayed within the sampling immersive environment. The 3D digital components may also include specific object property components for rendering the 3D objects, such as textures, color schemes, animations, motion paths, and physical parameters. The 3D digital components are captured as samples, which are stored in a sample palette data structure (SPDS) that collects and organizes samples. The captured samples may also include monochrome samples and palette samples. The samples may be viewed and accessed via a sample palette user interface (SPUI), which displays sample icons representing samples stored in the SPDS. Sampling suggestions may also be displayed within the sampling immersive environment.

[0191] During the remixing phase, the user may reuse / apply samples stored in the SPDS to modify 3D objects, immersive environments, and / or 3D applications in order to generate and render new 3D objects, new immersive environments, and / or new 3D applications. The user may add sampled objects to the remixed immersive environment via interaction with the SPUI to modify the remixed immersive environment. The user may apply one or more object-based samples to 3D objects displayed within the remixed immersive environment via interaction with the SPUI to modify one or more object properties of the 3D object, such as the texture, color scheme, animation, motion path, and / or physical parameters of the 3D object. The user may also apply palette samples to multiple 3D objects displayed within the remixed immersive environment via interaction with the SPUI to modify the color properties of multiple 3D objects. A revisit function is also provided that enables the user to revisit the sampled immersive environment from which the sampled object was originally sampled.

[0192] At least one technical advantage of the disclosed technology over the prior art is that the disclosed technology enables sampling and acquisition of 3D digital components from an immersive environment, which is not possible using prior art methods. In addition, the disclosed technology enables a designer / user to navigate an immersive environment, select 3D objects or other 3D digital components to be sampled and stored in a sample palette data structure (SPDS), which can then be accessed via a sample palette user interface (SPUI). Once accessed, the sampled 3D objects or 3D digital components can be reused or modified and applied in different immersive environments or 3D applications. In this way, the disclosed technology does not require a designer to design and generate each 3D object of an immersive environment or 3D application, as is the case with the prior art. Therefore, the disclosed technology can significantly reduce the computer resources required to design and generate 3D objects included in an immersive environment or 3D application, and can also significantly reduce the overall designer time and effort required to create an immersive environment or 3D application. These technical advantages provide one or more technical improvements over prior art methods.

[0193] Various aspects of the subject matter described herein are set forth in the following numbered clauses.

[0194] 1. In some embodiments, a computer-implemented method for capturing one or more samples within a three-dimensional (3D) immersive environment includes: rendering and displaying a first 3D object within a first 3D immersive environment, wherein the first 3D object includes at least a first component for rendering and displaying the first 3D object; and capturing the at least first component as a first sample; and storing the first sample in a sample data structure to allow one or more additional operations to be performed on the first sample.

[0195] 2. The computer-implemented method of clause 1, wherein the first component comprises metadata of a 3D model associated with the first 3D object.

[0196] 3. A computer-implemented method as described in clause 1 or 2, wherein the first component includes metadata of a first property associated with the first 3D object.

[0197] 4. A computer-implemented method as described in any of clauses 1 to 3, wherein the first property includes a texture, an animation, a motion path, or a set of physical parameters associated with the first 3D object.

[0198] 5. The computer-implemented method of any of clauses 1 to 4, wherein the sample data structure stores a plurality of samples captured from a plurality of different 3D immersive environments.

[0199] 6. A computer-implemented method as described in any of clauses 1 to 5, wherein the sample data structure includes multiple entries corresponding to multiple samples, each entry having a set of data fields for the corresponding sample, wherein the set of fields includes at least one of a sample identifier field, an associated object field, a sample metadata field, or a sample icon field.

[0200] 7. A computer-implemented method as described in any of clauses 1 to 6, wherein the sample data structure stores a plurality of samples that can be viewed and accessed via a sample user interface, the sample user interface displaying a plurality of sample icons that visually represent the plurality of samples.

[0201] 8. A computer-implemented method as described in any of clauses 1 to 7, further comprising: receiving a selection of a first point within the first 3D immersive environment; in response, capturing a monochrome sample associated with the first point; and storing the monochrome sample within the sample data structure.

[0202] 9. A computer-implemented method as described in any of clauses 1 to 8, further comprising: receiving a selection of multiple 3D objects within the first 3D immersive environment; in response, capturing palette samples associated with the multiple 3D objects; and storing the palette samples within the sample data structure.

[0203] 10. A computer-implemented method as described in any of clauses 1 to 9, further comprising displaying at least one sampling suggestion within the first 3D immersive environment, wherein the at least one sampling suggestion comprises at least one of a suggested 3D object, a suggested texture, a suggested animation, a suggested motion path, a set of suggested physical parameters, or a suggested image.

[0204] 11. In some embodiments, one or more non-transitory computer-readable media include instructions that, when executed by one or more processors, cause the one or more processors to capture one or more samples within a three-dimensional (3D) immersive environment by performing the following steps: rendering and displaying a first 3D object within a first 3D immersive environment, wherein the first 3D object includes at least a first component for rendering and displaying the first 3D object; and capturing the at least first component as a first sample; and storing the first sample in a sample data structure to allow one or more additional operations to be performed on the first sample.

[0205] 12. The one or more non-transitory computer-readable media of clause 11, wherein the first component comprises metadata of a 3D model associated with the first 3D object.

[0206] 13. One or more non-transitory computer-readable media as described in clause 11 or 12, wherein the first component includes metadata of a first property associated with the first 3D object.

[0207] 14. One or more non-transitory computer-readable media as described in any of clauses 11 to 13, wherein the first property includes a texture, an animation, a motion path, or a set of physical parameters associated with the first 3D object.

[0208] 15. The one or more non-transitory computer-readable media of any of clauses 11 to 14, wherein the sample data structure stores a plurality of samples captured from a plurality of different 3D immersive environments.

[0209] 16. One or more non-transitory computer-readable media as described in any of clauses 11 to 15, wherein the sample data structure stores a plurality of samples of a plurality of different sample types, wherein the plurality of samples are organized within the sample data structure based on sample type.

[0210] 17. One or more non-transitory computer-readable media as described in any of clauses 11 to 16, wherein the sample data structure stores a plurality of samples that can be viewed and accessed via a sample user interface, the sample user interface displaying a plurality of sample icons that visually represent the plurality of samples.

[0211] 18. One or more non-transitory computer-readable media as described in any of clauses 11 to 17, wherein the first 3D object includes a sub-portion of a second 3D object deconstructed into multiple sub-portions within the first 3D immersive environment, and the multiple sub-portions include the first 3D object.

[0212] 19. One or more non-transitory computer-readable media as described in any of clauses 11 to 18, wherein the first sample includes object metadata for rendering the first 3D object, and the non-transitory computer-readable medium also includes: rendering and displaying a second 3D immersive environment; and modifying the second 3D immersive environment by applying the first sample within the second 3D immersive environment to render and display the first 3D object within the second 3D immersive environment.

[0213] 20. In some embodiments, a computer system includes: a memory that stores instructions; and at least one processor that is coupled to the memory and, when executing the instructions, captures one or more samples within a three-dimensional (3D) immersive environment by performing the following steps: rendering and displaying a first 3D object within a first 3D immersive environment, wherein the first 3D object includes at least a first component for rendering and displaying the first 3D object; and capturing the at least first component as a first sample; and storing the first sample in a sample data structure to allow one or more additional operations to be performed on the first sample.

[0214] Any and all combinations of claimed elements recited in any claim and / or any elements described in this application, in any manner, are within the contemplated embodiments and scope of protection.

[0215] Descriptions of the various embodiments have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments.

[0216] Aspects of the present embodiment may be embodied as a system, method, or computer program product. Therefore, aspects of the present disclosure may take the form of a complete hardware implementation, a complete software implementation (including firmware, resident software, microcode, etc.), or an implementation combining software and hardware aspects, which may all be generally referred to herein as "modules" or "systems." In addition, any hardware and / or software technology, process, function, component, engine, module, or system described in the present disclosure may be implemented as a circuit or a group of circuits. In addition, aspects of the present disclosure may take the form of a computer program product embodied in one or more computer-readable media having computer-readable program code implemented thereon. In various embodiments, software constructs and entities ( For example , engines, modules, GUI, etc.) are stored in a memory / memory shown in the relevant system diagrams and executed by the processors shown in those same system diagrams.

[0217] Any combination of one or more computer-readable media may be utilized. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electronic, non-temporary, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. More specific examples (non-exhaustive list) of computer-readable storage media would include the following: an electrical connection with one or more wires, a portable computer floppy disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer-readable storage medium may be any tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus.

[0218] Aspects of the present disclosure are described above with reference to the flowchart illustration and / or block diagram of the method, device (system) and computer program product according to the embodiments of the present disclosure. It should be understood that each frame in the flowchart illustration and / or block diagram and the combination of frames in the flowchart illustration and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine. The instruction enables the function / action specified in one or more frames of the flowchart and / or block diagram to be realized when executed by the processor of a computer or other programmable data processing device. Such processors can be, but are not limited to, general-purpose processors, special-purpose processors, special-purpose processors or field programmable gate arrays.

[0219] The flow charts and block diagrams in the accompanying drawings illustrate the architecture, functions and operations of possible implementations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram may represent a module, a fragment or a portion of a code, and the code includes one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the box may not occur in the order indicated in the accompanying drawings. For example, the two boxes shown in succession may actually be executed substantially simultaneously, or the boxes may sometimes be executed in reverse order, depending on the functionality involved. It should also be noted that each box in the block diagram and / or flow chart illustration and the combination of boxes in the block diagram and / or flow chart illustration can be implemented by a system based on special-purpose hardware or a combination of special-purpose hardware and computer instructions that performs the specified function or action.

[0220] While the foregoing is directed to embodiments of the present disclosure, other and further embodiments of the present disclosure may be envisaged without departing from the basic scope of the present disclosure, and the scope of the present disclosure is determined by the claims that follow.

Claims

1. A computer-implemented method for capturing one or more samples within a three-dimensional (3D) immersive environment, the method comprising: Rendering and displaying a first 3D object within a first 3D immersive environment, wherein the first 3D object comprises at least a first component for rendering and displaying the first 3D object; and capturing the at least first component as a first sample; and The first sample is stored in a sample data structure to allow one or more additional operations to be performed on the first sample.

2. The computer-implemented method of claim 1, wherein: The first component includes metadata of a 3D model associated with the first 3D object.

3. The computer-implemented method of claim 1, wherein: The first component includes metadata of a first property associated with the first 3D object.

4. The computer-implemented method of claim 3, wherein: The first property includes a texture, an animation, a motion path, or a set of physical parameters associated with the first 3D object.

5. The computer-implemented method of claim 1, wherein: The sample data structure stores a plurality of samples captured from a plurality of different 3D immersive environments.

6. The computer-implemented method of claim 1, wherein: The sample data structure includes multiple entries corresponding to multiple samples, each entry having a set of data fields for the corresponding sample, wherein the set of fields includes at least one of a sample identifier field, an associated object field, a sample metadata field, or a sample icon field.

7. The computer-implemented method of claim 1, wherein: The sample data structure stores a plurality of samples viewable and accessible via a sample user interface that displays a plurality of sample icons visually representing the plurality of samples.

8. The computer-implemented method of claim 1, further comprising: receiving a selection of a first point within the first 3D immersive environment; in response, capturing a monochromatic sample associated with the first point; as well as The monochrome samples are stored in the sample data structure.

9. The computer-implemented method of claim 1, further comprising: receiving a selection of a plurality of 3D objects within the first 3D immersive environment; In response, capturing palette samples associated with the plurality of 3D objects; as well as The palette samples are stored in the samples data structure.

10. The computer-implemented method of claim 1, further comprising displaying at least one sampling suggestion within the first 3D immersive environment, wherein: The at least one sampling suggestion includes at least one of a suggested 3D object, a suggested texture, a suggested animation, a suggested motion path, a set of suggested physical parameters, or a suggested image.

11. One or more non-transitory computer-readable media comprising instructions that, when executed by one or more processors, cause the one or more processors to capture one or more samples within a three-dimensional (3D) immersive environment by performing the following steps: Rendering and displaying a first 3D object within a first 3D immersive environment, wherein: The first 3D object includes at least a first component for rendering and displaying the first 3D object; as well as capturing the at least first component as a first sample; as well as The first sample is stored in a sample data structure to allow one or more additional operations to be performed on the first sample.

12. One or more non-transitory computer-readable media as claimed in claim 11, wherein: The first component includes metadata of a 3D model associated with the first 3D object.

13. The one or more non-transitory computer-readable media of claim 11, wherein: The first component includes metadata of a first property associated with the first 3D object.

14. One or more non-transitory computer-readable media as claimed in claim 13, wherein: The first property includes a texture, an animation, a motion path, or a set of physical parameters associated with the first 3D object.

15. The one or more non-transitory computer-readable media of claim 11, wherein: The sample data structure stores a plurality of samples captured from a plurality of different 3D immersive environments.

16. The one or more non-transitory computer-readable media of claim 11, wherein: The sample data structure stores a plurality of samples of a plurality of different sample types, wherein the plurality of samples are organized within the sample data structure based on sample type.

17. The one or more non-transitory computer-readable media of claim 11, wherein: The sample data structure stores a plurality of samples viewable and accessible via a sample user interface that displays a plurality of sample icons visually representing the plurality of samples.

18. The one or more non-transitory computer-readable media of claim 11, wherein: The first 3D object includes a sub-portion of a second 3D object deconstructed into a plurality of sub-portions within the first 3D immersive environment, the plurality of sub-portions including the first 3D object.

19. The one or more non-transitory computer-readable media of claim 11, wherein: The first sample includes object metadata for rendering the first 3D object, the non-transitory computer-readable medium further comprising: Rendering and displaying a second 3D immersive environment; and The second 3D immersive environment is modified by applying the first sample within the second 3D immersive environment to render and display the first 3D object within the second 3D immersive environment.

20. A computer system comprising: a memory storing instructions; as well as At least one processor is coupled to the memory and, when executing the instructions, captures one or more samples within a three-dimensional (3D) immersive environment by performing the following steps: Rendering and displaying a first 3D object within a first 3D immersive environment, wherein the first 3D object comprises at least a first component for rendering and displaying the first 3D object; and capturing the at least first component as a first sample; and The first sample is stored in a sample data structure to allow one or more additional operations to be performed on the first sample.

Images