Incremental propagation using coverage for distributed collaborative content creation applications

Abstract

In various examples, coverage-based data change propagation for a distributed collaborative content creation platform is provided. An overlay layer may be generated that includes a representation of a series of overlay primitives that may be applied to a baseline Universal Scene Description (USD) scene definition layer to render an update / current version of the scene. The digital content creation application may record edits when applying modifications (overlays) to the scene definition primitives. A content creation application CCA may propagate the edit to other content creation application CCA at other client nodes subscribing to a collaborative content creation platform CCCP by sending a new layer message to a server of the collaborative content creation platform CCCP instructing the server to add the overlay into a scene graph of a root USD file. Other content creation application CCA subscribing to the root USD file receives a message from a server indicating that a new overlay layer has been added, and may selectively apply propagated overlay primitives.

Description

Background Technology

[0001] Digital content creation applications (CCAs) are software applications used to create, edit, manipulate, and / or render three-dimensional (3D) digital content to build 3D virtual worlds (such as those used for animation, game development, filmmaking, and virtual reality). To create a 3D virtual world, multiple different CCAs (which can be hosted or accessed by different client nodes) can be used together to create and enhance various aspects of the features and elements appearing in the virtual world. For example, a CCA for 3D modeling based on computer-aided design (CAD) can be used to design and define the structure of object features; another CCA can be used to create two-dimensional (2D) textures that can be applied to the surfaces of object features; yet another CCA can be used to apply lighting effects (such as surface reflections and / or shadow casting); and a third CCA can be used to create background terrain and / or a dome for the 3D scene where the object features reside. Distributed collaborative content creation platforms (CCCPs), such as NVIDIA's Omniverse, provide tools for establishing public interactive environments where individual content designers can use multiple CCAs to collaboratively create and edit various features of elements in a virtual 3D world in real time. Summary of the Invention

[0002] Embodiments of this disclosure relate to coverage-based data change propagation for CCCP systems and applications.

[0003] Compared to existing distributed CCCPs, this invention discloses a system and method for generating overlay layers comprising a series of overlay primitives that can be applied to one or more baseline Common Scene Description (USD) scene definition layers to render an updated / current version of the scene. More specifically, when a digital CCA is editing or otherwise manipulating elements of a scene rendered by CCCP, the CCA records the edit as a modification (overlay) to primitives defined in one or more scene definition layers associated with those features. The CCA can propagate the edit to other CCAs at other client nodes subscribed to CCCP by sending a new layer message to the CCCP server, instructing the server to add the overlay layer to the scene graph of the root USD file.

[0004] (For example, one, some, or each) Overlay primitives indicate edits made to definition primitives in the scene definition layer corresponding to those edited features. In other words, in response to one or more edits applied by CCA, an overlay layer is generated that includes representations of a series of overlay primitives (e.g., directed acyclic graphs) that can be applied to the baseline scene definition layer to render an updated version of the scene.

[0005] Other CCAs subscribing to the root USD file receive messages from the server indicating that a new overlay has been added. CCAs receive messages about the new layer from the CCCP server and can optionally read and selectively apply the overlay primitives propagated via the new overlay added to the root USD file. Because the payload of the overlay is reserved for propagating overlay primitives, the overlay is relatively sparse compared to a scene definition layer, which may also carry definition primitives and / or other types of primitive and scene definition information. Therefore, the propagation of messages including overlays is a relatively lightweight operation, consuming significantly fewer computational and / or network resources compared to transmitting the entire USD layer. Attached Figure Description

[0006] The system and method for propagating data changes based on coverage for distributed collaborative content creation platform systems and applications will be described in detail below with reference to the accompanying drawings, wherein:

[0007] Figure 1 This is a data flow diagram of a process for an example collaborative environment including a content creation platform, according to embodiments of this disclosure;

[0008] Figure 2 This is a data flow diagram of the process for generating a USD overlay layer according to an embodiment of the present disclosure;

[0009] Figure 3 The process for overlay propagation according to some embodiments of this disclosure is illustrated;

[0010] Figure 4 This is a flowchart of a method for propagating overlay-based data changes in an example collaborative environment including a content creation platform, according to embodiments of this disclosure;

[0011] Figure 5 This is a block diagram of an example computing device applicable to implementing some embodiments of the present disclosure; and

[0012] Figure 6 This is a block diagram of an example data center applicable to implementing some embodiments of this disclosure. Detailed Implementation

[0013] This invention discloses a system and method for propagating data changes based on coverage for distributed collaborative content creation platform systems and applications. Distributed collaborative content creation platforms (CCCPs), such as NVIDIA's Omniverse, provide tools to establish a common interactive environment where multiple digital content creation applications (CCAs) running on different client nodes are available for individual content designers to collaboratively create and edit various features of elements within a virtual 3D world in real time. Currently, real-time collaboration between heterogeneous CCAs using the currently available CCCP can be challenging because different CCAs typically use different data formats to represent the content they create. For example, a first content designer using a first CCA might modify elements represented in the common interactive environment provided by the CCCP. To share content with contributors operating CCAs on other client nodes of the CCCP, the first CCA can transfer the representation of the content to the CCCP's central database using a feature-based native file format. Since content designers using the same CCA instance on another client node can easily import and understand the first CCA's native file format and the feature edits performed on the elements, the pair of CCAs can maintain synchronized representations of elements (on one or each application) by exchanging updated file versions using their native file formats. However, this is not necessarily the case when collaborators across different client nodes of CCCP use a set of heterogeneous CCAs. For a set of heterogeneous CCAs, because these CCAs may not share native file formats, one CCA may not be able to directly detect and / or interpret all the operations performed by another CCA (e.g., editing of elements and features rendered in the CCCP's public interactive environment) simply by reading the shared native files, including those being edited.

[0014] To address this inconsistency, collaborative CCCPs can generate and distribute Boundary Presence Representation (BREP) models and / or tessellated representations of edited elements, so that one or more (e.g., each) of a set of heterogeneous CCAs can at least render a local representation of the edited elements for viewing at client nodes. That is, CCAs, and CCCPs, can maintain their own canonical underlying representations of elements within a shared interactive environment and receive real-time representations (often referred to as “delta information”) of changes made to that canonical underlying representation by other CCAs on other client nodes. However, propagating BREP and / or tessellated surface CAD representations to share delta information is a relatively inefficient process, involving the amount of memory required to represent the current state of objects appearing in the shared interactive environment. As the scene complexity rendered in the shared interactive environment increases, the amount and complexity of using BREP and / or tessellated surface CAD representations to communicate and store delta information also increases accordingly. Furthermore, while BREP and / or tessellated surface CAD representations can reflect changes in the appearance of features and / or elements rendered in a public interactive environment, they cannot provide a comprehensive and clear description to the CCA at the client node, nor can they accurately describe which features of specific elements have changed and how they have changed.

[0015] A Universal Scene Description (USD) is a software framework and file format used to describe the geometry, materials, physics, and behavior of a 3D virtual world scene. Primitives (“prims”) and content are organized into scene description containers called “layers” (collections of primitives (“prims”) and their attributes). In some USD implementations, USD layers may be populated with data converted from the native file format used by the Content Architect (CCA) that provides content to the public interactive environment. That is, at the CCCP client node, when a content designer uses a CCA to create and / or edit elements that appear in the public interactive environment, content may be saved in the native file format of that CCA and further used to populate one or more USD layers corresponding to the elements created and / or edited by that CCA. USD layers may define contributions to the scene graph of a root USD data structure (e.g., a file containing a scene graph), which is used to render one or more aspects of elements in the public interactive environment. A scene graph is a data structure comprising a set of graph nodes arranged in a graph or tree structure for organizing the logical and / or specific features of a computer graphics scene. Each node can represent an object, an attribute, or a transformation matrix applied to each of its child nodes. Therefore, a USD-compatible CCA processing one or more USD layers of the root USD file can synchronize these scene definition layers with the CCCP database server. Other USD-compatible CCAs can also (on the CCCP database server node) subscribe to these scene definition layers, enabling them to collaboratively process (e.g., edit) the features of elements represented by primitives using copies of these layers received. However, as scenes rendered in a shared interactive environment become more complex (e.g., in terms of the number of elements and / or the number of defined features of those elements), the number of nodes and / or complexity of the scene graph associated with each scene definition layer may increase accordingly. Therefore, using message-exchanging scene definition layers to facilitate collaboration between CCAs is not conducive to the continuous expansion of scenes and may become unmanageable in terms of memory, bandwidth, and the time consumed in exchanging messages carrying scene definition layers (at both the server node and the individual CCAs).

[0016] Compared to existing distributed collaborative CCPs, embodiments of this disclosure provide overlay-based data change propagation for distributed collaborative content creation platform systems and applications. More specifically, when a CCA is editing or otherwise manipulating elements of a scene rendered by a CCCP, the CCA records the edit as a modification (overlay) to primitives defined in one or more scene definition layers associated with those features.

[0017] For example, elements of a scene within a public interactive environment may include objects with cubic geometry. Geometry (e.g., size), surface texture (e.g., 2D surface image), material composition, surface color and / or lighting environment (e.g., direction, color, and / or brightness of light emitted by a light source) and / or non-rendering properties (e.g., weight values, inventory, etc.) are non-limiting examples of cubic features, the properties of which may be individually defined by definition primitives included in the scene definition layer subscribed to by CCA via the CCCP database server node.

[0018] When CCA performs an edit operation on one or more of these features, these operations are tracked at the client node as a series of overlay primitives generated by CCA. Each overlay primitive indicates an edit made to a definition primitive in the scene definition layer corresponding to those edited features. For example, the scene definition layer might include a primitive in the definition of a cube that specifies the cube's surface as red. In the local rendering of the cube generated by CCA, and in the rendering of the cube generated by CCCP in a public interactive environment, the cube's surface is also rendered as red. The CCA user can then apply an edit to change the cube's surface color to blue. In this process, CCA can apply the change based on a non-destructive editing paradigm, where the baseline definition primitive in the scene definition layer continues to define the cube's color as red, but an overlay primitive is generated at the client node that instructs CCA to ignore the value of the baseline definition primitive that specifies the cube as red, and instead adopt the blue value specified by the overlay primitive generated in response to the edit made by CCA. In other words, in response to one or more edits applied by CCA, an overlay layer is generated, which includes a representation of a series of overlay primitives (e.g., directed acyclic graphs) that can be applied to the baseline scene definition layer to render an updated / current version of the scene.

[0019] Using the USD framework, a CCA can propagate edits to other CCAs at other client nodes subscribed to CCCP by sending a new layer message to the CCCP server node, instructing the server to add an overlay layer to the scene graph of the root USD file. Other CCAs subscribed to the root USD file receive the message indicating the addition of a new overlay layer from the server node. The CCA receives the message about the new layer from the CCCP server node and can optionally read and selectively apply the overlay primitives propagated via the new overlay added to the root USD file. Because the payload of the overlay layer is reserved for propagating overlay primitives, it is relatively sparse compared to a scene definition layer, which may also carry definition primitives and / or other types of primitive and scene definition information. Therefore, propagating messages including overlay layers is a relatively lightweight operation, consuming significantly fewer computational and / or network resources compared to transmitting the entire USD layer.

[0020] In some embodiments, a new layer message including an overlay may include an ASCII- or other text-based representation of the overlay primitives, or a representation compressed into a binary serialization or other format, to make the message smaller and / or consume fewer resources during transmission. In some embodiments, the overlay may be passed to a CCCP server node and / or propagated as a USD payload to one or more CCAs. CCCP may include one or more application programming interfaces (APIs) exposed to CCAs at client nodes to allow CCAs to perform one or more overlay propagation functions as described herein. For example, a CCA may invoke a messaging API to send a message including a new overlay to a CCCP server node to propagate the edits made by that CCA. Another API may implement a subscription function for CCAs to use to specify relevant layers in the root USD file, which the CCA will subscribe to to receive notifications when the corresponding new overlay becomes available.

[0021] (via messages from the CCCP server node) Receiving the propagated overlay CCA allows for precise identification of which elements and features were updated by the applied edit CCA, and the updated parameter values ​​for those features. The CCA receiving the propagated overlay can parse the overlay primitive sequence to identify edits in features associated with that CCA and ignore further processing of overlay primitives considered irrelevant. For example, the first CCA managing the surface texture of elements in a public interactive environment might ignore overlay primitives that have repositioned the element (e.g., position and / or rotation), since surface texture and object position are completely independent features of that element.

[0022] In some embodiments, since overlay primitives are presented sequentially in the order in which editing operations are performed within an overlay, the CCA receiving the propagated overlay can efficiently parse the overlay to find the most recent occurrence of an overlay primitive associated with a feature and apply that overlay primitive. Overlay primitives associated with the feature prior to the most recent overlay primitive can be simply considered obsolete and ignored. For example, if a first overlay primitive changes the color of a feature of a cube to red, but a subsequent overlay primitive changes the color of that feature to yellow, the CCA receiving the propagated overlay can ignore the first overlay primitive (changing the color of the feature to red) and apply the change indicated by a subsequent overlay primitive in the sequence (changing the color of the feature to yellow). Assuming there are no updated subsequent overlay primitives for that feature, in this case, the latest subsequent overlay primitive for that feature will be applied.

[0023] In some embodiments, an overlay may include timestamps applied to overlay primitives, indicating when an editing operation was performed and / or when the overlay was created. In these embodiments, the CCA receiving the propagated overlay may determine which overlay primitives to apply (e.g., the most recent) based on at least one or more timestamps. In some embodiments, based on the propagated overlay, the CCA may merge one or more overlay primitives back into the baseline definition primitives in the scene definition layer to establish the updated features as the new baseline definition. That is, parameters from the overlay primitives of the overlay can be used to modify definition primitives specified by other layers in the root USD file. In some embodiments, the propagated overlay may be provided by a server node of a distributed collaborative content creation platform using a file format plugin that exposes an application programming interface (API) that provides the receiving CCA with the underlying format architecture of the propagated overlay to facilitate reading and / or parsing the overlay primitives included in the propagated overlay.

[0024] In some embodiments, CCA may determine when to push new overlay messages to the CCCP server node based on one or more criteria. For example, CCA may push new overlay messages manually (e.g., based on user input), based on a predetermined frequency (e.g., once every specified time interval), and / or based on event-based criteria (e.g., the size of the overlay and / or the number of accumulated overlay primitives reaching a threshold).

[0025] In some embodiments, when two or more CCAs propagate their respective new overlay messages (including conflicting overlay primitives acting on the same feature), the CCCP server node can implement conflict management functionality (e.g., an algorithm). For example, if an overlay from a first CCA changes the surface color feature of a cube to blue, while another overlay from a second CCA changes the surface color feature of the same cube to orange, the conflict management functionality can implement a conflict protocol to determine which overlay will apply. For example, the conflict management functionality can send a conflict notification to users of one or both CCAs, informing them of the existence of the conflict so that they can coordinate to determine a solution and be notified of the selected solution (e.g., which overlay primitive will be retained, and / or which overlay primitive will be discarded).

[0026] In some embodiments, one or more server nodes of CCCP itself may execute software that generates a virtual rendering of a public interactive environment. Users (e.g., users of CCAs and / or users of other applications) and / or agents (e.g., agents implemented by computer algorithms and / or artificial intelligence) can interface through this virtual rendering to visualize and / or interact with a scene that includes one or more various elements included in the root USD file. That is, CCCP can maintain an analytical, real-time representation of the scene described in the root USD file and provide access to this scene as a public interactive environment to various client nodes. CCCP server nodes may generate computer-generated simulations that render the features of elements defined by one or more scene definition layers in the root USD file and / or modified by one or more overlay layers propagated by one or more CCAs. In some embodiments, users and / or agents operating within the virtual rendering of the public interactive environment generated by CCCP may also initiate editing of the features of elements in the scene (e.g., using CCAs directly hosted by CCCP). For example, a user of a CCCP-hosted CCA may move the position of one or more elements, which may represent a deviation of the element's positional features from the position defined by the baseline scene definition layer of the root USD file. In this scenario, the CCCP server node can generate one or more new overlays that capture overlay primitives, which in turn capture the edited positions of elements. These overlays generated by the CCCP server node can be propagated to the client nodes by adding them to the root USD file and notifying the subscribed CCA at the client node via a new layer message, in the same manner as the propagation of new overlays by CCAs hosted by the client nodes described herein.

[0027] In some embodiments, one or more CCAs coupled to a client node in a common interactive environment can generate edits to element features based on user (e.g., content creator) input. In some embodiments, such edits to features can be generated by a non-human agent. For example, in some embodiments, a CCA (client node hosted or CCP hosted) may include an artificial intelligence (AI)-based agent (e.g., an agent using a generative AI model) that perceives elements of the common interactive environment as input (e.g., using computer vision techniques) and generates changes to the elements of the common interactive environment based on these inputs. These changes can be captured as edits by the CCA and used to generate overlay primitives embedded in a propagated overlay. For example, in some embodiments, a generative AI agent may perceive an unordered collection of boxes in the common interactive environment and generate overlay primitives to reposition these boxes into a neat stack of boxes, and the CCA may propagate an overlay that a CCCP server node and / or other CCAs can use to organize their scene rendering.

[0028] In some embodiments, a bidirectional connector can be used to transfer information between a first CCA and a scene location within a common interactive environment. The transferred information may be fragmented or otherwise selected based on multiple different parameters, such as according to a defined workflow or the specification of a given type of CCA. In operation, a client node can use the first CCA and provide a set of information to the scene location, which can be accessed by a second CCA and a second client node, which can use either the first CCA or a different second CCA. Based on the information provided to the scene location, one or more features of the content from the first CCA can be viewed and / or modified within the second CCA, and then provided back to the scene location for the first client node to view. If each client node uses a different CCA, the amount of editing / modification may be limited based on the information identified and extracted through the scene location. However, if each client uses a common CCA, a copy of the original file can also be provided between the two client nodes, allowing for additional editing and collaboration. In this way, CCCP provides a collaborative environment where a subset of information is extracted from the presented content to allow access using different CCAs.

[0029] refer to Figure 1 , Figure 1 This is an example data flow diagram of a collaborative environment 100 including a content creation platform 102, according to some embodiments of this disclosure. It should be understood that the schemes and other schemes described herein are illustrative only. Other arrangements and elements (e.g., machines, interfaces, functions, sequences, functional groups, etc.) may be used in addition to or in place of the arrangements and elements shown, and some elements may be omitted entirely. Furthermore, many elements described herein are functional entities that may be implemented as discrete or distributed components, or combined with other components, and implemented in any suitable combination and location. The various functions described herein as being performed by entities may be performed by hardware, firmware, and / or software. For example, various functions may be performed by a processor executing instructions stored in memory.

[0030] Figure 1An example collaborative environment 100 that can be used with embodiments of the present disclosure is illustrated. In this example, the collaborative environment 100 includes a Collaborative Content Creation Platform (CCP) 102 through which one or more collaborators can operate CCAs 130 (shown as 130A-130N) distributed across one or more client nodes 104 (shown as 104A-104N) to collaborate and / or interact with one or more objects 106 rendered within a computer-generated common interactive environment 108. In some embodiments, CCCP 102 includes one or more CCCP server nodes (server 112) that include one or more processors that execute one or more CCCP applications 120 to perform one or more functions of CCCP 102 as described herein. One or more CCCP servers 112 may host one or more CCCP databases 114, which include a root USD data structure 116 (e.g., a file including scene graphs). CCCP applications 120 and / or CCA 130 may use this root USD data structure to render one or more elements of the scene (e.g., one or more objects 106) within a public interactive environment 108. In some embodiments, one or more client nodes 104A-104N may include networked computing devices (e.g., regarding...). Figure 5 The computing device 500 discussed is coupled to one or more CCCP servers 112 via at least one network 105 (e.g., the Internet). In some embodiments, one or more functions performed by one or more CCCP servers 112 and / or CCA 130 may be performed at least in part by a cloud computing platform 600, for example, referencing Figure 6 The described cloud computing platform. CCCP 102 can be integrated into one or more such distributed and / or cloud computing environments, and / or communicate with one or more such distributed and / or cloud computing environments, which allow one or more users to collaborate and / or view objects and / or scenes within a public interactive environment 108 via client nodes 104A-104N. In some embodiments, CCCP 102 can be deployed as one or more tools or features accessible within the public interactive environment 108. These tools and / or features can be provided by the underlying processing hardware of CCCP 102 (such as...). Figure 5 and Figure 6 As described, the hardware supports the execution of stored software instructions in response to one or more commands. These commands may be provided as input, such as by a user performing operations via CCA 130A-130N and / or CCCP application 120. In some embodiments, CCCP application 120 may include one or more applications that operate within the common interactive environment 108 in the same manner as described for CCA 130A-130N.

[0031] exist Figure 1 In this example, client nodes 104A-104N are using a heterogeneous set of CCA 130A-130N to collaborate with one or more objects 106 and / or other elements within a public interaction environment 108 defined by the root USD data structure 116. In this example, one or more objects 106 are shown as 3D volumes, which include features such as, but not limited to, their geometric boundaries, transformed positions in 3D space, etc. Other features of one or more objects 106 represented by the root USD data structure 116 may include, but are not limited to, material properties (e.g., one or more materials comprising one or more objects 106), weight, mass, texture, physical coefficients (e.g., coefficient of friction, coefficient of reflection, etc.), audio data (e.g., sounds emitted by one or more objects 106), inventory levels, and / or other characteristics.

[0032] exist Figure 1In this context, client nodes 104A-104N use CCCP 102 to collaborate on one or more objects 106. Each of these client nodes 104A-104N and its corresponding CCA 130A-130N may represent a user and / or software instructions executed according to one or more workflows. For example, client nodes 104A-104N may include a human user associated with an assembly task involving objects 106 as components. In some embodiments, client nodes 104A-104N and / or CCA 130A-130N may respond to other instructions or workflows. For example, client nodes 104A-104N may be components of a generative artificial intelligence system that collaborates to generate content within a public interactive environment 108. As previously described, client nodes 104A-104N may be using a diverse, non-homogeneous set of CCA 130A-130Ns to modify or otherwise collaborate on object 106. For example, each of client nodes 104A and 104B may be using the same first CCA as the CCA used to generate object 106. However, client node 104C may be using a second CCA, and client node 104N may be using a third CCA. To facilitate collaboration, embodiments of this disclosure may provide an overlay layer that can be used to pass a series of overlay primitives (e.g., a directed acyclic graph) that can be applied to a baseline scene definition layer to render an updated / current version of the scene within the common interaction environment 108. The overlay layer is propagated among client nodes 104 using new layer messages, so that any changes to element characteristics in the common interaction environment 108 can be passed into the common interaction environment 108 via the root USD data structure 116 and shared with various other CCAs, even if these CCAs are different from the software program initiating the change, without needing to propagate other primitives and scene definition information.

[0033] For example, as previously described, a public interactive environment 108 hosted by CCCP server 112 of CCCP 102 can be instantiated based on a root USD data structure 116 comprising multiple USD layers 118 (e.g., a file including a scene graph). USD layers 118 can define one or more aspects of the contribution to the scene graph of the root USD data structure 116 (used to render elements within the public interactive environment 108, such as objects 106). The root USD data structure 116 can include a scene graph data structure comprising a set of graph nodes arranged in a graph or tree structure for organizing logical and / or specific features of a computer graphics scene. USD layers 118 can include one or more graph nodes of the root USD data structure 116, thereby representing primitives associated with objects, attributes, or transformation matrices applicable to each child node of that node.

[0034] In some embodiments, a CCA 130A processing one or more USD layers 118 of the root USD data structure 116 may subscribe to (e.g., via an API provided by the CCCP application 120) one or more scene definition layers of these USD layers 118 to access one or more graph nodes associated with features of one or more objects 106. In some embodiments, the CCA 130A may synchronize or otherwise load those scene definition layers from the root USD data structure 116 into the local storage of the client node 104A (shown at USD scene definition layer 134). The CCA 130A may operate on the definition primitives included in the USD scene definition layer 134 as part of a workflow for collaboratively updating one or more objects 106 and / or other elements of the common interaction environment 108. When the CCA 130A performs operations to apply editing to features defined by the definition primitives of the USD scene definition layer 134, these operations are tracked at the client node 104A in the form of a series of overlay primitives generated by the CCA 130A. Such overlay primitives are collected in the form of overlay scene graphs represented by one or more USD overlay layers 136.

[0035] For example, refer to Figure 2 , Figure 2 A process is illustrated for generating a USD overlay layer 136 based on features (e.g., characteristics) of one or more objects 106 generated by updating the definition primitives represented by the USD scene definition layer 134. For example... Figure 2 As shown, the USD scene definition layer 134 may include a USD definition primitive 212, which specifies parameters for defining the object 202 of the public interaction environment 108. In this example, CCA 130 reads the USD scene definition layer 134 and identifies nodes including the USD definition primitive 212 to render the object 202 as a spherical object. Figure 2As shown, USD definition primitive 212 can specify parameters for object 202, defining the object as a sphere, which may have specified position, radius, rotation, scaling factor, and / or translation. A user and / or generative AI system may apply one or more instructions to CCA 130 to apply one or more scene edits 210 to one or more features of object 202 established by USD definition primitive 212. In response to one or more scene edits 210, CCA 130 generates a modified version of the public interactive environment 108', which includes a modified instance of object 202, shown as modified object 202', reflecting one or more scene edits 210. While CCA 130 processes scene edits 210, CCA 130 also tracks which definition primitives of the USD scene definition layer 134 are affected (e.g., modified) by scene edits 210 and generates one or more overlay primitives 214 representing incremental information. In other words, one or more overlay primitives 214 instruct CCA 130 to use parameters provided by one or more overlay primitives 214 when rendering the element (object), instead of the corresponding parameters specified for the element by USD definition primitive 212. In this example, scene editor 210 applies changes to the Y-plane translation value specified by USD definition primitive 212. Accordingly, CCA 130 generates an overlay primitive 214 specifying the updated Y-plane translation value, and this overlay primitive 214 is saved (e.g., attached) to USD overlay layer 136. As CCA 130 continues to render object 202 of the public interactive environment 108, the object will be rendered at least partially based on USD scene definition layer 134, which includes USD definition primitive 212. However, based on the graph nodes processed by USD scene definition layer 134, CCA 130 will identify and apply nodes of the overlay scene graph that include overlay primitive 214 to render modified object 202' (e.g., with edited Y-plane translation relative to object 202). In some embodiments, CCA 130 may also save modified versions of the modified object 202' and / or the public interaction environment 108' to a feature-based native file 132 at the client node 104.

[0036] When CCA 130 processes edits applied to elements in the public interactive environment 108, the overlay primitives generated from these edits can be accumulated (e.g., sequentially) in the USD overlay layer 136 at client node 104. Based on the USD framework, CCA 130A can propagate edits to other CCAs (e.g., CCA 130B-130N) at other client nodes subscribing to the USD layer 118 of the root USD data structure 116. As an example, Figure 3 The overlay propagation process according to some embodiments of this disclosure is illustrated. For example... Figure 3As shown, CCA 130A can initiate the propagation of one or more USD overlay layers 136 by sending a new layer message 310 to CCCP server 112 (e.g., to CCCP database 114) (e.g., via network 105). The new layer message 310 may include a propagated USD overlay layer 312 based on one or more USD overlay layers 136 including overlay primitives (e.g., overlay primitive 214). In some embodiments, CCA 130A may determine when to push the new layer message 310 to CCCP server 112 based on one or more criteria. For example, CCA 130A may push the new layer message 310 on a manual basis (e.g., based on user input), based on a predetermined frequency (e.g., once every specified time interval), and / or according to event-based criteria (e.g., the size of the USD overlay layer 136 and / or the accumulated number of overlay primitives 214 reaches a threshold). In response to receiving a new layer message 310, CCCP server 112 may add the overlay primitive content from the propagated USD overlay layer 312 as a new USD overlay layer 315 to the root USD data structure 116 (e.g., by adding one or more USD layers 315 that include the overlay scene graph of the propagated USD overlay layer 312).

[0037] CCCP server 112 may send a new layer notification 320 message to other CCAs 130B-130Ns subscribing to root USD data structure 116, indicating that a new USD overlay 315 has been added. CCAs 130B-130Ns receiving the new layer notification 320 may selectively read and apply overlay primitives propagated via the new USD overlay 315 added to the root USD file. In some embodiments, the new layer notification 320 may include the propagated USD overlay 312 (or another representation of the new USD overlay 315) as a payload passed to other CCAs 130B-130Ns.

[0038] Because the payloads of the new layer message 310 and / or the new layer notification 320 can be reserved for the propagated USD overlay layer 312 and the corresponding overlay primitives, the payloads of these messages are relatively sparse compared to messages carrying a scene definition layer that may include definition primitives and / or other types of primitives and scene definition information. Therefore, sharing one or more USD overlay layers 136, including overlay primitives generated by CCA 130A, is a relatively lightweight operation that consumes far fewer computational and / or network resources than transmitting a complete USD layer.

[0039] In some embodiments, the propagated USD overlay 312 may format the overlay primitives as ASCII or other text-based representations of the overlay primitives, or compress them into binary serialization or other formats to make the message smaller and / or consume fewer resources during transmission. In some embodiments, the propagated USD overlay 312 may be delivered to CCCP server 112 and / or propagated as a USD payload to one or more CCA 130A-130N servers.

[0040] In some embodiments, CCCP application 120 may provide one or more APIs exposed to CCAs 130A-130N at client nodes 104A-104N to allow CCAs 130A-130N to perform one or more overlay propagation functions as described herein. For example, a CCA may invoke the messaging API of CCCP 102 to send a message to CCCP server 112 including the propagated USD overlay 312 to propagate the edits made by that CCA. Another API provided by CCCP application 120 may implement a subscription function for use by CCAs 130A-130N to specify layers in the root USD data structure 116; when a corresponding new USD overlay 315 becomes available, CCAs 130A-130N subscribe to these layers to receive notifications (e.g., new layer notification 320).

[0041] By receiving the CCA of the propagated USD overlay 312, it is possible to accurately determine which features of which elements in the public interaction environment 108 have been updated by the application scenario-edited CCA, and what the updated parameter values ​​of these features are. Receiving the CCA of the public interaction environment 108 can resolve a series of overlay primitives to identify edits in features related to that CCA and ignore further processing of overlay primitives considered irrelevant. For example, because surface texture and object position are completely independent features of the element, the first CCA managing the surface texture of elements within the public interaction environment can ignore overlay primitives from the public interaction environment 108 that rearrange the element's position (e.g., position and / or rotation).

[0042] In some embodiments, because overlay primitives are presented sequentially in the propagated USD overlay layer 312 in the order of execution of edit operations, the CCA 130A-130N receiving the propagated USD overlay layer 312 can efficiently find the latest occurrence of the overlay primitive associated with the feature and apply that overlay primitive. Overlay primitives replaced by subsequent overlay primitives in the propagated USD overlay layer 312 may be considered outdated information and ignored. In some embodiments, the propagated USD overlay layer 312 may include a timestamp applied by the CCA, indicating the time when the edit operation was performed and / or the overlay was created. In these embodiments, the CCA receiving the propagated USD overlay layer 312 may determine which overlay primitives (e.g., the latest) to apply based on at least one or more timestamps. In some embodiments, based on the propagated USD overlay layer 312, the CCA may merge one or more overlay primitives back into baseline definition primitives in the USD scene definition layer 134 and / or the root USD data structure 116 to establish the updated feature as the new baseline definition. That is, CCCP application 120 can use parameters from one or more overlay primitives from the propagated USD overlay 312 to modify the defined primitives specified by other USD layers 118 of the root USD data structure 116. In some embodiments, the propagated USD overlay 312 may be provided by CCCP server 112 based on a file format plugin that exposes an API that provides the underlying format architecture of the propagated USD overlay 312 for receiving CCAs, so as to facilitate reading and / or parsing the overlay primitives included in the propagated USD overlay 312.

[0043] In some embodiments, CCCP application 120 may include a conflict management function (e.g., an algorithm) that operates when two or more CCAs 130A-130N send a new layer message 310 of a propagated USD overlay layer 312 having overlay primitives that act on the same feature. The conflict management function may send conflict notifications to users of one or both CCAs, informing them of the existence of the conflict so that they can coordinate to determine a solution and notify the conflict management function of the selected solution (e.g., which overlay primitives will be retained and / or discarded). In some embodiments, the conflict management function may also delay and / or otherwise order the propagated USD overlay layers 312 received from different CCAs 130. For example, if two different CCAs 130 interact with the same feature of one or more objects 106, the conflict management function can provide instructions that the corresponding definition primitive modified by multiple overlay primitives can execute the following commands: update according to the order in which the propagated USD overlays 312 are received; ignore the first propagated USD overlay 312 and use the second propagated USD overlay 312; ignore the second propagated USD overlay 312 and use the first propagated USD overlay 312; or various combinations thereof. For example, different propagated USD overlays 312 received from different CCAs 130A-130N within each other's threshold time periods can be evaluated, and then one or more rules can be applied, such as ignoring the first propagated USD overlay 312, because the probability of rendering the second propagated USD overlay 312 before seeing the result of the first propagated USD overlay 312 may be greater than a certain threshold. In addition, in various embodiments, a messaging service can be provided to notify users of rapid changes to different elements, which can be used to regulate or otherwise facilitate communication between CCA 130A-130N.

[0044] In some embodiments, one or more CCCP applications 120 may generate a rendering of a public interactive environment 108 through which users (e.g., users of CCA 130A-130N and / or users of other applications) and / or agents (e.g., agents implemented by computer algorithms and / or artificial intelligence) may interface to visualize and / or interact with a scene that includes one or more elements from various elements in the root USD data structure 116. That is, CCCP 102 may maintain a real-time representation of the analysis of the scene described by the root USD data structure 116 and provide this scene as the public interactive environment 108 to individual client nodes 104A-104N. CCCP applications 120 may generate computer-generated simulations that render element features defined by one or more scene definition layers of the root USD data structure 116 and / or modified by one or more propagated USD overlay layers 312 propagated by one or more CCA 130A-130N. In some embodiments, users and / or agents operating in the virtual rendering of the public interactive environment 108 produced by CCCP 102 can also initiate editing of element features in the scene (e.g., using CCCP application 120 which includes a CCA directly hosted by CCCP server 112). For example, a user of a CCA hosted by CCCP 102 can move the position of one or more elements, which can represent a deviation of the element's positional features from the position defined by the baseline scene definition layer of the root USD data structure 116. In this case, the CCA hosted by CCCP 102 can generate one or more new overlays that capture overlay primitives that capture the edited positions of the elements. Those overlays produced by the CCA hosted by CCCP 102 can be propagated to client nodes by adding a new USD overlay 315 to the root USD data structure 116 and notifying the subscribed CCA at the client node via a new layer notification 320, in the same manner as the propagation of the propagated USD overlay 312 by CCA 130A-130N hosted by client nodes described herein.

[0045] Therefore, the systems and methods disclosed herein can be used to enhance and / or improve collaboration among users performing different, often incompatible, underlying software systems in a public environment.

[0046] Now for reference Figure 4 , Figure 4 This is a flowchart illustrating a method 400 for propagating overlay-based data changes for a distributed collaborative content creation platform, according to some embodiments of this disclosure. (The text here refers to...) Figure 4 The features and elements described in method 400 can be used in combination, in combination with, or substituted for elements of any other embodiment discussed herein, and vice versa. Furthermore, Figure 4 The functionality, structure, and other descriptions of the elements of the embodiments described herein may be applied to similar or similar named or described elements in any of the accompanying drawings and / or embodiments described herein, and vice versa.

[0047] Each block of the method 400 described herein includes a computational process that can be executed using any combination of hardware, firmware, and / or software. For example, various functions can be executed by one or more processors that include processing circuitry and execute instructions stored in memory. These methods can be additionally or alternatively implemented as computer-usable instructions stored on a computer storage medium. These methods can be provided as standalone applications, services, or managed services (standalone or in combination with another managed service) or plug-ins to another product, to name a few. Furthermore, as an example, method 400 is for applications including... Figure 1 This is described in the context of the collaborative environment 100 of the content creation platform 102. However, these methods may additionally or alternatively be performed by any system or combination of systems, including but not limited to the systems described herein.

[0048] In some embodiments, method 400 may primarily be used for: tracking one or more changes associated with one or more scene primitives across one or more instances of one or more content creation applications, these scene primitives being organized into one or more scene definition layers representing one or more elements of a virtual scene environment; and propagating the one or more changes to one or more client nodes of a Collaborative Content Creation Platform (CCCP), these client nodes having subscribed to the one or more scene primitives through CCCP; wherein the propagation includes: generating an overlay feature scene graph based at least on the one or more changes; creating at least one overlay layer including a representation of the overlay feature scene graph; and instructing a CCCP server node to add the at least one overlay layer to the representation of the virtual scene environment.

[0049] Method 400 includes, at block B402, accessing a scene graph representing a computer-generated environment, the computer-generated environment comprising one or more scene elements, each scene element comprising one or more scene features, wherein each of the one or more scene features is defined by one or more scene primitives included in at least one or more scene layers of the scene graph. The scene graph may include one or more scene definition layers, wherein the one or more scene definition layers include one or more definition primitives that define one or more properties of the one or more features. The one or more scene definition layers may be based at least on a root scene definition framework hosted by a server node of a collaborative content creation platform. In some embodiments, the root scene definition framework may include a root USD data structure, such as, but not limited to, the Open Universal Scene Description (OpenUSD) data structure. For example, as per [reference to...] Figure 1 As discussed, one or more CCCP servers 112 of CCCP 102 may host one or more CCCP databases 114, which include a root USD data structure 116 (e.g., a file including a scene graph). This root USD data structure may be used by CCCP application 120 and / or content creation application (CCA) 130 to render one or more elements (e.g., one or more objects 106) of a scene within a public interactive environment 108. The public interactive environment 108 hosted by CCCP server 112 of CCCP 102 may be instantiated based on a root USD data structure 116 (e.g., a file including a scene graph) that includes multiple USD layers 118. USD layers 118 may define contributions to the scene graph of the root USD data structure 116 (which is used to render one or more aspects of the elements (e.g., objects 106) within the public interactive environment 108). The root USD data structure 116 may include a scene graph data structure comprising a set of graph nodes arranged in a graph or tree structure for organizing logical and / or specific features of a computer graphics scene. USD layers 118 may include one or more graph nodes from the root USD data structure 116, thus representing primitives associated with objects, attributes, or transformation matrices applied to each child node of that node. In some embodiments, a CCA 130A processing one or more USD layers 118 of the root USD data structure 116 may subscribe (e.g., via an API provided by CCCP application 120) to one or more scene definition layers (e.g., scene definition scene graphs) of USD layers 118 to access one or more graph nodes associated with features of one or more objects 106. In some embodiments, the CCA 130A may synchronize or otherwise load those scene definition layers from the root USD data structure 116 into the local memory of the client node 104A (shown at USD scene definition layer 134).

[0050] Method 400, at box B404, includes accessing an overlay feature scene graph based at least on tracking one or more changes associated with the one or more scene primitives. For example, the method may generate an overlay feature scene graph based at least on tracking one or more changes associated with one or more scene definition primitives. This overlay scene graph may include a USD data structure layer. For example, the CCA 130A may operate on definition primitives included in the USD scene definition layer 134 as part of a workflow for collaboratively updating one or more objects 106 and / or other elements of the public interaction environment 108. The method may generate one or more overlay primitives to generate the overlay feature scene graph based at least on tracking the one or more changes associated with the one or more scene definition primitives. When the CCA 130A performs operations to apply editing to features defined by the definition primitives of the USD scene definition layer 134, these operations are tracked at the client node 104A in the form of a series of overlay primitives generated by the CCA 130A. Such overlay primitives are collected in the form of an overlay scene graph represented by one or more USD overlay layers 136. The overlay feature scene graph (e.g., defined by USD overlay layer 136) may include one or more overlay primitives to overlay parameters of at least one scene primitive among one or more scene primitives of a computer-generated environment, the at least one scene primitive being associated with at least one of the following: the geometry of one or more elements of the computer-generated environment, the surface texture of one or more elements of the computer-generated environment, the material composition of one or more elements of the computer-generated environment, the surface color of one or more elements of the computer-generated environment, the lighting environment of one or more elements of the computer-generated environment, the weight value of one or more elements of the computer-generated environment, or the inventory quantity of one or more elements of the computer-generated environment. In some embodiments, one or more processors execute one or more generative artificial intelligence (AI)-based agents within a computer-generated environment, wherein one or more generative AI-based agents generate one or more modifications associated with one or more scene primitives based at least on computer-based perception of the computer-generated environment.

[0051] Method 400 includes, at box B406, at least one overlay layer for creating a scene graph, the at least one overlay layer including a representation of the overlay feature scene graph. For example, refer to... Figure 2 , Figure 2 A process is illustrated for generating a USD overlay layer 136 based on features (e.g., characteristics) of one or more objects 106 generated from the definition primitives represented by the USD scene definition layer 134. For example... Figure 2As shown, the USD scene definition layer 134 may include USD definition primitives 212 that specify parameters for defining objects 202 of the public interactive environment 108. When CCA 130 processes edits applied to elements of the public interactive environment 108, overlay primitives generated from these edits may be accumulated (e.g., sequentially) into the USD overlay layer 136 at client node 104. Based on the USD framework, CCA 130A may propagate edits to other CCAs (e.g., CCAs 130B-130N) at other client nodes subscribing to the USD layer 118 of the root USD data structure 116. In some embodiments, at least one overlay is generated by a first instance of a first CCA that has subscribed to at least a portion of a computer-generated environment to the CCCP and propagated to at least one of: one or more second instances of the first CCA, or one or more instances of at least one second CCA that has subscribed to the at least portion of the computer-generated environment to the CCCP. In some embodiments, at least one overlay is encoded according to a 3D graphics data format. At least one overlay may be generated by an application hosted on a collaborative content creation platform and propagated by server nodes to one or more digital content creation (DCC) applications that subscribe to the root scene definition framework. At least one overlay layer may represent a temporal sequence of overlay primitives, which are generated based at least on the temporal order of one or more changes associated with one or more scene definition primitives. In some embodiments, the method may include applying a timestamp to at least one overlay layer, indicating the time when one or more changes associated with one or more scene definition primitives were made.

[0052] Method 400, at box B408, includes propagating at least one overlay to one or more client nodes of a Collaborative Content Creation Platform (CCCP), each client node executing at least one instance of a Content Creation Application (CCA), wherein the computer-generated environment is encoded using a three-dimensional (3D) graphics data format. In some embodiments, the 3D graphics data format includes an Open Universal Scene Description (OpenUSD). In some embodiments, the method may propagate at least one overlay to one or more client nodes coupled to the server nodes of the Collaborative Content Creation Platform, at least based on at least one overlay added by a server node instructing the CCCP (e.g., added to a root scene definition framework). As an example, Figure 3 The process for overlay propagation according to some embodiments of this disclosure is illustrated. For example... Figure 3As shown, CCA130A can initiate the propagation of one or more USD overlay layers 136 by sending a new layer message 310 to CCCP server 112 (e.g., to CCCP database 114) via network 105. The new layer message 310 may include a propagated USD overlay layer 312 based on one or more USD overlay layers 136 including overlay primitives (e.g., overlay primitive 214). Messages sent to server nodes can cause server nodes to add at least one overlay to the root scene definition framework. For example, in response to receiving the new layer message 310, CCCP server 112 may add overlay primitive content from the propagated USD overlay layer 312 as a new USD overlay layer 315 to the root USD data structure 116 (e.g., by adding one or more USD layers 315 to the overlay scene graph including the propagated USD overlay layer 312).

[0053] CCCP server 112 may transmit a new layer notification 320 message to other CCAs 130B-130N subscribing to root USD data structure 116, indicating that a new USD overlay 315 has been added. Therefore, in some embodiments, the method may include processing one or more second overlays received from the server node and selectively applying one or more overlay primitives defined by the one or more second overlays to one or more scene definition primitives of one or more scene definition layers. CCAs 130B-130N receiving the new layer notification 320 may selectively read and apply the overlay primitives propagated by the new USD overlay 315 added to the root USD file. In some embodiments, CCAs 130B-130N receiving the new layer notification 320 may parse one or more second overlays to find the most recent occurrence of an overlay primitive associated with a scene definition primitive of one or more scene definition layers in the one or more second overlays, thereby applying the most recent occurrence of the overlay primitive to the scene definition primitive.

[0054] In some embodiments, the systems and methods described herein can be performed within an immersive simulation environment (e.g., NVIDIA's DriveSIM platform, NVIDIA's Omniverse platform, etc.) using simulated data (e.g., simulated sensor data from simulated sensors of virtual or simulated machines). For example, simulated sensor data and / or map data can be used to identify elements of interest (e.g., objects, regions of interest, etc.) within a public interactive environment 108 provided by CCCP 102, and this information can be used to perform operations associated with one or more virtual machines within the environment (e.g., navigation). These simulated operations can be used to test the performance of underlying algorithms, systems, and / or processes before deploying them to the real world. In some cases, simulations based on the public interactive environment 108 can be used to generate synthetic training data, for example, including training data from objects, systems, structures, regions of interest, and / or sub-regions of interest within the simulation. The synthetic training data (supplementing or replacing real-world data) can then be processed to determine geometry, visualizations, and / or other information associated with such subjects of interest. In any example, such as when a simulated environment is used for testing, validation, training, etc., the simulated environment and / or associated training data may be rendered or otherwise generated using one or more optical transport algorithms, such as ray tracing and / or path tracing algorithms. In some embodiments, the simulated environment and / or one or more of its objects, features, and components may be generated or managed within a 3D content collaboration platform (e.g., NVIDIA's Omniverse) for industrial digitization, generative physics AI, and / or other use cases, applications, or services. For example, a content collaboration platform (e.g., CCCP 102) or system may include systems for using or developing generic scene descriptions (USD) (e.g., OpenUSD) data to manage objects, features, scenes, etc., within simulated environments, digital environments, etc. In some embodiments disclosed herein, the platform may include realistic physical simulations, such as using NVIDIA's PhysX SDK, to simulate real physics and physical interactions with simulations hosted on the platform. Content collaboration platforms can integrate OpenUSD descriptions with ray tracing / path tracing / light transport simulations (e.g., NVIDIA's RTX rendering technology) into software tools and simulation workflows to build, train, deploy, or test AI systems, such as systems for testing, validating, training (e.g., machine learning models, neural networks, etc.) and / or other tasks related to automobiles, robots, machines, or other applications.

[0055] The systems and methods described herein can be used for a variety of purposes, including but not limited to: machine control, machine mobility, machine driving, synthetic data generation, model training, perception, augmented reality, virtual reality, mixed reality, robotics, security and surveillance, simulation and digital twins, autonomous or semi-autonomous machine applications, deep learning, environmental simulation, object or participant simulation and / or digital twins, data center processing, conversational AI, optical transport simulation (e.g., ray tracing, path tracing, etc.), collaborative content creation of 3D assets, cloud computing, generative AI, and / or any other suitable application.

[0056] The disclosed embodiments may be included in a variety of different systems, such as automotive systems (e.g., control systems for autonomous or semi-autonomous machines, perception systems for autonomous or semi-autonomous machines), systems implemented using robots, aviation systems, medical systems, marine systems, smart area monitoring systems, systems for performing deep learning operations, systems for performing simulation operations, systems for performing digital twin operations, systems implemented using edge devices, systems including one or more virtual machines (VMs), systems for performing synthetic data generation operations, systems implemented at least partially in a data center, systems for performing conversational AI operations, systems implementing one or more language models (e.g., one or more large language models (LLMs) and / or one or more visual language models (VLMs)), systems for performing optical transmission simulations, systems for performing collaborative content creation of 3D assets, systems implemented at least partially using cloud computing resources, and / or other types of systems.

[0057] In some examples, the machine learning models described herein (e.g., deep neural networks, language models, LLM, VLM, multimodal language models, perceptual models, tracking models, fusion models, Transformer models, diffusion models, encoder-only models, decoder-only models, encoder-decoder models, neural rendering field (NERF) models, etc.) can be packaged as microservices, such as inference microservices (e.g., NVIDIA NIMs), which may include containers (e.g., operating system (OS) level virtualization packages) that may include an application programming interface (API) layer, a server layer, a runtime layer, and / or at least one model "engine". For example, an inference microservice may include the container itself and the model (e.g., weights and biases). In some cases, such as when one or more machine learning models are small enough (e.g., have a sufficiently small number of parameters), the model may be included within the container itself. In other examples, such as when one or more models are large, the one or more models may be hosted / stored in the cloud (e.g., in a data center) and / or hosted locally and / or at the edge (e.g., on a local server or computing device, but outside the container). In these embodiments, the one or more models may be accessed via one or more APIs (such as REST APIs). Therefore, in some embodiments, the one or more machine learning models described herein may be deployed as inference microservices to accelerate model deployment on any cloud, data center, or edge computing system while ensuring data security. For example, an inference microservice may include one or more APIs, pre-configured containers for simplified deployment, an optimized inference engine (e.g., execution software built using standardized AI model deployments, such as NVIDIA's Triton Inference Server, and / or one or more APIs for high-performance deep learning inference, which may include inference runtimes and model optimizations for low latency and high throughput for production applications, such as NVIDIA's TensorRT), and / or enterprise management data for telemetry (e.g., including identity, metrics, health checks, and / or monitoring). The one or more machine learning models described herein, along with the accelerated infrastructure, may be included as part of a microservice, capable of deployment with a single command and / or orchestrated and automatically scaled on the accelerated infrastructure via a container orchestration system (e.g., reaching data center scale on a single device). Therefore, an inference microservice may include one or more machine learning models (e.g., models optimized for high-performance inference), inference runtime software for executing one or more machine learning models and providing output / response to inputs (e.g., user queries, prompts, etc.), and enterprise management software for providing health checks, identity and / or other monitoring. In some embodiments, the inference microservice may include software for performing in-situ replacements and / or updates to one or more machine learning models.When replacing or updating, the software performing the replacement / update maintains the user configuration of the inference runtime software and the enterprise management software.

[0058] Example computing device

[0059] Figure 5 This is a block diagram of an example computing device 500 suitable for implementing some embodiments of the present disclosure. The computing device 500 may include an interconnect system 502 directly or indirectly coupled to: a memory 504, one or more central processing units (CPUs) 506, one or more graphics processing units (GPUs) 508, a communication interface 510, input / output (I / O) ports 512, input / output components 514, a power supply 516, one or more presentation components 518 (e.g., one or more displays), and one or more logic units 520. In at least one embodiment, one or more computing devices 500 may include one or more virtual machines (VMs), and / or any component of its components may include virtual components (e.g., virtual hardware components). For a non-limiting example, one or more GPUs 508 may include one or more vGPUs, one or more CPUs 506 may include one or more vCPUs, and / or one or more logic units 520 may include one or more virtual logic units. Accordingly, one or more computing devices 500 may include discrete components (e.g., a full GPU dedicated to computing device 500), virtual components (e.g., a portion of the GPU dedicated to computing device 500), or combinations thereof. In some embodiments, one or more aspects of client node 104 and / or one or more CCCP servers 112 may be executed by or implemented using computer device 500.

[0060] although Figure 5 The various blocks are shown connected via an interconnect system 502 with wiring, but this is not intended to be limiting and is merely for clarity. For example, in some embodiments, a presentation component 518, such as a display device, can be considered an I / O component 514 (e.g., if the display is a touchscreen). As another example, CPU 506 and / or GPU 508 may include memory (e.g., memory 504 may represent a storage device other than the memory of GPU 508, CPU 506, and / or other components). Therefore, Figure 5The computing devices mentioned are merely illustrative. No distinction is made between categories such as "workstation," "server," "laptop," "desktop," "tablet," "client device," "mobile device," "handheld device," "game console," "electronic control unit (ECU)," "virtual reality system," and / or other device or system types, as all of these are considered within the same category. Figure 5 Within the scope of computing devices.

[0061] Interconnect system 502 may represent one or more links or buses, such as address buses, data buses, control buses, or combinations thereof. Interconnect system 502 may include one or more bus types, such as Industry Standard Architecture (ISA) buses, Extended Industry Standard Architecture (EISA) buses, Video Electronics Standards Association (VESA) buses, Peripheral Component Interconnect (PCI) buses, Peripheral Component Interconnect Fast (PCIe) buses, and / or another type of bus or link. In some embodiments, there is a direct connection between components. As an example, CPU 506 may be directly connected to memory 504. Furthermore, CPU 506 may be directly connected to GPU 508. In cases where there is a direct or point-to-point connection between components, interconnect system 502 may include a PCIe link to perform the connection. In these examples, a PCI bus is not required in computing device 500.

[0062] The memory 504 may include any medium of a wide variety of computer-readable media. A computer-readable medium can be any available medium that can be accessed by the computing device 500. Computer-readable media may include volatile and non-volatile media, as well as removable and non-removable media. For example and without limitation, computer-readable media may include computer storage media and communication media.

[0063] Computer storage media may include volatile and non-volatile media and / or removable and non-removable media, implemented in any way or by any method or technique for storing information such as computer-readable instructions, data structures, program modules, and / or other data types. For example, memory 504 may store computer-readable instructions (e.g., representing programs and / or program elements, such as an operating system). Computer storage media may include, but is not limited to, RAM, ROM, EEPROM, flash memory or other storage technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage devices, magnetic tape cassettes, magnetic tape, disk storage devices or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by computing device 500. As used herein, computer storage media does not include the signal itself.

[0064] Computer storage media may contain computer-readable instructions, data structures, program modules, and / or other data types in modulated data signals such as carrier waves or other transmission mechanisms, and include any information transport medium. The term "modulated data signal" can refer to a signal whose characteristics are set or altered in a manner that encodes information into that signal. For example and without limitation, computer storage media may include wired media such as wired networks or direct wired connections, and wireless media such as sound, RF, infrared, and other wireless media. Any combination of the above should also be included within the scope of computer-readable media.

[0065] CPU 506 may be configured to execute at least some computer-readable instructions to control one or more components of computing device 500 to perform one or more of the methods and / or processes described herein. Each of CPU 506 may include one or more cores (e.g., one, two, four, eight, twenty-eight, seventy-two, etc.) capable of processing a large number of software threads simultaneously. CPU 506 may include any type of processor and may include different types of processors depending on the type of computing device 500 implemented (e.g., processors with fewer cores for mobile devices and processors with more cores for servers). For example, depending on the type of computing device 500, the processor may be an advanced RISC machine (ARM) processor implemented using Reduced Instruction Set Computing (RISC) or an x86 processor implemented using Complex Instruction Set Computing (CISC). In addition to one or more microprocessors or supplementary coprocessors such as math coprocessors, computing device 500 may also include one or more CPUs 506.

[0066] In addition to or in place of CPU 506, GPU 508 may be configured to execute at least some computer-readable instructions to control one or more components of computing device 500 to perform one or more methods and / or processes described herein. One or more GPUs 508 may be integrated GPUs (e.g., having one or more CPUs 506) and / or one or more GPUs 508 may be discrete GPUs. In embodiments, one or more GPUs 508 may be coprocessors of one or more CPUs 506. Computing device 500 may use GPUs 508 to render graphics (e.g., 3D graphics) or perform general-purpose computing. For example, GPUs 508 may be used for general-purpose computing on a GPU (GPGPU). GPUs 508 may include hundreds or thousands of cores capable of processing hundreds or thousands of software threads simultaneously. GPUs 508 may generate pixel data for outputting an image in response to rendering commands (e.g., rendering commands received from CPUs 506 via a host interface). GPUs 508 may include graphics memory, such as display memory, for storing pixel data or any other suitable data, such as GPGPU data. Display memory may be included as part of memory 504. 508 may include two or more GPUs operating in parallel (e.g., via a link). The link may connect the GPUs directly (e.g., using NVLINK) or via a switch (e.g., using NVSwitch). When combined, each GPU 508 may generate pixel data or GPGPU data for different portions of the output or different outputs (e.g., a first GPU for a first image and a second GPU for a second image). Each GPU may include its own memory or may share memory with other GPUs.

[0067] In addition to or replacing CPU 506 and / or GPU 508, logic unit 520 may be configured to execute at least some computer-readable instructions to control one or more components of computing device 500 to perform one or more methods and / or processes described herein. In embodiments, CPU 506, GPU 508, and / or logic unit 520 may perform any combination of methods, processes, and / or portions thereof, discretely or jointly. One or more logic units 520 may be part of and / or integrated with one or more of CPU 506 and / or GPU 508, and / or one or more logic units 520 may be discrete components or otherwise external to CPU 506 and / or GPU 508. In embodiments, one or more logic units 520 may be coprocessors of one or more CPUs 506 and / or one or more GPUs 508. In some embodiments, one or more functions of client node 104, one or more CCA130 and / or CCCP102 may be executed or implemented using software instructions executed by one or more CPUs 506, one or more GPUs 508 and / or one or more logic units 520.

[0068] Examples of logic unit 520 include one or more processing cores and / or components thereof, such as data processing unit (DPU), tensor core (TC), tensor processing unit (TPU), pixel vision core (PVC), vision processing unit (VPU), graphics processing cluster (GPC), texture processing cluster (TPC), streaming multiprocessor (SM), tree traversal unit (TTU), artificial intelligence accelerator (AIA), deep learning accelerator (DLA), arithmetic logic unit (ALU)), application-specific integrated circuit (ASIC), floating-point unit (FPU), input / output (I / O) element, peripheral component interconnect (PCI) or peripheral component interconnect fast (PCIe) element, etc.

[0069] Communication interface 510 may include one or more receivers, transmitters, and / or transceivers that enable computing device 500 to communicate with other computing devices via electronic communication networks, including wired and / or wireless communications. Communication interface 510 may include components and functions that enable communication via any of several different networks, such as wireless networks (e.g., Wi-Fi, Z-Wave, Bluetooth, Bluetooth LE, ZigBee, etc.), wired networks (e.g., via Ethernet or wireless bandwidth communication), low-power wide area networks (e.g., LoRaWAN, SigFox, etc.), and / or the Internet. In one or more embodiments, logic unit 520 and / or communication interface 510 may include one or more data processing units (DPUs) to directly transmit data received via a network and / or via interconnect system 502 to one or more GPUs 508 (e.g., their memory).

[0070] I / O port 512 enables computing device 500 to be logically coupled to other devices, including I / O component 514, presentation component 518, and / or other components, some of which may be built into (e.g., integrated into) computing device 500. Illustrative I / O component 514 includes microphones, mice, keyboards, joysticks, game pads, game controllers, satellite dish antennas, scanners, printers, wireless devices, and so on. I / O component 514 can provide a Natural User Interface (NUI) for processing user-generated air gestures, voice, or other physiological input. In some instances, the input may be transmitted to appropriate network elements for further processing. The NUI can implement any combination of voice recognition, stylus recognition, facial recognition, biometric recognition, on-screen and adjacent-screen gesture recognition, air gestures, head and eye tracking, and touch recognition associated with the display of computing device 500 (described in more detail below). Computing device 500 may include depth cameras such as stereo camera systems, infrared camera systems, RGB camera systems, touchscreen technology, and combinations thereof for gesture detection and recognition. In addition, computing device 500 may include an accelerometer or gyroscope that enables motion detection (e.g., as part of an inertial measurement unit (IMU)). In some examples, the output of the accelerometer or gyroscope may be used by computing device 500 to render immersive augmented reality or virtual reality.

[0071] Power supply 516 may include a hard-wired power supply, a battery power supply, or a combination thereof. Power supply 516 may supply power to computing device 500 so that the components of computing device 500 can operate.

[0072] The presentation component 518 may include a display (e.g., a monitor, touchscreen, television screen, head-up display (HUD), other display types, or combinations thereof), speakers, and / or other presentation components. The presentation component 518 may receive data from other components (e.g., GPU 508, CPU 506, DPU, etc.) and output that data (e.g., as images, video, sound, etc.). In some embodiments, one or more CCA130s may render one or more objects 106 and / or other elements of the common interactive environment 108 on one or more presentation components 518 of the client node 104.

[0073] Example Data Center

[0074] Figure 6 An example data center 600 that can be used in at least one embodiment of this disclosure is illustrated. Data center 600 may include a data center infrastructure layer 610, a framework layer 620, a software layer 630, and / or an application layer 640. In some embodiments, one or more functions of client node 104, one or more CCA 130s, and / or CCCP 102 may be performed by or implemented using data center 600.

[0075] like Figure 6As shown, the data center infrastructure layer 610 may include a resource coordinator 612, grouped computing resources 614, and node computing resources (“nodes CR”) 616(1)-616(N), where “N” represents any complete positive integer. In at least one embodiment, nodes CR 616(1)-616(N) may include, but are not limited to, any number of central processing units (CPUs) or other processors (including DPUs, accelerators, field-programmable gate arrays (FPGAs), graphics processing units or graphics processing units (GPUs), etc.), memory devices (e.g., dynamic read-only memory), storage devices (e.g., solid-state drives or disk drives), network input / output (NW I / O) devices, network switches, virtual machines (VMs), power modules and / or cooling modules, etc. In some embodiments, one or more nodes CR 616(1)-616(N) may correspond to servers having one or more of the aforementioned computing resources. Furthermore, in some embodiments, nodes CR616(1)-616(N) may include one or more virtual components, such as vGPU, vCPU, etc., and / or one or more of nodes CR616(1)-616(N) may correspond to virtual machines (VMs). In some embodiments, one or more functions of client node 104, one or more CCA130 and / or CCCP 102 may be executed or implemented using software instructions executed by nodes CRs 616(1)-616(N).

[0076] In at least one embodiment, the grouped computing resources 614 may include individual groups (not shown) of node CR616 housed in one or more racks, or a plurality of racks (also not shown) housed in data centers in various geographic locations. Individual groups of node CRs within the grouped computing resources 614 may include computing, networking, memory, or storage resources that can be configured or allocated to support groups of one or more workloads. In at least one embodiment, several node CR616, including CPUs, GPUs, DPUs, and / or other processors, may be grouped within one or more racks to provide computing resources to support one or more workloads. One or more racks may also include any number of power modules, cooling modules, and / or network switches, in any combination.

[0077] Resource coordinator 612 can be configured or otherwise controlled to control one or more nodes CR616(1)-616(N) and / or groups of computing resources 614. In at least one embodiment, resource coordinator 612 may include a Software Design Infrastructure (SDI) management entity for data center 600. Resource coordinator 612 may include hardware, software, or some combination thereof.

[0078] In at least one embodiment, such as Figure 6 As shown, framework layer 620 may include a job scheduler 628, a configuration manager 634, a resource manager 636, and / or a distributed file system 638. Framework layer 620 may include a framework of software 632 supporting software layer 630 and / or one or more applications 642 of application layer 640. Software 632 or application 642 may respectively include web-based service software or applications, such as services or applications provided by Amazon Web Services, Google Cloud, and Microsoft Azure. Framework layer 620 may be, but is not limited to, a free and open-source software web application framework, such as Apache Spark, which can use distributed file system 638 for large-scale data processing (e.g., "big data"). TM (Hereinafter referred to as "Spark"). In at least one embodiment, job scheduler 628 may include Spark drivers to facilitate the scheduling of workloads supported by various layers of data center 600. Configuration manager 634 may be able to configure different layers, such as software layer 630 and framework layer 620 including Spark and distributed file system 638 for supporting large-scale data processing. Resource manager 636 is able to manage cluster or grouped computing resources mapped to or allocated to support distributed file system 638 and job scheduler 628. In at least one embodiment, cluster or grouped computing resources may include grouped computing resources 614 on data center infrastructure layer 610. Resource manager 636 may coordinate with resource coordinator 612 to manage these mapped or allocated computing resources.

[0079] In at least one embodiment, the software 632 included in the software layer 630 may include software used by at least a portion of nodes CR616(1)-616(N), grouped computing resources 614, and / or the distributed file system 638 of the framework layer 620. One or more types of software may include, but are not limited to, Internet web page search software, email virus scanning software, database software, and streaming video content software.

[0080] In at least one embodiment, one or more applications 642 included in application layer 640 may include one or more types of applications used by at least a portion of nodes CR616(1)-616(N), grouped computing resources 614, and / or the distributed file system 638 of framework layer 620. One or more types of applications may include, but are not limited to, any number of genomics applications, cognitive computing and machine learning applications, including training or inference software, machine learning framework software (e.g., PyTorch, TensorFlow, Caffe, etc.), and / or other machine learning applications used in conjunction with one or more embodiments.

[0081] In some embodiments, one or more functions of CCA130 and / or CCCP 102 described herein may be performed by software 632 and / or application 642, or may be implemented using software 632 and / or application 642.

[0082] In at least one embodiment, any of the configuration manager 634, resource manager 636, and resource coordinator 612 can perform any number and type of self-modification actions based on any amount and type of data acquired in any technically feasible manner. Self-modification actions can mitigate potentially poor configuration decisions by data center operators of data center 600 and can prevent underutilization and / or poor performance of the data center.

[0083] Data center 600 may include tools, services, software, or other resources to train one or more machine learning models or to use one or more machine learning models to predict or infer information according to one or more embodiments described herein. For example, a machine learning model can be trained by calculating weight parameters based on a neural network architecture using the software and / or computing resources described above with respect to data center 600. In at least one embodiment, information can be inferred or predicted using trained or deployed machine learning models corresponding to one or more neural networks using the resources described above with respect to data center 600 by using weight parameters calculated through one or more training techniques such as, but not limited to, those described herein.

[0084] In at least one embodiment, the data center 600 may use a CPU, application-specific integrated circuit (ASIC), GPU, FPGA, and / or other hardware (or corresponding virtual computing resources) to perform training and / or inference. Furthermore, one or more of the aforementioned software and / or hardware resources may be configured as a service to allow a user to train or perform information inference, such as image recognition, speech recognition, or other artificial intelligence services.

[0085] Example network environment

[0086] A network environment suitable for implementing embodiments of this disclosure may include one or more client devices, servers, network-attached storage (NAS), other backend devices, and / or other device types. Client devices, servers, and / or other device types (e.g., each device) may be... Figure 5 This is implemented on one or more instances of computing devices 500—for example, each device may include similar components, features, and / or functions of one or more computing devices 500. Furthermore, in the case of implementing backend devices (e.g., servers, NAS, etc.), the backend devices may be included as part of a data center 600, examples of which are described in this document. Figure 6 To describe in more detail.

[0087] Components of a network environment can communicate with each other via one or more networks, which can be wired, wireless, or both. A network may include multiple networks or one of multiple networks. For example, a network may include one or more wide area networks (WANs), one or more local area networks (LANs), one or more public networks such as the Internet and / or the Public Switched Telephone Network (PSTN), and / or one or more private networks. In cases where the network includes a wireless telecommunications network, components such as base stations, communication towers, or even access points (and other components) can provide wireless connectivity.

[0088] A compatible network environment may include one or more peer-to-peer network environments (in which case the server may not be included in the network environment) and one or more client-server network environments (in which case one or more servers may be included in the network environment). In a peer-to-peer network environment, the functionality described herein with respect to the server can be implemented on any number of client devices.

[0089] In at least one embodiment, the network environment may include one or more cloud-based network environments, distributed computing environments, combinations thereof, etc. A cloud-based network environment may include a framework layer, a job scheduler, a resource manager, and a distributed file system implemented on one or more servers, which may include one or more core network servers and / or edge servers. The framework layer may include a framework of software supporting the software layer and / or one or more applications supporting the application layer. The software or applications may respectively include web-based service software or applications. In embodiments, one or more client devices may use web-based service software or applications (e.g., by accessing the service software and / or applications via one or more application programming interfaces (APIs)). The framework layer may be, but is not limited to, free and open-source software web application frameworks, such as those used for large-scale data processing (e.g., "big data") using a distributed file system.

[0090] A cloud-based network environment can provide cloud computing and / or cloud storage for any combination of the computing and / or data storage functions (or one or more portions thereof) described herein. Any of these different functions can be distributed across multiple locations from a central or core server (e.g., distributed across one or more data centers at the state, region, country, global, etc.). The core server may assign at least a portion of the functionality to the edge server if the connection to the user (e.g., a client device) is relatively close to the edge server. A cloud-based network environment can be private (e.g., limited to a single organization), public (e.g., available to many organizations), and / or a combination thereof (e.g., a hybrid cloud environment).

[0091] One or more client devices may include the information described in this article. Figure 5 The components, features, and functions of one or more example computing devices 500 described are at least some of the components, features, and functions described. By way of example and not limitation, the client device may be embodied as a personal computer (PC), laptop computer, mobile device, smartphone, tablet computer, smartwatch, wearable computer, personal digital assistant (PDA), MP3 player, virtual reality headset, global positioning system (GPS) or device, video player, camera, surveillance equipment or system, vehicle, ship, spacecraft, virtual machine, drone, robot, handheld communication device, hospital equipment, gaming equipment or system, entertainment system, vehicle computer system, embedded system controller, remote control, electrical appliance, consumer electronics device, workstation, edge device, any combination of these described devices, or any other suitable device.

[0092] This disclosure can be described in the general context of machine-usable instructions or computer code, including computer-executable instructions such as program modules, which are executed by a computer or other machine such as a personal digital assistant or other handheld device. Typically, a program module, including routines, programs, objects, components, data structures, etc., refers to code that performs a specific task or implements a specific abstract data type. This disclosure can be practiced in a wide variety of system configurations, including handheld devices, consumer electronics, general-purpose computers, more specialized computing devices, etc. This disclosure can also be practiced in distributed computing environments in which tasks are performed by remote processing devices linked via a communication network.

[0093] As used herein, statements relating to two or more elements with respect to “and / or” should be interpreted as referring to only one element or a combination of elements. For example, “element A, element B, and / or element C” can include only element A, only element B, only element C, element A and element B, element A and element C, element B and element C, or element A, B, and C. Furthermore, “at least one of element A or element B” can include at least one of element A, at least one of element B, or at least one of element A and at least one of element B. Further, “at least one of element A and element B” can include at least one of element A, at least one of element B, or at least one of element A and at least one of element B.

[0094] This document describes in detail the subject matter of this disclosure to meet legal requirements. However, the description itself is not intended to limit the scope of this disclosure. Rather, the inventors have envisioned that the claimed subject matter may also be embodied in other ways to include steps different from or similar combinations of steps described herein in conjunction with other current or future techniques. Moreover, although the terms “step” and / or “box” may be used herein to imply different elements of the method employed, these terms should not be construed as suggesting any particular order among or between the various steps disclosed herein, unless the order of the steps is explicitly described.

Claims

1. One or more processors, including processing circuitry, said processing circuitry being used to: Access a scene graph representing a computer-generated environment, the computer-generated environment comprising one or more scene elements, the one or more scene elements comprising one or more scene features, wherein, Each of the one or more scene features is defined by one or more scene primitives included in at least one scene layer of the scene graph. Access to the overlay feature scene graph is based at least on tracking one or more changes associated with the one or more scene primitives; Create at least one overlay layer for the scene graph, the at least one overlay layer including a representation of the overlay feature scene graph; as well as The at least one overlay is propagated to one or more client nodes of the collaborative content creation platform CCCP, each client node executing at least one instance of the content creation application CCA, wherein the computer-generated environment is encoded using a three-dimensional 3D graphics data format.

2. The processors according to claim 1, wherein, The 3D graphics data format includes OpenUSD, an open general scene description.

3. The processors according to claim 1, wherein, The one or more processors are also used to synchronize the one or more scene layers of the scene graph.

4. The processors according to claim 1, wherein, The one or more processors are further configured to generate one or more overlay primitives, at least based on tracking the one or more changes associated with the one or more scene primitives, to generate the overlay feature scene map.

5. The processors according to claim 1, wherein, The one or more processors are also configured to send messages to at least one server node of the collaborative content creation platform CCCP, such that the at least one server node adds the at least one overlay.

6. The processors according to claim 1, wherein, The at least one overlay is generated by a first instance of a first content creation application (CCA) that subscribes to at least a portion of the computer-generated environment on the collaborative content creation platform (CCCP), and is propagated to at least one of the following: one or more second instances of the first content creation application (CCA), or one or more instances of a second content creation application (CCA) that subscribes to at least a portion of the computer-generated environment on the collaborative content creation platform (CCCP).

7. The processors according to claim 1, wherein, The at least one overlay layer is encoded according to a three-dimensional 3D graphics data format.

8. The processors according to claim 1, wherein, The overlay feature scene map includes one or more overlay primitives to overlay parameters of at least one of the one or more scene primitives of the computer-generated environment, wherein at least one of the one or more scene primitives is associated with at least one of the following: The geometry of one or more elements of the computer-generated environment. The surface texture of one or more elements of the computer-generated environment. The material composition of one or more elements of the computer-generated environment. The surface color of one or more elements of the computer-generated environment. The lighting environment of one or more elements of the computer-generated environment. The weight values ​​of one or more elements of the computer-generated environment, or The inventory of one or more elements of the computer-generated environment.

9. The processors according to claim 1, wherein, The at least one overlay layer represents a time series of overlay primitives, which are generated based at least on the temporal order of the one or more changes associated with the one or more scene primitives.

10. The processors according to claim 1, wherein, The one or more processors are also configured to apply timestamps in the at least one overlay layer, the timestamps indicating the time when the one or more changes associated with the one or more scene primitives were made.

11. The processors according to claim 1, wherein, The one or more processors are also used for: Process one or more second overlay layers; as well as Selectively apply one or more overlay primitives of the one or more second overlay layers to the one or more scene primitives of the one or more scene layers.

12. The processors of claim 11, wherein, The one or more processors are further configured to parse the one or more second overlay layers to find the last occurrence of an overlay primitive associated with a scene primitive of the one or more scene layers in the one or more second overlay layers, and to apply the last occurrence of the overlay primitive to the scene primitive.

13. The processors according to claim 1, wherein, The one or more processors are further configured to execute one or more generative artificial intelligence (AI) agents within the computer-generated environment, wherein the one or more generative AI agents generate the one or more changes associated with the one or more scene primitives based at least on computer-based perception of the computer-generated environment.

14. The processors according to claim 1, wherein, The processing circuit is included in at least one of the following: Control systems for autonomous or semi-autonomous machines; Sensing systems for autonomous or semi-autonomous machines; A system used to perform simulation operations; Systems used to perform digital twin operations; A system for performing optical transmission simulation; A system for performing collaborative content creation of 3D assets; A system used to perform deep learning operations; A system used to perform remote operations; Systems used to perform real-time stream processing; A system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content; Systems implemented using edge devices; Systems implemented using robots; Systems used to perform conversational AI operations; A system that implements one or more Visual Language Models (VLMs); A system that implements one or more large language model LLMs; A system for generating synthetic data; Systems for generating synthetic data using AI; A system that includes one or more virtual machines (VMs); A system that is at least partially implemented in a data center; or A system that utilizes cloud computing resources at least in part.

15. A system comprising one or more processors, said one or more processors being configured to: One or more overlay primitives are generated based on at least one or more changes associated with one or more scene primitives of one or more scene layers that track and jointly define the elements of the virtual scene, said one or more scene layers being defined using a three-dimensional 3D graphics format and hosted on one or more server nodes of a collaborative content creation platform. Create at least one overlay layer, the overlay layer including a representation of the one or more overlay primitives; as well as The at least one overlay is propagated to one or more client nodes coupled to the one or more server nodes of the collaborative content creation platform, based at least on instructing the one or more server nodes to add the at least one overlay to the one or more scene layers that jointly define the virtual scene.

16. The system according to claim 15, wherein, The at least one overlay layer represents the one or more overlay primitives based at least on a scene graph data structure representing the virtual scene.

17. The system according to claim 16, wherein, The one or more processors are also configured to generate the one or more overlay primitives, at least based on tracking the one or more changes associated with the one or more scene primitives, to generate an overlay feature scene map.

18. The system according to claim 15, wherein, The at least one overlay layer represents a time series of overlay primitives, which are generated based at least on the temporal order of the one or more changes associated with the one or more scene primitives.

19. The system according to claim 15, wherein, The system is included in at least one of the following: Control systems for autonomous or semi-autonomous machines; Sensing systems for autonomous or semi-autonomous machines; A system used to perform simulation operations; Systems used to perform digital twin operations; A system for performing optical transmission simulation; A system for performing collaborative content creation of 3D assets; A system used to perform deep learning operations; A system used to perform remote operations; Systems used to perform real-time stream processing; A system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content; Systems implemented using edge devices; Systems implemented using robots; Systems used to perform conversational AI operations; A system that implements one or more Visual Language Models (VLMs); A system that implements one or more large language model LLMs; A system for generating synthetic data; Systems for generating synthetic data using AI; A system that includes one or more virtual machines (VMs); A system that is at least partially implemented in a data center; or A system that utilizes cloud computing resources at least in part.

20. A method, the method comprising: In one or more instances of one or more content creation applications, track one or more changes associated with one or more scene primitives, which are organized in one or more scene definition layers representing one or more elements of a virtual scene environment; as well as The one or more changes are propagated to one or more client nodes of the Collaborative Content Creation Platform (CCCP), which subscribe to the one or more scene primitives through the CCCP. The propagation includes generating an overlay feature scene graph based at least on the one or more changes, creating at least one overlay layer including a representation of the overlay feature scene graph, and instructing a server node of the CCCP to add the at least one overlay layer to the representation of the virtual scene environment.

Images