Distributed command execution in a multi-location studio environment

Abstract

A content production management system in a distributed studio environment includes a command interface module and a command queue management module. The command interface module is configured to render a user interface for a set of content production entities associated with a set of content production volumes in the distributed studio environment. The command queue management module is configured, upon execution of software instructions, to perform operations of receiving from the command interface module commands targeted to a target content production entity, assigning a synchronous execution time to the command, enqueuing the command in a command queue associated with the target content production entity according to the synchronous execution time, and enabling the target content production entity to execute the command from the command queue according to the synchronous execution time.

Description

[Technical field]

[0001] The present disclosure relates to facilitating remote command and control via distributed synchronous command queues.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Provisional Patent Application No. 63 / 229674, filed August 5, 2021. The entire disclosures of the above applications are incorporated herein by reference. [Background technology]

[0003] Film studios typically employ green screen, blue screen, or, more recently, LED wall technology to shoot content for film, television, motion capture, or other purposes. Such technology allows for enhanced special effects and unique filming backdrops. When studios rely on green or blue screen technology, directors cannot see the final composition of the content until post-production work is completed, which can take weeks, if not months, to complete, resulting in additional costs and delays in production, and potentially delaying any necessary reshoots. When studios rely on LED wall technology, directors may be able to see portions of the final composition of the content (e.g., video backgrounds) before post-production work is completed.

[0004] The background discussion provided herein is intended to generally present the context of the present disclosure. To the extent described in this Background section, neither the work of the currently named inventors nor aspects of the description that may not be admitted as prior art at the time of filing are admitted expressly or impliedly as prior art to the present disclosure. Summary of the Invention [Means for solving the problem]

[0005] A content production management system in a distributed studio environment or ecosystem includes at least one computer-readable medium, at least one command interface module, and at least one command queue management module in communication with the command interface module. The at least one command interface module is configured to render or present at least one user interface to a content production entity set. Each content production entity from the content production entity set is associated with a content production volume from a content production volume set in the distributed studio environment. The at least one command queue management module is configured to perform operations of receiving a command set from the at least one command interface module upon execution of software instructions, and assigning an execution time, and possibly a synchronous execution time, to each command of the command set. Each command of the command set targets a target content production entity from the content production entity set. The at least one command queue management module is further configured to perform operations of enqueuing each command of the command set in a command queue associated with the target content production entity according to its respective synchronous execution time, and enabling the target content production entity to execute each command of the command set from the command queue according to its respective synchronous execution time, upon execution of the software instructions. The command queue is stored on a computer readable medium.

[0006] Also disclosed is a computerized method for managing a command set in a content production management system in a distributed studio environment. The content production management system includes a command interface module and a command queue management module in communication with the command interface module. The command interface module is configured to render or present at least one user interface to a content production entity set, each content production entity from the content production entity set being associated with a content production volume from a content production volume set in the distributed studio environment. The method includes receiving a command set from the command interface module and assigning, with the command queue management module, a synchronous execution time to each command of the command set. Each command of the command set is targeted to a target content production entity of the content production entity set. The method further includes, with the command queue management module, enqueuing, in a command queue associated with the target content production entity according to its respective synchronous execution time, each command of the command set from the command queue according to its respective synchronous execution time, and enabling the target content production entity to execute each command of the command set from the command queue according to its respective synchronous execution time.

[0007] Further areas of applicability of the present disclosure will become apparent from the detailed description, claims and drawings. The detailed description and specific examples are intended for purposes of illustration only and are not intended to limit the scope of the present disclosure.

[0008] The present disclosure will become more fully understood from the detailed description and the accompanying drawings, wherein: [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a functional block diagram of a distributed studio environment in which the principles of the present disclosure can be implemented. [Diagram 2] 2 is a functional block diagram of a command queue management module of the distributed studio environment of FIG. 1 in accordance with the principles of the present disclosure. [Diagram 3] 11 is a flowchart of an example operation for categorizing commands in a distributed studio environment in accordance with the principles of the present disclosure. [Figure 4] 11 is a flowchart of an example operation for allocating execution time for commands in a distributed studio environment in accordance with the principles of the present disclosure. [Diagram 5] 11 is a flowchart of an example operation for executing commands in a distributed studio environment in accordance with the principles of the present disclosure. [Figure 6] FIG. 1 is a front view of an LED wall according to the principles of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] In the drawings, reference numbers may be reused to identify similar and / or identical elements.

[0011] Introduction This disclosure describes a solution for directing or coordinating entities associated with multiple content production volumes in a distributed content production environment. For example, a film shooting session may include many pieces of equipment, extras, stage crew, actors, props, lighting, light sensors, microphones, cameras, displays, devices, robots, etc., that must coordinate activities within a single shooting volume in one studio, among multiple volumes in one studio, and / or among multiple volumes in multiple remote, possibly geographically distributed (e.g., Los Angeles (USA) and London (UK)) studios. This coordination of activities can be accomplished even when there is latency between the multiple remote studios. For example, when entities are located across multiple remote studios, possibly across vast geographic distances (e.g., 10 km, 100 km, 1000 km, etc.), and one or more latencies exist between them, the solution can provide synchronized direction or coordination to entities across the remote studios to ensure that content production efficiency is maintained or improved, particularly with respect to content integration. Thus, if each studio has a volume containing an LED wall, and each volume has moving elements (e.g., cameras, booms, robots, actors, props, etc.), the LED walls and moving elements in each studio can operate in sync with one another so that their individual portions of content can be quickly incorporated into a near-final, if not final, rendering of the desired content. In other words, if the LED wall in one studio is shifted in a particular way as the camera moves, the LED wall and camera in another studio can be shifted and moved in a similar or coordinated way. Thus, content shot from multiple remote studios can be combined or integrated to create a single seamless production and rendered to a director or other stakeholder during the shooting session.For example, by combining multiple frames of a film shot from a remote studio into a single frame, a director can make real-time adjustments (e.g., lighting adjustments, sound adjustments, camera adjustments, actor and / or prop space adjustments, etc.) during shooting and reshoots, thereby reducing the time (and cost) required for post-production activities. Thus, it should be appreciated that the subject matter of the disclosed invention provides for the synchronization coordination of many target devices or other target entities, even when taking into account communication latencies between various entities in their remote, geographically distributed studios.

[0012] Solutions can be implemented to direct or coordinate content production entities distributed around the world. For example, solutions can employ a dynamically resourced global stage network with always-on connectivity that routes data through localized data centers. For example, a shoot with actors on a stage in London, UK, with content and set pieces from another stage in Los Angeles, US, and with elements of a virtual art department captured from computing devices in Sydney, Australia, can be brought together and streamed into a "virtual control room / brain bar" or NORAD-like command center by collaborators, both physically on-set at any of those locations, and remote participants in a content production management system (possibly in a virtual reality environment). In such examples, physical or virtual GPUs can be employed to scale resources to the local stage. In addition, communication between remote locations on the network can be provided via fat-pipe point-to-point connectivity. In some examples, each stage can operate independently of the network as its own silo, sending all footage to a virtual control room for combination into a single picture.

[0013] In addition, it should be understood that the solutions may be employable across the full spectrum of real and virtual worlds: for example, sets, entities, volumes, etc. herein may be implemented in physical reality (PR), mixed reality (MR), mixed physical reality (MPR), augmented reality (AR), virtual reality (VR), projected reality (PrR), etc.

[0014] In some examples, the solution can be implemented as a content production management system that includes a centralized command and control computer module. In such examples, the centralized command and control computer module can act as a "NORAD" for the production session in that it provides multiple user interfaces to one or more stakeholders (e.g., director, producer, writer, artist, cast, stage manager, etc.) representing different entities (e.g., actors, props, and / or equipment such as cameras, camera tracking systems, microphones, lighting, light sensors (e.g., Pantone® HUEY™, Pantone® CAPSURE™, Parrot® SEQUOIA™, etc.), motion sensors, speakers, LED walls / ceilings / floors, display modules, etc.) possibly located across multiple remote studios, generates a set of commands for one or more target entities, and provides the commands for execution by the target entities or corresponding target devices. These commands can be sequenced according to one or more factors. For example, these commands may be executed in a time synchronized manner, taking into account latencies between various entities to ensure that the commands are executed properly relative to one another, regardless of the physical location of the executing device. One non-limiting example of latencies between various entities at physical locations (e.g., Location A, Location B, Location C, and Stakeholder (SH) location) is shown in Table 1 below.

[0015] [Table 1]

[0016] For example, a content production management system may include at least one command interface module and at least one command queue management module. The command interface module may support various video / audio conferencing software platforms (e.g., Google Hangouts™, Teams™, Skype™, Zoom™, etc.). For example, stakeholders in the command interface module may have real-time interactions with actors, grips, choreographers, makeup artists, set crews, etc. in a remote studio via the video / audio conferencing software platform.

[0017] In some examples, multiple command interface modules may be employed. In such examples, each command interface module may be provided with permissions, access control lists, etc. for functionality determined based on stakeholder requirements. Thus, a command interface module employed by a director may have different permissions, control options, etc. than a command interface module employed by a producer, writer, etc.

[0018] The command interface modules can render or present one or more user interfaces for a set of content production entities associated with a set of content production volumes in a distributed studio environment. The user interfaces can be rendered based on data feeds from one or more studios, content production volumes, content production entities, etc. For example, each command interface module can render one usage interface for a single content production entity. In other examples, a single command interface module can render multiple usage interfaces corresponding to multiple content production entities.

[0019] A content producing entity herein may include, for example, any suitable controllable device. For example, a content producing entity may include props such as vehicles, robots, production equipment such as cameras, lighting systems, sound systems, sensors, moving platforms, moving walls, LED walls (e.g., background LED walls, foreground LED walls, static LED walls, moving LED walls, etc.), LED ceilings, floors, gimbals, display modules, and user devices such as personal computing devices (e.g., smartphones, smart watches, smart glasses, VR headsets, MR headsets, tablets, laptops, etc.). In some examples, users such as actors, grippers, choreographers, makeup artists, set personnel, etc. may utilize the content producing entity (e.g., personal computing devices, display modules, LED walls / ceilings / floors, etc.) to receive commands from one or more stakeholders. For example, users may be controlled by commands through off-screen visual indicators, digital floor markings, in-ear audio instructions, smart glasses, MR headsets, portions of LED walls outside the camera's field of view, smart watches, etc.

[0020] In some examples, the LED wall, ceiling and / or floor may include touch panel technology, tracking technology (e.g., Mo-Sys™, StYpe™, Vicon™, OptiTrack™, structured light tracking, unstructured light tracking, silhouette tracking, blob detection and tracking, face tracking, audio tracking, kinematic tracking, etc.), countdown or count-up timers for different commands (described further below), and the like. For example, the LED wall, ceiling and / or floor may display a specific command for a particular individual on the set based on that individual's tracked location. In addition, the countdown or count-up timer may be moved around on the LED wall and / or floor as the individual moves, so that the timer remains near the individual, thereby signaling to the individual when to take an action corresponding to the command. In addition to timers, cues or other instructions, markers may be placed or moved on the LED wall so that they are visible to a corresponding target entity (e.g., actor, camera, device, etc.). The entity may observe such markers and take a corresponding action. For example, markers can be placed on an LED wall to show actors where to look while shooting a sequence.

[0021] A content creation volume herein may include, for example, any area and / or space from which motion capture data can be acquired. Volumes may be large, small, built into a set, purpose-built, outdoors, a stage, a studio, a room, inside a vehicle, etc. Volumes may include, for example, outdoor (or indoor) settings, stages such as sound stages, LED walls (e.g., background LED walls, foreground LED walls, etc.), LED floors, LED ceilings, green screens, public spaces, streets, parks, beaches, buildings, forests, swamps, etc.

[0022] In some examples, the content production volumes may be distant from one another. For example, two or more content production volumes may be geographically separated (e.g., 10 km, 100 km, 1000 km, or more). For example, one volume (associated with one studio) may be located in Los Angeles, USA, Sydney, Australia, etc., and another volume (associated with another studio) may be located in London, UK, New York City, USA, etc. Additionally and / or alternatively, two or more content production volumes may be separated by latency, such as, for example, network communication latency. For example, one volume may be separated from another volume by, for example, about 5 ms, about 10 ms, about 50 ms, about 100 ms, about 1000 ms latency, or more.

[0023] Additionally, the content production volume may be separate from the command interface module and / or the command queue management module. For example, the content production volume may be geographically separated from the command interface module. For example, the volume may be located in Sydney, Australia and the command interface module may be located in London, England. Additionally and / or alternatively, the content production volume and the command interface module may be separated by latency. For example, the volume may be separated from the command interface module by about 5 ms, about 10 ms, about 50 ms latency, or more.

[0024] The command interface module can then generate a command set, for example, based on user input provided to a user interface or automatically as provided by other devices in the ecosystem via one or more APIs. The command set may include one or more commands. In some examples, at least one of the commands may be generated automatically in response to a defined event. Each command in the command set may be targeted to a particular (e.g., target) content production entity from a content production entity set or a corresponding target device.

[0025] Commands herein may include any suitable commands for controlling, directing, or managing entities during a production. For example, commands may include instructions for positioning and / or moving cameras, camera properties (e.g., video resolution, camera exposure adjustment, frame rate, white balance, ISO, aperture, shutter speed, bit rate, focus, etc.), microphone properties (e.g., microphone volume, microphone boost, microphone input level, pickup pattern, etc.), equipment, props, robots, robot controllers, actuators, servos, etc., setting and / or adjusting sound levels, turning lights on / off, adjusting light brightness levels, adjusting lighting color, brightness and color to update content displayed on an LED wall, and / or instructions for directing humans (e.g., actors, grip people, choreographers, makeup people, set people, etc.). In some examples, commands may include network provisioning commands, virtualization commands, sensor commands, device commands, script-related commands, program-related commands, human directing commands, choreography-related commands, script commands, controller commands, previs-related commands, etc.

[0026] The command queue management module communicates with the command interface module. In some examples, the command queue management module may be coupled to the command interface module via a wired (or wireless) connection and data may be transmitted therebetween. For example, the command queue management module may receive a set of commands from the command interface module. As desired, these commands may be stored in a database, a local file, a remote file, the cloud, and / or another suitable type of data structure in a computer-readable medium.

[0027] The command queue management module or other device may also assign a synchronous execution time to each command in the received command set. Thus, commands may be issued, transmitted, executed, etc., to different target entities at precise times. This allows for synchronous execution of the commands, as desired. The term "queue" is used euphemistically to refer to a data structure stored in at least one computer-readable memory for storing a list of ordered commands. It should be understood that the command queue may be a FIFO, FILO, deck (i.e., double-ended queue), or other form of data structure that allows access and / or management of ordered commands. In a more preferred embodiment, the command queue is at least ordered according to execution time. However, in some embodiments, commands in a command queue may have other trigger criteria or dependencies. For example, if a series of events are staged to occur in relation to one another, the enablement or execution of one command may depend on the execution or completion of another command, ensuring that the executing entities perform their obligations in relation to one another in the appropriate manner according to the desired order defined by the staged. Thus, commands in a command queue may be linked to other commands in the same queue or other queues. Additionally, commands in the command queue can be linked to the states of other commands (eg, pending, completed, running, paused, etc.), thereby forming the trigger criteria for other commands.

[0028] The synchronous execution times of the commands can be determined based on multiple factors (e.g., including latency, time-related factors, sequencing of the commands, execution times of other commands, or other trigger factors). Latency factors can include, for example, communication latency, execution latency, network latency, human error delay, human reaction time, certain human factors, etc. In some examples, the synchronous execution times of one or more commands can be synchronized with respect to absolute time. In such examples, a Network Time Protocol (NTP), Global Positioning System (GPS), etc. can be employed to obtain close to absolute time to ensure that the commands are synchronized with respect to absolute time (e.g., with respect to a master clock, GPS clock, etc.). In doing so, the command set can be time-stamped based on absolute time. In other examples, the synchronous execution times of one command can be synchronized with respect to the synchronous execution times of another command. For example, one command can be assigned an execution time of X and another command can be assigned an execution time of X+5ms. In yet another example, the synchronized execution times of the command sets can be synchronized globally, synchronized for different content production volumes, synchronized for content production entities, synchronized for geographic locations, and the like.

[0029] The command queue management module can also enqueue (e.g., insert, etc.) each command of the command set into a command queue according to its respective synchronous execution time. A command queue can be associated with a particular (e.g., target, device, entity, etc.) content production entity or a corresponding target device. Thus, each targeted content production entity can have a corresponding command queue for sorting, prioritizing, etc., or managing the sequenced commands.

[0030] The command queue can be stored on a computer-readable medium. In some examples, the computer-readable medium can be located locally to the command queue management module or can be remote from the queue management module. For example, the computer-readable medium can be associated with a target content production entity (e.g., located at a target content production entity, part of a target content production entity, etc.), associated with a particular volume (e.g., located at a particular volume, part of a particular volume, etc.), etc. In some examples, the computer-readable medium can be implemented on a server separate from the command queue management module.

[0031] In some examples, the command queue management module can instantiate the command queue, which can be done on the computer-readable medium on which the command queue is initially stored or on another computer-readable medium.

[0032] The command queues herein (individually or collectively) can be considered workflow management systems and / or state machines. For example, command queues can be implemented using various data and / or time management techniques, such as synchronized, asynchronous, isochronous, real-time, choreographed, etc. data transmission, so that, for example, one command can trigger another command to remain synchronized.

[0033] For example, multiple state machines may be employed, with each state machine associated with one remote studio, one scene shot at one remote studio, the entire production, etc. Each state machine includes a set of states, with each state having one or more commands. In some examples, each of the commands in a state must be executed or optionally bypassed before the state machine transitions to the next state. For example, if a state includes two commands, the two commands must be executed or bypassed before the state machine transitions to a next state having a different set of commands for execution. In some embodiments, the transition to the next state may be delayed, advanced, etc., based on a trigger condition, regardless of whether a command in a state has been executed. For example, the trigger condition may be based on a determined latency (e.g., latency between different remote studios), as described herein. In such examples, a trigger condition that forces a state machine associated with one remote studio to transition to a next state may maintain synchronization between other remote studios and / or with state machines associated with the entire production.

[0034] Additionally, the command queues herein may include various features for implementing data and / or time management techniques. For example, the command queue may include countdown timers, countup timers, activation criteria, permissions, external triggers (e.g., initiated by a person, device, etc.), links to a notarized ledger, links to associated command queues, links to previously executed queues, links to future queues, etc. All queues or related data may be stored in a notarized ledger (e.g., a distributed ledger, blockchain, hashgraph, smart contract system, etc.), which may include all queue times, their information, the time the information was presented, the time the information was delivered, all queues presented and their presentation times, the time the information was generated by the system or entered into the system by a person or device. The ledger can later be used to analyze what went wrong with a film shot and why. For example, the ledger can be used to detect hardware and software issues, to detect malicious behavior, to locate the cause of any latency issues, to detect where the maximum latency occurred, to provide auditing services for those paying for the production, etc. In this sense, storing the command queue or other command queue related information in a notarized ledger (e.g., a distributed ledger, blockchain, hashgraph, Ethereum, Bitcoin, etc.) provides authentication or verification of the production at a later time. In some embodiments, the actual production is stored off-ledger (e.g., IPFS, etc.), while the ledger itself may have blocks or other data that point to the off-ledger storage location. This off-ledger storage approach is advantageous because it allows large amounts of data to be managed separately from the notarized ledger to reduce ledger bloat.

[0035] In some examples, the command queue management module can transmit commands to a targeted content production entity. For example, the command queue management module can transmit one or more commands, all commands, etc. of a command set to a particular content production entity over a network. Thus, the command queue management module can individually distribute commands for execution and / or placement in a local queue (e.g., in a device memory, a camera memory, on an LED wall, in a game engine, etc.) of the targeted content production entity. In other examples, the command queue management module can transmit entire command queues, rather than individual commands, over one or more networks. For example, the command queue management module can transmit (e.g., over a network) to a particular content production entity a command queue associated with that production entity.

[0036] In addition, the command queue management module can enable the targeted content production entity to execute each command of the command set from the command queue according to its respective synchronous execution time. For example, if the command queue is stored on a computer-readable medium located remotely from the command queue management module (e.g., the targeted content production entity, an associated content production volume, etc.), the command queue management module can transmit a signal to the targeted content production entity to execute a command from the command queue.

[0037] The content production management system can provide more than three-dimensional control over the target content production entities. For example, the content production management system can control only the three-dimensional movement of one or more target content production entities, control the three-dimensional movement and timing of one or more target content production entities, etc. For example, through commands, the target content production entities such as gimbals, moving platforms, LED walls / floors / ceilings, cameras, lights, microphones, robots, etc. can be moved as required. For example, a first studio can include a background LED wall (e.g., a first target content production entity, etc.) for displaying mountains, sky, landscapes, etc., a second studio can include a moving rowboat (e.g., a second target content production entity, etc.), and a third studio can include a foreground LED wall (e.g., a third target content production entity) for displaying water and fish. Each of these three components can be coordinated in synchrony (or with a known synchronization delay, e.g., 100 milliseconds), so that the three-dimensional movement of the three components can be controlled through commands. It should be understood that while these various entities may be located in the same geographical location, it is specifically considered that the entities and studios are distributed over geographical distances. Furthermore, the command and control center where the content is integrated or presented to the stakeholders may be located remotely from the studios. Thus, in a more interesting embodiment, the synchronization of the various entities in the studio may be synchronized relative to an observer in the command and control center, rather than globally synchronized relative to absolute time. In other words, the execution of commands in the command queues for the various distributed entities may include appropriate execution time delays so that the resulting data from the various studios, when sent to the command and control center, arrives at the same time, is presented at the same time, or is provided to the stakeholders at a preferred time.

[0038] Content from different (e.g., remote) studios / sets can be combined (or time-aligned and then combined) into one shot and provided to a centralized command and control computer module ("NORAD") and / or each of the studios for viewing. Combining content shot from different studios / sets can reduce the time associated with post-production work. For example, an action scene may be shot in a spaceship where the main character wears chrome armor and uses a glass sword. This type of scene is not suitable for a green screen or blue screen because the reflections of the armor and sword will be blue or green, creating problems in the footage. However, if the background of the scene is rendered on an LED wall relative to the camera's tracking position, the background image will have the correct refraction and reflection without requiring extensive post-production work. This can be achieved in real time using game engine software (e.g., UnReal® game engine, Unity® game engine, Godot game engine, Blender™ renderer, etc.). In addition, if a scene includes extras and / or props surrounding the main character, these elements can be shot in a studio separate from the main character and then combined with the main character content during production. Such an arrangement can eliminate travel to different locations, reduce scheduling issues, etc.

[0039] In some examples, some content producing entities need to know the status or state of other content producing entities. For example, a camera may capture content on an LED wall. If the camera is moved or the settings are changed (e.g., to a different angle, a different latitude position, a different longitude position, a different orientation, a different altitude, a different zoom position, a different ISO setting, a different shutter speed, a different aperture setting, a different frame rate, a different white balance setting, a different focus, etc.), it may be desirable to update the LED wall to take into account the different perspective that the camera should see. In some examples, the LED wall may be updated through commands based on transformation matrices, projection matrices, etc. used to render 3D realistic environments for, e.g., video games, architectural design, simulations, 3D animations, etc. For example, the LED wall may be updated through commands related to the operation of a virtual camera of a game engine used to render the LED wall.

[0040] In another example, a previs compiler can be created that shows what is expected during production if, for example, a camera is moved. For example, errors can be identified in the previs compiler based on the camera movement, and then corrective action can be taken such that the LED wall is updated when the camera is moved (in the actual production). It should be understood that these capabilities can be manifested via commands in the command queue. For example, the previs compiler can be configured to generate coordinated state machines, each state machine being associated with one remote studio, one scene shot in one remote studio, the entire production, etc., as described herein. A state of a state machine associated with one remote studio can include a command to move the camera in the studio. If the command is executed (e.g., the camera is moved), this state machine and / or another state machine associated with another remote studio can transition to a new state and execute the commands therein, for example, to update the LED wall, move devices or actors based on the camera movement, etc. Furthermore, in some embodiments, such actions can be combined with simulations. For example, a command to move the camera can be simulated to show the behavior resulting from such a movement. In such an example, a state machine may execute a command to simulate a camera movement, and then the same state machine and / or another state machine associated with another remote studio may transition to a new state and execute the commands therein, e.g., updating an LED wall, moving devices or actors based on the simulated camera movement, etc.

[0041] In addition, it may be desirable to calibrate the cameras via commands or command queues from the different studios so that the combined content is not distorted. Such calibration via commands can be done for camera angles, camera focus, camera lenses, etc. For example, distortion can result from lenses. This distortion can be compounded (in some cases) by video content from one studio being placed as a backdrop on an LED wall in another studio. In some examples, this problem can be solved by calculating the distortion before rendering each backdrop on the LED wall and calibrating the lenses accordingly based on the submitted commands. In addition, skew distortion can occur during filming if the camera angles are not aligned. In some examples, this problem can be solved by relying on a skew matrix or the difference in angle between the original camera that captured the content and the camera that is capturing the content relative to the backdrop rendered on the LED wall.

[0042] It may also be desirable to calibrate the lighting from different studios to ensure that the actual objects in the film appear correctly. For example, the lighting from different studios can be calibrated via command to ensure that extras, objects, creatures (e.g., aliens, dragons, etc.) do not appear differently than the lead actors or appear out of place in the combined content.

[0043] Additionally, production elements from the content production management system can be minted as non-fungible tokens (NFTs). For example, actual production events and / or specific actions (e.g., moments, scenes, bloopers, lines, VFX, etc.) can be minted as one or more NFTs. Additionally, a final edit file can be minted and used to generate and store NFTs. Thus, one aspect of the inventive subject matter can include creating commands to mint or cause the minting of NFTs during production. This can be accomplished by submitting the NFT to a distributed ledger, smart contract ledger, or other notarized ledger. For example, NFTs can be minted according to ERC-721 (see URL eips.ethereum.org / EIPS / eip-721 for Ethereum Improvement Protocol), ERC-1155 (see URL eips.ethereum.org / EIPS / eip-1155), or ERC-998 (see URL eips.ethereum.org / EIPS / eip-998) multi-token set and then recorded on the Ethereum blockchain. Although Ethereum is used as an example, it should be understood that other forms of notarized ledgers (including, for example, Solana, XPR, Hyperledger, Cardano, etc.) can also be used. Such an approach provides a computer-based technique or method for creating a market for NFTs representing the actual production of a film or movie. Additionally, the approach provides a computer-based system for establishing chain of custody of digital content, verifying digital content, authenticating digital content, conducting transactions associated with digital content, or managing digital content.Such NFTs can be tied to a particular producing entity or content creator, for example, via techniques disclosed in U.S. Patent No. 1,121,0383, filed June 28, 2021, to Witchey et al., entitled "Content Authentication and Validation via Multi-Factor Digital Tokens, Systems, and Methods," which is incorporated herein by reference in its entirety.

[0044] Block Diagram 1, a distributed studio environment 100 according to the principles of the present disclosure is shown. The distributed studio environment 100 (sometimes referred to as ecosystem 100) includes a content production and management system 102 and at least two studios 104A, 104B, each of which has a computing system 120A, 120B, a volume 106A, 106B, a camera 132A, 132B, and one or more controllable devices 134A, 134B. As shown, each computing system 120A, 120B includes a communication interface module 122A, 122B, a control module 124A, 124B, a sensing module 126A, 126B, a control interface module 130A, 130B, a content display interface module 136A, 136B, and a computer-readable medium (e.g., memory 138A, 138B). 1, distributed studio environment 100 is presented in one arrangement, other embodiments may include distributed studio environments arranged in other manners depending, for example, on the number, type and / or configuration of connected studios. For example, it should be understood that as many of the components shown may be present as is practical.

[0045] At this point, it should be understood that many of the various elements in the distributed studio environment 100 are computing devices having at least one processor and at least one non-transitory computer-readable memory that stores software instructions. Furthermore, the various elements can be communicatively coupled via one or more network connections (e.g., wired, wireless, optical, 802.11, cellular networks, etc.). Thus, the various elements can cooperate through the execution of their respective software instructions. In addition, it should be understood that the collection of software instructions can include one or more applications (e.g., software applications, programs, scripts, apps, adapters, APIs, etc.) distributed among the elements. In some embodiments, the collection of software instructions can include a software suite having individual programs designed to cooperate. From a hardware perspective, the various elements can include client devices, server devices, peer devices, adapter devices, module devices, controller devices, or other types of computing devices that form the platform on which the subject matter of the invention is built.

[0046] The computer-based nature of the environment 100 imposes technical limitations on the operation of the environment 100. More specifically, the distributed nature of the environment 100 imposes latency issues during the coordination of various devices, such as within the same studio 104A, 104B, between different studios 104A, 104B, between one or both of the studios 104A, 104B and the content production management system 102, to name just a few. Thus, to address such technical limitations associated with latency, the environment 100 is capable of determining specific latency values ​​between the various devices and retiming command execution as necessary to ensure that the various devices are coordinated, as described herein. Other computer-related technical limitations imposed on the environment 100 to overcome may include, for example, bandwidth limitations in the transfer of large amounts of data (e.g., images, high-definition video, etc.), storing data on storage devices (e.g., SSD, HDD, flash, RAM, etc.), indexing or accessing data within the environment 100 or data stored on the storage devices, or other related issues. Thus, environment 100 is configurable to address these and other technological limitations through specific and unique interactions, devices, and the like, as described herein.

[0047] In the example of FIG. 1, system 102 and studios 104A and 104B are separate from each other. For example, content production management system 102 may be located in New York City, USA, studio 104A may be located in London, UK, and studio 104B may be located in Los Angeles, USA. In some embodiments, studios 104A and 104B may be geographically separated (e.g., 10 km, 100 km, 1000 km, or more), while studios 104A and 104B may be temporally separated (e.g., 5 ms, 10 ms, 100 ms, 1000 ms, or more) as measured by latency to each other or to content production management system 102. For example, studio 104A may be a volume that is physically separate from studio 104B, while still being in the same production facility (e.g., the same production lot, etc.). However, due to network connection infrastructure, computer system infrastructure, or other factors, two studios may be separated in terms of latency even if they are physically close to each other (e.g., less than 10 km, less than 1 km, less than 100 m, etc.) Thus, the latency between distributed devices can create serious coordination issues both between the devices and in providing seamless integration as a coordination point (at a remote facility under the control of the director of film production).

[0048] In the particular example of FIG. 1, the volumes 106A, 106B include background LED screens or walls. However, it should be understood that one or both of the volumes 106A, 106B may include other suitable types of screens, such as green screens, blue screens, curved (or other non-planar) background LED screens, LED ceilings, LED floors, flat or curved (or other non-planar) foreground screens, etc. The LED screens preferably include one or more computers that control the content displayed on the screens. In a more preferred embodiment, the LED display computer operates according to instructions received from one or more rendering engines (e.g., Unity®, UnReal®, Blender, etc.). In a particularly preferred embodiment, the rendering engine includes a game engine (e.g., the UnReal® game engine).

[0049] Examples of LED-based production volumes include NantStudios (see URL www.nantstudios.com), a pioneer in real-world virtualization production via giant LED walls. For example, FIG. 6 illustrates one exemplary LED-based production volume 600 implemented by NantStudios. As shown, the volume 600 includes a ceiling LED wall 602, an LED stage area (e.g., floor, foreground, etc.) 604, and a background LED wall 606. In the illustrated embodiment of FIG. 6, the background LED wall 602 is a curved wall that extends between the ceiling LED wall 602 and the LED stage area 604. In some embodiments, the LED stage area may be 24,000 square feet and the background LED wall 606 may be 145 feet long by 20 feet high, with a 270 degree oval shaped volume of 52 feet wide by 65 feet long. Alternatively, the LED stage area 604 and the background LED wall 606 may have any other suitable size. Although the LED-based production volume 600 of FIG. 6 is shown and described in one arrangement, other embodiments may include LED-based production volumes arranged in other manners depending, for example, on the number, type, size, configuration, etc. of connected LED walls, stage areas, etc.

[0050] 1, content production and management system 102 includes a communication interface module 108, a command interface module 110, a command queue management module 112, an input device 114, a display module 116, and computer-readable media (e.g., memory 118). The command interface module 110 and the command queue management module 112 may function similarly to the command interface module and command queue management module described herein.

[0051] For example, the command interface module 110 can render or present one or more user interfaces on the display module 116 to content production entities associated with the content production volumes 106A, 106B in the distributed studio environment 100. For example, the command interface module 110 can render a list of commands, updates provided from one or both of the studios 104A, 104B, etc., such that an observer (e.g., a director, a producer, etc.) viewing the display module 116 can determine or respond with a course of desired action (e.g., initiating one or more commands, stopping filming, changing the position and / or movement of actors, content and / or devices, etc.).

[0052] In addition, the command interface module 110 can generate commands, for example, based on user input provided at the input device 114 or automatically according to one or more rule systems. For example, the commands generated by the command interface module 110 can be data structures or objects instantiated in one or more non-transitory computer-readable media and acting as instructions, directives, etc., to instruct or cause one or more content production entities or their devices to perform a particular task. In some embodiments, one or more compilers can be employed to compile the commands. For example, the commands can first be created in a computer language (e.g., an object-oriented computer language or a non-object-oriented computer language). One or more of the compilers can then compile the commands, thereby converting the commands into a machine-readable language. The commands can be compiled in the command interface module 110 and / or one or both of the computing systems 120A, 120B. Thus, command interface module 110 and any one (or all) of computing systems 120A, 120B may include a compiler stored on one or more non-transitory computer-readable media (e.g., memory 118, memory 138A, memory 138B, etc.). Commands (before or after being compiled) are transmitted to command queue management module 112 and then transmitted to studios 104A, 104B via communications interface module 108, as described further below.

[0053] In some embodiments, the commands may be targeted to specific (e.g., target) content production entities or their devices associated with the content production volumes 106A, 106B in the distributed studio environment 100. These entities may include, for example, cameras 132A, 132B, LED screens, controllable devices 134A, 134B, computing devices, robots, etc. located in the studios 104A, 104B. In some scenarios, the entities may include humans, in which case the commands may be rendered by the corresponding target devices in a human understandable manner (e.g., visual cues, audio, text, images, symbols, marks, etc.), as discussed further below.

[0054] The command queue management module 112 receives commands from the command interface module 110 and can receive or assign a synchronous execution time to each command. The assigned synchronous execution time can be based on, for example, latency, pace of filming, pace of performance, time-related factors, sequencing of commands, execution times of other commands, etc., as described herein. For example, latency between various devices, systems 102 and / or studios 104A, 104B in the environment 100 can be measured using various techniques, including obtaining time via GPS sensors, NTP, ping time, measuring round trip time (RTT) associated with data packets, time to first byte (TTFB), based on observed human behavior (e.g., a human observer viewing the display module 116, previs 199, or other techniques. In other embodiments, latency can also be measured based on running one or more simulations or rehearsals to account for human-imposed latency. The execution time of a command can be measured using a variety of techniques, including obtaining time via GPS sensors, NTP, ping time, measuring round trip time (RTT) associated with data packets, time to first byte (TTFB), based on observed human behavior (e.g., a human observer viewing the display module 116, previs 199, or other techniques. In other embodiments, latency can also be measured based on running one or more simulations or rehearsals to account for human-imposed latency. , it should be understood that these represent when commands should be executed to ensure that the resulting action produces content that can be integrated or viewed at the time of use (e.g., in the content production and management system 102, in a previs 199 associated with the content production and management system 102) contemporaneously with content generated from the execution of other commands, regardless of the physical location where the other commands were executed. Thus, it should be understood that each studio may execute local commands that are out of sync with the local commands of other studios, but that the content received at the time of use will appear to be in sync, or that the content is at least received according to the requirements of the viewer (e.g., director, producer, etc.).

[0055] The command queue management module 112 can then enqueue the command into a command queue associated with the target content production entity (e.g., camera 132A, 132B, LED screen, controllable device 134A, 134B, etc.). For example, the module 112 can enqueue the command into a command queue stored in memory 118. In other examples, the module 112 can enqueue the command into a command queue in another computer-readable medium (e.g., memory 138A, memory 138B, etc.) that is remote from the system 102. In some embodiments, the system 102 and / or computing systems 120A, 120B can include one or more buffers for temporarily storing commands or command queues to enable proper enqueuing of commands (or queues) relative to the timing at which the commands are generated. For example, if multiple state machines are compiled, their relative commands are generated and buffered. The commands can then be enqueued from the buffer according to their appropriate execution times.

[0056] The command queue management module 112 can then transmit the commands to the studios 104A, 104B via the communication interface module 108. For example, the module 112 can transmit the commands to each control module 124A, 124B (e.g., individually and / or collectively in one or more command queues). In some examples, the transmitted commands can be stored or buffered in local command queues in the memory 138A, 138B. It should be understood that the commands can be transmitted according to one or more techniques. In some cases, individual commands are sent to a remote location for incorporation into the local command queue, in other cases, a corresponding command queue with the commands is transmitted to the remote location, in yet other cases, the remote location can query the command queue management module 112 for the commands, and so forth.

[0057] The command queue management module 112 may include various components to implement these features. For example, FIG. 2 illustrates an example of an embodiment of the command queue management module 112 of FIG. 1. As shown, the command queue management module 112 includes a command synchronization module 202 and a command execution module 206. In the example of FIG. 2, the command synchronization module 202 receives commands from the command interface module 110 and assigns a synchronous execution time to the command, which preferably includes a time formatted for local execution (e.g., absolute local time, time relative to other local events, etc.). As previously discussed, the execution time can be generated based on observed conditions including, for example, GPS time values, NTP, or other suitable time measurement techniques. The command synchronization module 202 then passes the commands and their synchronous execution times to the memory 118 and enqueues the commands in a command queue stored in the memory 118. The command execution module 206 timely retrieves the commands and / or command queues from the memory 118 and transmits the commands (and / or command queues) to the targeted content production entities or their corresponding target devices via the communication interface module 108 as described herein. The number of command queues may vary depending on the situation, configuration, or other factors. Thus, 1, 2, 20, 100, 1000, or more command queues may be under management. Furthermore, each command queue may contain 1, 2, 20, 100, 1000, or more commands as required by the complexity of the production event or as required by the corresponding state machine.

[0058] Returning to FIG. 1 , each computing system 120A, 120B communicates with each other and with the content production management system 102. For example, the communication interface module 122A of the computing system 120A provides bidirectional communication (e.g., data reception, data transmission, and data transmission) with the computing system 120B and the content production management system 102 (via the communication interface module 108), and the communication interface module 122B of the computing system 120B provides bidirectional communication (e.g., data reception, data transmission, and data transmission) with the computing system 120A and the content production management system 102 (via the communication interface module 108). The communication interface module 108, like other communication interface modules in the ecosystem 100, may include interfaces for wired or wireless connectivity (Ethernet, USB, RS-232, 802.11, Bluetooth, WiGIG, IrDA, cellular modem, radio, or other types of communication connectivity interfaces).

[0059] 1, each detection module 126A, 126B receives data from one or more sensors 128A, 128B located in its respective studio 104A, 104B. For example, the sensors 128A, 128B may provide data related to lighting characteristics, audio characteristics, prop locations, camera positions, etc. within the studio 104A, 104B. Data received from the sensors 128A, 128B may be passed to the content production management system 102 via the control modules 124A, 124B and communication interface modules 122A, 122B.

[0060] The command interface module 110 can generate at least some of the commands based on data received from the sensors 128A, 128B to make desired adjustments (e.g., real-time adjustments) to the lighting system, sound system, cameras 132A, 132B, rendered content of the volumes 106A, 106B, actors, prop locations, etc. in the studios 104A, 104B. These commands can be generated automatically (e.g., based on defined events, etc.), in response to user input from one of more stakeholders, etc. The commands can be transmitted (e.g., individually and / or collectively in a command queue) to the computing systems 120A, 120B as described above. Additionally, commands in a command queue can also include trigger conditions, e.g., based on observed sensor data. Thus, such commands can include additional execution requirements other than execution time. For example, commands can be queued for a range of allowable execution times and based on when particular sensor data was observed (e.g., an actor moved, a prop moved, someone spoke, etc.).

[0061] The control modules 124A, 124B can distribute the transmission commands to various targeted content production entities or their corresponding target devices. For example, the control modules 124A, 124B can distribute the transmission commands to the cameras 132A, 132B in the studios 104A, 104B, the LED screens in the volumes 106A, 106B, the controllable devices 134A, 134B, and / or other targeted content production entities via the control interface modules 130A, 130B. The commands can then be executed by the corresponding entities (e.g., devices, etc.) according to their synchronous execution times, execution time spreads, or other trigger criteria. In some examples, the commands can be executed only after the entities are enabled to execute them (e.g., by an enable signal received from the command queue management module 112). Thus, in some embodiments, the commands can be triggered based on the satisfaction of multiple factors that define the command trigger criteria (e.g., time, other commands, audio triggers, computer vision recognition, other sensor data, previs instructions, etc.).

[0062] The controllable devices 134A, 134B may include any suitable target content production entity. For example, the controllable devices 134A, 134B may include props such as vehicles, robots, production equipment such as cameras, lighting systems, sound systems, sensors, moving platforms, moving walls, LED walls, LED ceilings, LED floors, gimbals, display modules, and user devices such as personal computing devices (e.g., smartphones, smart watches, smart glasses, tablets, laptops, etc.).

[0063] 1, control modules 124A, 124B render content on LED screens via content display interface modules 136A, 136B, respectively. The content for rendering can be adjusted, updated, etc. as needed, e.g., based on generated commands, as described above.

[0064] The content for rendering (including the adjusted and / or updated content) may be the same or different for studios 104A, 104B. For example, LED screens in volumes 106A, 106B may both display the same content (e.g., a scenic background with props such as stars 142A). In such an example, camera 132A may capture the content for rendering in volume 106A with actor 140A performing, and camera 132B may capture the same content for rendering in volume 106B with actors 144B, 146B performing. In some examples, any one of actors 140A, 140B, 144B, 146B may be a physical actor, a virtual actor that is part of the content for rendering, etc.

[0065] In other examples, the LED screens in volumes 106A, 106B may display different content. For example, the LED screen in volume 106A may display a scenic background with stars 142A, and the LED screen in volume 106B may display a foreground (without stars 142B). It should be understood that while the content displayed on the different LED screens may differ from one another, the different content may still be part of the same content production effort. Thus, different content captured by different cameras may be brought back into the content product management system 102 for integration or other uses.

[0066] In some examples, captured film from studio 104A can be passed to studio 104B as a possible input for control purposes. For example, one of cameras 132A, 132B can be driven based on input from the other of cameras 132A, 132B. For example, camera 132A (e.g., a slave camera) can be mechanically driven based on input from camera 132B (e.g., a main camera) such that camera 132A moves in coordination with, relative to, etc., camera 132B during a scene. In such examples, content captured from camera 132A (including actors 140A and / or stars 142A) can be transmitted to content display interface module 136B, camera 132B, control module 124B, etc., to be added to the final rendering, rendered on an LED wall in volume 106B (via content display interface module 136B), etc. In some embodiments, content captured from camera 132A in studio 104A may be stored or buffered in memory 138B in studio 104B.

[0067] When the content display interface module generates a background for a volume that includes content captured from a camera, the content display interface module can adjust the captured content to ensure that the necessary visual information is properly displayed on the volume's LED panels according to corresponding commands. Such adjustment commands can include scaling the captured content to scale relative to actors or props taking into account the local camera's field of view, rotating and tilting the captured content to match the volume's horizon and the local camera capturing the content, making sure all actors are standing perpendicular to the ground, adjusting the lighting, color, exposure and contrast of the captured content to match the current volume so that the captured content is not out of place, moving the content to make sure it is not floating in space, stuck on the floor or too far to the side so that it is captured correctly, or verifying that the placement of the content matches the previs, script or storyboard for the scene.

[0068] 1, to render volume 106B in location B, content display interface module 136B receives captured content from volume 106A in location A via interface modules 122A, 122B and control modules 1241, 124B. Content display interface module 136B moves captured actor 140A to the planned previs placements as defined for the scene. These example placements are shown in previs 199, with one actor's location depicted at 140C, a star's location depicted at 142C, another actor's location depicted at 144C, and another actor's location depicted at 146C.

[0069] The location of Actor 140A in the scene is adjusted to match the previs, but otherwise Actor 140A captured from Location A may be projected or rendered behind Actor 146B on an LED panel that does not match the previs 199. Once the captured content is in its final position for filming, based on the position of Camera 132B and the orientation at which the scene was captured, it can be rotated to match Volume 106B in Location B, its lighting, color, contrast and brightness can be adjusted to match Volume B 106B, any additional visual effects can be added, and rendered with all other content on the LED panel so that filming can begin in Volume B 106B.

[0070] The process of taking the captured content and tracking items (e.g., cameras, lights, light sensors, other sensors, microphones, actors, props, etc.) for rendering on the LED panels of volumes 106A, 106B may be done semi-manually with some input from a person (e.g., from the ingestion of a previs 199, a script, one or more state machines, or other choreography documents). The content display interface module, with the help of the rendering engine, establishes and performs the necessary steps to adjust the captured content using the coordinates and orientations of the captured content (e.g., microphones, cameras, actors, props, lights, light sensors, other sensors, movement, LED wall displays, etc.) at Location A relative to Location A's camera 132A, and the coordinates and orientations of all items at Location B relative to Location B's camera 132B. This can be achieved automatically by using 3D mathematics (e.g., polar coordinates, Euler angles, quaternions, etc.), artificial intelligence (e.g., neural networks, classifiers, etc.), speech / audio recognition, and computer vision (e.g., gesture recognition, edge detection, face detection, facial expression recognition, silhouette detection, motion detection, activity detection, object recognition, object tracking, gaze tracking, item classification, 3D reconstruction, blob detection, pattern recognition, segmentation, etc.) to capture data (e.g., audio data, image data, orientation data, coordinate data, setting data, tracking data, etc.) for all items in the volume (e.g., microphones, cameras, actors, props, lighting, light sensors, extras, crew, etc.).

[0071] For example techniques relating to the use of computer-based recognition techniques, see U.S. Pat. No. 10,540,772, filed Feb. 25, 2019, entitled “Feature Trackability Ranking, Systems and Methods” by Wnuk; U.S. Pat. No. 10,346,712, filed Oct. 5, 2018, entitled “Metric-Based Recognition, Systems and Methods” by Siddiqui; U.S. Pat. No. 10,410,088, filed Sep. 15, 2017, entitled “Invariant-Based Dimensional Reduction of Object Recognition Features, Systems and Methods” by Wnuk et al.; U.S. Pat. No. 10,572,724, filed Feb. 25, 2019, entitled “Activity Recognition Systems and Methods” by Wnuk et al.; and U.S. Pat. No. 10,572,724, filed Feb. 25, 2019, entitled “Robust Feature Identification for Image-Based Object Recognition Systems and Methods” by Song et al. No. 10,719,731, filed June 24, 2019, entitled "Image-Based Feature Detection Using Edge Vectors" by Song et al., and U.S. Pat. No. 10,229,342, filed December 15, 2017, entitled "Image-Based Feature Detection Using Edge Vectors" by Song et al., both of which are incorporated herein by reference in their entireties.

[0072] Using the captured data, the system (e.g., content production management system 102, control module 124A or 124B, etc.) can instruct actors, crew, extras, microphones, cameras, lights, light sensors, and LED panels at each location via commands executed on corresponding target devices. Additionally, the system can utilize audio and / or visual cues to capture one or more of: displaying instructions to actors on a screen, creating or lighting arrows, turning lights on, turning lights off, providing audio instructions to actors / extras / camera operators, providing haptic feedback, displaying instructions to crew, generating VFX on LED panels, etc. Commands corresponding to such cues can be timed to account for latencies caused by one or more of the following: information transmission latencies, network latencies, latencies in rendering content on the LED panels, latencies due to human reaction time to receiving commands, latencies for camera movement, latencies for camera acceleration, latencies in repositioning captured content on other or secondary volumes and their corresponding LED panels, planned latencies as defined in the script or previs, or latencies due to other causes. The system can be configured to receive the necessary information (e.g., a script with the actors' lines, sounds that will be present at the locations, previs images of the scenes, actors and their costumes at each location, expected actor movements in the scene, locations of props, or more) to generate estimated timings of execution of commands to manage the shoot and timings of commands for various types of cues required by the humans involved and to create a list of events and their timings for the shoot in the various volumes. The system can begin a process or render a cue as soon as an appropriate trigger is activated (e.g., as soon as the director says action) and continue acting, largely by executing commands, if not fully autonomously.The system may cue individual actors, extras, support staff or other personnel while also managing rendering systems, lighting, cameras or other devices. Additionally, the system may adjust shot timing based on estimated latency, or adjust timing during a shoot based on actual observed latency by passing latency information to other locations involved in the shoot.

[0073] The pace or rate of the content production effort can be tracked in real time while the actors are performing the required scenes. For example, a script can be compiled down to one or more expected observations and expected actions that should be performed in response to the execution of commands. For example, each studio and each scene can have one or more corresponding state machines, each state having a set of commands to execute, and each transition including expected actions or observations that should be present before transitioning to the new state. The compiled script can then be translated, possibly via the command interface module 110, into commands that are then placed in the corresponding queue. The content production management system 102 or other appropriately enabled devices can use computer speech or audio recognition algorithms running on one or more processors to observe the pace of filming or timing required for various commands, including cuing actors, moving cameras, triggering events, rendering visual effects, rendering content to LED panels, adjusting microphone inputs, moving vehicles, adjusting lighting, and providing instructions to extras and crew at various locations, to ensure precise or accurate timing throughout the volume of the ecosystem. The content creation system 102 or other devices in the system may also use computer vision algorithms to recognize the actors, their gestures (see U.S. Pat. No. 1,057,2724, referenced above), the direction they are facing, their location within the volume, their facial expressions, and based on this information, determine where they are in the script or adjust some or all of the timing or cues as needed. Additionally, devices in the ecosystem may be able to move the camera to ensure that shots are captured as specified by the director, script, previs, or other creative authority.For example, if it is observed that the scene being captured is progressing at a slower or faster pace than expected, the commands in the command queue can have their execution times altered to slow down or speed up the capture of other volumes involved in the capture as necessary.

[0074] In some embodiments, the commands may also include one or more execution priority values. For example, each command may include a particular priority value selected from a plurality of priority values. The priority values ​​may include, for example, a numerical range, an alphabetical range, or another suitable range of values ​​to order the importance of the command relative to other commands. For example, one command A may include a priority value of 1, and another command B may include a priority value of 5, indicating that command B is less important than command A. Thus, if production slows down, a command with a lower execution priority (e.g., command B above) may be removed from execution to ensure that production returns to expected timing. Such an embodiment provides for the creation of a "good enough" production, possibly as part of a rehearsal or as part of a director's optional set of actions.

[0075] In some embodiments, cues can be a very important part of the generation of content in a controlled or managed scene, and the content management system can utilize many types of cues or techniques to provide such cues. Non-audio cues can be important for actors and extras being filmed, since audio cues may not be possible because they would interfere with the dialogue of the script. Furthermore, using earphones for audio may be undesirable as it may be visible on camera. However, audio cues may be used for extras, camera operators, or crew. Thus, visual or tactile cues are more suitable for actors. For example, visual cues can be rendered or presented according to one or more commands on LED displays in a large 3D LED panel volume (e.g., volume 106A and / or volume 106B). Thus, actors can look in almost any direction and see the cues being displayed or rendered, and in some cases, read messages that are displayed as part of the cues. Such messages can be strategically placed according to corresponding command parameters to direct the actor to look in a particular direction and allow the camera to capture the desired shot, where the camera is pointed away from the visual cue rendered on the LED. Furthermore, cues can be generated via haptic devices (possibly small devices attached to the actor's clothing) that can signal the actor to move in a certain direction or to stop. Such small electronic devices (e.g., Yost LABS 3-space Mini Bluetooth, etc.) or smart clothing (TactSuit, Woojer Vest, TeslaSuit, see URL teslasuit.io, etc.) provide content display interface modules (e.g., Display Interface Module 136A and / or Display Interface Module 136B) to communicate with all actors and reduce the number of retakes.

[0076] Other cues to devices (e.g., cameras, robots, vehicles, etc.) may require additional consideration in forming the corresponding commands. For example, a camera command may include a movement to prepare the camera before taking a picture. Thus, the command used to generate the camera cue may need to consider the acceleration of the camera (or other higher order derivatives of the distance traveled) as part of the latency for the corresponding command, while also considering additional time to start moving the camera before, during, or after taking a picture. More specifically, the camera or other equipment may need time to accelerate to a particular speed or velocity, since the equipment is not always in the right position at an instant or has the desired motion parameters at an instant. The motion of cameras and other props (e.g., vehicles, robots, environmental objects (such as swings or swinging trees)) may also require time to reach the appropriate motion, which may increase the expected latency of the corresponding command. It should be understood that in many scenarios, the motion is not linear, which may change the way in which the command is triggered. In other words, the execution time of the command may be a function of the acceleration or other higher order derivatives (e.g., d 3 x / dt 3 , d 4 x / dt 4 Of particular interest is that some studios may not have the space to track the camera outside of the volume, forcing actor and prop movements, so props, cameras or actors can be tilted, moved on tracks, pedestals, etc., as required to fit the seven basic camera movements (i.e. zoom, pan, tilt, dolly, track, pedestal, rack focus).

[0077] Digital content (e.g., film, audio, stills, etc.) captured from one or both of studios 104A, 104B may be transmitted to the content production and management system 102 and / or to another location. The film captured from both studios 104A, 104B may then be combined, integrated, or otherwise managed as described herein. For example, the captured film from both studios 104A, 104B can be combined using computer vision algorithms based on feature tracking or based on stitching the content together based on identified features (such as the scale-invariant feature transform (SIFT) algorithm, the speeded-up robust features (SURF) algorithm, Features from Accelerated Segment Test (FAST), Binary Independent Elementary Features (BRIEF), ORB Object FAST and Rotated BRIEF, Canny Operator, Canny edge detector, etc.). Such vision algorithms, packages or APIs (e.g., Microsoft Computer vision, AWS Rekognition, Google Cloud Vision, IBM Watson, Cloud Sight, ClarifaiV2, Kairos, EmoVu, ImageVision, Animetrics, NANT's iD platform API, OpenCV, etc.) can be configured to track the background and separate the actual actors and props from the background based on a priori identified tracking features (see U.S. Pat. No. 1,054,0772 referenced above). In such cases, the background can be upgraded to a higher resolution if desired.This can be done in real time, for example, based on the camera coordinates. In some examples, the stakeholders (e.g., director, etc.) may zoom out further than what the camera captured, since the coordinates may be used to generate the required environment. Among other devices in the ecosystem, the content production and management system 102 can be configured to modify, via software instructions, how the film is combined or integrated from different volumes. Such adjustments can be small, such as adjusting relative lighting, to larger changes, such as removing an object from the digital content derived from the volume (e.g., removing overlapping star 142A and leaving only star 142B or vice versa) or moving actor 140B left or right so that it appears as if the actor is holding hands with one of the two actors 144B, 146B. Given this approach, post-production can be reduced through coordinated distributed command queues, since integration occurs in near real time during production. The technical advantages are clear. The disclosed approach provides computer-based techniques that leverage distributed command queues, and in some cases, computer vision systems, to reduce tracking of the timing, location, or orientation of objects within the distributed volumes. As a result, the content production management system 102 achieves a more realistic production that requires less post-production work in the production available to remote directors or other stakeholders. Examples of techniques for rendering distinct streams of content when used locally that can be adapted for use with the subject matter of the invention include those disclosed in U.S. Patent Application Publication No. 2021 / 0304418, filed March 31, 2021, entitled "Digital Representation of Multi-Sensor Data Stream," by Soon-Shiong, which is incorporated herein by reference in its entirety. For example, content generated from the execution of a synchronization command can be placed in a buffer queue in preparation for synchronous rendering when used.

[0078] flowchart An exemplary command categorization operation is illustrated in Figure 3. The command categorization operation may be implemented, for example, in a command queue management module of a content production management system (e.g., command queue management module 112 of Figures 1 and 2), among other devices. In other embodiments, the command categorization operation of Figure 3 may be implemented, for example, in a computing system associated with a studio (e.g., one or both of computing systems 120A, 120B associated with studios 104A, 104B of Figure 1). In yet other embodiments, one or more steps of the command categorization operation may be implemented in a command queue management module (e.g., command queue management module 112 of Figures 1 and 2), and one or more other steps of the command categorization operation may be implemented in a computing system associated with a studio (e.g., one or both of computing systems 120A, 120B associated with studios 104A, 104B of Figure 1).

[0079] The control begins at 304 where the control queries for a set of commands. The query for the set of commands at 304 may be implemented, for example, via the command queue management module 112 of FIG. 1 and / or one or both of the computing systems 120A, 120B. For example, one or more commands may be generated by a command interface module as described herein based on input from stakeholders or other sources. The generated commands may be stored or buffered in one or more data structures or databases, such as command queues, in one or more non-transitory computer-readable media (e.g., memory 118 associated with the content production management system 102 of FIG. 1), as described herein. In other embodiments, the generated commands may be transmitted (individually or collectively) to one or more remote locations (e.g., to computing systems 120A, 120B of FIG. 1) and then stored or buffered in one or more data structures in one or more non-transitory computer-readable media (e.g., memory 138A, memory 138B associated with computing systems 120A, 120B of FIG. 1, etc.) as described herein. The command set may then be retrieved from one or more data structures through a request or query implemented, for example, by command queue management module 112, control module 124A, control module 124B of FIG. 1, etc., depending on where the commands are stored or buffered. In some embodiments, queries to the command queue or database may be made using keywords, SQL queries, attributes, or other techniques.

[0080] Control then proceeds to 308 where control determines whether the query returned a command. If no command was returned, control returns to 304, and if a command was returned, control proceeds to 312. At 312, control selects a command (e.g., the first command) of the command set. It should be understood that while a stakeholder (perhaps a director) may provide input related to the command, the command may also originate from a computing device of the ecosystem. For example, in some embodiments, the command may be generated via a computer enabled with software instructions that can understand scripts (e.g., OCR, script compiler, etc.), previs compilation, state machine management modules, rendering engines, or other types of management systems. The computer can then translate such information into one or more commands targeted to specific target devices at specific locations within various volumes. The commands may include many features to facilitate production. As discussed, the commands may be targeted to specific entities. Furthermore, a command may include multiple instructions, which may be encoded in a machine-understandable format (e.g., script, compiled object code, XML or JSON encoded instructions, etc.) so that the target device (or an appropriate adapter) can execute the command instructions. A command may also include attributes (e.g., metadata, attribute-value pairs, etc.), priority values, keywords, etc., which describe the nature of the command and may modify the command behavior or be used as the basis for queries. For example, a command may include specific attributes and priority values ​​that indicate the purpose of the command and the importance of the command relative to other commands. In addition, a command may include an execution time (e.g., absolute time, relative time, local time, execution time tolerance, etc.) that helps ensure that all commands are executed at the appropriate time relative to each other.In addition to an execution time, a command may include one or more additional trigger criteria, which may include necessary or optional conditions that must be met before the command is executed. For example, in addition to a particular time, a command may require that an actor be in a particular location within a volume (e.g., a particular voxel, etc.) before the command is executed (a VFX explosion fires, a camera movement begins, a prop moves, etc.).

[0081] Next, at 316, the control determines a target content creation entity for the selected command. For example, the control may determine the target content creation entity based on input from stakeholders, who / what the command is telling them to do what, etc. With respect to the script, the entity may be identified by character name or set direction. Additionally, in some embodiments, various entities may be identified via a digital storyboard showing particular characters, desired camera positions (e.g., coordinates, camera identifiers, etc.), props, or other entities associated with the production. After the target content creation entity is determined, the control proceeds to 320.

[0082] At 320, the control categorizes and stores the selected command according to the determined target content production entity. For example, if the control determines that the target content production entity for the command is a movable platform, the control may categorize (e.g., label, etc.) the command accordingly and store the command in a command queue associated with the movable platform. In some embodiments, the categorized command may be stored with other related commands such that the command is part of a state of a state machine associated with a remote studio, a scene filmed at a remote studio, an entire production, etc., as described herein. Control then proceeds to 324.

[0083] The corresponding command queue may be located locally in the content production management system, for example, or stored remotely at one of the remote studio locations, or may be buffered for use. In some embodiments, the command queue may be stored in the memory of the particular device that is the target entity of the command. The command queue may be implemented as a data structure in which the commands are ordered according to one or more command attributes (e.g., execution time, trigger criteria, etc.). Examples of acceptable data structures may include linked lists, queues, dequeues, vectors, maps, trees, or other data types, to name just a few. For small devices that may have minimal memory, the device may be equipped with an adapter, a device with a larger memory, that is communicatively coupled to the small device and that can store the command queue. The adapter may then submit commands to the small device for execution. Such an adapter may be referred to as a device server, with a network interface exposed to the outside of the small device and an interface for the small device (e.g., RS-232, USB, etc.). Exemplary device server products include those offered by Lantronix®, Digi International®, or Moxa®, or in some cases are implemented using commercially available hardware such as a Raspberry PI or using proprietary devices.

[0084] At 324, control determines whether the query returned additional commands. If no additional commands were returned, control returns to 304, otherwise control proceeds to 328. At 328, control selects the next command from the query (e.g., a second command). Control then returns to 316, where control determines the target content authoring entity for the selected command, as described above.

[0085] An example operation for allocating execution time for a command is shown in Figure 4. The example operation of Figure 4 may be implemented, for example, in a command queue management module (e.g., command queue management module 112 of Figure 1) of a content production management system, among other appropriately enabled devices. The control elements of Figure 4 may be reordered to suit a particular implementation. For example, a command may be generated prior to obtaining a target entity set.

[0086] The control begins at 404 where the control queries for a set of target content production entities (e.g., devices, cameras, equipment, actors, robots, etc.). For example, the target content production entities located in different studios (possibly remote studios) may be known or identifiable to the content production management system. In some embodiments, one or more databases may store a set of potential entities and / or their corresponding target devices, possibly indexed by their capabilities or by characteristic attributes. The entity database may be populated from entities inferred from scripts, previs, or other sources outlining the requirements for a set of entities (e.g., defined in a JSON file, listed in a table, etc.). Additionally, each site may have their own entity database, which may facilitate non-joining studios to collaborate with each other. Queries to the entity database may be made using keywords, SQL queries, attributes, or other techniques. Entities in the database may be tagged or indexed with relevant attributes to make them searchable. For example, entities may be tagged with attributes including location, capabilities, associated scripts, scene information, cost of use, or other types of information. Cost of use information is provided for production studios to track usage by third parties for billing purposes. It is believed that one aspect of the present disclosure includes providing auditing or billing services based on the allocation or usage of production equipment as evidenced by corresponding execution commands.

[0087] Control then proceeds to 408 where control determines whether the query returned a targeted content creation entity. If no targeted content creation entity was returned, control ends, otherwise control proceeds to 412. At 412, control selects a targeted content creation entity of the targeted content creation entity set from the query (e.g., the first targeted content creation entity).

[0088] Next, at 416, the control determines whether a command is associated with the selected target entity. For example, each command may be categorized and stored in a command queue associated with a particular target entity, as described above. The command may be determined based on the capabilities of the entity. Consider scenarios where an entity is tagged with or includes an exposed API. The API may be expressed via a markup language (e.g., XML, JSON, YAML, etc.) from which the command is derived or otherwise identified. Thus, the control may access one or more command queues associated with the selected target entity and determine whether the command is stored in the command queue. If the command is associated with the selected target entity at 416, the control proceeds to 420; if the command is not associated with the selected target entity, the control proceeds to 448.

[0089] At 420, the control determines a command destination associated with the selected target entity. For example, the control may determine an IP address, a particular network endpoint, a location ID, a volume ID, a voxel ID, a camera capture ID, a URL, a UUID, a GUID, and / or another identification marker associated with the selected target entity. In some embodiments, the command destination may include addresses that follow a hierarchical structure. For example, the addresses may be structured in a manner similar to <studio>, <volume>, <device>, <queue>, where "studio" represents a studio identifier or name, "volume" identifies a particular capture volume within the studio, "device" represents a particular device identifier (e.g., MAC address, UUID, GUID, IP address, etc.), and "queue" may be used to represent a particular command queue within or associated with a device.

[0090] Control then proceeds to 424. At 424, the control selects a stored command (e.g., the first stored command) associated with the selected target entity. Next, at 428, the control assigns a synchronous execution time or other trigger condition for the selected command. For example, the control can assign a synchronous execution time for the selected command based on, for example, latency, time-related factors, command sequencing, other command execution times, etc., as described herein. The synchronous execution time for a command can be an absolute time (e.g., a globally determined time, a local time, etc.), a relative time, or a combination of both. For example, if the command for the selected target entity depends on one or more other entities, the relative execution time can be a relative time relative to the other entities completing those commands, and can be adjusted to trigger the relative time after those entities complete their commands. Thus, as previously discussed, the execution criteria for a command can depend on additional factors other than time. In addition, if the system fails to detect completion of the command by the other entities within a specified time, the system can have a fallback absolute synchronous execution time that it will use to execute the command in that case. The execution time may take into account communication or network latencies between various devices or studios in the ecosystem, and the latencies may be measured using various techniques including obtaining time via GPS sensors, NTP, ping time, measuring round trip time (RTT), time to first byte (TTFB), or other techniques. Individual latencies may be added (or subtracted) to arrive at the execution time. When the execution time is determined, consideration is preferably given to when and where the corresponding captured digital content is to be presented or viewed.Thus, if a stakeholder (e.g., director, producer, etc.) is remote from the distributed studio and content is to be presented to the stakeholder, the latency used to calculate the execution time is likely to be to the stakeholder's computing device as well as to the remote device. The latency can also be measured based on running one or more simulations or rehearsals to account for human-imposed latency.

[0091] Control then proceeds to 432. At 432, control optionally transmits the command and its execution time to a destination for the selected target entity, possibly over a network. In such an example, the command is passed individually to the selected target entity or device. The command may be packaged as a serialized object (e.g., binary, XML, JSON, YAML, etc.) and transmitted over one or more protocols (e.g., HTTP, SSL, TCP / IP, UDP / IP, etc.).

[0092] Control then proceeds to 436. At 436, control determines whether additional commands are associated with the selected target entity. If additional commands are associated with the selected target entity, control proceeds to 440, and if additional commands are not associated with the selected target entity, control proceeds to 444. At 440, control selects the next stored command associated with the selected target entity (e.g., the second stored command). Control then returns to 428 to assign a synchronous execution time to the selected command.

[0093] At 444, control optionally transmits all of the commands associated with the selected target entity and their execution times to a destination for the selected target entity. In such examples, the commands may be passed collectively to the selected target entity. In some examples, a command queue containing all of the commands may be passed to the selected target entity. As with the commands, the command queue may be packaged or serialized for transmission over the network. Control then proceeds to 448.

[0094] At 448, control determines whether the query returned additional targeted content creation entities. If no additional targeted content creation entities were returned, control ends; if additional targeted content creation entities were returned, control proceeds to 452. At 452, control selects the next targeted content creation entity of the targeted content creation entity set from the query (e.g., a second targeted content creation entity). Control then returns to 416.

[0095] In Fig. 5, an exemplary command execution operation is shown. The exemplary command execution operation of Fig. 5 may be implemented, for example, in a remote studio. For example, the command execution operation may be implemented using a control module located in the remote studio having processor hardware, etc. associated with a target content production entity of the remote studio. If the target entity is a device (e.g., a camera, a device adapter, etc.), the device may execute the command directly. If the target entity includes a human, the corresponding target computing device may present a command (e.g., a cue, a prompt, etc.) to the human. Such commands may include visual commands (e.g., rendered on a display, etc.), audio commands, haptic commands, or other modalities observable by a human. Such commands or cues may be rendered, for example, on an LED wall within the field of view of the human (but outside the field of view of the camera).

[0096] Control begins at 504 where control queries a command set. For example, commands received from a content production management system can be stored in local and / or remote memory associated with the control module, the target content production entity, etc. For example, the command can be to pop (i.e., dequeue) from the front of a corresponding queue. Control then proceeds to 508 where control determines whether the query returned a command. If no command was returned, control returns to 504, and if a command was returned, control proceeds to 512. At 512, control selects a command (e.g., the first command) of the command set from the query. Results of the execution can be archived, for example, possibly as an NFT or data on a notary ledger, possibly for future use (e.g., audit, billing, machine learning training data, review, etc.).

[0097] Next, at 516, control identifies an execution time for the selected command. For example, the execution time for the selected command may have been previously assigned and stored in local and / or remote memory associated with the control module, the targeted content production entity, etc. Each execution device may have access to a clock (e.g., GPS, internal clock, NTP, etc.) to monitor the current time (possibly a globally synchronized time). When the command's execution time is reached and other trigger criteria are met, the device executes the command. Control then proceeds to 520, where control executes the selected command according to its execution time. Control then proceeds to 524.

[0098] At 524, control determines whether the query returned additional commands. If no additional commands were returned, control returns to 504; if additional commands were returned, control proceeds to 528. At 528, control selects the next command in the command set from the query (e.g., the second command). Control then returns to 516 to identify the execution time for the selected command.

[0099] Additional Considerations The preceding discussion has presented the subject matter of the invention primarily in the context of a distributed studio environment. In addition to the above, the following discussion expands upon embodiments of the subject matter or provides additional details regarding the subject matter of the invention.

[0100] Given the disclosed ecosystem that includes multiple facilities and devices distributed over geographic distances, the content production management system can provide network management capabilities to ensure that all devices can communicate with each other efficiently, regardless of location. One aspect of network management includes managing latency between interconnected devices. As previously discussed, latency can be measured between at least two devices using empirical techniques. In addition to measuring latency, some embodiments provide for configuring or provisioning a network connecting the devices to have a desired latency or other level of service. Exemplary techniques that can be adapted to provision a network are described in U.S. Patent No. 9,917,728, filed Jan. 13, 2015, by Wittenschlaeger, entitled "Software-Enabled Fabric Enablement," and U.S. Patent No. 10,826,796, filed Sep. 26, 2017, by Charnock et al., entitled "Virtual Circuits in Cloud Networks." These patents are incorporated herein by reference in their entireties. For example, a content production management system can be configured to have a latency threshold that must not be exceeded (e.g., not to exceed 100 ms, not to exceed 500 ms, etc.). In response to such configuration parameters, the content production management system can select a network channel, possibly up to a particular wavelength of light on an optical fiber, that can provide the required level of latency service. The selected channel can then be dedicated to the production effort, possibly for a specified or lease time. Thus, one aspect of the inventive subject matter is to determine an execution time for a command based on a provisioned network latency between at least two devices.

[0101] Beyond latency management, there are other aspects of network management that may be deeply considered in the disclosed ecosystem. Additional areas of network management may include inventorying or querying available equipment operating as part of or connected to the network, generating alerts based on observed device states or observed command activity associated with the network, monitoring behavior of target entities on the network, recovering various devices on the network, securing device communications on the network, reporting information obtained over the network, or other management activities. For example, commands executed by various target entities (e.g., devices, actors, etc.) before, during, or after a production event may be reported to a content production management system for display to stakeholders. Such behavior may be reported via SNMP or a management information base (MIB). For example, U.S. Patent No. 1,0958,512, filed March 12, 2020, entitled "Fine Grained Network Management to Edge Device Features" by Soon-Shiong et al., which is incorporated herein by reference in its entirety, describes a method for reporting such device behavior via a MIB. Additionally, network management activities (particularly observing commands, command execution, or communication with devices regarding command queues) can be presented to stakeholders via a dashboard through software such as OpenNMS (see URL www.opennms.com). Consider a scenario where a director creates a set of commands to be used for production in two or more studios. As each device executes those commands, the observed execution results can be sent back to the director's dashboard on the display module of the content production management system (e.g., via SNMP, XML, JSON, OpenNMS, etc.). The dashboard can show the current state of each command queue and associated commands versus their execution time.To inform the director of deviations from execution times, such deviations can be highlighted and / or the visual presentation of the command queue can be altered.

[0102] Also, additional details regarding commands may provide insight into various embodiments of the inventive subject matter. Commands may generally be considered as data structures or objects instantiated in one or more non-transitory computer-readable memories. If an object-oriented computer language (e.g., C++, Java, Python, etc.) is used to create commands, the commands may be instantiated from one or more command classes. Thus, individual commands may be considered as separate manageable objects. The command classes may include additional data members (e.g., variables, constants, methods, APIs, functions, etc.) to easily manage the commands collectively or individually. In other cases where a non-object-oriented language (e.g., C, assembly, etc.) is used to create commands, the commands may be created in memory using the language's data structure syntax (e.g., in C, one may use typedef, struct, typedef struct, etc.).

[0103] The difference between a target entity and a target device that executes a command should be understood. In some cases, the target entity and the target device may be the same physical device, such as a camera. However, the target entity (e.g., an actor) is not a target device or device. In such cases, the target device is more likely to be a device that is configured or programmed to present a cue to the target entity (e.g., an actor, etc.) based on one or more associated commands. Thus, the target device (e.g., a display screen, an LED wall, a speaker, etc.) may execute a command to present or render an instruction (e.g., a cue, etc.) to the target entity (e.g., an actor, a staff member, a crew member, etc.). The target entity may then take an action based on the received instruction. Thus, a command may include one or more attributes that associate the command with a target device and / or a target entity. Additionally, a command may have attributes that define a desired result (i.e., a particular movement or action to be taken by the target entity) to be achieved after execution of the command or after acting on the cue. For example, a command class may have one or more member instance attributes indicating a target device type (e.g., device class, manufacturer, model, etc.) or target entity type, so that available device commands may be instantiated according to the capabilities of a target device associated with the target entity. Such capabilities may be discovered via one or more discovery protocols or protocols that may be adapted to be discovered (e.g., SSDP, SDP, SNMP, ICMP, proprietary discovery protocols, etc.). Upon execution of the device command, the observed execution results by the device may be reported to the content production management system. Considering that the target device and the target entity may be different, there may be a delay between the execution of the device command by the target device and the observation of the target entity taking the corresponding action.The delay between execution and action may be taken into account in determining the execution time of a command. Such an approach is advantageous in identifying or diagnosing problems within an ecosystem, or in adapting, possibly in real time, to action delay problems encountered.

[0104] Additional command attributes (e.g., member data variables, instance variables, etc.) that may be associated with a command extend beyond execution time. Additional command attributes that may be used to manage commands may include one or more of the geographic location where the command is executed, the target voxel of the command (e.g., location within a 3D volume, etc.), the expected duration of command execution, a tolerance or criterion for command execution indicating when execution is considered successful or unsuccessful, pointers or links to other commands or command queues that allow cascaded execution of interdependent commands, just to name a few.

[0105] Yet another type of command attribute may include one or more command states, which may be part of a state machine embodiment. A command state indicates the current situation in which a command exists before, during, or after its execution. Exemplary command states may include NULL, where the command has been instantiated but not fully defined. A state of PENDING may indicate that the command is waiting to be executed. A state of EXECUTING may indicate that the command is currently being executed. Such states may be useful in scenarios where a command may take time to complete (e.g., an actor walking around in a filming volume). A state of EXECUTED or COMPLETE may indicate that the command has finished. Another type of command state (possibly a primary or secondary state) may indicate whether execution was successful or not (possibly via a SUCCESS or FAIL state). The various states that a command may have may depend on the target entity or target device. In some embodiments, a command may follow a state machine (possibly an entity-specific or device-specific state machine where only certain state transitions are possible based on command execution or observed execution results). In other embodiments, a command may have states defined by a stakeholder or other interface.

[0106] Commands can be generated in many ways. As discussed above, one or more stakeholders can define commands via a user interface on the content production management system. However, it should be understood that the commands are ultimately generated for at least one computing device. For example, a stakeholder can select one or more commands from a menu for execution on a target device. The selected command can then be instantiated with necessary attributes or characteristics, as discussed above. Still further, one or more commands can be automatically generated by such computing device. Continuing with the previous example, the selected command may require additional support commands to be executed first, such as requiring a camera to be moved around within the volume, which may result in a physical collision with other equipment for the filming of the scene. Thus, the system can recognize a collision due to the physical constraints of the filming volume and then generate additional device commands (e.g., move a prop, etc.) targeted at the colliding equipment or the colliding target entity. Thus, an aspect of the inventive subject matter is believed to include identifying commands that may cause a collision within the volume and automatically generating commands with corresponding execution times to address the anticipated collisions. When a conflict is found, the system can employ backward chaining reasoning in terms of the execution time of the current command to determine the execution of the predicted command required to resolve the conflict.

[0107] As mentioned above, the disclosed system can employ one or more cinematography compilers that can convert a priori defined cinematography information into device commands along with their desired execution times. Consider a script or screenplay. Such documents typically follow a highly structured format that can be translated into device commands, possibly based on a corresponding state machine. For example, the script can be analyzed via optical character recognition (OCR) to identify characters (e.g., actors), set direction, or other information. More specifically, when a key feature is found in the script, the compiler can auto-generate the associated device command (perhaps from a line of dialogue a device command is instantiated that represents a cue to the actor). Once instantiated, the director or other stakeholder can use a content production management system to materialize the commands for deployment to the volume. In addition to compiling the script, the previs can also include an initial set of device commands in preparation for deployment to the target volume. The compiled previs commands can again be materialized via the content production management system. It should be appreciated that the content production management system may include attributes related to available equipment (e.g., devices, cameras, etc.) and attributes related to specific volumes that can be used to help populate compiled previs commands to ensure that the commands properly match the real-world target volume. Thus, commands originating from compiled documents, previs, or other structured information may be populated with initial execution times based on a priori requirements, which may be modified to accommodate the real-world filmed volume. Additionally, each production entity, or the ecosystem as a whole, may operate according to one or more state machines, with state transitions occurring as commands are executed, in some cases.

[0108] An interesting aspect of the execution of commands of specific execution times within a distributed filming environment is the ability to monitor observed execution times and observed execution results to ascertain variability against expectations. Monitoring of execution results can be accomplished through target devices reporting actual execution times and / or observed results. The observed results can then be compared to expected results stored in the content production management system. The expected results can be defined by stakeholders, by compiled command sets, or by other techniques. Various devices (e.g., cameras, microphones, sensors, etc.) in the ecosystem can observe the results of the execution through capturing sensor data and running one or more algorithmic implementations on the sensor data (e.g., activity recognition, image recognition, object recognition, natural language processing, OCR, classification techniques, prediction techniques, etc.). Deviation from expected results can be quantified by comparing attributes of the expected results with attributes of the actual results. Such differences (perhaps per attribute) can aggregate to produce a difference. In some scenarios, the differences can be quantified from the resulting attribute space through generating Euclidean, Hamming or other types of distances between attributes, while taking into account or normalizing various attribute values ​​as necessary.

[0109] In addition to commands, details about command queues may also be useful. Command queues may also be implemented as objects, in the computer science sense. Command queues may be instantiated from one or more command queue classes, possibly based on an underlying data structure that can store commands in a desired order (e.g., queue, dequeue, vector, map, tree, circular queue, stack, etc.). Additionally, an instantiated command queue may inherit one or more device-specific characteristics, possibly including available commands (i.e., device capabilities) that can be stored in the queue for execution. When queried via an API, an instance command queue may report a result set of commands available for the queue. With this approach, a content creation management system may ensure that only commands that work within the command queue are generated for the command queue. Commands may be "pushed" (e.g., enqueued, etc.) into the queue or "popped" (e.g., dequeued, etc.) from the queue according to the API of the corresponding data structure.

[0110] The command queue preferably stores commands in a specified order (in a more preferred embodiment, by execution time). For example, the command queue may place commands with the fastest execution time at the top of the queue and commands with the slowest execution time at the bottom of the queue. When the queue or queue manager determines that it is time for a command to be executed, the command may be popped from the top of the queue and executed. It should be understood that some commands may require some duration or time to complete their execution. In such cases, the EXECUTING command may be placed in a separate execution queue until execution is complete. Once execution is complete, the command may be removed from the execution queue and any necessary reporting may be made to registered stakeholders. It should be understood, therefore, that a device may employ multiple types of command queues to manage commands throughout their lifespan or as their state changes.

[0111] A command queue can be a passive command queue or an active command queue. A passive command queue operates based on calls to the queue's API outside of the passive command queue data structure. In other words, a passive command queue does not directly execute its own commands, but relies on a manager outside of the queue to execute commands. An active command queue can be thought of as a queue object that manages its own commands and ensures that the commands are executed as soon as the command's trigger criteria are met. Such queues operate in their own agency. For example, an active command queue may include one or more listeners that observe the influx of environmental sensor data, including the current time. When a queue's listener observes sensor data (e.g., internal state data, external state data, etc.) that meets the trigger criteria of a command (likely the command at the head of the queue) and / or the current time, the queue will execute the command. Command execution may take the form of transmitting a signal to the device, calling an API to the device, executing a script, executing one or more software instructions, or other forms of device execution. Additionally, active queues can report the results of execution to target devices or registered devices (e.g., content production management systems), and can notify other command queues that execution has started or completed so that pending commands in those other command queues can be freed for execution.

[0112] Depending on the nature of the target device, the device's corresponding command queue or queues may reside locally, be remote from the device, or be located on an intermediary device. Target devices with sufficient computing resources (e.g., RAM, CPU, cores, etc.) may have a programmatically available execution environment (e.g., VM, container, script engine, etc.) that allows or enables local instantiation of command queues and further enables local execution of commands. Other target devices lacking such resources may only be capable of responding to signals received from external sources. For example, a small device may only respond to signals received on a PIO pin. In such cases, the command queues may be located on an external queue management device that provides a signal on a PIO pin in response to execution of a command, thereby causing the small device to take an action. Such an external queue management device may be remotely located or implemented on a remote computer over a network (e.g., a content production management system, etc.). Still further, the external queue management device may be an intermediary device connected to the target device and the content management system. In such a scenario, the queue management device may act as a device server or adapter that receives command queue management functions from the content production management server and then manages the execution of the corresponding commands through sending signals to the target devices. As previously mentioned, such device servers or adapters may be obtained from a variety of vendors and then programmed or provisioned with the necessary command queue functionality. Additional devices that may act as such adapters may include Arduino™ or Raspberry Pi™ devices.

[0113] Yet another area to consider includes the management of various production volumes, each of which has its own specific characteristics, including size, shape, actual filming volume (e.g., stage, LED wall, etc.), total production volume available (e.g., areas for cameras or crew, etc.), available equipment and their properties, schedule, cost, or other set of attributes that describe the nature of the volume. Such volume characteristics can be useful in inputting those characteristics into commands or in determining where to place command queues to ensure that the production between volumes and choreography between all devices are properly synchronized when the commands are executed at their desired execution times.

[0114] There are many techniques for managing volumes. Particularly interesting techniques focus on voxel management of volumes. Voxel management involves breaking down a volume into voxels (i.e., volumetric cells), and in most cases the corresponding volume will be filled with a set of voxels of the volume. In other words, a voxel can be thought of as a 3D analogue of a 2D floor tile, where an area is covered by a set of floor tiles. Similarly, a volume can be filled with a set of voxels. A voxel can be thought of as a virtual data object in the memory of a computing device (e.g., the memory of a content creation management system). The voxels of a volume may all be the same size, different sizes, the same shape, different shapes, or may be sized and dimensioned as desired. The shapes can be any 3D shape (e.g., cubes, spheres, cuboids, irregular 3D shapes, tetrahedrons, cylinders, prisms, cones, Platonic solids, etc.). In some embodiments, the voxels are non-overlapping, while in other embodiments the voxels may overlap one another to some extent. Furthermore, the voxels may completely or partially fill a volume, and at least a portion of the volume may require volume management with respect to commands or command queue management.

[0115] Considering that each volume may have a different physical shape, each volume may have a different voxel set. For example, a first studio focused on high fidelity foreground action may have a very fine voxel set, perhaps down to a certain 3D point (e.g., a 10 cm cube fills the volume), whereas a second studio focused on background, mostly static content, may require a coarser voxel set (perhaps a 100 cm cube). From the perspective of a content production management system, the voxel set of the first volume may be mapped to the voxel set of the second volume via a transformation (e.g., V2=T(V1), etc.), where the transformation T may be implemented as a computer-based function (e.g., one-to-one mapping, continuous mapping, discontinuous mapping, etc.). Such a transformation may cause or include transformations of cells (e.g., scaling, rotating, skewing, translating, shearing, reflecting, affine, etc.) to enable the desired mapping. While this example focuses on two volumes, it should be understood that multiple studio volumes can also be mapped to additional volumes (possibly virtual volumes leveraged by a content production management system to assemble the resulting content into a more complete content set or a more final integrated result).

[0116] The management of voxels in the filming volume creates several interesting capabilities with respect to the subject matter of the invention regarding commands and command queues. Consider a scenario where a stakeholder wants to incorporate content from multiple volumes, each of which has their own set of voxels. The voxel sets are incorporated together into a virtual production volume in the stakeholder's content production management system through appropriate transformations. The stakeholder also compiles the desired set of commands (e.g., from scripts, previs, state machines, etc.) along with corresponding command queues for the production of the content or a portion thereof. Rather than simply executing all the commands in a live production, the content production management system or other appropriately enabled computing device may perform a simulation of the production through simulated execution of commands in the various command queues in the virtual volume. The simulation may leverage a game engine, such as the UnReal® game engine or the Unity® game engine. As each command is executed, the impact on individual voxels (preferably in the virtual production volume) may be monitored. For example, monitoring voxels in a simulation may indicate whether an actor is moving through or occupying one or more expected voxels (or, alternatively, one or more unexpected voxels). This approach helps in identifying potential collisions while ensuring that command execution, along with their execution times, occur as expected or desired. In some embodiments, results from the execution of a simulation that executes the commands in the command queue create a voxel heatmap that indicates the occurrence or frequency of the impact that command execution has on a voxel. Such an approach is similar to previs but is believed to be advantageous because it allows for immediate comparison of the expected behavior of production volumes to physical reality, particularly with respect to how volumes should be integrated, something that is typically missing in previs.

[0117] The use of voxels in volume management creates other interesting capabilities besides command simulation management or command conflict management. In some embodiments, voxels may be individually addressable in a way that allows a content production management system to send commands or other messages to voxels or sets of voxels. For example, a group of voxels (e.g., one or more voxels, voxel blocks, voxel sets, etc.) may be assigned an IP address (or other addressable identifier, possibly within a well-defined namespace) and treated as a virtual device. A group of voxels may also have one or more corresponding command queues. Such an approach is advantageous in situations where equipment, devices, actors, or other target entities move in a more non-deterministic manner (e.g., unscripted, ad-lib, etc.). As the target entity moves through the voxel, commands may be transmitted to the target entity's corresponding target device. Thus, in this example, commands and their execution times may be delayed until the system (e.g., controllable device, camera, game engine, etc.) detects the presence of an entity within the voxel. Upon detection, the system instructs the target device to execute the command.

[0118] conclusion By employing one or more aspects of the teachings herein, travel associated with content production can be reduced. For example, actors can be able to film in a nearby studio instead of traveling around the world. Additionally, by employing one or more aspects of the teachings, the final product at the end of filming can be much closer to the final production of a movie, television program, or the like, than with conventional systems. This can reduce post-production time and costs, for example. For example, the production of a movie using green screen technology may require approximately 400 days of work, while the production of the same movie using LED wall technology may require approximately 50 days of work. However, the use of a distributed studio environment disclosed herein can significantly reduce post-production time, building props, studio setup configurations, and the like, compared to conventional systems.

[0119] It should be understood that the teachings disclosed herein can be employed in a variety of applications. For example, while the teachings herein are described primarily in relation to content creation environments, it should be understood that the teachings can be employed in other applications involving remote users, systems, etc. For example, the teachings can be employed in physical training applications (e.g., involving remote trainers and trainees), team training applications, educational applications (e.g., remote learning applications, simulations, etc.), military applications, news, motion capture applications, e-sports (e.g., tournaments, virtual camera operators, replays, etc.), and the like.

[0120] The foregoing description is merely exemplary in nature and is not intended to limit the disclosure, its application, or uses in any way. The broad teachings of the disclosure can be implemented in various forms. Thus, while this disclosure includes specific examples, the true scope of the disclosure should not be so limited, since other variations will become apparent from a study of the drawings, this specification, and the following claims. It should be understood that one or more steps in a method can be performed in a different order (or in parallel) without altering the principles of the disclosure. Furthermore, although each of the embodiments is described above as having certain features, any one or more of those features described with respect to the embodiments of the disclosure can be implemented in and / or combined with any feature of the other embodiments, even if the combination is not explicitly described. In other words, the described embodiments are not mutually exclusive, and one or more embodiments can be substituted for one another within the scope of this disclosure.

[0121] Spatial and functional relationships between elements (e.g., between modules) are described using a variety of terms, including "connected," "engaged," "interfaced," and "coupled." Unless expressly described as "direct," when a relationship between a first element and a second element is described in the above disclosure, the relationship encompasses a direct relationship between the first element and the second element, with no other intervening elements between them, and also encompasses an indirect relationship between the first element and the second element, with one or more intervening elements (spatial or functional). Reference to at least one of A, B, and C should be interpreted as meaning the logic (A or B or C) using a non-exclusive logical OR, and not as meaning "at least one of A, at least one of B, and at least one of C."

[0122] In the diagram, the direction of the arrow, indicated by the arrowhead, generally indicates the flow of information (such as data or instructions) of the subject of the diagram. For example, an arrow may point from element A to element B when element A and element B exchange various information, but what is relevant to the diagram is information transmitted from element A to element B. This unidirectional arrow does not imply that other information is not transmitted from element B to element A. Furthermore, for information transmitted from element A to element B, element B may send a request for that information or an acknowledgment of receipt of that information to element A. The term subset does not necessarily have to be a proper subset. In other words, a first subset of a first set may be coextensive with (equal to) the first set.

[0123] In this application, including the following definitions, the term "module" or "controller" may be substituted for the term "circuitry." The term "module" may refer to, be a part of, or include processor hardware (shared, dedicated, or group) that executes code, software instructions, or other executable data, and memory hardware (shared, dedicated, or group) that stores code executed by the processor hardware.

[0124] The module may include one or more interface circuits. In some examples, the interface circuit may implement a wired or wireless interface to connect to a local area network (LAN) or a wireless personal area network (WPAN). Examples of LANs are the Institute of Electrical and Electronics Engineers (IEEE) standard 802.11-2016 (also known as the WIFI wireless networking standard) and the IEEE standard 802.3-2015 (also known as the ETHERNET wired networking standard). Examples of WPANs include the IEEE standard 802.15.4 (including the ZIGBEE standard from the ZigBee Alliance) and the BLUETOOTH wireless networking standard from the Bluetooth Special Interest Group (SIG) (including the Core Specification versions 3.0, 4.0, 4.1, 4.2, 5.0 and 5.1 from the Bluetooth SIG).

[0125] Modules can communicate with other modules using interface circuits. Although in this disclosure, modules are depicted as logically communicating directly with other modules, in various implementations, modules can actually communicate through a communication system. A communication system includes physical and / or virtual network equipment (such as hubs, switches, routers, and gateways). In some implementations, a communication system is connected to or traverses a wide area network (WAN) such as the Internet. For example, a communication system may include multiple LANs connected to each other over the Internet or point-to-point leased lines using technologies such as Multiprotocol Label Switching (MPLS) and Virtual Private Networks (VPNs).

[0126] In various implementations, the functionality of a module may be distributed among multiple modules connected via a communication system. For example, multiple modules may implement the same functionality distributed by a load balancing system. In a further example, the functionality of a module may be split between a server (also known as remote or cloud) module and a client (or user) module. For example, a client module may include a native application or a web application that runs on a client device and is in network communication with a server module.

[0127] The term code, as used above, may include software, firmware, and / or microcode and may refer to programs, routines, functions, classes, data structures, and / or objects (which may in some cases form a software suite). Shared processor hardware encompasses a single microprocessor that executes some or all code from multiple modules. Group processor hardware encompasses a microprocessor that executes some or all code from one or more modules in combination with additional microprocessors. References to multiple microprocessors encompass multiple microprocessors on separate dies, multiple microprocessors on a single die, multiple cores of a single microprocessor, multiple threads of a single microprocessor, or combinations of the above.

[0128] Shared memory hardware encompasses a single memory device that stores some or all code from multiple modules. Group memory hardware encompasses memory devices that, in combination with other memory devices, store some or all code from one or more modules.

[0129] The term memory hardware is a subset of the term computer-readable medium. The term computer-readable medium, as used herein, does not encompass transitory electrical or electromagnetic signals propagated through a medium (such as on a carrier wave). Thus, the term computer-readable medium is considered to be tangible and non-transitory. Non-limiting examples of non-transitory computer-readable media are non-volatile memory devices (such as flash memory devices, erasable programmable read-only memory devices or masked read-only memory devices), volatile memory devices (such as static random access memory devices or dynamic random access memory devices), magnetic storage media (such as analog or digital magnetic tape or hard disk drives) and optical storage media (such as CDs, DVDs or Blu-ray discs).

[0130] The apparatus and methods described in this application may be implemented, in part or in whole, by a special-purpose computer created by configuring a general-purpose computer to perform one or more specific functions embodied in a computer program. The functional blocks and flow chart elements described above serve as software specifications that can be converted into a computer program by the routine work of a skilled engineer or programmer.

[0131] A computer program includes processor-executable instructions stored on at least one non-transitory computer-readable medium. A computer program may also include or rely on stored data. A computer program may include a basic input / output system (BIOS) that interacts with the hardware of a special-purpose computer, device drivers that interact with particular devices of the special-purpose computer, one or more operating systems, user applications, background services, background applications, and the like.

[0132] A computer program may include (i) written text to be parsed, such as HTML (Hypertext Markup Language), XML (Extensible Markup Language), or JSON (JavaScript Object Notation), (ii) assembly code, (iii) object code generated from source code by a compiler, (iv) source code for execution by an interpreter, (v) source code for compilation and execution by a just-in-time compiler, etc. By way of example only, source code may be written using syntax from languages ​​including C, C++, C#, Objective-C, Swift, Haskell, Go, SQL, R, Lisp, Java®, Fortran, Julia, Perl, Pascal, Curl, OCaml, JavaScript®, HTML5 (Hypertext Markup Language Fifth Edition), Ada, ASP (Active Server Pages), PHP (PHP: Hypertext Preprocessor), Scala, Eiffel, Smalltalk, Erlang, Ruby, Flash®, Visual Basic®, Lua, MATLAB, SIMULINK, or Python®.

Claims

1. 1. A content production management system in a distributed studio environment, comprising: At least one computer readable medium; at least one command interface module configured to render at least one user interface to a content production entity set, wherein each content production entity from the content production entity set is associated with a content production volume from a content production volume set within the distributed studio environment; at least one command queue management module in communication with the command interface module, the command queue management module being configured to, upon execution of the software instructions: an operation of receiving a set of commands from the at least one command interface module, each command of the command set targeted to a target content production entity from the set of content production entities; assigning a synchronous execution time to each command of the command set; enqueueing each command of the command set according to its respective synchronous execution time into a command queue associated with the target content production entity, the command queue being stored on the computer-readable medium; transmitting, over a network, the set of commands or the queue of commands to the targeted content production entity; enabling the targeted content production entity to execute each command of the command set from the command queue according to its respective synchronized execution time; at least one command queue management module configured to execute Including, the system.

2. 2. The system of claim 1, wherein the at least one command queue management module is further configured to perform an operation to enable the targeted content production entity to execute each command of the command set by transmitting one or more enable signals to the targeted content production entity upon execution of software instructions.

3. The system of claim 1 or 2, wherein the synchronous execution times of at least one command of the command set are synchronized with respect to an absolute time.

4. The system of claim 1 or 2, wherein the synchronous execution time of at least one command of the set of commands is synchronized with respect to the synchronous execution time of another command of the set of commands.

5. The system of claim 1 or 2, wherein the synchronous execution times of the command sets in the command queue are globally synchronized.

6. at least two commands of the command set in the command queue are synchronized to at least two content creation volumes from the content creation volume set; The system of claim 1 , wherein the at least two content creation volumes are separate from one another.

7. The system of claim 6 , wherein the at least two content production volumes are geographically separated.

8. 8. The system of claim 6 or 7, wherein the at least two content creation volumes are separated by a latency of at least 5 ms.

9. The system of claim 1 , wherein the at least one command interface module is geographically separated from at least one content production volume of the set of content production volumes.

10. The system of claim 1 , wherein the at least one command interface module is separated from at least one content creation volume of the set of content creation volumes by a latency of at least 5 ms.

11. 2. The system of claim 1, wherein the targeted content production entity comprises at least one of a computing device, an LED wall, a camera, a sensor, a gimbal, a robot, a vehicle, a display module, a controlled lighting system, a controllable sound system, a movable wall, and a movable platform.

12. The system of claim 1 , wherein the set of content creation volumes includes at least one of an outdoor setting, a stage, an LED wall, and a street.

13. The system of claim 1 , wherein the command set includes at least one of network provision commands, virtualization commands, sensor commands, device commands, scripts, programs, human instructions, choreography commands, and previs commands.

14. 2. The system of claim 1, wherein the at least one command queue management module is further configured to perform the operation of instantiating the command queue within the at least one computer-readable medium upon execution of software instructions.

15. 2. The system of claim 1, wherein the command queue includes at least one of a countdown timer, a countup timer, an activation criterion, a permission, a link to a notary ledger, a link to an associated command queue, a link to a previously executed queue, and a link to a future queue.

16. The system of claim 1 , wherein the at least one command queue management module is further configured to perform an operation to determine a latency associated with the targeted content production entity upon execution of software instructions.

17. 17. The system of claim 16, wherein the at least one command queue management module is configured to perform an operation to assign the synchronous execution time to each command of the command set based on the determined latency upon execution of a software instruction.

18. 1. A computerized method for managing a command set in a content production management system in a distributed studio environment, the content production management system including a command interface module and a command queue management module in communication with the command interface module, the command interface module configured to render at least one user interface for a content production entity set, each content production entity from the content production entity set being associated with a content production volume from a content production volume set in the distributed studio environment; receiving a set of commands from the command interface module, each command of the command set targeted to a target content production entity of the set of content production entities; using the command queue management module to assign a synchronous execution time to each command in the command set; enqueuing, using the command queue management module, each command of the command set according to its respective synchronous execution time into a command queue associated with the target content production entity; transmitting the command set or the command queue from the command queue management module to the target content production entity over a network; enabling the targeted content production entity to execute each command of the command set from the command queue according to its respective synchronized execution time; A method comprising:

19. The method of claim 18 , wherein enabling the targeted content production entity to execute each command of the command set comprises transmitting one or more enable signals to the targeted content production entity.

20. 20. The method of claim 18 or 19, wherein the synchronous execution time of at least one command of the set of commands is synchronized relative to an absolute time or is synchronized relative to the synchronous execution time of another command of the set of commands.

21. at least two commands of the command set in the command queue are synchronized to at least two content creation volumes from the content creation volume set; The method of claim 18 , wherein the at least two content creation volumes are separate from one another.

22. The method of claim 21 , wherein the at least two content creation volumes are geographically separated.

23. 23. The method of claim 21 or 22, wherein the at least two content creation volumes are separated by a latency of at least 5 ms.

24. The method of claim 18 , wherein the command interface module is geographically separated from at least one content-creation volume of the set of content-creation volumes.

25. 20. The method of claim 18, wherein the command interface module is separated from at least one content creation volume of the set of content creation volumes by a latency of at least 5 ms.

26. 20. The method of claim 18, further comprising instantiating the command queue in at least one computer-readable medium.

27. The method of claim 18 , further comprising determining a latency associated with the targeted content creation entity.

28. 28. The method of claim 27, wherein assigning the synchronous execution time to each command of the command set comprises assigning the synchronous execution time based on the determined latency.

Images