Cloud-based remote video production platform

Abstract

A platform and process for creating and modifying video editing workflows uses an object-oriented, cloud-based system that simplifies the video editing workflow creation and editing process by providing users with definable input sets and definable job assignments associated with specific digital assets.

Description

[Technical field]

[0001] Related Applications This application claims the benefit of priority to U.S. Provisional Application No. 63 / 316,332, which is incorporated herein by reference in its entirety.

[0002] The present invention relates to a method and platform for creating and managing a video production workflow from concept to completion. [Background technology]

[0003] Multi-camera video productions require significant equipment and labor. Preparing for broadcast requires significant setup time and effort. Creating a video production often requires a truckload of equipment along with dedicated audio and video (A / V) specific hardware, along with a team of highly trained personnel on-site. By its nature, most A / V equipment is designed to be operated in-location, either at the site of a live event or in a dedicated production studio.

[0004] Economic and efficiency considerations (e.g., project budgets and shortages of skilled labor) have created an increasing need to facilitate remote collaboration between video crew members, in addition to remote operation away from the actual filming location. While limited remote control capabilities may exist in some traditional A / V equipment, most implementations are highly constrained in functionality and require a lot of relay equipment to accommodate remote use cases that the equipment was never intended for. These factors make remote production with an off-site crew complex, expensive, and difficult to achieve.

[0005] Video editing involves a complex series of steps and processes that begin with the identification of digital assets (i.e., digital files containing content to be incorporated into a video) and end with the final production version. This process involves moving digital assets from one device to another, and finally getting the video files to the person who will edit them. Assigning them to editors and then to projects, and finally tracking their progress and final approval, traditionally required one or more humans to create an ad-hoc list of statuses and assignments, which is prone to errors and does not lend itself well to providing timely status updates.

[0006] Pre-production, production, and editorial / post-production are all impacted by an increasing number of digital video assets while managing the entire process with a limited number of people. This increase in digital content creation creates a backlog of content that is difficult to manage and allocate to specific projects and people. Pre-production, production, and editorial / post-production are all impacted by an increasing number of digital video assets while managing the entire process with a limited number of people.

[0007] The industry has recently begun to adopt information technology (IT) for transmitting video, audio, and control signals over Internet Protocol (IP)-based infrastructure. In addition to being useful for remote production use cases, IT-based workflows have significant potential cost and efficiency advantages. However, widespread adoption has been hindered by the inherent limitations of traditional A / V equipment and production methods, as well as the technical knowledge required to comply with IP standards. Video technicians are accustomed to plugging A / V cables into specialized A / V devices to perform specialized A / V work. The transition to plugging IT devices into an IP network presents knowledge gaps and hurdles for these personnel.

[0008] Existing applications have many shortcomings that can impede users' ability to have timely and detailed awareness of project and file status. These applications are typically only able to apply media management functions, resulting in clumsy movement of digital assets from one system to the next to complete the video production workflow process. This results in multiple systems being required to support users' needs to ingest content from recording sources, through a series of file copies between separate systems, and finally obtain the final production version of those digital assets. This shortcoming exists within current video production workflows, allowing for discontinuity between file names, projects, and related data. What is needed is a system that applies changes in a cascading manner while preserving relationships. This is similar to the data quality concept in modern database management systems known as "referential integrity," where changes to data in one place are automatically reflected in other related records.

[0009] All of these shortcomings work together to increase the difficulty of creating and tracking assets, jobs, and status. Thus, there is a need in the field for new orchestration layer management capabilities to streamline the video production process, make it easier to use, and promote collaboration. There is a strong need for an ecosystem that is easily navigable by traditional A / V technicians while maximizing IT workflows. Summary of the Invention

[0010] The cloud-based video production platform of the present invention transforms the traditional video production model by converting video, audio, and control signals directly to IP protocols on the camera platform. An example of a suitable camera platform is described in International Publication No. WO2021 / 195641 (referred to as "AIRstation™"), the disclosure of which is incorporated herein by reference. Within the video production platform of the present invention (which may be referred to as the "AIRcloud™ system"), IP signals are connected to virtual computing resources via wired and / or wireless connections. This approach allows the entire video production ecosystem to be virtualized. Signals can be transported using a ubiquitous IP infrastructure, turning dedicated A / V hardware into on-demand virtual computing resources. This approach significantly reduces the on-site equipment footprint, the manpower required for installation, and creates opportunities for a distributed workforce. In this system, equipment can be operated locally, remotely, or a combination thereof, via a robust Internet connection.

[0011] To effectively deploy a video-over-IP ecosystem, a comprehensive and easy-to-use management system is required. The AIRcloud™ system of the present invention is an integrated tool for IP-based video production. It allows remote operation of A / V and IT resources, defining user roles, enabling permissions, provisioning cameras, computing resources, and storage, managing media, and managing accounts and payments. It is an ecosystem that encompasses the management and use of the entire A / V workflow of pre-production, production, and post-production in a cloud-based environment.

[0012] The system of the present invention uses software components that include multiple individual smaller software modules, a relational database, and network-based connections. User account management, user access rights, user groups, and job assignments are defined within the system and used to create and edit video production workflows. One of the key improvements provided by the platform of the present invention is that digital asset information, i.e., information about the content of a digital file and how it was created (which may include file metadata, camera and lens metadata, IMU and encoder metadata, and user metadata), is automatically captured as it is generated. The modularity of the system allows for the automatic routing of uploaded content to externally or internally connected services (e.g., transcription systems, or computer vision analysis for object identification) so that functions can be automatically performed on the digital asset before or after an editor makes changes to the digital asset. Digital assets are assigned to one or more users through the creation of a project job. During the post-production process, if one or more digital assets are identified that require an editor to make specific changes to the asset, the digital asset is assigned by the manager to a user-defined project job. This project job is further assigned to an editor. An editor may change the status of the project job when they begin the editing process and / or when the edits are complete, which submits the project job for review. A manager may then review the edits to the digital asset and take other actions (e.g., accept the work, change the status, and download the asset). If the work is not accepted as complete, the manager may provide new instructions or comments, change the status, and be notified when the editor has completed the revisions. This iterative process may continue across multiple revisions until the digital asset is accepted.

[0013] One of the key innovations of the AIRcloud™ system is its ability to automate traditionally manual tasks. In traditional broadcast workflows, most functions must be performed manually by a human operator. In contrast, with the AIRcloud™ system, many of these functions can be automated across pre-production, production, and post-production workflows.

[0014] In one aspect of the system of the present invention, a platform for managing a video production workflow includes an interface in communication with a network, the interface configured to input and send instructions to and receive information from a plurality of modules, each module in communication with the network, the modules including a management module configured to distribute instructions to and receive status information from other modules, an off-site camera control module, a remote editing module, a remote production module, a client review and collaboration module, a distribution / broadcast module, and at least one camera in communication with the network, the management module used by a project manager to assign jobs, access and distribute digital assets, and communicate with platform users. The platform users may include one or more of administrators, producers, technical directors, camera operators, sound engineers, graphics operators, and clients. In some embodiments, the instructions and information may be configured to trigger one or more automated actions in one or more of the plurality of modules and the at least one camera. The plurality of modules may further include a machine learning module configured for artificial intelligence (AI)-assisted asset identification and classification. The camera may be configured to be controlled via a remotely located camera operator.

[0015] In another aspect of the invention, a platform for managing a video production workflow includes an interface in communication with a network, the interface configured to input and send instructions to and receive information from a plurality of modules, each module in communication with the network, the module including a management module configured to provide instructions to and receive status information from the other modules, an off-site camera control module, a remote editing module, and at least one digital file source in communication with the network, the management module including a user interface (UI) used by a project manager to assign jobs, access and distribute digital assets, and communicate with platform users, the interface further configured to distribute completed video production to a content delivery network (CDN) in communication with the network, the network may include a combination of wired and wireless connections.

[0016] The platform users may include one or more of an administrator, a project manager, a producer, a technical director, a camera operator, an audio operator, an on-screen or voice-over talent, and a client. The instructions and information are configured to trigger one or more automated actions in one or more of the plurality of modules and the at least one digital file source. The at least one digital file source generates one or more digital files including one or a combination of an image, a sequence of images, and audio, and metadata describing the image, the sequence of images, or the audio. The metadata may include one or a combination of identifiers consisting of a name, a date, a time, a format, a client, a project, a subject, a location, a setting under which the one or more digital files were created, an editing history, editing permissions, and copy permissions. In some embodiments, the metadata may include one or more of a camera setting, a camera lens setting, a camera robot controller setting, a camera sensor reading, and an audio level.

[0017] In some embodiments, the plurality of modules includes a machine learning module configured for artificial intelligence (AI)-assisted asset identification and classification. The machine learning module may be configured to extract information from one or more digital files and perform analysis including one or more of object detection, keyword tagging, face detection, video segment identification, computer vision analysis of human performance, body mechanics, emotion perception, and voice analysis. In other embodiments, the machine learning module may be further configured to extract audio information from the one or more digital files and perform analysis of the audio content of the audio information for automatic speech-to-text transcription or to create keywords or commands for further action.

[0018] The plurality of modules further includes a remote production module configured to manage the broadcasting of the video production to the CDN. A client review and collaboration module configured to provide edited content to a client for review and acceptance or rejection of the edited content prior to delivery of the completed video production.

[0019] In some embodiments, the at least one digital file source is a camera configured to be controlled via a remotely located camera operator, the remotely located camera operator being in communication with the interface. In other embodiments, the at least one digital file source is an audio recorder, which may be integrated within the camera. The at least one digital file source may be a plurality of digital file sources, each associated with a unique code to establish a link to the predefined site layout. The unique code may be configured to be scanned by a mobile device of a platform user, whereby scanning the unique code causes the predefined site layout to be displayed on the mobile device, and the assigned location of the digital file source associated with the unique code is displayed within the predefined site layout.

[0020] In some embodiments, the management module may be further configured to allow a project manager to assign access restrictions to control each user's access to elements and digital assets within the platform as needed for the platform user's assigned job. The management module may be further configured to collect status information for the platform components and to create a Master Status Dashboard (MSD) to display real-time status and settings for at least one digital file source, communication, and one or more of performance data. In some implementations, the status and settings displayed on the MSD are one or more of camera settings, CPU status, connection status, stream health, available storage, recording status indicator, and uptime.

[0021] In some embodiments, the remote editing module may be configured to manage the workflow for the job, including editing, review, and finalization of the project, and the remote editing module is further configured to deliver the digital assets to a designated platform user when the job is assigned to the designated platform user.

[0022] In yet another aspect of the invention, a method for managing a video production workflow over a network includes providing an interface configured to input and send instructions to and receive information from a plurality of modules, each module in communication with the network, the modules including a management module configured to provide instructions to and receive status information from the other modules, an off-site camera control module, a remote editing module, and at least one digital file source in communication with the network, the management module providing a user interface (UI) including a user interface (UI) used by a project manager to assign jobs, access and distribute digital assets, and communicate with platform users, and delivering a completed video production to a content delivery network (CDN) in communication with the network. The network may include a combination of wired and wireless connections.

[0023] The platform users may include one or more of an administrator, a project manager, a producer, a technical director, a camera operator, an audio operator, and a client. The instructions and information are configured to trigger one or more automated actions in one or more of the plurality of modules and the at least one digital file source. The at least one digital file source creates one or more digital files including one or a combination of an image, a sequence of images, and audio, and metadata describing the image, sequence of images, or audio. The metadata may include one or a combination of identifiers: name, date, time, format, client, project, subject, location, settings under which the one or more digital files were created, editing history, editing permissions, and copy permissions. In some embodiments, the metadata may include one or more of camera settings, camera lens settings, camera robot controller settings, camera sensor readings, and audio levels.

[0024] In some embodiments, the plurality of modules includes a machine learning module configured for artificial intelligence (AI)-assisted asset identification and classification. The machine learning module may be configured to extract information from one or more digital files and perform analysis including one or more of object detection, keyword tagging, face detection, video segment identification, computer vision analysis of human performance, body mechanics, emotion perception, and voice analysis. In other embodiments, the machine learning module may be further configured to extract audio information from the one or more digital files and perform analysis of the audio content of the audio information to perform one or more of automatic speech-to-text transcription and generation of keywords or commands for further action.

[0025] The plurality of modules further includes a remote production module configured to manage the broadcasting of the video production to the CDN. A client review and collaboration module configured to provide edited content to a client for review and acceptance or rejection of the edited content prior to delivery of the completed video production.

[0026] In some embodiments, the at least one digital file source is a camera configured to be controlled via a remotely located camera operator, the remotely located camera operator being in communication with the interface. In other embodiments, the at least one digital file source is an audio recorder, which may be integrated within the camera. The at least one digital file source may be a plurality of digital file sources, each associated with a unique code to establish a link to the predefined site layout. The unique code may be configured to be scanned by a mobile device of a platform user, whereby scanning the unique code causes the predefined site layout to be displayed on the mobile device, and the assigned location of the digital file source associated with the unique code is displayed within the predefined site layout.

[0027] In some embodiments, the management module may be further configured to allow a project manager to assign access restrictions to control each user's access to elements and digital assets within the platform as needed for the platform user's assigned job. The management module may be further configured to collect status information for the platform components and to create a Master Status Dashboard (MSD) to display real-time status and settings for at least one digital file source, communication, and one or more of performance data. In some implementations, the status and settings displayed on the MSD are one or more of camera settings, CPU status, connection status, stream health, available storage, recording status indicator, and uptime.

[0028] In some embodiments, the remote editing module may be configured to manage the workflow for the job, including editing, review, and finalization of the project, and the remote editing module is further configured to deliver the digital assets to a designated platform user when the job is assigned to the designated platform user. [Brief description of the drawings]

[0029] [Figure 1A] FIG. 2 is a high-level block diagram of the basic modules of an embodiment of the platform AIRcloud™ system of the present invention. [Figure 1B] FIG. 1 is an alternative system diagram showing groupings of major operations within the platform. [Figure 2A] FIG. 2 is a diagrammatic view of a main dashboard in accordance with an embodiment of the system of the present invention. [Figure 2B] FIG. 1 is a high level flow diagram showing a typical sequence of operations for a post-production workflow, including digital asset upload, user assignment, job creation, editing and update processes, to create an efficient workflow using the system of the present invention. [Figure 3A] FIG. 1 illustrates a typical sequence for a photography setup using the system of the present invention. [Figure 3B] FIG. 1 illustrates a typical sequence for a photography setup using the system of the present invention. [Figure 3C] FIG. 1 illustrates a typical sequence for a photography setup using the system of the present invention. [Figure 3D] FIG. 1 illustrates a typical sequence for a photography setup using the system of the present invention. [Figure 3E] FIG. 1 illustrates a typical sequence for a photography setup using the system of the present invention. [Figure 4A] 11 is a sample screenshot of live video / audio routing using the routing functionality of the system of the present invention. [Figure 4B] 11 is a sample screenshot of live video / audio routing using the routing functionality of the system of the present invention. [Diagram 5] FIG. 1 is a diagrammatic view of a sample screenshot of a multi-view display with images received from assigned cameras. [Figure 6A] FIG. 13 is a diagram of a sample multiview feature. [Figure 6B] FIG. 13 is a diagram of a sample multiview feature. [Figure 6C] FIG. 13 is a diagram of a sample multiview feature. [Figure 7] FIG. 13 is a diagram of a sample program delivery feature according to an embodiment of the system of the present invention. [Figure 8] 11A-11C are diagrams of sample screenshots of a camera launchpad feature in accordance with an embodiment of the present system. [Figure 9] FIG. 1 is a diagram of an exemplary process for accessing camera information by scanning a QR code. [Figure 10] 11A-11C are diagrammatic views of sample screenshots of the edit setup feature in accordance with an embodiment of the inventive system. [Figure 11] 1 is a sample screenshot of a media asset management (MAM) feature according to an embodiment of the system of the present invention. [Figure 12] 1 is a sample screenshot of a Master Status Dashboard (MSD) according to an embodiment of the system of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0030] In the following detailed description, reference is made to the accompanying drawings, which provide example embodiments of the system of the present invention.

[0031] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items. As used herein, the singular forms "a," "an," and "the" are intended to include the plural as well as the singular, unless the context clearly indicates otherwise. As an example for clarity, when discussing an object, "one or more objects," "at least one object," or "multiple objects" are also included within the meaning of the term, unless the object is expressly described as a single object.

[0032] Definitions: Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which this invention belongs. Furthermore, it should be understood that the meaning of a term (e.g., as defined in a commonly used dictionary) should be interpreted as consistent with its meaning in the context of the art and this disclosure, and should not be interpreted in an ideal or overly formal sense, unless expressly so defined herein. For the sake of further clarity, but not as a limitation, further definitions are provided for the following terms.

[0033] "User" means a person acting in one or more roles on a Project, which may include project managers, administrators, producers, technical directors, camera operators, sound operators, and support personnel who may be permitted to interact with the Platform.

[0034] "Role" refers to a job assignment for a project and may include, for example, graphics operator, replay operator, client, field crew, editor, in addition to the users mentioned above.

[0035] "Project" means an event or series of events including the capture and production of video and / or audio. "Project" may also include post-production and editing.

[0036] "Project Manager" means one or more persons responsible for the planning and execution of a project, including managers, producers and technical directors.

[0037] "Instance" means a computer, machine, or equipment. For example, an "instance" may be a virtual machine in the cloud, a machine in a remote location away from the set, or a machine on-site with a camera.

[0038] "Asset" means any person, camera, equipment, computer instance, or digital file associated with a Project or User.

[0039] "Digital file source" means a device that is the source of an image, video, and / or audio file (collectively "digital file"), as well as their associated data (e.g., the elements and features of the digital file and metadata describing how it was created). For purposes of this description, "digital file source" includes not only traditional cameras for direct image and / or audio detection / collection (e.g., live footage), but also storage devices or other sources of digital files created separately in time or as part of an independent effort using an independent camera and stored on a storage device, or computer-generated digital files where no digital file or only a portion of the digital file was created using a traditional camera. For example, an existing animation may be a digital file.

[0040] "Authorization" means permission / access to a Protected Asset.

[0041] "Online" means assets that are available for immediate use.

[0042] "Offline" means an asset that is not available for immediate use.

[0043] "Remote" in the context of production and editing means work that takes place away from the event location and / or away from traditional workplaces.

[0044] "Broadcast" refers to the simultaneous transmission of an AV signal to one or more recipients by wired or wireless means, including, without limitation, over-the-air and live stream transmissions.

[0045] In describing the present invention, it should be understood that several techniques and steps are disclosed. Each of these has its own advantages, and each can also be used in conjunction with one or more, or possibly all, of the other disclosed techniques. Thus, for the sake of clarity, this description refrains from unnecessarily repeating all possible combinations of the individual steps. Nevertheless, the specification and claims should be read with the understanding that such combinations are anticipated and therefore within the scope of the present invention and claims.

[0046] FIG. 1A illustrates a high-level block diagram of the basic elements of an embodiment of the video production platform 100 of the present invention. The heart of the platform is the AIRcloud™ interface 110, which provides a single interface to multiple cloud services. The interface 110 provides communication for remote pre-production, production, and post-production, content management, secure storage / archive / retrieval transcoding packaging for distribution, machine learning control assistance, and analytics. The interface 110 communicates bidirectionally with each of the following platform modules: camera(s) 102 (which are typically (but not necessarily) multiple cameras and may include robotic controls), off-site camera control 104, integrated workflow management 106 that may be accessed by an administrator ("admin") control device 105 (e.g., a laptop, tablet, mobile device, desktop computer, workstation, or other computing device), client review and collaboration 108, remote editing and post-production 112, remote production 116, distribution / broadcast 120, where the crew controls multiple aspects of the broadcast from an off-site location, and machine learning analytics and assistance 122. The functionality and details of the various modules are described below.

[0047] FIG. 1B shows a higher level diagram illustrating the grouping of major operations managed by the AIRcloud™ interface of the present invention. The interface 110 provides management of modules to facilitate and execute several major operation groups in the overall video production sequence that ends with a send-off for distribution to a content delivery network (CDN). The "setup / wrap" operation group 130 includes planning, crew assignment, transportation, shoot setup and tear-down, i.e., pre-production. The operation group 130 primarily involves interfacing with the management module 105 for pre-production activities. The "production" operation group 140 includes camera operation, switching, replay, graphics, audio, and communications. The interface 110 manages modules 102, 104, 108, 112, and 116, and then the hand-off to the CDN for the distribution operation 150. The interface 110 provides real-time diagnostic visibility across the entire production chain.

[0048] Access to the platform portal begins with a landing page of login username and password. The login process may be secured using one or more known authentication methods. Examples of authentication methods include Single Sign-On (SSO), Security Assertion Markup Language (SAML), Authoce, and OpenID. Other methods are known to those skilled in the art. This access may be accomplished by an administrator controller 105 or any other communication device assigned or associated with the user. In some projects, users may be pre-deployed and already assigned credentials to log on to the system. For users who may not yet have an account, a "New User Registration" button links to an account creation screen where various information is entered (e.g., without limitation, name, company, email address, nickname, phone number, hometown, etc.). Users may upload an image to be used as their avatar / icon. If no image is provided, a two-letter initial may be substituted or the user may be assigned a user number. The final step is to provide a login password. After an approved login, the user is directed to the platform dashboard, as shown diagrammatically in FIG. 2A.

[0049] The dashboard 202 is the starting point and central hub for users to navigate the various features of the platform portal, sometimes referred to as the "AIRcloud™ Portal." Once a user has been assigned a project role, one or more associated project icons 204 are displayed on the dashboard. The user can then access project details, contact other team members, and any features accessible via permissions and roles. Permissions are automatically generated based on the user's defined role within the project, or can be customized by an administrator or producer.

[0050] Project functionality may include, without limitation, multi-view interface selection, program feed display, camera control, virtual workstation control, shoot setup configuration, audio / video routing, media asset management, post production setup and configuration, communications setup and configuration, and detailed project overview.

[0051] In pre-production, to create a new project, an administrator or producer selects "New Project" by clicking "New Project" 206 on the dashboard. An icon image can then be uploaded if appropriate and various information about the project can be entered, such as, without limitation, project name, client, project ID, shoot date(s), and location. Examples of these are shown as project boxes 210. All projects that a particular user is involved in are displayed on the dashboard. Selecting a project icon 204 on the dashboard 202 opens a navigation menu 220 for projects. Each of the buttons 221-230 in the menu 220 leads the user to a different module or function within the overall system (each of which is described in more detail below).

[0052] The "Shoot Setup" button 226 on menu 220 launches the Shoot Setup screen, FIG. 3A, which is used by the manager / producer or technical director ("TD") to assign roles and access privileges to users. Pre-production for multi-camera projects is typically a laborious, manual process. The logistics of getting equipment and personnel to the site and positioning them on-site is a significant undertaking. The difficulty and complexity are amplified when crew members are working remotely from off-site, creating security issues.

[0053] The Film Setup feature is a core component of the AIRcloud™ system's provisioning and authorization automation, where cameras, computer instances, connected devices and personnel are securely provisioned and assigned to a project.

[0054] Using the Shot Setup screen 300 shown in Figures 3A-3E, a producer (or administrator) uploads a background image or site plan for the project. A camera icon is selected from a list of available resources and placed in its appropriate location. An operator from the available crew list can then be attached to the camera(s) for which the operator is responsible. Compute instances and other equipment for switchers, playback, sound, graphics, etc. can be provisioned and crew assigned. On-camera talent, on-site technicians, clients, and other roles can also be assigned in the Shot Setup interface. New users can be invited to join the project by using the "Add" (+) feature. Once the invitee creates their user profile, they are added to the list of available personnel assets for the project. Cameras, compute instances, and other connected equipment can be added to the available equipment selection list by using the "Add" (+) feature. "Placeholders" for any assets can also be provisioned and later replaced.

[0055] 3A-3E show examples of film setups. After logging in and selecting a project, the "Film Setup" button 226 is selected in the navigation menu 220. In FIG. 3A, the "Conference Room" site plan is selected from the list of available site plans from the "Site Plans" menu 302. Selecting this site plan displays a diagram 304 of a conference room with a stage / platform area and audience seating. The diagram may be generic or custom programmed for a specific location. In FIG. 3B, the AIRstation™ camera #3524 is selected from the list of available equipment in the equipment menu 308 and placed in a predetermined location on the site plan shown as camera 310. Once positioned, the camera can be rotated to indicate orientation and a production number may be associated with the camera. Positioning can be easily accomplished by a drag-and-drop action, i.e., the manager / producer uses a finger (on a touch screen) or a stylus, mouse, trackball, or similar interface to select the desired equipment and drag it onto the desired location in the site plan. When a camera is selected, a switcher instance (Live Switching Instance 01 on the Equipment menu) can be automatically provisioned. In FIG. 3C, "CAM03" is the name assigned to the camera 310 for the production at that time. "User02" (322) is then selected from the list of available crew members in the Crew Menu 320 and connects to "CAM03." The act of assigning a crew member to a job position also activates permissions that allow access to information and other actions necessary to perform the assigned work.

[0056] Referring to FIG. 3D, by selecting from the crew menu 320 and using the same drag and drop action, “User 26” (324) is assigned as the technical director and “User 22” (326) is assigned as the on-site technician. A similar process is followed for the rest of the equipment and crew by simply dragging and dropping the equipment and camera operators to the desired positions within the site layout. In the completed setup example shown in FIG. 3E, five cameras 311-315 selected from the equipment menu 308 are positioned around the auditorium stage and a user is assigned to each camera. “User 34” is shown as the director / producer and may be physically located on-site or off-site. Additional functions and features that may be provided within the crew menu include communication buttons for each crew member. For example, next to each crew member's identifier are one or more buttons for instant voice communication, messaging, and / or emailing that person. This allows for quick communication between crew members so that quick adjustments can be made as needed. Another feature that may be displayed within the filming setup screen is the camera status. Color-coded or other visually distinctive indicators may be provided for each camera and each crew member. For example, a green dot next to a camera or crew member means that they / they are online and available for selection, a yellow dot indicates that they are offline and available for selection. A red dot may indicate that the camera or crew member's registration has expired or is otherwise unavailable for selection. Other types of status indicators may be used, as would be apparent to one of ordinary skill in the art.

[0057] A powerful feature of the AIR system of the present invention is the automation that is triggered as a result of assets being provisioned and assigned. A single manual act can trigger multiple automated actions. In one example, referring to FIG. 4A, once a camera platform icon is assigned to a location in a site plan, a switcher instance can be automatically provisioned and audio and video signals from the camera are automatically routed to a specific network input port (TCP / IP) on the switcher instance. In this example, camera #1 (402) is routed at router 406 to network input port: 3861 (404) of the switcher instance.

[0058] The results of the automatic routing process for multiple assets can be viewed by selecting the "Routing" button 221 in the navigation menu 220. This displays the Live Video / Audio Routing screen (an example is shown in FIG. 4B) with details about the cameras assigned to the project. In the example shown, five cameras are displayed with information about each camera, including the destination(s) to which the camera output is sent. When cameras are assigned on the Shot Setup screen, a multi-view source is automatically created with the same name, as shown in FIG. 5. An origin source is automatically created in the AIR Hub (in the Integrated Workflow Management 106). When a switcher instance is created on the Shot Setup screen, the video switcher program output is automatically created as a source in the AIR Hub, and a multi-view source is automatically created from the video switcher output.

[0059] At this point in the workflow, a camera operator (user) is assigned to camera #1 and permissions for control of that camera are automatically configured. When the camera operator logs into the system and selects a camera for control, the IP address and appropriate permissions and settings for the camera platform are automatically entered into the AIR mobile application.

[0060] For other users, their permissions will be automatically populated as well: for example, a technical director who controls a cloud switching instance will automatically be assigned permission for remote control of that instance when they are assigned the technical director role for a project, and will also be automatically assigned permission to route any audio / video signals related to the project.

[0061] This automation removes multiple manual steps in the setup process, saving time and reducing the chance of human error. Communications and Notifications: If necessary, notifications can be automatically sent to the appropriate crew members when modifications are made to any aspect of the project. Similarly, with a communications system, permissions can be automatically provisioned for users and users are routed to the appropriate groups for real-time communications once they are assigned project roles.

[0062] "Multiview" refers to multiple camera video feeds being displayed within one given space or screen. Multiview is common in broadcast production trucks or broadcast stage control rooms. Traditionally, each video feed is sent to an individual monitor rather than displaying multiple video feeds aggregated on a single screen. Hardware (e.g. Blackmagic Design video switchers) offers an aggregated view on a single output as a standard feature. However, the multiview feeds are often not available to technicians outside the control room space.

[0063] The ability to view feeds from multiple cameras simultaneously in real time is critical to the production process. Producers and directors need this capability to select the correct shot(s) for the program feed, set proper camera angles, prepare replays, display graphics, and for other purposes. Other project technicians can benefit from viewing these real-time feeds, but this is rarely available due to the expense and complexity of sending multiple feeds to multiple destinations. Camera operators are sometimes given a "program return" video feed that shows the master program output, usually with graphics superimposed on the camera operator's own camera, which the camera operator uses to appropriately frame the shot to accommodate the placement of the graphics. Camera operators often cannot see what the other cameras in a multi-camera setup are capturing in real time. The availability of such a feature would help them adjust framing, see events outside of the direct camera view, and anticipate framing needs. This is especially true when the camera operator is not directly behind the camera or is off-site.

[0064] With the inventive approach, when entering production, a user can select the "Multiview" button 230 in the navigation menu 220 (FIG. 2A) to open the Multiview feature, where the user can select from pre-configured Multiviews or available program video sources and condense them onto a single screen in the "Multiview" window. An example of a Multiview window is shown diagrammatically in FIGS. 6A-6B. In one embodiment, the Multiview is delivered over Ethernet via a simple web browser window. The user can customize which video feeds are displayed in each subsection or "drop zone" and view the configuration in a full screen view. Referring to the Multiview Setup screen 600 shown in FIG. 6A, video sources 601-605 (from "CAM01"-"CAM05") can be dragged and dropped into a reduced display pattern. For example, display pattern options 4-up, 6-up, and 9-up are displayed and available for the user to select. These pattern options may be set as defaults or customized options may be set through the Setup feature. In the figure, a 6-up display 610 has been selected and a feed from the primary switcher output is used for the program feed 612. After the source has been selected, clicking the Display button 620 on the multiview setup screen opens the "6-up" display shown in Figure 6B with footage from the selected source. Figure 6C shows an example of an actual 6-up multiview stream for a football game.

[0065] The "Program Feed" button 229 on the navigation menu 220 (FIG. 2A) opens a new window with a video stream. This feed can be sourced from the program output of a video switcher or from an end content delivery network (CDN). This feed can be used for multiple purposes including a "reliability return" for technical monitoring of the viewer broadcast quality. As shown in FIG. 7, the program feed corresponds to the video displayed in FIG. 6B as program feed 612.

[0066] Establishing remote control of a camera platform can be complex. An uninterrupted control connection must be maintained and latency kept to an absolute minimum. Interruption of control signals can be catastrophic to video quality. Serial control connections are limited in their practical range. Conversely, IP-based protocols are not constrained by distance, but face challenges in traversing NAT (Network Address Translation) implementations and firewalls. Network administrator privileges at the host location are typically required for port forwarding. Most organizations do not allow port forwarding because it exposes their network to security risks. These challenges make establishing a camera platform control connection difficult.

[0067] The camera control feature of the inventive system avoids these problems with a pre-configured, easy-to-use access link for remote camera control devices. Selecting the "Camera Launch Pad" button 228 on the navigation menu 220 (FIG. 2A) opens a zero-configuration link between the remote control camera platform and the operator (which in this example uses the WebRTC real-time video and signaling protocol). WebRTC uses a feature known as ICE (Interactive Connection Establishment) to enable direct connections to peers. ICE achieves this using STUN (Session Traversal Utilities for NATs) and / or TURN (Traversal Using Relays Around NATs) servers. STUN is a protocol that allows discovery of the public IP addresses of peers and determines whether there are any restrictions that prevent peer-to-peer connectivity. If there are restrictions that prevent direct peer-to-peer connectivity, the connection is relayed through a server that uses the TURN protocol. Relaying incurs additional latency and is less desirable than STUN, since there is at least one additional connection point or "hop". For this reason, the use of TURN is a secondary (fallback) method for remote connectivity. In some embodiments, the remote connectivity of the system of the present invention may be established using a Virtual Private Network (VPN). A VPN system is fundamentally different from WebRTC, but operates similarly in that the connection is automatically upgraded to a peer-to-peer connection. If peer-to-peer connectivity is not possible, an intermediary (relay) server may be used between the AIRstation™ and the mobile app. Although two methods of remote connectivity are referenced, the approach of the present invention is flexible enough to work with other new and existing remote connectivity methods that allow for NAT traversal.

[0068] With the Camera Launchpad feature, this complex NAT traversal technology is transparent to users, who can access remote camera platforms by following three simple steps.

[0069] The camera operator logs into the portal as described above, selects the assigned project, and then selects the camera to use on the Launchpad screen.

[0070] Referring to Figure 8, on the camera control screen, a site map of the project is displayed with icons for the equipment and personnel involved in the production. This is a read-only version of the shot setup screen with a location overview, providing vital information for camera operators who are not physically on-site with the shooting scenario.

[0071] The Launchpad interface indicates which cameras are authorized for user control by combining indicators for online / offline status for the camera platform with visual differentiation. The visual differentiation can be based on color coding alone or in combination with opacity, brightness, or patterns (e.g., dashed lines). For example, units available and authorized for user control may be displayed as icons with 100% opacity, while units not authorized for this particular user may be displayed with reduced (e.g., 50%) opacity. A green dot may be used to indicate that the system is authorized and connected to the main control device. A yellow dot may indicate that the asset (camera) is authorized but not connected. A gray dot may indicate that the camera is already under control by another user. A red dot may mean that the system is not authorized for external control, and a blue dot may be used to indicate that the system is connected but not assigned. A user can submit a request for permission to control a camera that he or she is not yet authorized for, and the permission request can be automatically forwarded to an administrator for approval. This feature eliminates the need to manually enter IP addresses and access codes, reducing the opportunity for human error and providing quick, unrestricted access to the camera control hardware.

[0072] Clicking or double-clicking on the unit launches an application for operating that camera. In some embodiments, this application is described in International Publication No. WO2021 / 195641. It is used with the AIRstation™ robotic camera system. To control the camera platform (AIRstation™ or other), the user selects an authorized camera icon for control. The mobile device's browser then launches the AIR camera control mobile application via direct communication between the web browser and the mobile operating system. This may be accomplished by a sophisticated process known as "deep linking," in which an "intent URI" (uniform resource identifier) ​​is used to securely launch the mobile app. URIs are known to those skilled in the art to identify logical or physical resources used by web technologies. Credentials for control are automatically provisioned and access is pre-approved.

[0073] In the example shown in Figure 8, the green dot associated with CAM03 (labeled "Green Dot") indicates that CAM03 is online and available for selection by User02. When User02 (also associated with the green dot) selects CAM03, the AIR control mobile application is launched and credentials are automatically transferred allowing control of the platform. Other users and cameras involved in the exemplary project are labeled according to the color coding scheme described above, providing information to users viewing the Launchpad interface to facilitate teamwork when executing projects.

[0074] For the system to function effectively, the camera platform must be placed as per the site plan. To assist the on-site crew in placing the camera platform in the correct location, the system may use QR codes, bar codes, BLE (Bluetooth Low Energy), radio frequency ID (RFID) tags, or other identification schemes that can be physically or electronically associated with the equipment to provide a link to view a predefined site plan. For example, in the case of a system implementation utilizing a camera robot equipped with an AIRstation™, a QR code displayed on the back of the unit is scanned to show the technician exactly where the camera platform should be placed on the site. For example, in FIG. 9, scanning the QR code 902 displays a diagram 904 with the pre-planned location of the camera 906 in the site plan, shown here within a highlighted circle 908 for clarity. Further information may also be displayed, such as setup information including the specified lens type and settings, camera height, along with appropriate networking parameters and assigned permissions. Camera identification information may also be displayed. For equipment not pre-assigned, a simple one-click interface allows the question "where does this go?" to be directed to the appropriate administrator. In other embodiments, the cameras (AIRstation™ or other camera systems) may have an associated QR code, or similar code, which the on-site technician can scan to bring up a camera connection screen that displays the desired camera assignments and placement. This information can then be used by an off-site user to provide instructions to the on-site technician for the desired positioning of the cameras.

[0075] Video production often requires routing video and audio signals to more than one destination. Routing of audio and video signals has traditionally been handled by dedicated A / V specific hardware. See, for example, U.S. Patent No. 9,191,721 (Holladay, et al.), the disclosure of which is incorporated herein by reference. Companies such as Haivision (Montreal, QC, Canada) have improved on this concept with products such as Haivision Hub™, a hardware / software product that allows data streams in its SRT (Secure Reliable Transport) format to be routed to various destinations through a simple interface. A drawback of Haivision Hub™ is that it is only designed to ingest streams transported using the SRT protocol.

[0076] Selecting the "Routing" button 221 in the navigation menu 220 takes the user to the AIRhub™, a matrix router for live video and audio assets, an example of which is shown in Figure 4B. This AIRcloud™ A / V router (designated 456a...456x) uses a simple drag-to-connect interface to ingress and egress A / V feeds in multiple formats and protocols (SRT, RIST, RTMP, etc.). The AIRhub™ also allows for the routing of video and audio feeds to switcher instances, replay instances, digital object storage devices, and / or other endpoints.

[0077] To change or add additional sources and destinations, the user adds an endpoint (451) from the available destinations (450), places the endpoint in the routing area (452), and then drags a connector (453) from the source to the destination. Additional destination endpoints can be configured and added to the available list by selecting the "Add Destination" feature (454). For switcher outputs, video and audio are similarly routed by drag and connect. Using this feature, camera feeds can be routed into switcher instances, replay instances, graphics programs, program outputs and / or live camera feeds can be delivered to a CDN, camera feeds can be routed to digital storage endpoints, etc.

[0078] The routing scheme of the AIRcloud™ system of the present invention also records these sources and destinations as metadata that can be recalled at a later date for media management, digital rights management, and other purposes.

[0079] Selecting the "Virtual Workstation" button 227 in the AIRcloud™ portal navigation menu 220 (FIG. 2A) launches functionality for controlling virtual workstations. Users with appropriate permissions can launch a remote desktop interface to control cloud-based or off-site compute instances. These instances may run computer-based video and audio switching and mixing software, animation software, editing software, live graphics contribution, video playback / slow motion, etc.

[0080] Examples of suitable computer-based video switching and instant replay software that may be used with the system of the present invention include vMix™ (StudioCoast Pty Ltd., Robina, Queensland, AU), OBS (Open Broadcaster Software, OBS Project, open source software distributed under the General Public License, v2.), VizVector (Vizrt), and SimplyLive Replay (Reidel Communications). Examples of editing software include Adobe® Premiere® and Apple® Final Cut Pro®.

[0081] Our AIRcloud™ system enables more complex and high performance routing and switching because it is not constrained by traditional video hardware limitations of inputs, outputs and layers. The system can scale exponentially on demand by daisy-chaining compute instances together. Additionally, it is not constrained by geography.

[0082] The ability to communicate with all team members simultaneously in real time is critical to the production process. Dedicated intercom systems exist from multiple manufacturers for this purpose (Clear-Com, RTS, etc.), but they are tied to proprietary hardware, limited in range, and complex or impossible to deploy to crew members working remotely. For remote access users, maintaining secure communications can also be a challenge.

[0083] Selection of the "Communication" button 224 in the navigation menu 220 (FIG. 2A) launches a communications ("Communications") feature that allows authorized users to communicate with other team members in real time, via voice and text, on any internet-connected device. In one embodiment, full-duplex audio may be included to allow intermittent voice communication with the rest of the production team. The communications feature is automatically provisioned when crew members are assigned to project roles and is integrated with the overall project setup for easy, all-in-one deployment. The communications feature facilitates pre-prep and shoot day communications. Groups can be created, for example, a camera operator on-site for a shoot can set up a group and quickly link all members. In some embodiments, the communications feature can be compatible with standard communications operating systems (e.g., Apple® iOS, Android®, Mac®, Microsoft® Windows®, and other common communications formats).

[0084] Once principal photography is complete, many media projects must undergo a post-production process. This can include assembling an entire program from scratch using the camera's original footage and / or working from a live-switched program master to modify and fine-tune selections made in real time. This process can also include extracting highlight clips from the program feed and / or from individual camera recordings, adding graphics and animation, modifying program audio, repurposing captured content, as well as many other use cases.

[0085] The editing process typically involves aggregating video, audio, and graphics assets in media storage devices connected to the editing workstation. When multiple editors are working on a project, these assets and editing system configurations must be replicated across all workstations. This is often a manual process that is time-consuming, laborious, and potentially error-prone. A significant portion of an editor's effort is spent manually collecting assets, manually categorizing them, manually managing media, manually uploading and reviewing edits, and then manually interpreting and organizing feedback from reviewers to make changes. Versioning of assets can also be an issue.

[0086] Products such as EditShare™ (Watertown, MA) offer products that improve certain aspects of the editing process by allowing media to be shared over a local area network (LAN) or, in some cases, replicated over a wide area network (WAN). Frame.io (Adobe® Inc., New York, NY) is one example of a source of video review and collaboration platform that offers improved media management as well as media ingest processes through its C2C (camera-to-cloud) product. The main drawback of these products is that they still require significant manual intervention, as well as manual verification that files have been securely transferred. They also require workflows to adhere to a replication model to distribute and consume media to multiple endpoints.

[0087] The post-processing capabilities of the AIRcloud™ system of the present invention eliminate the need to replicate media by virtualizing the entire post-production process. Instead of sending media to various computer storage endpoints for use, the AIRcloud™ system brings computing power and editors to where the media resides. By centralizing the media in cloud-based storage directly connected to the cloud compute instance, any editor logging into the virtual compute instance(s) can access the same assets without the need for replication. Consolidating the back-end process reduces the need to replicate media and helps streamline version control. It also ensures that the editing system configuration and media assets are the same across all editors and contributors to the editing process.

[0088] The AIRstation™ camera platform's integration with AIRcloud™ storage enables media uploads from the field to a cloud storage instance. It also provides bit-level file integrity validation to ensure the entire file is uploaded and that it is an exact copy of the original. It also supports asynchronous uploads, allowing you to start a file upload, pause it, and continue uploading at a later time.

[0089] Selecting the "Post Setup" button 223 in the navigation menu 220 (FIG. 2A) launches an edit setup screen 1001, a sample of which is shown in FIG. 10. For workflow management, the edit process can be provisioned with instances, crews, and permissions using methods similar to those used in live production. Project media folders from the shoot are automatically allowed to be read and / or written by the editor. Further media folders can be automatically created for internal review and client approval. Virtual edit instances can be created and assigned to editors with access to the video, audio, animation, graphics, data, and text assets required to complete the project. In the example shown in FIG. 10, an editor 1002 has been assigned video and associated data stored in two media folders, a first folder 1004 designated as read only and a second folder 1006 designated as read / write (no deletion). As shown on the screen, the editor 1002 is an active user. On the right side of the screen is a menu 1020 of people (users) who have participated in the project. Below the menu 1020 are buttons that the producer may use to provision an edit instance or to add or connect additional media storage (1024) for use by the editor 1002. These project participants may be automatically populated in this menu with permissions enabled at the start of the project, or may be dragged, dropped, or removed according to the producer's preferences. The producer may select users as needed and discuss actions or explanations of their contributions to the project using the various communication features available to select phone, text, or email communication. Reviewers may be auto-created when an editor job is assigned. These users are the authorized reviewers for project change notes and approvals.When edits are ready to be reviewed, they can be made available for selection and viewing by users associated with the review activity. Completed edits can be created automatically, notifying reviewers when content is available for feedback and approval. This unified system also enables automation of the review and approval process.

[0090] FIG. 2B shows a typical post-production process flow, where in step 250, the project manager 105 inputs information identifying more editors and managers to be involved in the project. This allows the identified workers to be provided with assets. This step may be used to modify the original input. In step 251, the manager creates assignments or "jobs" for the people identified in step 250. One or more jobs may be identified, with the same or different teams assigned to each job. The digital assets created for the project are uploaded to the cloud (step 252) for access by the assigned users, and in step 253, the uploaded assets are assigned to the jobs created in step 251. In step 254, editor(s) are assigned to the job(s) and have access to the digital assets corresponding to the jobs. The editors make the assignment to access the digital assets and edit the digital assets corresponding to the assigned jobs (step 255), and when complete, send a status update to the workflow management module 106 to create a notification that the job is ready for review. In step 257, the manager or administrator reviews the edited content to determine whether it is complete. At this point, a notification that the edited content is ready for review may be sent for client review via the client review and collaboration module 108. If the manager, administrator, or client determines that the edited content is unacceptable, it is returned to the editor (step 255) and a notification is sent to the editor informing them that more work is required. If the edited content is deemed complete by the reviewer, the manager / administrator makes an entry via the interface 110 to update the status as "completed" (step 258). In step 259, the completed asset is downloaded for its intended "performance", i.e., broadcast or display.

[0091] Once an edited version of the project is complete, the editor 1002 can designate the file as "ready for review." A text-based notification can be automatically generated for the reviewer. The reviewer (internal 1010 and client 1012) can then log into the AIRcloud™ system of the present invention where they are directed to the file for edit review. The file playback interface allows for time-marked feedback. Timestamped notation is important for clarity. Time-marked feedback is available to all reviewers as well as approved project personnel. With indexed, time-stamped, collated feedback viewable by all contributors and the project editor(s), the review process is greatly streamlined.

[0092] The aggregation, classification, storage, and retrieval of media assets are some of the more labor-intensive parts of the post-production process. These are traditionally manual tasks performed by a combination of producers, editors, and assistant editors. In recent years, improvements in machine learning algorithms have allowed for some automation of image classification, but how to trigger the process remains cumbersome.

[0093] The AIRcloud™ Media Asset Management ("MAM") system enhances existing post-production methods by automating asset classification, movement, storage, and retrieval. This functionality can be selected using the "Media" button 222 in the navigation menu 220. Traditionally fragmented processes are centralized and standardized with a unified process that enables rigorous classification, tracking, and automated movement of media assets.

[0094] The AIRcloud™ MAM feature allows for searching of assets by name, date, format, client, project or other identifiers, tagging of assets with keywords and / or further metadata, manual and automatic movement of assets, automatic correction of permissions, copying of assets to editing systems, and moving of assets to archives or storage. From the moment an asset enters the system, its movement through routing, switching, storage, and editing can be tracked by this feature. This comprehensive tracking of assets throughout their entire lifecycle adds an unprecedented level of detail to asset metadata.

[0095] FIG. 11 shows a sample screenshot of the MAM feature for a film festival. Video assets may be selected by clicking on various thumbnails. MAM screen 1100 includes a details pane 1104 that shows information about the selected image, keywords that can be used to search for the image, notes describing the image, project name and client, and other information that may be useful in identifying or selecting video or audio files associated with the project. A search box 1102 allows for entry of search parameters. Assets may be moved and stored in one or more folders 1106 by dragging and dropping image thumbnails into the folders, as indicated by arrow 1110.

[0096] Artificial intelligence (AI)-assisted asset identification and classification can be automated for assets ingested into the system to extract useful information from video and audio. The AIRcloud™ system's approach greatly simplifies the use of automated classification tools that use machine learning and artificial intelligence for functions (e.g., object detection, keyword tagging, face detection, video segment identification, as well as computer vision analysis of human performance, body mechanics, emotion perception, etc.). Keywords and metadata collected from these AI tools are important tools for finding, organizing, and protecting media assets. Further AI-assisted features may employ advanced audio analysis and annotation. The AIRcloud™ system of the present invention may also include automatic voice-to-text transcription by integrating a third-party transcription application. This feature can further streamline the process of uploading video / audio files and downloading the results. In addition, the voice detection feature may be used to trigger other commands in the system. For example, a mention of a particular product or person may trigger instructions to insert an image of the mentioned product or person into the video.

[0097] Online storage, near-line backup, and long term offline archive storage are connected to the system so that automated backup and archiving of assets can be programmed upon project completion.

[0098] MAM functionality also integrates with approval and review processes, storing all project change notes and associating them with assets. When an asset or project is marked as "approved" by a reviewer(s), it can automatically trigger various actions required to complete the asset and / or project for production, distribution, and archiving. Asset and project indexing allows easy retrieval of complete assets that have been archived offline by issuing a "Restore Project" command.

[0099] The media asset management (MAM) functionality within the AIRcloud™ system of the present invention can be integrated with third-party asset management systems, such as Frame.io (AdobeR), catDV™ (Quantum Corporation), etc., via an API.

[0100] Existing third-party asset management systems (e.g., Frame.io) have similar MAM capabilities to the AIRcloud™ system, but are poorly integrated into the overall production / post-production workflow and lack the provisioning pieces, cloud instance routing, storage interconnection, automated permissions, etc.

[0101] Metadata, such as, without limitation, pan / tilt / roll positioning from motor encoders, speed and position data from the camera platform inertial measurement unit (IMU), lens data for focus and aperture, zoom motor data for lens focal length, camera metadata (e.g., ISO, F-stop, recording format, codec, bit rate, frame rate, white balance, color profile, shutter angle, exposure value offset, RGB information, autofocus settings, audio levels), user pan / tilt gesture information, user zoom / focus / aperture information, switcher video and audio sources, and graphics selection information, can all be captured in real time and used immediately in combination with the MAM functionality or the metadata can be leveraged for other post-production processes. These and other camera features and capabilities are described in the aforementioned International Publication No. WO2021 / 195641, which is incorporated herein by reference. This information can be used as a dataset to improve system performance, train autonomous systems, and automate production processes.

[0102] The status of all connected AIRstations™ as well as all deployed virtual compute instances can be displayed on a Master Status Dashboard (“MSD”) screen. FIG. 12 shows a sample screenshot of the MSD 1200. This dashboard displays all camera settings, CPU status, connection state, stream health 1208 (bandwidth, jitter, packet loss), crew communications, and other system performance data in real-time or near real-time. Examples of camera settings that may be displayed on the MSD include recording format 1210, codec 1212, white balance 1216, ISO 1214, F-stop 1218, geographic location 1220, etc., as shown. Further examples of status that may be reported on the MSD include memory, CPU, available storage, recording status indicators, and uptime. The utility of the screen MSD is that at a glance, the settings and status of multiple platform components can be immediately seen and various settings can be confirmed or adjusted. The Master Status Dashboard can also be accessed by technicians anywhere in the production chain, allowing unprecedented performance visibility across the entire broadcast workflow. This addresses the common problem of content delivery technicians being unable to diagnose video impairments that occur on the contributing side of the pipeline.

[0103] The present disclosure is to be considered as an exemplification of the invention and is not intended to limit the invention to the specific embodiments illustrated in the drawings or by the following description.

Claims

1. 1. A platform for managing a video production workflow, comprising: an interface for communicating with a network, the interface configured to input and transmit instructions to a plurality of modules and to receive information from a plurality of modules, each module in communication with the network, the modules: a management module configured to provide instructions to and receive status information from other modules; an off-site camera control module; A remote editing module; at least one digital file source in communication with said network; 1. A platform wherein the management module includes a user interface (UI) used by a project manager to assign jobs, access and distribute one or more digital files, and communicate with platform users, the interface further configured to deliver completed video productions to a content delivery network (CDN) in communication with the network.

2. The platform of claim 1 , wherein the platform users include one or more of an administrator, the project manager, a producer, a director, a technical director, a camera operator, a sound operator, an on-screen or voice-over talent, and a client.

3. The platform of claim 1 , wherein the instructions and information are configured to trigger one or more automated actions in one or more of the plurality of modules and the at least one digital file source.

4. 2. The platform of claim 1, wherein the one or more digital files include one or a combination of an image, a series of images, and audio, as well as metadata describing one or more of the images, the series of images, the audio, and information regarding the creation of the one or more digital files.

5. 5. The platform of claim 4, wherein the metadata includes one or a combination of identifiers consisting of name, date, time, format, client, project, subject, location, setting in which the one or more digital files were created, editing history, editing permissions, and copy permissions.

6. The platform of claim 4 , wherein the metadata includes one or more of camera settings, camera lens settings, camera robot controller settings, camera sensor readings, and audio levels.

7. 10. The platform of claim 1, wherein the plurality of modules further comprises a machine learning module configured for artificial intelligence (AI) assisted asset identification and classification.

8. 8. The platform of claim 7, wherein the machine learning module is configured to extract information from the one or more digital files and perform analysis including one or more of object detection, keyword tagging, face detection, video segment identification, computer vision analysis of human performance, body mechanics, emotion perception, and voice analysis.

9. 8. The platform of claim 7, wherein the machine learning module is further configured to extract audio information from the one or more digital files and perform an analysis of a phonetic content of the audio information to perform one or more of: automatic speech-to-text transcription and generation of keywords or commands for further action.

10. The platform of claim 1 , wherein the plurality of modules further comprises a remote production module configured to manage the broadcasting of the video production to the CDN.

11. 2. The platform of claim 1, wherein the plurality of modules further comprises a client review and collaboration module configured to provide edited content to a client for review and acceptance or rejection of the edited content prior to delivery of the completed video production.

12. The platform of claim 1 , wherein the at least one digital file source is a camera configured to be controlled via a remotely located camera operator, the remotely located camera operator in communication with the interface.

13. The platform of claim 1 , wherein the at least one digital file source is an audio recorder.

14. The platform of claim 1 , wherein the audio recorder is integrated into a camera.

15. 2. The platform of claim 1, wherein the management module is further configured to enable the project manager to assign access restrictions to control each platform user's access to elements within the platform and the one or more digital files as needed for the platform user's assigned job.

16. The platform of claim 1 , wherein the network includes a combination of wired and wireless connections.

17. 2. The platform of claim 1, wherein the at least one digital file source comprises a plurality of digital file sources, each digital file source being associated with a unique code to establish a link to a given site plan.

18. 20. The platform of claim 17, wherein the unique code is configured to be scanned by a mobile device of a platform user, and scanning the unique code causes the predetermined site map to be displayed on the mobile device, and an assigned location of the digital file source associated with the unique code is displayed within the predetermined site map.

19. 2. The platform of claim 1, wherein the management module is further configured to collect status information for platform components and create a master status dashboard (MSD) to display real-time status and settings for one or more of the at least one digital file source, communications, and performance data.

20. 20. The platform of claim 19, wherein the status and settings displayed on the MSD include one or more of camera settings, CPU status, connection status, stream health, available storage, recording status indicator, and uptime.

21. 2. The platform of claim 1, wherein the remote editing module is configured to manage a workflow for a job including editing, reviewing, and finalizing the project, the remote editing module being further configured to deliver digital assets to a designated platform user when a job is assigned to the designated platform user.

22. 1. A method for managing a video production workflow over a network, comprising: providing an interface configured to input and transmit instructions to and receive information from a plurality of modules, each module in communication with the network, the modules comprising: a management module configured to provide instructions to and receive status information from other modules; an off-site camera control module; A remote editing module; at least one digital file source in communication with said network; providing a user interface (UI), the management module including a user interface (UI) used by a project manager to assign jobs, access and distribute one or more digital files, and communicate with platform users; and delivering the completed video production to a content delivery network (CDN) in communication with said network.

23. 23. The method of claim 22, wherein the platform users include one or more of an administrator, the project manager, a producer, a technical director, a camera operator, a sound operator, and a client.

24. 23. The method of claim 22, wherein the instructions and information are configured to trigger one or more automated actions in one or more of the plurality of modules and the at least one digital file source.

25. 23. The method of claim 22, wherein the one or more digital files include one or a combination of an image, a sequence of images, audio, and metadata describing the image, the sequence of images, or the audio.

26. 26. The method of claim 25, wherein the metadata includes one or a combination of identifiers consisting of name, date, time, format, client, project, subject, location, setting in which the one or more digital files were created, editing history, editing permissions, and copy permissions.

27. 26. The method of claim 25, wherein the metadata includes one or more of camera settings, camera lens settings, camera robot controller settings, camera sensor readings, and audio levels.

28. 22. The method of claim 21, wherein the plurality of modules further comprises a machine learning module configured for artificial intelligence (AI) assisted asset identification and classification.

29. 30. The method of claim 28, wherein the machine learning module is configured to extract information from the one or more digital files and perform analysis including one or more of object detection, keyword tagging, face detection, video segment identification, computer vision analysis of human performance, body mechanics, emotion perception, and audio analysis.

30. 30. The method of claim 28, wherein the machine learning module is further configured to extract audio information from the digital file and perform an analysis of a phonetic content of the audio information to perform one or more of automatic speech-to-text transcription and generation of keywords or commands for further action.

31. 23. The method of claim 22, wherein the plurality of modules further comprises a remote production module configured to manage the broadcasting of the video production to the CDN.

32. 23. The method of claim 22, wherein the plurality of modules further comprises a client review and collaboration module configured to provide edited content to a client for review and acceptance or rejection of the edited content prior to delivery of the completed video production.

33. 23. The method of claim 22, wherein the at least one digital file source is a camera configured for control via a remotely located camera operator, the remotely located camera operator in communication with the interface.

34. 23. The method of claim 22, wherein the at least one digital file source is an audio recorder.

35. The method of claim 22, wherein the audio recorder is integrated into a camera.

36. 23. The method of claim 22, wherein the administration module is further configured to enable the project manager to assign access restrictions to control each platform user's access to elements within the platform and the digital assets as needed for the platform user's assigned job.

37. 23. The method of claim 22, wherein the network includes a combination of wired and wireless connections.

38. 23. The method of claim 22, wherein the at least one digital file source includes a plurality of digital file sources, each digital file source being associated with a unique code to establish a link to a given site plan.

39. 40. The method of claim 38, wherein the unique code is configured to be scanned by a mobile device of a platform user, and scanning the unique code causes the predetermined site layout to be displayed on the mobile device, and an assigned location of the digital file source associated with the unique code is displayed within the predetermined site layout.

40. 23. The method of claim 22, wherein the management module is further configured to collect status information for platform components and create a master status dashboard (MSD) to display real-time status and settings for one or more of the at least one digital file source, communications, and performance data.

41. 41. The method of claim 40, wherein the status and settings displayed on the MSD include one or more of camera settings, CPU status, connection status, stream health, available storage, recording status indicator, and uptime.

42. 23. The method of claim 22, wherein the remote editing module is configured to manage a workflow for a job including editing, reviewing, and finalizing the project, and the remote editing module is further configured to deliver digital assets to a designated platform user when a job is assigned to the designated platform user.

Images