Internet enabled method for automation, advanced control, monitoring, and management of cameras

Abstract

A comprehensive system and method designed to enhance the management, control, and monitoring of pan-tilt-zoom (PTZ) and static cameras, along with other production elements is provided. The present invention includes scale management for efficient large-scale operations, precise PTZ control through digital pixel bounding box conversion, and automated camera positioning using machine learning models with remote processing. Additionally, the present invention facilitates onsite PTZ movement emulation via joystick, remote monitoring and management through a centralized web-based portal, and extends system capabilities through a third-party API on a cloud-based platform. This integrated approach significantly improves automation, flexibility, and control in video production environments, thereby boosting overall quality and efficiency.

Description

[0001]Internet Enabled Method for Automation, Advanced Control, Monitoring, and Management of Cameras FIELD OF THE INVENTION The present invention relates generally to the field of digital video production technology. More specifically, the present invention relates to a system and method for controlling, monitoring, and managing pan-tilt-zoom (PTZ) and static cameras, as well as other production elements, utilizing innovative techniques and cloud-based services to optimize functionality and enhance operational efficiency. BACKGROUND OF THE INVENTION There is a demand for a system and method that can control, monitor, and manage pan-tilt-zoom (PTZ) and static cameras, as well as other production elements. In the field of digital video production, the ability to effectively control, monitor, and manage camera systems and production elements is crucial for achieving high- quality outputs. Traditional systems often rely on manual control and local processing, which can limit flexibility and efficiency, especially in complex or dynamic filming environments. The advent of pan-tilt-zoom (PTZ) cameras has introduced a higher degree of maneuverability in capturing video, allowing operators to remotely adjust the camera's pan, tilt, and zoom functions. However, managing these cameras, along with static cameras and other production elements, often requires substantial hardware and can be labor-intensive and error-prone when handled manually. Furthermore, the increasing demand for live broadcasting and the need for rapid setup and deployment of video production infrastructures call for more sophisticated, automated solutions. These solutions must not only provide comprehensive control over the cameras and associated production elements but also ensure high reliability and low latency in real-time applications. Thus, there is a need to develop systems and methods for controlling, monitoring, and managing pan-tilt-zoom (PTZ) and static cameras. The present invention is intended to address problems associated with and / or otherwise improve on conventional system and method through an innovative system and method incorporating other problem-solving features. SUMMARY This summary is provided to introduce a selection of concepts in a simplified form, that are further described below in the Detailed Description. This summary is not intended to identify key features or essential features of the claimed subject matter. Nor is this summary intended to be used to limit the claimed subject matter’s scope. The present invention provides a system for controlling and managing PTZ and static cameras using cloud-based technology. In one embodiment, the present invention provides a method 1 comprising: providing a universal interface for interacting with and automating production elements from multiple vendors; classifying base drivers into media egress drivers and control drivers; defining a set of functionalities in a standardized API that can be accessed and / or modified by a Platform API; translating commands from the base control driver into a protocol supported by a device using a Protocol Driver; translating media from the base media driver into a media format supported by the Platform Media Ingress; centralizing the management of protocol definitions in a cloud service and a universal schema; simplifying the process for users by utilizing a cloud-based Device Registry, structured as a database, containing driver configurations and commands, options, and details for each camera, with information sent from the cloud as structured data. The present invention disclose six additional methods which, in certain embodiments, may be integrated with other techniques. BRIEF DESCRIPTION OF THE DRAWINGS FIG.1 is an illustration of an online platform consistent with various embodiments of the present disclosure. FIG.2 is a block diagram of a system in accordance with some embodiments. FIG.3 is an illustration showing anatomy of a PTZ device driver. FIG.4 is an illustration of generalized structure of drivers. FIG.5 is an illustration of one embodiment showing a method for providing an automatic control of an array of Pan-Tilt-Zoom (PTZ) and static cameras using machine learning models. FIG.6 is an illustration of one embodiment showing a method for providing a third-party API on top of a cloud-based camera management platform. FIG.7 is a block diagram of a computing device for implementing the methods disclosed herein, in accordance with some embodiments. DETAIL DESCRIPTIONS OF THE INVENTION All illustrations of the drawings are for the purpose of describing selected versions of the present invention and are not intended to limit the scope of the present invention. As a preliminary matter, it will readily be understood by one having ordinary skill in the relevant art that the present disclosure has broad utility and application. As should be understood, any embodiment may incorporate only one or a plurality of the above- disclosed aspects of the disclosure and may further incorporate only one or a plurality of the above-disclosed features. Furthermore, any embodiment discussed and identified as being “preferred” is considered to be part of a best mode contemplated for carrying out the embodiments of the present disclosure. Other embodiments also may be discussed for additional illustrative purposes in providing a full and enabling disclosure. Moreover, many embodiments, such as adaptations, variations, modifications, and equivalent arrangements, will be implicitly disclosed by the embodiments described herein and fall within the scope of the present disclosure. Accordingly, while embodiments are described herein in detail in relation to one or more embodiments, it is to be understood that this disclosure is illustrative and exemplary of the present disclosure, and are made merely for the purposes of providing a full and enabling disclosure. The detailed disclosure herein of one or more embodiments is not intended, nor is to be construed, to limit the scope of patent protection afforded in any claim of a patent issuing here from, which scope is to be defined by the claims and the equivalents thereof. It is not intended that the scope of patent protection be defined by reading into any claim limitation found herein and / or issuing here from that does not explicitly appear in the claim itself. Thus, for example, any sequence(s) and / or temporal order of steps of various processes or methods that are described herein are illustrative and not restrictive. Accordingly, it should be understood that, although steps of various processes or methods may be shown and described as being in a sequence or temporal order, the steps of any such processes or methods are not limited to being carried out in any particular sequence or order, absent an indication otherwise. Indeed, the steps in such processes or methods generally may be carried out in various different sequences and orders while still falling within the scope of the present disclosure. Accordingly, it is intended that the scope of patent protection is to be defined by the issued claim(s) rather than the description set forth herein. Additionally, it is important to note that each term used herein refers to that which an ordinary artisan would understand such term to mean based on the contextual use of such term herein. To the extent that the meaning of a term used herein—as understood by the ordinary artisan based on the contextual use of such term—differs in any way from any particular dictionary definition of such term, it is intended that the meaning of the term as understood by the ordinary artisan should prevail. Furthermore, it is important to note that, as used herein, “a” and “an” each generally denotes “at least one,” but does not exclude a plurality unless the contextual use dictates otherwise. When used herein to join a list of items, “or” denotes “at least one of the items,” but does not exclude a plurality of items of the list. Finally, when used herein to join a list of items, “and” denotes “all of the items of the list.” The following detailed description refers to the accompanying drawings. Wherever possible, the same reference numbers are used in the drawings and the following description to refer to the same or similar elements. While many embodiments of the disclosure may be described, modifications, adaptations, and other implementations are possible. For example, substitutions, additions, or modifications may be made to the elements illustrated in the drawings, and the methods described herein may be modified by substituting, reordering, or adding stages to the disclosed methods. Accordingly, the following detailed description does not limit the disclosure. Instead, the proper scope of the disclosure is defined by the claims found herein and / or issuing here from. The present disclosure contains headers. It should be understood that these headers are used as references and are not to be construed as limiting upon the subjected matter disclosed under the header. The present disclosure includes many aspects and features. Moreover, while many aspects and features relate to, and are described in the context of methods and systems for a system and method for controlling, monitoring, and managing pan-tilt-zoom (PTZ) and static cameras, embodiments of the present disclosure are not limited to use only in this context. In general, the method disclosed herein may be performed by one or more computing devices. For example, in some embodiments, the method may be performed by a server computer in communication with one or more client devices over a communication network such as, for example, the Internet. In some other embodiments, the method may be performed by one or more of at least one server computer, at least one client device, at least one network device, at least one sensor, and at least one actuator. Examples of the one or more client devices and / or the server computer may include, a desktop computer, a laptop computer, a tablet computer, a personal digital assistant, a portable electronic device, a wearable computer, a smartphone, an Internet of Things (IoT) device, a smart electrical appliance, a video game console, a rack server, a super- computer, a mainframe computer, mini-computer, micro-computer, a storage server, an application server (e.g. a mail server, a web server, a real-time communication server, an FTP server, a virtual server, a proxy server, a DNS server, etc.), a quantum computer, and so on. Further, one or more client devices and / or the server computer may be configured for executing a software application such as, for example, but not limited to, an operating system (e.g. Windows, Mac OS, Unix, Linux, Android, etc.) in order to provide a user interface (e.g. GUI, touch-screen based interface, voice based interface, gesture based interface, etc.) for use by the one or more users and / or a network interface for communicating with other devices over a communication network. Accordingly, the server computer may include a processing device configured for performing data processing tasks such as, for example, but not limited to, analyzing, identifying, determining, generating, transforming, calculating, computing, compressing, decompressing, encrypting, decrypting, scrambling, splitting, merging, interpolating, extrapolating, redacting, anonymizing, encoding and decoding. Further, the server computer may include a communication device configured for communicating with one or more external devices. The one or more external devices may include, for example, but are not limited to, a client device, a third party database, a public database, a private database, and so on. Further, the communication device may be configured for communicating with the one or more external devices over one or more communication channels. Further, the one or more communication channels may include a wireless communication channel and / or a wired communication channel. Accordingly, the communication device may be configured for performing one or more of transmitting and receiving of information in electronic form. Further, the server computer may include a storage device configured for performing data storage and / or data retrieval operations. In general, the storage device may be configured for providing reliable storage of digital information. Accordingly, in some embodiments, the storage device may be based on technologies such as, but not limited to, data compression, data backup, data redundancy, deduplication, error correction, data finger-printing, role based access control, and so on. Further, one or more steps of the method disclosed herein may be initiated, maintained, controlled, and / or terminated based on a control input received from one or more devices operated by one or more users such as, for example, but not limited to, an end user, an admin, a creator, fans thereof. Further, the user as defined herein may refer to a human, an animal, or an artificially intelligent being in any state of existence, unless stated otherwise, elsewhere in the present disclosure. FIG.1 is an illustration of an online platform 100 consistent with various embodiments of the present disclosure. By way of non-limiting example, the online platform 100 may be hosted on a centralized server 102, such as, for example, a cloud computing service. The centralized server 102 may communicate with other network entities, such as, for example, a mobile device 106 (such as a smartphone, a laptop, a tablet computer, etc.), other electronic devices 110 (such as desktop computers, server computers, etc.), databases 114, and sensors 116 over a communication network 104, such as, but not limited to, the Internet. Further, the databases 114 may be associated with government agencies. Further, users of the online platform 100 may include relevant parties such as, but not limited to, end-users, administrators, service providers, service consumers, and so on. Accordingly, in some instances, electronic devices operated by the one or more relevant parties may be in communication with the platform. A user 112, such as the one or more relevant parties, may access online platform 100 through a web based software application or browser. The web based software application may be embodied as, for example, but not be limited to, a website, a web application, a desktop application, and a mobile application compatible with a computing device 2600. FIG.2 is a block diagram of a system 200 in accordance with some embodiments. Accordingly, the system 200 may include a communication device 202, a processing device 204, and a storage device 206. Further, the communication device 202 may be configured for the present invention which may include at least one user device in some embodiments. Further, the at least one user device may include a smartphone, a tablet, a desktop, a laptop. The present invention, hereinafter referred to as “Internet Enabled Method for Automation, Advanced Control, Monitoring, and Management of Cameras,” pertains to a comprehensive system and method for controlling, monitoring, and managing pan-tilt- zoom (PTZ) and static cameras, as well as other production elements, through innovative techniques and integration with cloud-based services. 1.0 A Method for managing PTZ Cameras, Static Cameras, and other production elements at Scale: The video production process often involves significant fragmentation when dealing with hardware elements such as cameras, microphones, switchers, mixers, and lighting. In a typical production environment, dozens of elements from multiple vendors may be utilized. The objective of this invention is to create a universal interface for interacting with and automating all aspects of the production process, necessitating a well-thought- out approach to drivers. 1.1 Base Drivers: Every device has one or more base drivers corresponding to the device type. Base drivers are classified into two categories: media egress drivers and control drivers. Media egress drivers convert media, such as audio or video, into a standardized format compatible with the Platform API. Control drivers enable devices to be manipulated through the Platform API, such as moving a PTZ Camera or configuring lighting color. The base drivers may include: - [Media Egress] Video Source - Enables the device to be used as a video source within the platform, allowing the platform to manipulate image settings such as exposure, contrast, and white balance. - [Control] PTZ Device - Allows the platform to manipulate the position of a PTZ device, such as a PTZ Camera or a PTZ Gimbal. - [Media Egress] Audio Source - Enables the device to be used as an audio source within the platform, allowing the platform to manipulate audio settings such as gain. - [Control] Lighting Device - Allows the platform to manipulate lighting levels and color of a lighting device. - [Control] Robotic Lighting Device - Enables the platform to manipulate the positioning of a robotic light. - [Control] Robotic Crane Device - Allows the platform to manipulate the positioning of a robotic crane, typically used to hoist a Video Source or an Audio Source. - [Control] Robotic Slide Device - Enables the platform to manipulate the positioning of a robotic slide device, generally used to position and move a Video Source. - [Control] Cable Camera - Allows the platform to manipulate the positioning of a robotic cable camera. Base drivers define a set of functionalities in a standardized API that can be accessed and / or modified by the Platform API. Using the platform API, a universal interface can be created for managing all these devices (outlined in section 6.0). When a device is added, it may inherit multiple base drivers depending on its capabilities. For example, a PTZ Camera is both a Video Source (due to the camera element) and a PTZ Device (due to the PTZ element). A Crane Camera may inherit three base drivers: a Video Source, a PTZ Device (to point the camera mounted on the crane), and a Robotic Crane Device (to position the crane itself). A device may also inherit multiple base drivers of the same type to control hardware of the same type. For instance, a slide camera might be 2-Axis. A horizontal robotic slide allows the camera to move left and right. Attached to the horizontal slide may be a vertical slide that enables the camera to move up and down. Attached to the vertical slide, there may be mounted a PTZ Camera. A device of this type would inherit four base drivers: 1 Video Source, 1 PTZ Device, and 2 Robotic Slide Devices (Horizontal and Vertical). The device registry (outlined in section 1.4) defines which base driver a device inherits. 1.2 Protocol Drivers: Base drivers provide a standardized API for the Platform API to communicate with devices. Protocol Drivers translate commands from the base driver into a protocol supported by the device, or translate commands or media from the device into a format that the base driver understands. Protocol support depends on the device type. For example, with PTZ Camera Control, the two primary protocols are VISCA and ONVIF. When the base driver receives a command from the platform API to move the device to the left, the base driver sends that command to the Protocol Driver. If the camera uses the VISCA protocol, the Protocol Driver translates the command into a VISCA RELATIVE_MOVE_LEFT command. Once the camera has finished moving, it may report its new position. The Protocol Driver receives the status update command from the device or inquires about its current status and translates the response into a format that the Base Driver understands. This applies to any base driver type or classification. For example, lighting may be controlled using the MIDI protocol. The base lighting driver may receive a command from the platform API to turn on a stage light. The base driver passes this information to the protocol driver, which translates the information into the MIDI protocol and sends the command to the device. This also applies to media egress. A camera may use various protocols to transmit its video and audio. The base driver expects the media to be in a specific format before sending the media stream to the Platform API. The Protocol Driver connects to the device using the configured protocol (e.g., RTSP, RTP, RTMP, SRT, NDI, USB, PCI, etc.) and standardizes the media for consumption by the base driver. 1.3 Cloud Protocol Definitions: Various control protocols have different variations and versions, with some vendors having custom commands within existing protocols or entirely custom protocols. To maximize hardware support, it is often necessary to customize the protocol driver for each brand and model of the device. These customizations are referred to as Protocol Definitions. To manage this at scale, centralizing the management of these protocol definitions in a cloud service that can be accessed and updated at any time becomes necessary. Protocol Definitions are written in a common schema, such as Protobuf or JSON. They can be used to configure the protocol driver, enabling or disabling certain features or overriding the command payload and structure for one or multiple commands. This approach allows the Protocol Drivers to remain flexible and minimizes the amount of device-specific information required within the driver itself. As protocol definitions are maintained at a cloud level, they can be updated on the fly in real time without needing to perform a software update for the driver itself. An example of a protocol definition is as follows: { "visca":{ "config":{ "extended_headers":true, "expect_return_response":false, "min_command_interval":250 }, "has_tally_light":true }, "TALLY_LIGHT_ON":"01047C 01", "TALLY_LIGHT_OFF":"01047C 00" } } } this example, we configure the VISCA Protocol Driver to enable tally light functionality (which is non-standard in VISCA), specify that the camera uses extended headers (a variant of VISCA commands), and set the minimum interval between commands to the camera to be 250ms (which varies depending on the brand and model). Additionally, we override the VISCA Commands for turning on and off the tally lights. Since there is no standard command in VISCA to enable or disable tally lights, this needs to be explicitly given. This approach also works in instances where two manufacturers implement the same feature but have different commands for it. 1.4 Device Registry: Considering the complexity of device drivers and the various components that connect together to create a single standardized API for communicating with all devices of the same class, it is essential to simplify the process for users. The Device Registry serves this purpose. It is a cloud-based repository of Device Brands and Models along with their respective driver configurations. If a device is registered in the device registry, a user only needs to select the brand and model for their device to be set up automatically. The Base Driver(s), Protocol Driver(s), and Protocol Definitions are configured using the information in the registry without the need to download additional software. Additionally, the Device Registry provides essential information about the device, such as calibration parameters (defined in Section 2). 1.5 Agent Bridge and Embedded Device Bridges: Due to the nature of most devices, accessing a device typically requires software to be locally running on-site. The driver pipeline runs within the Agent Bridge application. The agent bridge can be executed on a computer on-site, a dedicated hardware piece, or be embedded within the device itself. The agent bridge includes the entirety of the driver pipeline, from the base driver to the protocol driver and the vendor-specific overrides. 1.6 Diagrams: The FIG.3 and FIG.4 outlines the different components of drivers and how they interact with each other. It is important to note that the first diagram depicts the structure for a PTZ Device Driver. However, this structure can also be applied to other types of devices while following the same base structure, as outlined in the Generalized Structure of Drivers diagram. In FIG.3, the Platform API 301 interacts with the PTZ Device Base Driver 302, which provides the standard API for the Platform API’s use. The Base Driver communicates with the PTZ Device Protocol Driver 303, which ingests a protocol definition 304 from the Cloud Device Registry and applies Vendor Specific Overrides 305. This adapted protocol driver then communicates with the Camera 306. From the Camera’s perspective, it sends out Audio and Video. The Audio Protocol Driver 307 and Video Protocol Drivers 308 ingest this media and standardize it so that it can be ingested by the Base Audio 309 and Base Video Drivers 310. The media then goes through an on- site processing pipeline for manipulation before being sent to the Cloud Media Ingress 311, which allows the media to be accessed from the Platform API 301. In FIG.4, the same process is outlined, but not limited to PTZ Cameras; it applies to any supported device type. The Platform API 301 communicates with the Base Driver 401 of the device type, and the Base Driver 401 communicates with the Protocol Driver 402, which can ingest cloud protocol definitions 304. The adapted driver (Vendor specific Driver overrides) then communicates directly with the device (e.g., camera) 306. If the device 306 has the ability to egress media, that media is sent to the Protocol Egress Driver 403, where the content is standardized before being sent to the Base Egress Driver 404. Eventually, the media reaches (via On site Media Manipulation 405) the Cloud Media Ingress 406 to be made accessible through the Platform API 301. The method 1 can be provided with the diagram and the components described above. In preferred embodiment, the method 1 is provided, wherein the method 1 comprises steps of: 1. providing a universal interface for interacting with and automating production elements from multiple vendors; 2. classifying base drivers; 3. defining a set of functionalities in a standardized API that can be accessed and / or modified by a Platform API; 4. translating commands from the base control driver into a protocol supported by a device using a Protocol Driver; 5. translating media from the base media driver into a media format supported by the Platform Media Ingress; 6. centralizing the management of protocol definitions in a cloud service and a universal schema; and 7. simplifying the process for users by utilizing a cloud-based Device Registry, In some embodiments, the base drivers can be classified into media egress drivers and control drivers. In some embodiments, the cloud-based Device Registry can be structured as a database, containing driver configurations and commands, options, and details for each camera, with information sent from the cloud as structured data. 2.0 A method of controlling PTZ Cameras Whereby a digital pixel bounding box is converted to PTZ Values: Fast Frame is a feature that allows the user to quickly identify a specific frame and get there quickly. This means that the user can select within the current frame of the camera his next shot and decide when to transition to it. To achieve this, the camera is provided with the ability to map rectangular coordinates (x, y) in the two-dimensional space of the image to spherical coordinates (angles) in the movement space with 3 degrees of freedom of PTZ cameras (pan, tilt, zoom). To provide the cameras with this coordinate conversion capacity, an automated calibration process is necessary in which the following information is used. ● Manufacturer's hardware specifications. ● Data obtained through movements executed in the camera. ● Data obtained through video processing using computer vision. 2.1 Calibration process: In general, the calibration process is divided into 2 parts: ● Pan Tilt Calibration ○ The focal length of the camera is calculated using computer vision. ○ The coordinate conversion equation is deduced. ● Zoom Calibration ○ Data is collected using computer vision. ○ The equation that models the behavior of the zoom is deduced. 2.1.1 Pan Tilt Calibration 2.1.1.1 Calculation of the focal length: To calculate the focal length there are two options that are selected depending on the environment where the camera is located; checkerboard as a calibration tool and search for features and patterns in images by moving the camera to find deviations between them. The objective is to find the focal length in the x and y axes through a matrix like the one shown in M1. M1. Focal length matrix.Mapping of rectangular coordinates to spherical coordinates: Obtaining the focal distance, we proceed to find the equations that map the coordinates of the two-dimensional space on the image to the space of movement of the camera's axes. 2.1.1.2 Rotations and transformations: Due to the fact that during the operation the camera will not always start from the reference frame of the origin, a system of rotations and transformations is implemented that allow the conversion to be transferred from any other position to the origin, making it possible to handle the same system of equations for the conversion of coordinates. 2.1.2 Zoom Calibration: For zoom calibration, concurrent image processing is used executing movements in the camera (specifically zoom) after finding patterns that allow it to be tracked consistently during the execution of camera movements. The objective of image processing and data collection is to find a function that models the distance changes between the same points in the sequence (Eq 1). Eq 1. Proportion as a a sequence of images. In one embodiment, the method 2 can be provided, wherein the method 2 may comprise steps of: 1. selecting a specific frame within a current frame of a camera and calculating first positions the camera needs to go to in order to reach the specific frame; 2. selecting a specific position within the current frame of a camera and calculating second positions the camera needs to go to in order to center the specific position; 3. mapping rectangular coordinates in a two-dimensional space of an image to spherical coordinates in a movement space of one or more PTZ cameras; 4. performing an automated calibration process using manufacturer's hardware specifications, data obtained through camera movements, and data obtained through video processing; 5. calculating a focal length of the camera using lens calibration, involving moving the PTZ camera in small increments and comparing the distance between key points to model lens distortion; and 6. implementing a system of rotations and transformations to handle coordinate conversion from any position to an original position. In some embodiments, the first positions are Pan / Tilt / Zoom positions. In some embodiments, the second positions are Pan / Tilt / Zoom positions. 3.0 A method of automatically positioning PTZ Cameras through the use of Machine Learning models, whereby the processing is done on a remote server over the internet: Auto Tracking technology allows the user to select a subject and automate its tracking achieving cinematic motion without the need for human intervention. Some advantages of automatic tracking are the following: • Efficiency: Auto-tracking eliminates the need for human intervention in tracking a subject, which saves time and allows operators to do more with less. • Shot Quality Auto-tracking allows for smooth and cinematic camera movements, which can be typically difficult to do using a PTZ camera. Additionally, our technology in particular provides the following benefits: ● Real time tracking with ultra low latency. ● It is based on cloud processing and consequently: ○ It gives any PTZ camera the ability to auto-track. ○ The user does not need specialized hardware to use the tracking. It is possible to run the service from any computer that can correctly view the video from a PTZ camera. At a high level, the automatic tracking service is structured through 3 main parts: 1. An administration service for automatic tracking agents. 2. 2. A main or core algorithm that performs all the necessary video processing in the remote instances (auto tracking agent). 3. 3. A local tracking driver that interprets the auto tracking agent's information and executes control actions on the PTZ cameras accordingly. 3.1 Orchestration Service: The orchestration service is the process that manages auto-tracking instances in the cloud. These instances are the actual servers running the auto-tracking service in the cloud, which are started on demand according to the use that is given to the automatic tracking by the users. The controllers are dynamically started and allocated on hardware-accelerated remote instances via the server orchestration API and workflows. Depending on the number of cameras connected via the local agent, the control room will request the same number of workers / controllers to auto-track on those cameras. 3.2 Cloud Auto Tracking Service: It is the process or set of processes running in the cloud to track subjects. For this to be possible, this service must interact with other services in charge of specific tasks. The general process is described below: • A local agent sends the video of a connected camera using a pipeline specially designed to handle low latency. • The video enters a video streaming service using WebRTC. • The video is converted by another service from WebRTC to TCP / IP making it possible to process it later by the auto tracking service. • The auto tracking service processes the video and extracts the location information of the subjects or subject according to its status. • The location information is sent to the auto tracking driver in the local agent using NATs. • The auto tracking driver interprets the information and interacts within the local agent with the PTZ controller and the camera control driver. • The camera controller driver sends the corresponding movement commands to the camera. 3.2.1 Controller: An Auto Tracking Controller (also called Controller) is a process that runs on a remote instance and is in charge of performing all the functions of the Auto Track cloud service on a single camera. Its features include handling state synchronization communication, video converter to join video streaming service room and pull webrtc, auto track / ipc process manager and handle server orchestration integration. In summary, it is the central piece that drives the auto tracking process and surrounding components for a specific camera. 3.2.2 Auto Tracking process (Core): It is the main process, algorithm or set of algorithms which performs the actual tracking computer vision algorithms given a video feed from a single camera. It can be thought of as a single process unit which has as input the camera video and as output the coordinates of the person(s) being tracked. The tracking algorithm consists of several stages / algorithms in which each frame of the video signal is processed. Among the set of nuclear algorithms, 3 main ones can be distinguished; people detection, tracking and specific detection of joints / key points of the person being tracked. Each algorithm plays an important role in the functionalities offered to the user through auto tracking in the studio. Auto Tracking states machine The auto tracking process consists on five states but the subordinate can only iterate over 4 (READY->SHUTDOWN) of them depending on the behavior that wants: • INI. - Auto tracking is starting up, it’s disconnected from the controller so you can’t move to other detection modes yet. It will automatically move to READY as soon as possible. • READY. - The auto tracking process is running with the controller communication established and ready to interact with state sync, the control room or encoder agent. • SEARCHING_FOR_SUBJECT. - The auto tracking process is detecting all the people in the scene, has assigned an UUID for each person and is outputting the coordinates of the bounding boxes for each one in the corresponding nats coordinates subject. • TRACKING_SUBJECT. - The auto tracking is in tracking mode, it’s outputting only the coordinates of the bounding box of the person of interest in the corresponding NATs coordinates subject. • SHUTDOWN. - The auto tracking is shutting down the process, you can’t recover the process from here, you would need to start the auto tracking process again. The Auto Tracking only can move between the following states: 3.3 Auto Tracking Driver- PTZ Controller: It is the entity within the local agent that interprets the auto tracking cloud service information and executes control actions on the PTZ cameras accordingly. The calculation of the movements is divided into 2; Pan Tilt and Zoom. Following is calculations with relevant variables. Where is the current frame in the camera, is the Deadband, is the subject bounding box, y are the height and width of the Deadband respectively, width of the subject bounding box respectively, of coordinates of the subject bounding box center, is the set of coordinates of the Deadband center, is the distance between the edge of the subject's bounding box and the edge of is the distance between the edge of the Deadband and the edge of the frame. Eq 2 It represents the calculation of the error to be used in the PID through the PTZ controller in relation to the pan-tilt movements. Eq 2. Error in function of the distances of the different frames entities. 3.3.1 Deadband: Area around a tracked subject within which a motion of the tracked subject does not lead to a corresponding tracking movement. This is useful when a tracked subject is continually walking / rocking back and forth over a period of time. We should understand the deadband as the space (or threshold) of freedom that the subject has to move without the camera changing position. The key is that its reference is the subject and not the image (frame). The deadband itself must not be used to define in which position of the image you want the subject to be. 3.3.2 Pan / tilt movement calculation: As long as the frame of the subject S is inside the dead band C, the output of the system is within the boundaries of the set point, the error is zero (e (t ) = SP – y (t ) = 0) and no camera movement is performed. When the subject starts moving and S is no longer inside C (conditioned with hysteresis), we calculate the error of the system by getting the number of pixels that S is outside of C on each axis, and then normalizing it according to the distance between the edge of the dead band and the edge of the screen H. So the error between the current subject position and the desired position is computed as: Eq 3. Error calculation for the pan movement. This normalization gives more dynamic movement if the subject and the dead band are closer to one edge of the screen, reacting faster so the subject stays always in frame. The error value is then used as input in the control function to get the speed at which the camera should move. The constant P, I and D values were tuned through several tests to get the best movement response possible. This error-to-speed process is done for both types of movement, pan and tilt. 3.3.3 Zoom movement calculation: For the zoom behavior the set point of the system is the proportion of the height of the subject in reference to the entire screen (hs / hH) at time the tracking starts (conditioned with hysteresis). This means that the control will try to maintain the apparent height of the subject by performing zoom-ins and zoom-outs at different speeds. In this stage we do not consider the Deadband, since the height of C never changes with respect to H in a normal use case. That means it is equivalent to compare the subject height with the dead band height and to compare it with the screen height. At the start of the tracking process, we assign the set point as the current proportion of the subject height and for each iteration we compute the error in zoom as: This error is then used as an input to the control function to get the speed at which the zoom will move. If the subject moves away from the camera and hs is smaller than the original, the error will be positive and the zoom speed required to reach the set point will be positive. In the other case, the zoom speed will be negative. In one embodiment, the method 3 can be provided, wherein the method 3 may comprise steps of: 1. allowing a user to select a subject and automate its tracking to achieve cinematic motion without human intervention; 2. providing real-time tracking with ultra-low latency based on cloud processing; 3. managing auto-tracking instances in the cloud through an orchestration service; 4. processing video frames using a set of core algorithms; 5. calculating the exact position the camera needs to move to in order to reach the subject; and 6. positioning the camera cinematically and responsively. In some embodiments, the processing video frames may include people detection, tracking, and specific detection of joints / key points. In some embodiments, the method 3 may include method 2 to calculate the exact position the camera needs to move to in order to reach the subject. In some embodiments, the method 3 may include method 4 to position the camera cinematically and responsively. 4.0 A method of moving PTZ Cameras through the use of onsite processing whereby absolute PTZ positions are achieved by emulating relative movement from a PTZ joystick: The PTZ controller is at the core of the enhanced movements offered in the product, giving cameras the ability to execute movements that would not be possible using traditional control. Plays all motion functions via continuous motion commands providing additional capabilities such as; adding ramps, smoothing movements, pausing and resuming movements, undoing movements (going in reverse to the executed movement), among others. Its main function is to build the trajectories according to the behavior configured for the active movement function. 4.1 Functions of movement: Motion functions are those functions that are characteristic of our product and have enhanced control over the cameras. To achieve this, among other things, the motion functions are flexible and parameterizable. Here are some examples in terms of configuration. • Click to center; custom ramps, trajectory types (linear, bell), motion priority (time, smoothness), synchronized motion, selective motion (direction priority, e.g. first pan then tilt motion), pause motion, reverse motion. • Fast Frame & Presets; custom ramps, types of trajectory (linear, bell), motion priority (time, smooth), types of movement (TriSync, desynchronized), selective motion (e.g. finish pan / tilt first then zoom), pause motion, reverse motions. • Quick shots; custom ramps, pause motion, reverse motions. • Auto Tracking; dynamic speed and direction. • Tracing: This movement function is only possible due the reproduction of continuous movements since it is able to link different movement functions. Each movement function could require different processing that the PTZ controller identifies in order to properly parameterize the packages to be used in the necessary calculations. For example, a time-prioritized Fast Frame instruction will make use of the coordinate converter and path builder to minimize time. Whereas an Auto Tracking instruction will make use of PIDs and build the path dynamically as the target changes position. A Fast Frame motion synchronized (TriSync) instruction with ramps could make use of PIDs and the path builder that equalizes the motion times in all 3 degrees of freedom. To achieve the above, the PTZ controller must make use of the calibration parameters. The output of this system is a set of directions, velocities and time (in some cases) that allow to describe a trajectory to reach the objective. In a very general way, the process in which the PTZ Controller is involved can be summarized in the following steps: • User-activated movement functions come from the UI or from Cloud Services such as Auto Tracking. These functions, after prior processing, will deliver different types of simple information (such as rectangular coordinates, set points, thresholds and other parameters) to the Encoder Agent at the driver level. • The information at the driver level is processed according to what is required by the function activated (for example, being movement functions, nothing about the video protocol would be required). • At the driver level, the camera catalog contains information about the calibration parameters that are required for the PTZ Controller. • The PTZ controller makes use of the necessary calibration parameters and applies the functions available in the PID and coordinate converter packages so that it can use the parameter values as required to finally use the path builder. The path builder is the entity that generates the output of the PTZ controller; it has multiple configurations that will allow it to reach the objective in various ways. PTZ Controller core functions Description Switch-Case Identify active movement functions Calibration params Query the calibration parameters for the specific device PID & Coordinates converter Use the PID & Coordinates converter parametrization instances that are necessary and parameterize them according to the active movement function Path builder Build the trajectory according to the active motion configuration. • After building the trajectory, the commands would be sent according to the control protocol configured at the driver level for the camera in question (PTZ Protocol). • The camera will execute the movement commands. 4.2 Speed Standardization: An important feature that our product offers to users is the uniformity and consistency in the movements of the added cameras. To achieve this behavior, it uses calibration and uses units of measurement common to all devices. In addition to speed standardization, normalization is the way that regardless of the added device (make / model) and supported protocols, the implementation will remain the same. To standardize the speed we must use a common unit of measurement in the devices, degrees per second. So all camera speeds and corresponding control protocol will be mapped to degrees per second. Regarding speed normalization, the range used must be the same in all instances that use and refer to speed. Specifically, the following is true: • In the UI the available velocity range is (-1, 1). • All drivers in the agent receive speeds in the range (-1, 1). The normalized speed value can come indirectly from the UI through some movement function such as Joystick or through the auto tracking service. In any case it will go through the PTZ controller. When the PTZ controller works with the auto Tracking service the PID output is normalized in the same required range having the constants calibrated. Regarding speed normalization, the PTZ controller has the following responsibilities: Inputs: • Normalized value of speed: . • Range of standardized speed in degrees per second (common values used along all the supported cameras): . • Constants that allow building a mapping function to the range of speed values corresponding to the camera and the control protocol in question: . • Standard range of speeds of the protocol (it comes from the protocol definition itself). Processing: • Internally, the PTZ controller driver reconstructs the mapping function and applies it to obtain the resulting speed value corresponding to the protocol and camera. • Then it needs to normalize that value according to the standard maximum speed of the protocol. Output: ● A normalized value in the range (− 1, 1) Nomenclature used as reference: ● : normalized value of speed. ● : speed in degrees per second. ● : minimum value of speed in the camera for that protocol. ● : maximum value of speed in the camera for that protocol. ● : value of speed in the camera for that specific protocol. The standardized speed range, the calibration constants and the protocol speed range will come from the device registry service. 4.2.1 Calibration: In order to be able to provide the user with a uniform and consistent behavior in the movement of all the cameras added to the UI product, they must be calibrated. Camera calibration consists of the following: • Speed calibration. o Pan speed. o Tilt speed o Zoom speed. • Zoom model. To be able to calibrate the devices and later use the calibration information in the standardization and normalization of speeds, 2 mechanisms are needed; a calibration driver and a package of calibration parameters. 3.2.1.1 Calibration driver: The calibration driver is in charge of executing the calibration routines and reports the results to the calibration params package. It is an agent service that allows automatically executing movements on the camera while the cloud video processing service obtains the necessary information using computer vision to later execute the necessary calculations and define the corresponding calibration parameters. 3.2.1.2 Calibration Params package: The calibration params package is in charge of the following: ● Load. – register the resulting calibration parameters in the catalog for the camera in question (device registry service). ● Download. – consult the calibration parameters of the device registry service (if they already exist). ● Register the calibration parameters locally so that they are exposed to other drivers that may require them. The workflow of the calibration params is represented in FIG.7. 4.3 Path builder: The Path builder is very important within the PTZ controller since it is the one that generates the output of the system. The output or path generated will depend on the configuration of the active motion function. 4.4 Packages: Packages are used as required by the active function of movement. The PTZ controller is in charge of parameterizing the functions of the packages. Both packages are in charge of performing calculations only. 4.4.1 PID: The PID module or package is in charge of carrying out the calculations to control the movements and that these are executed in the desired way (usually a smooth movement and within a closed or open loop). The behavior of the PID is configurable according to the function to be built. To generate a bell-shaped trajectory, create a ramp, or be in a feedback loop for Auto Tracking. More than one PID may be used as required by the movement function (for example one for each direction of movement). The PID has as input the following: ● Rectangular coordinates of the current position. ● Set Point of the desired position. It computes an error function, Which is used to estimate the output, a set of 3 values with magnitude and direction used as factors for the PTZ controller to assign movement speeds. This function can be overloaded depending on the case to create different behaviors for the movement functions or used to control only a degree of freedom. 4.3.2 Coordinates converter: The movement functions give information about the position in pixels of the target or desired position, these are rectangular coordinates. However the camera moves in a different space so the PTZ controller needs to convert the coordinates before building the corresponding trajectory. The movement space of a PTZ camera is represented in FIG.26. This is the task of the coordinate converter (also named Fast Frame model). Eq 7. Output format of the Coordinates converter. The coordinate transformation will depend on the camera model, so the coordinate converter uses the calibration parameters contained in the device registry (camera catalog of our product) as part of the driver definition. In one embodiment, the method 4 can be provided, wherein the method 3 may comprise steps of: 1. building trajectories according to the behavior configured for the active movement function and translating them to camera commands using custom software that mimics the relative motions sent by hardware PTZ Controllers; 2. providing enhanced control over cameras through flexible and parameterizable motion functions; 3. identifying active movement functions and querying calibration parameters for the specific device; 4. standardizing and normalizing speed values across different devices and protocols; and 5. modeling the device’s motion using a series of motion calibration measurements to estimate its position at any given point in time. In some embodiments, the parameterizable motion functions includes a function of click to center, allowing users to center a specific area with configurable motion using the custom motion controller. In some embodiments, the parameterizable motion functions includes a function of Fast Frame & Presets, enabling users to frame and zoom to a specific area or recall saved presets with configurable motion. In some embodiments, the parameterizable motion functions includes a function of Quick shots, providing smooth, continuous pan, tilt, and zoom motions. In some embodiments, the parameterizable motion functions includes a function of Auto Tracking, calculating camera position based on subject pixel location and using velocity matching to maintain centering. In some embodiments, the parameterizable motion functions includes a function of Tracing, recording and repeating user-performed camera motions. 5.0 A method for the automatic control of an array of Pan-Tilt-Zoom (PTZ) and static cameras using machine learning models, wherein the processing occurs on a remote server via the internet: The present invention provides a system and method for autonomously controlling camera arrays, including PTZ and static cameras, through the utilization of machine learning models. The system is designed to automatically track subjects, determine which subjects are of interest for the final composition, and make decisions on camera framing and selection for live or post-production applications. This is achieved using a combination of machine learning models, including language models (LMs), to interpret and analyze video and audio input data. In one embodiment, the system utilizes a data pipeline for the autonomous production algorithm. Video input from one or more cameras, either live-streamed or pre- recorded, is transmitted to a remote processing server via the internet. The video input data, which may include frames and audio, is then analyzed by a series of machine learning models that extract relevant information about the scene. This information is used to generate a prompt that describes the scene to a language model (LM). The extracted information may include, but is not limited to: - Subject identification and tracking, based on facial recognition and clothing / appearance analysis - Subject action detection, such as walking, standing, dancing, speaking, jumping, etc. - Subject facial analysis, focusing primarily on expression recognition - Object identification and tracking across multiple frames - Audio transcription and diarization, allowing the LM to determine which subject is speaking and the content of their speech - Audio action detection, such as music playing, people speaking, yelling, etc. - Image captioning, enabling the description of specific regions of the image as text for LM interpretation - Optical character recognition (OCR), allowing the LM to read text within specific regions of the image - Image and video embeddings, enabling the LM to directly interpret visual and auditory data The LM processes the generated input and can query additional machine learning models to gather more information about the scene or adjust parameters of the data pipeline. With the gathered information, the LM makes decisions on which subjects and objects should be framed or auto-tracked, as well as the configuration of the framing, such as composition alignment (left, right, or center), zoom level (tight, wide, or mid shot), and tracking speed (fast for action or slow for cinematic effect). The system may also be configured to provide multiple framing and auto-tracking options for users in an assisted shot selection mode. After receiving the LM's instructions, the video is either digitally cropped to achieve the desired framing or PTZ camera commands are sent to adjust the camera position accordingly. In addition to framing configuration, the LM can be used to select which camera should be live (switching) or which pre-recorded video should be used for a specific time segment (splicing). The system can be further enhanced by incorporating user preferences, such as filming style, preset styles (e.g., action film, sporting event, or serious occasion), or emulating a specific director's style. The process end to end can be visualized in FIG.5. In FIG.5, we can observe that there are 2 + n cameras (with no fixed limit). Camera 1 is a PTZ Camera 306, while Camera 2 is a Static Camera. The media is sent to the Autonomous Production Server 501 over the internet 500 (as described in Section 1). The Autonomous Production Server 501 performs a series of data extractions 501a from the ingested media (defined above). The extracted data is then forwarded to the Prompt Generator 502, which generates a prompt for the Language Model 503 describing the current scene. The Language Model 503 can output three different types of commands: Camera Specific Output Commands 503a, such as framing configurations or auto-tracking configurations 503aa (defined in Section 4.0); Scene Specific Output Commands 503b, such as determining which camera should be live or which clip should be spliced (output camera section 503bb); and Inquiry Commands 503c, which allow the Language Model 503 to request additional data extraction to gather more information about the scene before making a decision on the output commands. When an Inquiry Command 503c is used, the corresponding processing 503cc is performed, and the information is sent back to the Language Model 503 for it to issue its next set of commands. In one embodiment, the method 5 can be provided, wherein the method 3 may comprise steps of: 1. autonomously controlling camera arrays through the utilization of machine learning models; 2. automatically tracking subjects, determining subjects of interest, and making decisions on camera framing and selection; 3. analyzing video input data using a series of machine learning models to extract relevant information about the scene; 4. generating a prompt describing the scene to a language model (LM) for processing and decision-making; and 5. adjusting camera position or digitally cropping video based on the LM's instructions for desired framing or auto-tracking. In some embodiments, generating a prompt describing the scene includes information such as image of the scene (if vision model is used). In some embodiments, generating a prompt describing the scene includes metadata of the event (client name, studio name, device names). In some embodiments, generating a prompt describing the scene includes descriptions and locations of objects in the scene fed in from other models. 6.0 A method for controlling, monitoring, and managing PTZ and static cameras over the internet, allowing users to access the cameras via a web browser through a centralized management portal: The present invention provides a system and method for simplifying and streamlining the process of digital video production by centralizing the control and management of various production elements, including cameras, audio equipment, and other hardware and software components, into a single online dashboard. The system enables users to remotely access and control these elements, regardless of their physical distance from the venue or location. In one embodiment, the system utilizes a standardized API to facilitate communication between individual production elements and remote servers. This API enables devices to send, receive, and process information, as well as to transmit low- latency media. Devices can connect directly to the cloud servers or via a bridge software agent running on-site at the venue. The online dashboard, accessible through a web browser, allows users to perform a range of tasks, such as: - Viewing individual or multiple camera feeds - Accessing and mixing microphones and audio sources - Controlling PTZ camera positioning - Digitally cropping camera feeds - Recording video to the cloud or on-site storage - Streaming video to external sources - Managing video switching using hardware, software, or cloud-based switchers - Managing on-screen graphics - Accessing advanced features, such as Fast Frame, Auto Tracking, and Autonomous Production Furthermore, the system supports multi-user collaboration, enabling multiple users to work on the same equipment simultaneously through separate browser sessions. The invention aims to improve efficiency and convenience in digital video production by offering a centralized, browser-based solution for managing diverse production elements from different vendors, using various protocols, and requiring specialized configurations. In one embodiment, the method 6 can be provided, wherein the method 6 may comprise steps of: 1. centralizing the control and management of various production elements into a single online dashboard; 2. utilizing a standardized API to facilitate communication between individual production elements and remote servers; 3. enabling users to perform tasks such as viewing camera feeds, controlling PTZ camera positioning, managing video switching, recording video to the cloud or on-site storage, streaming video to external sources, managing on- screen graphics, and accessing advanced features like Fast Frame, Auto Tracking, and Autonomous Production through a web browser; 4. supporting multi-user collaboration, allowing multiple users to work on the same equipment simultaneously; 5. offering a centralized, browser-based solution for managing diverse production elements from different vendors. In some embodiments, the tasks include viewing camera feeds, controlling PTZ camera positioning, and managing video switching. In some embodiments, the tasks include recording video to the cloud or on-site storage. In some embodiments, the tasks include streaming video to external sources, managing on-screen graphics, and accessing advanced features like Fast Frame, Auto Tracking, and Autonomous Production through a web browser. 7.0 A method for providing a third-party API on top of a cloud-based camera management platform, enabling users to extend the capabilities of their camera systems without the need for additional software or hardware installations: The present invention offers a system and method for extending the functionality of camera systems and other production elements by providing a third-party API for integration with a cloud-based camera management platform. This API allows third-party vendors to develop and implement additional features and applications, facilitating innovative use cases and enhancing the capabilities of the base hardware and software. In one embodiment, the third-party API grants vendors access to individual camera feeds and control information, including image settings, PTZ positions, and digital crop states. Vendors can manipulate these values and control production elements in real time, enabling the creation of custom features and enhancements. The system also allows vendors to integrate with the browser-based control interface, displaying custom user interface elements that enable users to interactively control the plugin and its functionality. Vendors have the ability to produce one or more output video streams, which can be ingested through the Cloud Media Ingress. This allows for video and audio manipulation, injection of custom video or audio content, and creation of custom composite shots using input from multiple cameras and audio devices. Access to the third-party API is centrally managed, allowing users to grant specific permissions for each plugin and ensuring that plugins are vetted for user privacy and security. This feature enables users to access new abstractions and functionalities for their production elements, beyond the capabilities of their base hardware and software systems. The process end to end can be visualized in FIG.6. In the diagram, there are 2 + n cameras 306 (with no defined limit). Camera 2 and Camera N are connected to the internet 500 via the Agent Bridge (defined in Section 1) 600. Camera 1306 is connected directly to the internet and runs the Agent Bridge 600 software on the device. Media from the cameras 306 is ingested via the Cloud Media Ingress 406. The control information about the cameras, such as their PTZ positions, is ingested via the Cloud Control Ingress 501. This information, as well as media ingested by the Cloud Media Ingress 406, can then be sent to third-party API servers 503, which can create new composite video 504 that is ingested by the Cloud Media Ingest 406 and / or sent to the Third-Party Plugin's UI 505. Additionally, third-party APIs can communicate directly with the Cloud Control Egress 502, allowing them to manipulate the devices directly. The plugin UI 505 is rendered in the user's browser 506, which can also send commands to the Cloud Control Egress 502. The Cloud Control Egress 502 then communicates with the devices over the internet 500 before the command reaches the device. In one embodiment, the method 7 can be provided, wherein the method 7 may comprise steps of: 1. providing a third-party API for integration with a cloud-based camera management platform; 2. granting vendors access to individual camera feeds and control information for real-time manipulation; 3. allowing vendors to integrate custom user interface elements into the browser- based control interface; 4. enabling vendors to produce output video streams that can be ingested through the Cloud Media Ingress using protocols; and 5. centrally managing access to the third-party API, ensuring user privacy and security while enabling access to new functionalities. In some embodiments, protocols includes WebRTC, SRT, RTMP, and MP4 (file upload). With reference to FIG.7, a system consistent with an embodiment of the disclosure may include a computing device or cloud service, such as computing device 2600. In a basic configuration, computing device 2600 may include at least one processing unit 2602 and a system memory 2604. Depending on the configuration and type of computing device, system memory 2604 may comprise, but is not limited to, volatile (e.g. random-access memory (RAM)), non-volatile (e.g. read-only memory (ROM)), flash memory, or any combination. System memory 2604 may include operating system 2605, one or more programming modules 2606, and may include a program data 2607. Operating system 2605, for example, may be suitable for controlling computing device 2600’s operation. In one embodiment, programming modules 2606 may include image-processing module, machine learning module, etc. Furthermore, embodiments of the disclosure may be practiced in conjunction with a graphics library, other operating systems, or any other application program and is not limited to any particular application or system. This basic configuration is illustrated in FIG.7 by those components within a dashed line 2608. Computing device 2600 may have additional features or functionality. For example, computing device 2600 may also include additional data storage devices (removable and / or non-removable) such as, for example, magnetic disks, optical disks, or tape. Such additional storage is illustrated in FIG.7 by a removable storage 2609 and a non-removable storage 2610. Computer storage media may include volatile and non- volatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer-readable instructions, data structures, program modules, or other data. System memory 2604, removable storage 2609, and non- removable storage 2610 are all computer storage media examples (i.e., memory storage.) Computer storage media may include, but is not limited to, RAM, ROM, electrically erasable read-only memory (EEPROM), flash memory or other memory technology, CD- ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store information and which can be accessed by computing device 2600. Any such computer storage media may be part of device 2600. Computing device 2600 may also have input device(s) 2612 such as a keyboard, a mouse, a pen, a sound input device, a touch input device, a location sensor, a camera, a biometric sensor, etc. Output device(s) 2614 such as a display, speakers, a printer, etc. may also be included. The aforementioned devices are examples and others may be used. Computing device 2600 may also contain a communication connection 2616 that may allow device 2600 to communicate with other computing devices 2618, such as over a network in a distributed computing environment, for example, an intranet or the Internet. Communication connection 2616 is one example of communication media. Communication media may typically be embodied by computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and includes any information delivery media. The term “modulated data signal” may describe a signal that has one or more characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media may include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media. The term computer readable media as used herein may include both storage media and communication media. As stated above, a number of program modules and data files may be stored in system memory 2604, including operating system 2605. While executing on processing unit 2602, programming modules 2606 may perform processes including, for example, one or more stages of methods, algorithms, systems, applications, servers, databases as described above. The aforementioned process is an example, and processing unit 2602 may perform other processes. Other programming modules that may be used in accordance with embodiments of the present disclosure may include machine learning applications. Generally, consistent with embodiments of the disclosure, program modules may include routines, programs, components, data structures, and other types of structures that may perform particular tasks or that may implement particular abstract data types. Moreover, embodiments of the disclosure may be practiced with other computer system configurations, including hand-held devices, general purpose graphics processor-based systems, multiprocessor systems, microprocessor-based or programmable consumer electronics, application specific integrated circuit-based electronics, minicomputers, mainframe computers, and the like. Embodiments of the disclosure may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices. Furthermore, embodiments of the disclosure may be practiced in an electrical circuit comprising discrete electronic elements, packaged or integrated electronic chips containing logic gates, a circuit utilizing a microprocessor, or on a single chip containing electronic elements or microprocessors. Embodiments of the disclosure may also be practiced using other technologies capable of performing logical operations such as, for example, AND, OR, and NOT, including but not limited to mechanical, optical, fluidic, and quantum technologies. In addition, embodiments of the disclosure may be practiced within a general-purpose computer or in any other circuits or systems. Embodiments of the disclosure, for example, may be implemented as a computer process (method), a computing system, or as an article of manufacture, such as a computer program product or computer readable media. The computer program product may be a computer storage media readable by a computer system and encoding a computer program of instructions for executing a computer process. The computer program product may also be a propagated signal on a carrier readable by a computing system and encoding a computer program of instructions for executing a computer process. Accordingly, the present disclosure may be embodied in hardware and / or in software (including firmware, resident software, micro-code, etc.). In other words, embodiments of the present disclosure may take the form of a computer program product on a computer-usable or computer-readable storage medium having computer-usable or computer-readable program code embodied in the medium for use by or in connection with an instruction execution system. A computer-usable or computer-readable medium may be any medium that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer-usable or computer-readable medium may be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium. More specific computer-readable medium examples (a non-exhaustive list), the computer-readable medium may include the following: an electrical connection having one or more wires, a portable computer diskette, a random-access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, and a portable compact disc read-only memory (CD-ROM). Note that the computer-usable or computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory. While certain embodiments of the disclosure have been described, other embodiments may exist. Furthermore, although embodiments of the present disclosure have been described as being associated with data stored in memory and other storage mediums, data can also be stored on or read from other types of computer-readable media, such as secondary storage devices, like hard disks, solid state storage (e.g., USB drive), or a CD-ROM, a carrier wave from the Internet, or other forms of RAM or ROM. Further, the disclosed methods’ stages may be modified in any manner, including by reordering stages and / or inserting or deleting stages, without departing from the disclosure. Although the invention has been explained in relation to its preferred embodiment, it is to be understood that many other possible modifications and variations can be made without departing from the spirit and scope of the invention.

Claims

What is claimed is:

1. A method for managing PTZ Cameras, Static Cameras, and other production elements at Scale, the method comprising: providing a universal interface for interacting with and automating production elements from multiple vendors; classifying base drivers; defining a set of functionalities in a standardized API that can be accessed and / or modified by a Platform API; translating commands from the base control driver into a protocol supported by a device using a Protocol Driver; translating media from the base media driver into a media format supported by the Platform Media Ingress; centralizing the management of protocol definitions in a cloud service and a universal schema; and simplifying the process for users by utilizing a cloud-based Device Registry, 2. The method as claimed in claim 1, wherein the base drivers are classified into media egress drivers and control drivers.

3. The method as claimed in claim 1, wherein the cloud-based Device Registry is structured as a database, containing driver configurations and commands, options, and details for each camera, with information sent from the cloud as structured data.

4. A method of controlling PTZ Cameras Whereby a digital pixel bounding box is converted to PTZ Values, the method comprising: selecting a specific frame within a current frame of a camera and calculating first positions the camera needs to go to in order to reach the specific frame; selecting a specific position within the current frame of a camera and calculating second positions the camera needs to go to in order to center the specific position;mapping rectangular coordinates in a two-dimensional space of an image to spherical coordinates in a movement space of one or more PTZ cameras; performing an automated calibration process using manufacturer's hardware specifications, data obtained through camera movements, and data obtained through video processing; calculating a focal length of the camera using lens calibration, involving moving the PTZ camera in small increments and comparing the distance between key points to model lens distortion; and implementing a system of rotations and transformations to handle coordinate conversion from any position to an original position.

5. A method of automatically positioning PTZ Cameras through the use of Machine Learning models, whereby the processing is done on a remote server over the internet, the method comprising: allowing a user to select a subject and automate its tracking to achieve cinematic motion without human intervention; providing real-time tracking with ultra-low latency based on cloud processing; managing auto-tracking instances in the cloud through an orchestration service; processing video frames using a set of core algorithms; calculating the exact position the camera needs to move to in order to reach the subject; and positioning the camera cinematically and responsively.

6. The method as claimed in claim 5, wherein the processing video frames may include people detection, tracking, and specific detection of joints / key points.

7. A method of moving PTZ Cameras through the use of onsite processing whereby absolute PTZ positions are achieved by emulating relative movement from a PTZ joystick, the method comprising: building trajectories according to the behavior configured for the active movement function and translating them to camera commands using custom software that mimics the relative motions sent by hardware PTZ Controllers; providing enhanced control over cameras through flexible and parameterizable motion functions; identifying active movement functions and querying calibration parameters for the specific device; standardizing and normalizing speed values across different devices and protocols; and modeling the device’s motion using a series of motion calibration measurements to estimate its position at any given point in time.

8. The method as claimed in claim 7, wherein the parameterizable motion functions includes a function of click to center, allowing users to center a specific area with configurable motion using the custom motion controller.

9. The method as claimed in claim 8, wherein the parameterizable motion functions includes a function of Fast Frame & Presets, enabling users to frame and zoom to a specific area or recall saved presets with configurable motion.

10. The method as claimed in claim 9, wherein the parameterizable motion functions includes a function of Quick shots, providing smooth, continuous pan, tilt, and zoom motions.

11. The method as claimed in claim 10, wherein the parameterizable motion functions includes a function of Auto Tracking, calculating camera position based on subject pixel location and using velocity matching to maintain centering.

12. A method for the automatic control of an array of Pan-Tilt-Zoom (PTZ) and static cameras using machine learning models, wherein the processing occurs on a remote server via the internet, the method comprising: autonomously controlling camera arrays through the utilization of machine learning models; automatically tracking subjects, determining subjects of interest, and making decisions on camera framing and selection; analyzing video input data using a series of machine learning models to extract relevant information about the scene; generating a prompt describing the scene to a language model (LM) for processing and decision-making; and adjusting camera position or digitally cropping video based on the LM's instructions for desired framing or auto-tracking.

13. The method as claimed in claim 12, wherein generating a prompt describing the scene includes information such as image of the scene (if vision model is used).

14. The method as claimed in claim 13, wherein generating a prompt describing the scene includes metadata of the event (client name, studio name, device names).

15. The method as claimed in claim 14, wherein generating a prompt describing the scene includes descriptions and locations of objects in the scene fed in from other models.

16. A method for controlling, monitoring, and managing PTZ and static cameras over the internet, allowing users to access the cameras via a web browser through a centralized management portal, the method comprising: centralizing the control and management of various production elements into a single online dashboard; utilizing a standardized API to facilitate communication between individual production elements and remote servers; enabling users to perform tasks;supporting multi-user collaboration, allowing multiple users to work on the same equipment simultaneously; offering a centralized, browser-based solution for managing diverse production elements from different vendors.

17. The method as claimed in claim 16, wherein the tasks include viewing camera feeds, controlling PTZ camera positioning, and managing video switching.

18. The method as claimed in claim 17, wherein the tasks include recording video to the cloud or on-site storage.

19. A method for providing a third-party API on top of a cloud-based camera management platform, enabling users to extend the capabilities of their camera systems without the need for additional software or hardware installations, the seventh method comprising: providing a third-party API for integration with a cloud-based camera management platform; granting vendors access to individual camera feeds and control information for real-time manipulation; allowing vendors to integrate custom user interface elements into the browser- based control interface; enabling vendors to produce output video streams that can be ingested through the Cloud Media Ingress using protocols; and centrally managing access to the third-party API, ensuring user privacy and security while enabling access to new functionalities.

20. The seventh method as claim in claim 19, wherein the protocols includes WebRTC, SRT, RTMP, and MP4.

Images