System and method facilitating cloud-based video stream processing

Abstract

A computer system may receive a first request from a user device to start steaming content, the transfer request including a port number of a source device. The system may send a second request to the source device by using the port number of the source device to start streaming the content, and send the content streamed from the source device to a streaming service.

Description

[0001] SYSTEM AND METHOD FACILITATING CLOUD-BASED VIDEO STREAM PROCESSING

[0002] Priority Claim

[0003] This Application claims the benefit of U.S. Provisional Patent Application No. 63 / 524,389, filed June 30, 2023, which is incorporated by reference in its entirety (including its Appendices) into the present disclosure.

[0004] Technical Field

[0005] The present subject matter relates generally to information technology and computer networks and, more particularly, to systems and methods for securely and reliably passing a video stream from a source to a cloud-based video-processing service.

[0006] Background

[0007] Internet-of-things (loT) technology is rapidly expanding in its application and deployment across many sectors of commerce and industry. When applied to video streams, such as videos captured by video cameras (e.g., cameras in mobile phones, web-cams, surveillance / security cameras, vehicle-mounted cameras, and others), loT technology enables intelligent systems to be built that can apply more sophisticated processing techniques than what may be achieved on the local camera or camera-connected devices. Such processing may enable real-time monitoring and analysis, such as object or face detection, analysis of captured activity, and other advanced analytics, utilizing machinelearning and other advanced algorithms.

[0008] The captured video streams can be processed using cloud services, which provide the necessary computing power and storage capabilities. Cloud-based processing allows for efficient data management, scalability, and remote access to video feeds from anywhere. With the integration of loT cameras and cloud services, businesses and individuals can enhance security, streamline operations, and gain valuable insights from their video data.

[0009] The significance of processing video streams in a scalable and reliable way is crucial, especially when cameras are deployed in remote locations using various connectivity means, including cellular, Wi-Fi, Ethernet, and satellite networks. In such cases, there is a pressing need for a solution that securely ingests video streams from cameras to a cloud environment, enabling cloud applications to process the video streams efficiently.

[0010] There are multiple cloud services available for building a system that meets these requirements. For instance, Amazon Kinesis Video Stream allows users to store video streams produced by camera devices and build applications capable of consuming video streams in real-time and in parallel. Additionally, Amazon Simple Storage Service (S3) provides an option for storing video files for later consumption, among other services. By utilizing these cloud solutions, organizations can effectively manage and process video data from a diverse range of sources, ensuring optimal performance, reliability, and security.

[0011] In building systems for processing video streams, it is important to note that many cameras on the market do not natively support cloud application programming interfaces (APIs) to ingest video streams to a cloud service. While some cameras have the ability to push videos to a specified endpoint on their own, many surveillance cameras available today only support protocols like Real-Time Streaming Protocol (RTSP). This means that a client needs to remotely access the camera to retrieve the video stream.

[0012] Connecting these RTSP cameras directly to the public internet and exposing the RTSP port poses a significant security risk as malicious actors could potentially gain unauthorized access to the cameras. Furthermore, even if a secure connection to a camera's port can be established, users still need to implement logic to retrieve video from the camera and ingest it into a cloud service by calling the API. This logic must run somewhere and be maintained, adding complexity and additional responsibilities for the users.

[0013] These challenges emphasize the need for a comprehensive solution that addresses security, ease of use, and scalability when dealing with video streams from various camera sources.

[0014] Summary

[0015] In some aspects, the techniques described herein relate to a computer system including: computing hardware including at least one processor, memory, and an input / output device, the computing hardware being operative to: receive a first request from a user device to start steaming content, the transfer request including a port number of a source device; send a second request to the source device by using the port number of the source device to start streaming the content; and send the content streamed from the source device to a streaming service.

[0016] In related aspects, the techniques described herein relate to a computer system, further operative to establish a connection between the source device and the computer system using the port number of the source device.

[0017] In related aspects, the techniques described herein relate to a computer system, further including: a connection relay operative to establish the connection between the source device and the computer system, and a task runner operative to send the content streamed from the source device to the streaming service; and wherein the computer system is further operative to: assign a global Internet Protocol (IP) address and a second port number to a connection relay, and establish a second connection between the task runner and the connection relay using the global IP address and the second port assigned to the connection relay, and wherein the task runner is operative to request the connection relay through the second connection to send the second request to the source device.

[0018] In related aspects, the techniques described herein relate to a computer system, wherein the connection relay is operative to establish the connection between the source device and the computer system using an internal IP address of the source device.

[0019] In related aspects, the techniques described herein relate to a computer system, wherein the computing hardware is operative to send the content streamed from the source device includes invoking an Application Programming Interface (API) of the streaming service to ingest the streaming content.

[0020] In related aspects, the techniques described herein relate to a computer system, further operative to: receive from the user device a third request to terminate streaming the content; and close the connection established between the source device and the computer system in response to receiving the third request.

[0021] In related aspects, the techniques described herein relate to a computer system, further operative to at least one of: transcode a codec of the content streamed from the source device; reduce a data size of the content streamed from the source device; and remove part of information from the content streamed from the source device. In related aspects, the techniques described herein relate to a computer system, further operative to: provide an API to receive the first request from the user device.

[0022] In some aspects, the techniques described herein relate to at least one non-transitory machine-readable storage medium storing one or more computer programs which, when executed by a computer system, cause the computer system to: receive a first request from a user device to start steaming content, the transfer request including a port number of a source device; send a second request to the source device by using the port number of the source device to start streaming the content; and send the content streamed from the source device to a streaming service

[0023] In some aspects, the techniques described herein relate to an automated method performed by a computer system, the method including: receiving a first request from a user device to start steaming content, the transfer request including a port number of a source device; sending a second request to the source device by using the port number of the source device to start streaming the content; and sending the content streamed from the source device to a streaming service.

[0024] In related aspects, the techniques described herein relate to a method, further including: establishing a connection between the source device and the computer system using the port number of the source device.

[0025] In related aspects, the techniques described herein relate to a method, further including: establishing a connection relay for making the connection between the source device and the computer system, and implementing a task runner for sending the content streamed from the source device to the streaming service; assigning a global Internet Protocol (IP) address and a second port number to a connection relay, and establishing a second connection between the task runner and the connection relay using the global IP address and the second port assigned to the connection relay, and wherein the task runner is operative to request the connection relay through the second connection to send the second request to the source device.

[0026] In related aspects, the techniques described herein relate to a method, wherein establishing the connection between the source device and the computer system is achieved using an internal IP address of the source device. In related aspects, the techniques described herein relate to a method, wherein sending the content streamed from the source device includes invoking an Application Programming Interface (API) of the streaming service to ingest the streaming content.

[0027] In related aspects, the techniques described herein relate to a method, further including: receiving, from the user device, a third request to terminate streaming the content; and closing the connection established between the source device and the computer system in response to receiving the third request.

[0028] In related aspects, the techniques described herein relate to a method, further including at least one of: transcoding a codec of the content streamed from the source device; reducing a data size of the content streamed from the source device; and removing part of information from the content streamed from the source device.

[0029] In related aspects, the techniques described herein relate to a method, further including: providing an API to receive the first request from the user device.

[0030] Brief Description of the Drawings

[0031] The invention may be more completely understood in consideration of the following detailed description of various embodiments of the invention in connection with the accompanying drawings.

[0032] FIG. 1 is a high-level diagram illustrating a simple-case conventional cloud-based video processing arrangement using Amazon® Web Services (AWS).

[0033] FIG. 2 is a diagram illustrating a system facilitating video transfer according to one embodiment.

[0034] FIG. 3 is a diagram illustrating an exemplary hardware and software architecture of a computer system such as the one depicted in FIG. 2, in which various interfaces between hardware components and software components are shown.

[0035] FIGs. 4A-4B, together, are a sequence diagram illustrating the interaction of the various engines of FIG. 2 according to an example use-case scenario in the context of a connectivity provider that has an on-demand remote access service.

[0036] FIG. 5 is a data-flow diagram illustrating high-level operation of a video-transfer system according to some embodiments. FIG. 6 is a data-flow diagram illustrating a provisioning phase according to some aspects of the invention.

[0037] FIG. 7 is a data-flow diagram illustrating a video-transferring phase of operation according to some embodiments.

[0038] While the invention is amenable to various modifications and alternative forms, specifics thereof have been shown by way of example in the drawings and will be described in detail. It should be understood, however, that the intention is not to limit the invention to the particular embodiments described.

[0039] Description

[0040] Aspects of this disclosure are directed to systems and methods that address the challenges outlined above, as well as other challenges, and enables users to direct video streams from a camera device to a cloud service of their choice. It will be understood that the technological solutions described herein are not limited to working with video streams. Indeed, any streaming content from a variety of sources may be supported. For instance, audio feeds, sensor or other telemetry data, among other data types, may be facilitated. For the sake of brevity, the following embodiments describe the streaming content in terms of video streams. In some embodiments, the systems and methods described herein provide such simplicity of use for the user that video streams from a given source may be directed to a selected cloud service by making one API call or simply clicking on a web interface.

[0041] The solutions can be applied to a connectivity services provider such as a Mobile Network Operator (MNO), Mobile Virtual Network Operator (MVNO), or Internet Service Provider (ISP) who can access a device connected via its connectivity service.

[0042] FIG. 1 is a high-level diagram illustrating a simple-case conventional cloud-based video processing arrangement using Amazon® Web Services (AWS). As shown, video streams from various sources of video streams, such as a mobile phone, or network- connected security camera, (which are referred to as producers in the context of Amazon Kinesis Data Streams) may be enabled to use AWS’s services. In the example depicted, the producer devices may include functions provided through AWS’s software development kit (SDK) libraries. Each producer pushes its media data to the AWS cloud, which runs a Kinesis Data Streams application. The application processes the video-stream data, and can provide its output to consumers of the content in batch or continuous fashions. The outputs may be provided to Amazon Elastic Compute Cloud (EC2) instances, as shown, or directly to the video consumers over the public Internet. As discussed above, many video cameras or other sources of streaming media are not enabled to use AWS or similar service.

[0043] FIG. 2 is a diagram illustrating a system facilitating video transfer according to one embodiment. The system may be implemented as part of a computer system. The computer system may be one physical machine, or may be distributed among multiple physical machines, such as by role or function, or by process thread in the case of a cloud computing distributed model. In various embodiments, aspects of the invention may be configured to run in virtual machines that in turn are executed on one or more physical machines. It will be understood by persons of skill in the art that features of the invention may be realized by a variety of different suitable machine implementations.

[0044] Generally, the system includes various engines, each of which is constructed, programmed, configured, or otherwise adapted, to carry out a function or set of functions, as detailed below. The term engine as used herein means a tangible device, component, or arrangement of components implemented using hardware, such as by an application specific integrated circuit (ASIC) or field-programmable gate array (FPGA), for example, or as a combination of hardware and software, such as by a processor-based computing platform and a set of program instructions that transform the computing platform into a special-purpose device to implement the particular functionality. An engine may also be implemented as a combination of the two, with certain functions facilitated by hardware alone, and other functions facilitated by a combination of hardware and software.

[0045] In an example, the software may reside in executable or non-executable form on a tangible machine-readable storage medium. Software residing in non-executable form may be compiled, translated, or otherwise converted to an executable form prior to, or during, runtime. In an example, the software, when executed by the underlying hardware of the engine, causes the hardware to perform the specified operations. Accordingly, an engine is physically constructed, or specifically configured (e.g., hardwired), or temporarily configured (e.g., programmed) to operate in a specified manner or to perform part or all of any operations described herein in connection with that engine. In examples in which engines are temporarily configured, each of the engines may be instantiated at different moments in time. For example, where the engines comprise a general-purpose hardware processor core configured using software; the general-purpose hardware processor core may be configured as respective different engines at different times. Software may accordingly configure a hardware processor core, for example, to constitute a particular engine at one instance of time and to constitute a different engine at a different instance of time.

[0046] In certain implementations, at least a portion, and in some cases, all, of an engine may be executed on the processor(s) of one or more computers that execute an operating system, system programs, and application programs, while also implementing the engine using multitasking, multithreading, distributed (e.g., cluster, peer-peer, cloud, etc.) processing where appropriate, or other such techniques. Accordingly, each engine may be realized in a variety of suitable configurations, and should generally not be limited to any particular implementation exemplified herein, unless such limitations are expressly called out.

[0047] In addition, an engine may itself be composed of more than one sub-engines, each of which may be regarded as an engine in its own right. Moreover, in the embodiments described herein, each of the various engines corresponds to a defined functionality; however, it should be understood that in other contemplated embodiments, each functionality may be distributed to more than one engine. Likewise, in other contemplated embodiments, multiple defined functionalities may be implemented by a single engine that performs those multiple functions, possibly alongside other functions, or distributed differently among a set of engines than specifically illustrated in the examples herein.

[0048] Referring to FIG. 2, various engines which are part of the system are depicted. The system includes video transfer task manager 202, video transfer task runner 204, connection manager 206, connection relay 208, and secure link service 210. Video transfer task manager 202 is operative to receive a request from a user device to start a video transfer from a source device such as a camera, and to initiate the requested video transfer task on video transfer task runner 204. If necessary, video transfer task manager 202 asks on- demand remote access connection manager 206 to open a port to access the videostreaming service on the source device (e.g., using the RTSP protocol). Video transfer task runner 204 is operative to run a video transfer task in response to the request by video transfer task manager 202 until it is asked to stop or runs out of the specified time. In some implementations, if a task is implemented in a virtual environment such as a Docker container, or the like, video transfer task runner 204 can be implemented in a virtual environment such as a container execution environment (e.g., Kubernetes, Amazon ECS, or the like).

[0049] Connection manager 206 is an optional engine that may be used when video transfer task runner 204 cannot access the source device directly because of a firewall between the source device and video transfer task runner 204. Connection manager 206 can be omitted when video transfer task runner 204 and the source device reside in the same private network. In some implementations, connection manager 206 may be implemented using Soracom Napter when the system is realized in the Soracom platform. Additional details may be found in U.S. Pre-Grant Patent Publication No. US2021336967, which is incorporated by reference herein.

[0050] Connection relay 208 is an optional engine that may be used in conjunction with connection manager 206. Connection relay 208 references the configuration provided via connection manager 206, and relays requests directed to an external IP address (and a port) to an internal IP address and port of the source device. In this example, the internal IP address is an IP address that is assigned to a device (for example, camera 406) by a connectivity provider and that cannot be designated as a destination IP address from the Public Internet.

[0051] Secure link service 210 is another optional engine which is used to establish a secure link, such as a virtual private network (VPN), between the source device and video transfer task runner 204. When the source device is connected over the public Internet, the system employs secure link service 210 to authenticate the source device and to secure the connection between the source device and secure link service 210 that connects an environment where video transfer task runner 204 is deployed. Secure link service 210 enables the system to work in cases where a service provider does not support such connectivity and instead only serves cameras connected over the public Internet. In some embodiments which may be implemented on a Soracom (or similar) platform, secure link service 210 may be implemented with Soracom Arc, (or similar technology) as described in International Patent Application Pub. No. WO2022270228, or Japanese Patent Publication No. JP2023002448, each incorporated by reference herein.

[0052] Notably, the system depicted in FIG. 2 is scalable to permit multiple different user devices to initiate separate video transfers from various source devices. Likewise, the system is scalable to accommodate a plurality of different source devices. Accordingly, each user device can initiate video transfer from one or more selected source devices. Moreover, for each video session corresponding to a user device and a source device, a variety of available video-processing services, e.g., which may be cloud-based, can be available and the user device may select from among these available services when a video stream from a particular source is requested.

[0053] In a sense, video transfer task runner 204 can be considered an interface device that, to the source device, looks like a user device. Video transfer task runner 204 feeds the streaming video content (e.g., in its native format) to the selected one or more videoprocessing service(s) that the user device may access. The video-processing service may change video format, add effects or enhancements, resize or resample the content, analyze the content for certain object or subject detection, symbol reading, or activity recognition, to name just a few. Each video-processing service may be a virtualized (e.g., containerenvironment-based) instance that is specialized for a particular type of task.

[0054] Video transfer task manager 202 serves as a user-device interface with the overall system. It facilitates simple commands (e.g., API calls) to be provided from user device 402, and passes suitable information to the various engines of the system to establish connection(s) to the desired source(s), and indicates to video transfer task runner 204 where to feed the video streams for processing (e.g., to which video-processing service instance).

[0055] FIG. 3 is a diagram illustrating an exemplary hardware and software architecture of a computer system such as the one depicted in FIG. 2, in which various interfaces between hardware components and software components are shown. As indicated by HW, hardware components are represented below the divider line, whereas software components denoted by SW reside above the divider line. On the hardware side, processing devices 302 (which may include one or more microprocessors, digital signal processors, etc., each having one or more processor cores), are interfaced with memory management device 304 and system interconnect 306. Memory management device 304 provides mappings between virtual memory used by processes being executed, and the physical memory. Memory management device 304 may be an integral part of a central processing unit which also includes the processing devices 302.

[0056] Interconnect 306 includes a backplane such as memory, data, and control lines, as well as the interface with input / output devices, e.g., PCI, USB, etc. Memory 308 (e.g., dynamic random access memory - DRAM) and non-volatile memory 309 such as flash memory (e.g., electrically-erasable read-only memory - EEPROM, NAND Flash, NOR Flash, etc.) are interfaced with memory management device 304 and interconnect 306 via memory controller 310. I / O devices, including video and audio adapters, non-volatile storage, external peripheral links such as USB, Bluetooth, etc., camera / microphone data capture devices, fingerprint readers and other biometric sensors, as well as network interface devices such as those communicating via Wi-Fi or LTE-family interfaces, are collectively represented as I / O devices and networking 312, which interface with interconnect 306 via corresponding I / O controllers 314.

[0057] In a related embodiment, input / output memory management unit IOMMU 315 supports secure direct memory access (DMA) by peripherals. IOMMU 315 may provide memory protection by meditating access to memory 308 from I / O device 312. IOMMU 315 may also provide DMA memory protection in virtualized environments, where it allows certain hardware resources to be assigned to certain guest VMs running on the system, and enforces isolation between other VMs and peripherals not assigned to them.

[0058] On the software side, a pre-operating system (pre-OS) environment 316, which is executed at initial system start-up and is responsible for initiating the boot-up of the operating system. One traditional example of pre-OS environment 316 is a system basic input / output system (BIOS). In present-day systems, a unified extensible firmware interface (UEFI) is implemented. Pre-OS environment 316 is responsible for initiating the launching of the operating system or virtual machine manager, but also provides an execution environment for embedded applications according to certain aspects of the invention.

[0059] Virtual machine manager (VMM)Zoperating system (OS) 318 is system software that creates and controls the execution of virtual machines (VMs) 320A and 320B. VMM / OS 318 may run directly on the hardware HW, as depicted, or VMM / OS 318 may run under the control of an operating system as a hosted VMM. Each VM 320A, 320B includes an execution environment 322A, 322B, and application programs 324A, 324B. Each execution environment 322A, 322B may include an operating system (e.g., guest OS) which provides a kernel that operates via the resources provided by VMM 318 to control the hardware devices, manage memory access for programs in memory, coordinate tasks and facilitate multi-tasking, organize data to be stored, assign memory space and other resources, load program binary code into memory, initiate execution of the corresponding application program which then interacts with the user and with hardware devices, and detect and respond to various defined interrupts. In other implementations, each execution environment 322A, 322B is implemented as an application virtual machine which does not provide a kernel with its associated functionality, and instead relies on an operating system that is provided at a lower level, e.g., VMM / OS 318, where the OS kernel may be provided.

[0060] Also, each execution environment 322A, 322B provides device drivers, and a variety of common services such as those that facilitate interfacing with peripherals and networking, that provide abstraction for corresponding application programs 324A, 324B so that the applications do not need to be responsible for handling the details of such common operations. Each execution environment 322A, 322B additionally may provide a graphical user interface (GUI) that facilitates interaction with the user via peripheral devices such as a monitor, keyboard, mouse, microphone, video camera, touchscreen, and the like. In some embodiments, execution environment 322B may omit a GUI.

[0061] Each execution environment 322A, 322B may provide a runtime system that implements portions of an execution model, including such operations as putting parameters onto the stack before a function call, the behavior of disk input / output (I / O), and parallel execution-related behaviors.

[0062] In addition, each execution environment 322A, 322B may provide libraries that include collections of program functions that provide further abstraction for application programs. These include shared libraries, dynamic linked libraries (DLLs), for example.

[0063] Application programs 324A, 324B are those programs that perform useful tasks for users, beyond the tasks performed by lower-level system programs that coordinate the basis operability of the computer system itself. In various embodiments, each of the engines as depicted in FIG. 2 may be implemented as an application program 324A, 324B, or as a driver or other code of execution environment 322A, 322B, or as a driver of VMM / OS 318. In some embodiments, each of the execution environments 322A and 322B is a container and multiple execution environments (that is, multiple containers) run on one VM.

[0064] FIGs. 4A-4B, together, are a sequence diagram illustrating the interaction of the various engines of FIG. 2 according to an example use-case scenario in the context of a connectivity provider that has an on-demand remote access service. If the camera 406 is not a cellular device and uses secure link service 210 to authenticate and securely connect to the platform, an ID for secure link service 210 is used instead of a SIM ID. Establishing a secure link between the source device (camera 406) and the platform embodying the system is performed prior to the sequence as illustrated and thus omitted in the sequence diagram. If the camera 406 is reachable from video transfer task runner 204 without using connection manager 206, the steps involving connection manager 206 and connection relay 208 are omitted.

[0065] As shown in FIGs. 4A-4B, operations 1 -8 relate to initiating a video transfer task, operations 9-16 relate to video streaming, and operations 17-26 relate to terminating the video transfer task. It is assumed that camera 406 is waiting for a video transfer request to a preset port through the operations in FIGs. 4A-4B.

[0066] At operation 1 , the user device 402 calls video transfer task manager 202 to initiate a video transfer. Information passed to video transfer task manager 202 includes a SIM ID of the camera 406, the video-transfer protocol, port number, destination cloud service, endpoint, and authentication information. The SIM ID of the camera 406 identifies a device (camera 406) from which a video stream is to be transferred. The SIM ID may be an International Mobile Subscriber Identity (IMSI) stored in the SIM of the camera 406. The video-transfer protocol indicates a protocol supported by the camera 406. The port number indicates a port number (e.g., 554 for RTSP) used by the camera 406 to wait for a video transfer request. The port number used by the camera 406 may be preset in the camera 406 by an owner of the camera 406. The user of the user device 402 knows this port number prior to operation 1 . The destination cloud service indicates a cloud service to which the video stream from the camera 406 is to be transferred. The endpoint indicates an endpoint of the destination cloud service. The authentication information can be used by the video transfer task manager 202 to authenticate the user of the user device 402. The video transfer task manager 202 may perform operation 2 only if the user of the user device 402 is successfully authenticated by using the authentication information.

[0067] At operation 2, video transfer task manager 202 communicates a request to connection manager 206 to open a port of the camera 406 to which video transfer task runner 204 can connect. The request to connection manager 206 may include the internal IP address and port number of the camera 406. The video transfer task manager 202 may manage device information which associates an internal IP address and a SIM ID of each device serviced by the video transfer task manager 202. The video transfer task manager 202 may use this device information to specify the internal IP address of the camera 406 associated with the SIM ID included in the request received at operation 1 . The port number included in the request at operation 2 is the same as the port number received from the user device 402 at operation 1 .

[0068] At operation 3, connection manager 206 sends, to connection relay 208, the internal IP address and port number included in the request received at operation 2, and assigns, to connection relay 208, a global IP address and port number for accepting the remote access connection from the video transfer task runner 204. Connection manager 206 may select one of the global IP addresses which are not currently used and randomly selects a port number to assign them to connection relay 208.

[0069] At operation 4, connection relay 208 returns the assigned global IP address and port information to connection manager 206. The port information includes the port number assigned at operation 3. At operation 5, connection manager 206 returns the assigned global IP address and the port information to video transfer task manager 202 as a response to the request at operation 2.

[0070] In various embodiments, multiple connection relays are contemplated, each of which can reach different source devices. In such cases, connection manager 206 operates to select an appropriate connection relay depending on the network topology.

[0071] At operation 6, video transfer task manager 202 commands video transfer task runner 204 to start a video transfer task. The command may include the protocol, the global IP address, port number, destination cloud service, endpoint, and authentication information. The global IP address and port number are the same as those returned at operation 5. The protocol, the destination cloud service, the endpoint, and the authentication information included in the command are the same as those received at operation 1 .

[0072] At operation 7, video transfer task runner 204 confirms initiation of the task, and returns a task ID. For example, video transfer tank runner 204 starts a container process for the video transfer task requested at operation 6. At operation 8, video transfer task manager 202 communicates the task ID to the user device 402 to confirm initiation of the task.

[0073] In related embodiments, to support various combinations of protocols that the source device uses and that the destination cloud service to may use, the service provider may prepare multiple container images (one corresponding to each unique combination of protocols). In such an embodiment, the video transfer task manager works to select which container image to use depending on the request.

[0074] At operation 9, video transfer task runner 204 opens a connection to the connection relay 208 using the global IP address and port received at operation 6, by communicating this information to connection relay 208.

[0075] At operation 10, connection relay 208 establishes a connection to the camera 406 using the internal IP address and port number of camera 406 received at operation 3 to initiate capture of the video stream. At operation 11 , camera 406 acknowledges the request, and at operation 12 the acknowledgement is forwarded to video transfer task runner 204. At operation 13, the video stream is initiated to video transfer task runner 204 by camera 406. At operation 14, video transfer task runner 204 invokes an AP I of cloud service 404 to ingest the video stream. The cloud service 404 is specified by the destination cloud service and the endpoint received at operation 6. Video transfer task runner 204 may use the protocol received at operation 6. At operation 15, user device 402 requests the video stream, which is provided to it by cloud service 404 at operation 16.

[0076] At operation 17, user device 402 requests termination of the video stream. The request includes the task ID received at operation 8. Alternatively, the request may further include the SIM ID of camera 406. In this case, video transfer task manager 202 specifies a task ID of the ongoing video transfer task associated with the SIM ID of the camera 406. At operation 18, video transfer task manager 202 commands video transfer task runner 204 to terminate the video transfer task. The task ID is included with the command. At operation 19, video transfer task runner 204 communicates a command to close the connection established at operation 10 to connection relay 208, which relays the command to camera 406 at operation 20. Acknowledgements are provided at operations 21 -22, and at operation 23 video transfer task runner 204 commands cloud service 404 to close its connection. Acknowledgements are provided at operations 24-26 as shown.

[0077] In some embodiments, at operation 9, video transfer task runner 204 sends a request to the camera 406 further through a router or a gateway. In an environment in which multiple devices (for example, the camera 406) which have different port numbers can be accessed from video transfer task runner 204 through the router or the gateway, a port number is used to specify a device which starts video stream.

[0078] FIG. 5 is a data-flow diagram illustrating high-level operation of a video-transfer system according to some embodiments. As shown, the video-transfer system may be implemented within a network of a connectivity services provider, such as an ISP, MNO, or MVNO. Notably, the user device, with a single API call to open a video stream, sets in motion the operations discussed above, which cause the video transfer system to establish a session, open a video stream using a native interface and protocol of the video source, receive the video in the video source’s native format, and direct the video stream to one or more cloud-based services residing on an external network, which may further process the video (e.g., to convert the video into a format more suitable for advanced analysis or for delivery to the user device).

[0079] In an example use case, a basic operation (without processing) includes:

[0080] 1 . Receiving a chunk of a video stream;

[0081] 2. Calling the cloud service API to upload the chunk; and

[0082] 3. Repeat 1 and 2.

[0083] Additional operations that can be added between 1 and 2 include:

[0084] - Transcoding video codec from one to anotherto e.g., increase the compression ratio or adapt to a codec to one that the destination cloud service supports;

[0085] - Reducing video frame rate to reduce data size;

[0086] - Reducing video frame resolution to reduce data size; or - Pre-processing and removing information unnecessary for the downstream processing, e.g. if the motion detection is only the processing performed in the downstream, then color and audio information can be removed from the video.

[0087] FIG. 6 is a data-flow diagram illustrating a provisioning phase according to some aspects. As depicted, the video-transfer system includes an API gateway, a connection manager and a video transfer task manager, a database (e g., for storing mappings between global IP addresses and related ports, and internal IP addresses and related ports), a MAEB- proxy (as an implementation of a connection relay), and a P-GW. The API gateway responds to an API request from the user device to open a video stream from a source, and manages the video transfer task and its associated session information, including IP addresses and ports used by the video transfer task runner in the database and returning session ID information to the user device. The API gateway authenticates the request and forwards to the video transfer task manager that executes the sequence described in the above, including creating port mapping for the connection relay that can reach a packet data network gateway (P-GW) that the source device is connected to, which will work to direct the video stream to various services. The video stream may be converted from its native format into an Amazon Kinesis video stream to facilitate further processing in the cloud. In addition, the video transfer task manager may launch an elastic container service (ECS) to which the video stream may be provided for such processing.

[0088] FIG. 7 is a data-flow diagram illustrating a video-transferring phase of operation according to some embodiments using HTTP Live Streaming (HLS) to access a video steam uploaded to Amazon KVS. Using the provisioned session information, the user device may request transfer of the video stream from its source (e.g., camera) using HLS. The video stream is routed via the P-GW to the cloud services, e.g., Amazon ECS, where the video stream may be converted or analyzed, or otherwise processed. The output of this processing is directed, via the session manager and API gateway, to the user device for consumption, or elsewhere for further use.

[0089] Advantageously, aspects of the system resolve the challenges mentioned above and enable ingesting a video stream from a basic RTSP camera without compromising security or requiring the user device to run a special client to ingest video to a cloud service. Simply by making an API call, a video stream from an off-the-shelf RTSP camera becomes available in the cloud service of the user's choice and ready to be consumed by viewers or videoconsuming applications.

[0090] Moreover, in some embodiments, the video transfer task runner is implemented as a container execution environment, which allows the system to support various video streaming protocols and cloud services just by swapping (i.e., selecting differently configured) containers. In various examples, if the container is an adapter for an RTSP camera that ingests video to Amazon KVS, the video stream becomes available as a KVS stream. If the container is a server for receiving a video stream over RTMP and stores it to Amazon S3, the video stream can be stored as a series of files in an Amazon S3 bucket. The user can provide a custom container image to run their own logic or video stream adapter as well.

[0091] The system can be applied to support various types of video players because of its modular and dynamically-variable architecture. In a practical implementation, a cellular or other connectivity provider can deploy the system by implementing a remote access service and video transfer task manager. An ISP can also implement the secure video transfer service.

[0092] The embodiments above are intended to be illustrative and not limiting. In addition, although aspects of the present disclosure have been described with reference to particular embodiments, those skilled in the art will recognize that changes may be made in form and detail without departing from the scope of the inventive concepts as described herein.

[0093] Persons of ordinary skill in the relevant arts will recognize that the invention may comprise fewer features than illustrated in any individual embodiment described above. The embodiments described herein are not meant to be an exhaustive presentation of the ways in which the various features of the inventive concepts may be combined. Accordingly, the embodiments are not mutually exclusive combinations of features; rather, the inventive concepts may comprise a combination of different individual features selected from different individual embodiments, as will be understood by persons of ordinary skill in the art.

Claims

What is claimed is:1 . A computer system comprising: computing hardware including at least one processor, memory, and an input / output device, the computing hardware being operative to: receive a first request from a user device to start steaming content, the transfer request including a port number of a source device; send a second request to the source device by using the port number of the source device to start streaming the content; and send the content streamed from the source device to a streaming service.

2. The computer system according to claim 1 , further operative to establish a connection between the source device and the computer system using the port number of the source device.

3. The computer system according to claim 2, further comprising: a connection relay operative to establish the connection between the source device and the computer system, and a task runner operative to send the content streamed from the source device to the streaming service; and wherein the computer system is further operative to: assign a global Internet Protocol (IP) address and a second port number to a connection relay, and establish a second connection between the task runner and the connection relay using the global IP address and the second port assigned to the connection relay, and wherein the task runner is operative to request the connection relay through the second connection to send the second request to the source device.

4. The computer system according to claim 3, whereinthe connection relay is operative to establish the connection between the source device and the computer system using an internal IP address of the source device.

5. The computer system according to claim 1 , wherein the computing hardware is operative to send the content streamed from the source device includes invoking an Application Programming Interface (API) of the streaming service to ingest the streaming content.

6. The computer system according to claim 2, further operative to: receive from the user device a third request to terminate streaming the content; and close the connection established between the source device and the computer system in response to receiving the third request.

7. The computer system according to claim 1 , further operative to at least one of: transcode a codec of the content streamed from the source device; reduce a data size of the content streamed from the source device; and remove part of information from the content streamed from the source device.

8. The computer system according to claim 1 , further operative to: provide an API to receive the first request from the user device.

9. At least one non-transitory machine-readable storage medium storing one or more computer programs which, when executed by a computer system, cause the computer system to: receive a first request from a user device to start steaming content, the transfer request including a port number of a source device; send a second request to the source device by using the port number of the source device to start streaming the content; and send the content streamed from the source device to a streaming service10. An automated method performed by a computer system, the method comprising: receiving a first request from a user device to start steaming content, the transfer request including a port number of a source device; sending a second request to the source device by using the port number of the source device to start streaming the content; and sending the content streamed from the source device to a streaming service.11 . The method according to claim 10, further comprising: establishing a connection between the source device and the computer system using the port number of the source device.

12. The method according to claim 11 , further comprising: establishing a connection relay for making the connection between the source device and the computer system, and implementing a task runner for sending the content streamed from the source device to the streaming service; assigning a global Internet Protocol (IP) address and a second port number to a connection relay, and establishing a second connection between the task runner and the connection relay using the global IP address and the second port assigned to the connection relay, and wherein the task runner is operative to request the connection relay through the second connection to send the second request to the source device.

13. The method according to claim 12, wherein establishing the connection between the source device and the computer system is achieved using an internal IP address of the source device.

14. The method according to claim 10, wherein sending the content streamed from the source device includes invoking an Application Programming Interface (API) of the streaming service to ingest the streaming content.

15. The method according to claim 12, further comprising: receiving, from the user device, a third request to terminate streaming the content; and closing the connection established between the source device and the computer system in response to receiving the third request.

16. The method according to claim 10, further comprising at least one of: transcoding a codec of the content streamed from the source device; reducing a data size of the content streamed from the source device; and removing part of information from the content streamed from the source device.

17. The method according to claim 10, further comprising: providing an API to receive the first request from the user device.

Images