SYSTEM AND METHOD FOR SIMULTANEOUS MULTIVIEW VIDEO - Patent application

Abstract

A simultaneous multi-view video system is disclosed that is capable of receiving a first video stream from a first device, receiving a second video stream from a second device, and receiving a third video stream from a third device, extracting timing data from each of the first video stream, the second video stream, and the third video stream, determining time signatures associated with each of the first video stream, the second video stream, and the third video stream based on the timing data, generating post-processed data based on each of the first video stream, the second video stream, and the third video stream, aligning one or more time signatures encoded on the post-processed data, and generating a production data file including a plurality of time-synchronized video streams corresponding to each of the first video stream, the second video stream, and the third video stream.

Description

[Technical field]

[0001] Cross-reference to related applications

[0001] This application claims priority to and benefit from U.S. Provisional Patent Application No. 63 / 127,419, entitled “SYSTEMS AND METHODS FOR SIMULTANEOUS MULTIPLE POINT OF VIEW VIDEO,” filed December 18, 2020, which application is hereby incorporated by reference in its entirety for all purposes. Field

[0002] The present disclosure relates to video streaming tools, and more particularly, to video streaming tools for processing and integrating multiple simultaneous video streams. [Background technology]

[0002]

[0003] Traditional video recording and streaming systems often only provide one perspective of a given event. Traditional systems can synchronize multiple image streams based on simultaneously recorded audio data. In this regard, traditional systems can introduce synchronization errors in response to audio quality issues, room dynamics, or other external factors. Summary of the Invention

[0003]

[0004] In various embodiments, systems, methods, and articles of manufacture (collectively, the "system") for simultaneous multi-perspective video are disclosed. The system can receive a first video stream from a first device, a second video stream from a second device, and a third video stream from a third device. The system can extract timing data from each of the first video stream, the second video stream, and the third video stream. The system can determine a time signature associated with each of the first video stream, the second video stream, and the third video stream based on the timing data. The system can generate post-processing data based on each of the first video stream, the second video stream, and the third video stream. The system can align one or more time signatures encoded on the post-processing data and generate a production data file. The production data file includes a plurality of time-synchronized video streams corresponding to each of the first video stream, the second video stream, and the third video stream.

[0004]

[0005] In various embodiments, the system can perform operations or manipulations of the raw image data. In various embodiments, the system can enable active viewing, via a mobile device, of each of the first video stream, the second video stream, and the third video stream. In various embodiments, the system can enable selection and simultaneous display, via a mobile device, of each of the multiple time-synchronized video streams. In various embodiments, the timing data is WebRTC timing data. In various embodiments, the system can include a stream fusion module configured to synchronize and align data streams from the multiple image sensors. In various embodiments, the system can also include an application programming interface module, a data handler module, a video processing module, a user interface module, and a database module.

[0005]

[0006] The above features and elements may be combined in various combinations, and these combinations are not excluded, unless expressly stated otherwise herein. These features and elements, as well as the operation of the disclosed embodiments, will become more apparent with reference to the following description and accompanying drawings. [Brief description of the drawings]

[0006]

[0007] The subject matter of the present disclosure is particularly pointed out and distinctly claimed in the concluding portion of this specification. However, a more complete understanding of the present disclosure can be obtained by reference to the detailed description and claims when considered in conjunction with the drawings, in which like numerals refer to like elements. [Figure 1] FIG. 1 is a block diagram illustrating various system components of a simultaneous multi-view video system, in accordance with various exemplary embodiments. [Diagram 2] FIG. 1 is a block diagram illustrating services of a simultaneous multi-view video system in accordance with various exemplary embodiments. [Diagram 3] 1 is a sign in interface for a simultaneous multi-view video system according to various exemplary embodiments. [Figure 4] 1 is a device selection interface for a simultaneous multi-view video system according to various exemplary embodiments. [Diagram 5] 1 is a device selection interface for a simultaneous multi-view video system according to various exemplary embodiments. [Figure 6] 1 illustrates a recording interface of a simultaneous multi-view video system according to various exemplary embodiments. [Figure 7] 1 illustrates a recording interface of a simultaneous multi-view video system according to various exemplary embodiments. [Figure 8] 1 is a device selection interface for a simultaneous multi-view video system according to various exemplary embodiments. [Figure 9] 1 is a device selection interface for a simultaneous multi-view video system according to various exemplary embodiments. [Figure 10] 1 illustrates a recording interface of a simultaneous multi-view video system according to various exemplary embodiments. [Figure 11] 1 illustrates a recording interface of a simultaneous multi-view video system according to various exemplary embodiments. [Figure 12] 1 illustrates a recording interface of a simultaneous multi-view video system according to various exemplary embodiments. [Figure 13] 1 illustrates a recording interface of a simultaneous multi-view video system according to various exemplary embodiments. [Figure 14] 1 illustrates a configuration interface for a simultaneous multi-view video system according to various exemplary embodiments. [Figure 15] 1 illustrates a configuration interface for a simultaneous multi-view video system according to various exemplary embodiments. [Figure 16] 1 is an illustration of a sign-out interface for a simultaneous multi-view video system in accordance with various exemplary embodiments. [Figure 17] 1 is a profile interface of a simultaneous multi-view video system according to various exemplary embodiments. [Figure 18] 1 is a viewing interface for a simultaneous multi-view video system according to various exemplary embodiments. [Figure 19] 1 illustrates a configuration interface for a simultaneous multi-view video system according to various exemplary embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007]

[0027] In the detailed description of the various embodiments herein, reference is made to the accompanying drawings and screen shots. The accompanying drawings show various embodiments as illustrations. Although the various embodiments are described in sufficient detail to enable one skilled in the art to practice the present disclosure, it should be understood that other embodiments may be realized and that logical and mechanical changes may be made without departing from the spirit and scope of the present disclosure. That is, the detailed description herein is presented for purposes of illustration only and not limitation. For example, in any method or process description, the steps recited therein may be performed in any order and are not necessarily limited to the order presented. Furthermore, any function or step may be outsourced to or performed by one or more third parties. Furthermore, whenever a singular reference is made, the plural embodiment is also included, and whenever a reference is made to more than one component, one embodiment is also included.

[0008]

[0028] As used herein, “electronic communication” means communication with at least a portion of an electronic signal being physically coupled (e.g., “telecommunication” or “electrically coupled”) and / or communication without physical coupling but by electromagnetic fields (e.g., “inductive communication” or “inductively coupled” or “inductive coupling”). As used herein, “transmit” may include sending at least a portion of electronic data from one system component to another system component (e.g., via a network connection). Additionally, as used herein, “data,” “information,” and the like may include encompassing information, such as commands, queries, files, messages, data for storage, and the like, in digital or any other form.

[0009]

[0029] As used herein, "satisfy," "meet," "match," "associated with," or similar terms can include an exact match, a partial match, meeting certain criteria, matching a subset of data, correlation, satisfying certain criteria, correspondence, association, algorithmic relationship, etc. Similarly, as used herein, "authenticate" or similar terms can include exact authentication, partial authentication, authenticating a subset of data, correspondence, satisfying certain criteria, association, algorithmic relationship, etc.

[0010]

[0030] Systems, methods, and computer program products are provided. In the detailed description herein, when reference is made to "various embodiments," "one embodiment," "an embodiment," "example embodiment," and the like, it is intended to indicate that the described embodiment most likely includes a particular feature, structure, or characteristic, but that not all embodiments necessarily include that particular feature, structure, or characteristic. Moreover, such phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in connection with one embodiment, it is submitted that it is within the knowledge of one skilled in the art to implement such feature, structure, or characteristic in connection with other embodiments, whether or not expressly described. After reading the description, it will be apparent to one skilled in the art how to implement the present disclosure in alternative embodiments.

[0011]

[0031] In various embodiments, and with reference to FIG. 1, a simultaneous multi-view video synchronization system 100 is illustrated. The system 100 can include various computing devices, software modules, networks, and data structures that communicate with each other. The system 100 can also be used in conjunction with web services, utility computing, pervasive and individualized computing, security and identity solutions, autonomous computing, cloud computing, commodity computing, mobility and wireless solutions, open source, biometrics, grid computing, and / or mesh computing. The system 100 can include an application server 102, a user device 104, and a service 200. Any of these components can be outsourced and / or communicate with the data comparison application server 102 and / or the service 200 through a network, such as a first network 106 and a second network 108.

[0012]

[0032] The system 100 may be computer-based and may comprise a processor, a tangible non-transitory computer-readable memory, and / or a network interface, along with other suitable system software and hardware components. Instructions stored on the tangible non-transitory memory may cause the system 100 to perform various functions as described herein. In various embodiments, the application server 102 and / or the Flysview service (i.e., service 200) may be configured as a central network element or hub to access the various systems, engines, and components of the system 100. The application server 102 may comprise a network (e.g., network 106), a computer-based system, and / or software components configured to provide an access point to the various systems, engines, and components of the system 100. The application server 102 may operatively and / or electronically communicate with the user device 104 through the first network 106 and with the service 200 through the second network 108. In this regard, the application server 102 may facilitate communication from the user device 104 to the systems, engines, and components of the system 100, such as, for example, the services 200. In various embodiments, the application server 102 may receive commands and / or metadata from the user device 104 and may pass responses to the user device 104.

[0013]

[0033] In various embodiments, the application server 102 may include one or more computing devices, rack mounted servers, and / or virtual machines as described above and may provide load balancing, application services, web services, data query services, data transfer services, reverse proxy services, or other services that facilitate the delivery and receipt of data across the networks (106, 108).

[0014]

[0034] In various embodiments, the user device 104 may comprise software and / or hardware that communicates with the system 100 over a network (e.g., the network 106) and may comprise hardware and / or software configured to allow a user or the like to access the application server 102. The user device 104 may comprise any suitable device configured to allow a user to communicate with the network and the system 100. The user device may include, for example, a personal computer, a personal digital assistant, a cellular phone, a kiosk, a mobile device, or the like, and may allow a user to send a comparison request to the system 100. In various embodiments, the user device 104 may include an image sensor and / or a video recording device, such as, for example, a charge-coupled device. In various embodiments, the user device 104 described herein may execute web applications or native applications to communicate with the application server 102. The native application 110 may be installed on the user device 104, for example, via a download, a physical medium, or an app store. The native application 110 may utilize a development code base provided for use with an operating system and may execute system calls to manipulate data stored or displayed on the user device 104 and to communicate with the application server 102. A web application may be a web browser application that is compatible with and specifically written to run on a web browser. That is, a web application may be a browser-based application that operates in conjunction with the application server 102.

[0015]

[0035] In various embodiments, the native applications 110 executing on the user devices 104 can communicate with the application server 102 to support real-time updates. For example, data on the server 200 can be synchronized across various user devices 104 used by any number of users interacting with the application server 102 and / or the services 200. In this regard, the application server 102 can serve data from the services 200 to each of the user devices 104 and can also serve commands from the user devices 104 to the services 200. In various embodiments, the application server 102 can apply access permissions to restrict data transmitted across the networks (106, 108) and / or between various components of the system 100. A user can authenticate on the native application 110 by, for example, a username, password, two-factor authentication, a secret encryption key, a one-time password, security questions, biometrics, or other suitable authentication techniques known to those skilled in the art.

[0016]

[0036] 2, a block diagram of a service 200 of the system 100 is shown in accordance with various embodiments. The service 200 can include a software bus architecture 202, an application programming interface (i.e., API module 204), a data handler 206 module, a video processing module 208, a stream fusion module 210, a user interface module 212, and a database module 214.

[0017]

[0037] In various embodiments, API module 204 can be configured to provide a programmatic interface to any of the collections of system 100, service 200 services, components, modules, and / or engines.

[0018]

[0038] In various embodiments, the data handler 206 is configured to obtain data from one or more image sensors and process the data into a number of data feeds 207 for use by the various systems, engines, and components of the services 200. In various embodiments, the data feeds 207 may be real-time data feeds, such as video streams. The data handler may be capable of integrating with a variety of data sources. The data handler may extract raw timing data from the data feeds.

[0019]

[0039] In various embodiments, the video processing module 208 can be configured to collect and process data from the data handler 206. The video processing module 208 can receive raw image data 216 and perform various processes and manipulations of the raw image data 216. The video processing module 208 can generate post-processed data (PP data 218) for later use by the stream blending module 210.

[0020]

[0040] In various embodiments, the stream blending module 210 comprises an engine configured to synchronize and align streams from image sensors of various user devices. The stream blending module 210 can align one or more time signatures encoded in the PP data 218 to generate a production data 220 file. The production data 220 file includes multiple time-synchronized video streams corresponding to each of the raw data feeds. In various embodiments, the stream blending module 210 can generate the production data based on WebRTC timing data extracted by the data handler 206.

[0021]

[0041] In various embodiments, the user interface module 212 can provide output from the service 200 to the user device 104. In various embodiments, the user interface module 212 can provide output through two channels, such as, for example, a web-based application or a native application 110. The user interface may allow active viewing of a real-time stream from an image sensor. The user interface may allow on-demand viewing of stored production data 220. The user interface may allow selection and simultaneous display of one or more synchronized video streams.

[0022]

[0042] In various embodiments, the database module 214 can include any number of data structures or data elements, such as, for example, raw data 216, PP data 218, and production data 220. The database module 214 can be configured to maintain raw data 216, such as streaming data from an image sensor. The database module 214 can also be configured to maintain PP data 218, such as data extracted by the data handler 207 and already processed by the video processing module 208. The database module 214 can also be configured to maintain production data 220, such as generated by the stream fusion module 210.

[0023]

[0043] Benefits, other advantages, and solutions to problems have been described herein with respect to specific embodiments. However, any of the benefits, advantages, solutions to problems, and elements that may cause or make any benefit, advantage, or solution more pronounced should not be construed as essential, required, or essential features or elements of the disclosure. The scope of the disclosure is therefore not intended to be limited except by the appended claims. In the claims, reference to an element in the singular is intended to mean "one or more," rather than "one and only," unless expressly so stated. Furthermore, when phrases similar to "at least one of A, B, and C" or "at least one of A, B, or C" are used in the claims or specification, this is intended to mean that in an embodiment only A may be present, in an embodiment only B may be present, in an embodiment only C may be present, or in an embodiment any combination of elements A, B, and C may be present, e.g., A and B, A and C, B and C, or A and B and C. It is contemplated that the present disclosure includes methods, which may be embodied as computer program instructions on a tangible computer readable carrier, such as a magnetic or optical memory or a magnetic or optical disk. All structural, chemical, and functional equivalents of the elements of the various embodiments described above that are known to those skilled in the art are expressly incorporated herein by reference and are intended to be encompassed by the claims of the present application. Moreover, it is not necessary for a device or method to address each and every problem sought to be solved by the present disclosure, but rather to be encompassed by the claims of the present application. Furthermore, no element, component, or method step in this disclosure is intended to be dedicated to the public, regardless of whether that element, component, or method step is expressly recited in a claim.If a claim element is not expressly recited using the phrase "means for" or "step for," it is not intended that the element be invoked under 35 U.S.C. § 112(f). As used herein, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus making up a list of elements may include not only those elements, but also other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0024]

[0044] The term "non-transitory" should be understood to exclude only propagating transitory signals, per se, from the claims, and not to disclaim all standard computer-readable media other than propagating transitory signals, per se. In other words, the meaning of the terms "non-transitory computer-readable medium" and "non-transitory computer-readable storage medium" should be interpreted to exclude only those types of transitory computer-readable medium that In RE Nuijten found to fall outside the scope of patentable subject matter under 35 U.S.C. § 101.

[0025]

[0045] In various embodiments, the components, modules, and / or engines of system 100 may be implemented as micro-applications, or micro-apps. Micro-apps are typically deployed in the context of a mobile operating system, such as the WINDOWS® mobile operating system, the ANDROID® operating system, the APPLE® iOS operating system, the BLACKBERRY® Corporation operating system, and the like. The micro-apps may be configured to utilize the resources and associated hardware of a larger operating system, subject to a set of predefined rules that govern the operation of the various operating system and hardware resources. For example, if a micro-app desires to communicate with a device or network other than the mobile device or mobile operating system, the micro-app may utilize the operating system's communication protocols and associated device hardware, subject to the predefined rules of the mobile operating system. Additionally, if a microapp desires input from a user, the microapp can be configured to request a response from the operating system, which monitors various hardware components and then communicates detected input from the hardware to the microapp.

[0026]

[0046] The foregoing systems and methods may be described herein in terms of functional block components, screen shots, options, and various processing steps. It should be appreciated that such functional blocks may be implemented by any number of hardware and / or software components configured to perform the specified functions. For example, the system may employ various integrated circuit components, such as memory elements, processing elements, logic elements, look-up tables, etc., that may perform various functions under the control of one or more microprocessors or other control devices. Similarly, the software elements of the system may be implemented by any programming or scripting language, such as C, C++, C#, Java, JavaScript, JavaScript Object Notation (JSON), VBScript, Macromedia Cold Fusion, COBOL, Microsoft Active Server Pages, assembly, PERL, PHP, awk, Python, Visual Basic, SQL Stored Procedures, PL / SQL, any UNIX shell script, and Extensible Markup Language (XML), and the various algorithms may be implemented by any combination of data structures, objects, processes, routines, or other programming elements. It should further be noted that the system may employ any number of conventional techniques for data transmission, signaling, data processing, network control, and the like. Furthermore, the system may be used to detect or prevent security issues with client-side scripting languages, such as JavaScript, VBScript, and the like.

[0027]

[0047] The systems and methods are described herein with reference to screen shots, block diagrams, and flowchart illustrations of methods, apparatus, and computer program products according to various embodiments. It will be understood that each functional block of the block diagrams and flowchart illustrations, and combinations of functional blocks in the block diagrams and flowchart illustrations, respectively, can be implemented by computer program instructions.

[0028]

[0048] Thus, the functional blocks of the block diagrams and flowchart representations support combinations of means for performing the specified functions, combinations of steps for performing the specified functions, and program instruction means for performing the specified functions. It will also be appreciated that each functional block of the block diagrams and flowchart representations, and combinations of functional blocks in the block diagrams and flowchart representations, can be implemented by either special purpose hardware-based computer systems that perform the specified functions or steps, or by suitable combinations of special purpose hardware and computer instructions. Additionally, the process flow representations and descriptions thereof may also refer to user WINDOWS® applications, web pages, websites, web forms, prompts, and the like. Practitioners will appreciate that the steps described and illustrated herein may be comprise in any number of configurations, including the use of WINDOWS® applications, web pages, web forms, pop-up WINDOWS® applications, prompts, and the like. It should also be appreciated that what has been shown or described as multiple steps has been expanded for simplicity only and may be combined into one web page and / or WINDOWS® application. In other cases, steps illustrated and described as a single process step have only been combined for simplicity and may be separated into multiple web pages and / or WINDOWS® applications.

[0029]

[0049] In various embodiments, the software elements of the system may be implemented using a JAVASCRIPT® runtime environment configured to execute JAVASCRIPT® code outside of a web browser. For example, the software elements of the system may be implemented using a NODE.JS® component. The NODE.JS® program may implement various modules to operate various core functions. For example, a package management module such as NPM® may be implemented as an open source library to aid in orchestrating the installation and management of third-party NODE.JS® programs. The NODE.JS® program may also implement a process manager, such as Parallel Multithreaded Machines ("PM2"), resource and performance monitoring tools, such as Node Application Metrics ("appmerics"), library modules for building user interfaces, and / or any other suitable and / or desired modules.

[0030]

[0050] Middleware may include any hardware and / or software suitably configured to facilitate communication and / or process transactions between disparate computing systems. Middleware components are commercially available and known in the art. Middleware may be implemented with commercially available hardware and / or software, with custom hardware and / or software components, or with a combination thereof. Middleware may reside in a variety of configurations, may exist as a stand-alone system, or may be a software component residing in an Internet server. Middleware may be configured to process transactions between various components of an application server and any number of internal or external systems for any of the purposes disclosed herein. WEBSPHERE® MQTM (formerly MQSeries) by IBM® Corporation (Armonk, NY) is an example of a commercially available middleware product. Enterprise Service Bus ("ESB") applications are other examples of middleware.

[0031]

[0051] The computers discussed herein may also provide suitable web sites or other internet-based graphical user interfaces that are accessible by users. In one embodiment, MICROSOFT Corporation's Internet Information Server (IIS), Transaction Server (MTS) services, and SQL SERVER database are used in conjunction with the MICROSOFT operating system, WINDOWS NT web server software, SQL SERVER database, and MICROSOFT commercial servers. In addition, components such as ACCESS software, SQL SERVER database, ORACLE software, SYBASE software, INFORMIX software, MYSQL software, INTERBASE software, and the like, may also be used to provide an Active Data Objects (ADO) compliant database management system. In one embodiment, the APACHE® web server is used in conjunction with the LINUX® operating system, a MYSQL® database, and the PERL®, PHP, Ruby, and / or PYTHON® programming languages.

[0032]

[0052] For the sake of brevity, conventional data networking, application development, and other functional aspects of the present system (as well as components of the individual operating components of the system) may not be described in detail herein. Furthermore, the connecting lines shown in the various figures contained herein are intended to represent example functional relationships and / or physical couplings between the various elements. It should be noted that many alternative or additional functional relationships or physical connections may be present in an actual system.

[0033]

[0053] In various embodiments, the methods described herein are implemented using various specific machines described herein. The methods described herein can be implemented using the following specific machines and machines developed in the future in any suitable combination as would be readily recognized by one skilled in the art. Furthermore, as will be apparent from this disclosure, the methods described herein can result in obtaining various variations of certain articles.

[0034]

[0054] The various system components discussed herein may include one or more of the following: a host server or other computing system including a processor for processing digital data, a memory coupled to the processor for storing digital data, an input digitizer coupled to the processor for inputting digital data, an application program stored in the memory and accessible by the processor to direct the processing of the digital data by the processor, a display device coupled to the processor and the memory for displaying information derived from the digital data processed by the processor, and a plurality of databases. As used herein, the various databases may include client data, merchant data, financial institution data, and / or similar data useful in the operation of the system. As will be appreciated by those skilled in the art, a user computer may include an operating system (e.g., WINDOWS, UNIX, LINUX, SOLARIS, MACOS, etc.), as well as various conventional support software and drivers that typically accompany a computer.

[0035]

[0055] The system, or any part(s) or function(s) thereof, may be implemented using hardware, software, or a combination thereof, and may be implemented in one or more computer systems or other processing systems. However, the operations performed by the embodiments may be expressed in terms such as matching or selecting, and are generally associated with mental operations performed by a human operator. In most cases, no such human operator capability is necessary or desirable for any of the operations described herein. Rather, the operations may be machine operations, any of which may be performed or improved by artificial intelligence (AI) or machine learning. Artificial intelligence may generally refer to the study of agents (e.g., machines, computer-based systems, etc.) that perceive the world around them, form plans, and make decisions to achieve their goals. Foundations of AI include mathematics, logic, philosophy, probability, linguistics, neuroscience, and decision theory. Many fields, such as computer vision, robotics, machine learning, and natural language processing, fall under the umbrella of the field of AI. Useful machines for implementing various embodiments include general purpose digital computers or similar devices.

[0036]

[0056] The computer system may also include a communications interface. The communications interface allows software and data to be transferred between the computer system and external devices. Examples of such communications interfaces may include a modem, a network interface (such as an Ethernet card), a communications port, and the like. The software and data transferred through the communications interface are in the form of signals and may be electronic, electromagnetic, optical, or other signals that can be received by the communications interface. These signals are provided to the communications interface through a communications path (e.g., a channel). The channel carries the signals and may be implemented using wires, cables, optical fibers, telephone lines, cellular links, radio frequency (RF) links, wireless, and other communications channels.

[0037]

[0057] In various embodiments, the server may include an application server (e.g., WEBSPHERE®, WEBLOGIC®, JBOSS®, POSTGRES PLUS ADVANCED SERVER®, etc.). In various embodiments, the server may include a web server (e.g., Apache, IIS, GOOGLE® Web Server, SUN JAVA® System Web Server, JAVA® Virtual Machine running on a LINUX® or WINDOWS® operating system).

[0038]

[0058] A web client includes any device or software that communicates over any network, such as, for example, any device or software discussed herein. A web client can include Internet browsing software installed in a computing unit or system to perform online transactions and / or communications. These computing units or systems can take the form of a computer or set of computers, although other types of computing units or systems can be used, including personal computers, laptops, notebooks, tablet smart phones, cellular phones, personal digital assistants, servers, pooled servers, mainframe computers, distributed computing clusters, kiosks, terminals, point-of-sale (POS) devices or terminals, televisions, or any other device capable of receiving data over a network. The web client may include an operating system (e.g., WINDOWS®, WINDOWS MOBILE® operating system, UNIX® operating system, LINUX® operating system, APPLE® OS® operating system, etc.) as well as various conventional support software and drivers that typically accompany a computer. The web client may also run MICROSOFT® INTERNET EXPLORER® software, MOZILLA® FIREFOX® software, GOOGLE CHROME™ software, APPLE® SAFARI® software, or any other of the myriad software packages available for browsing the Internet.

[0039]

[0059] As will be appreciated by those skilled in the art, a web client may or may not be in direct contact with a server (e.g., an application server, a web server, etc., as discussed herein). For example, a web client may access the services of a server through other servers and / or hardware components that may have direct or indirect connections to an Internet server. For example, a web client may communicate with a server through a load balancer. In various embodiments, the web client access is through a network or the Internet using a commercially available web browser software package. In this regard, the web client may be in a home or business environment with access to a network or the Internet. The web client may implement security protocols such as Secure Sockets Layer (SSL) and Transport Layer Security (TLS). The web client may implement a variety of application layer protocols, including HTTP, HTTPS, FTP, and SFTP.

[0040]

[0060] The various system components may be appropriately coupled to a network, independently, separately, or collectively, via a data link, including, for example, a connection through a local loop to an Internet Service Provider (ISP) as commonly used in conjunction with standard modem communications, cable modem, DISH NETWORK®, ISDN, Digital Subscriber Line (DSL), or various wireless communication methods. It is noted that the network may be embodied as other types of networks, such as an interactive television (ITV) network. Additionally, the system contemplates the use, sale, or distribution of any goods, services, or information over any network having similar functionality as described herein.

[0041]

[0061] The system also contemplates use in connection with web services, utility computing, pervasive and personalized computing, security and identity solutions, autonomous computing, cloud computing, commodity computing, mobility and wireless solutions, open source, biometrics, grid computing, and / or mesh computing.

[0042]

[0062] Any of the communication, input, storage, database, or display discussed herein can be facilitated through a website having web pages. The term "web page" as used herein is not intended to limit the types of documents and applications that may be used to interact with a user. For example, a typical website may contain, in addition to standard HTML documents, various forms, JAVA applets, JAVASCRIPT programs, Active Server Pages (ASP), Common Gateway Interface Script (CGI), Extensible Markup Language (XML), Dynamic HTML, Cascading Style Sheets (CSS), AJAX (Asynchronous Javascript and XML) programs, helper applications, plug-ins, and the like. The server may include a web service that receives requests from a web server, which requests include a URL and an IP address (192.168.1.1). The web server retrieves the appropriate web pages and sends the data or applications for those web pages to the IP address. A web service is an application that can interact with other applications through a communication medium such as the Internet. Web services are typically based on standards or protocols such as XML, SOAP, AJAX, WSDL, and UDDI. Web service methods are well known in the art and are well covered in many standard texts. For example, representational state transfer (REST), or RESTful, web services can provide one way to enable interoperability between applications.

[0043]

[0063] The Web client computing unit may also be equipped with an Internet browser connected to the Internet or an intranet using standard dial-up, cable, DSL, or any other Internet protocol known in the art. Transactions initiated at the Web client may pass through a firewall to prevent unauthorized access from users of other networks. Additional firewalls may also be deployed between various components of the CMS for added security.

[0044]

[0064] Encryption can be performed by any of the techniques currently available in the art or that may become available, such as Twofish, RSA, El Gamal, Schorr signature, DSA, PGP, PKI, GPG (GnuGP), HPE format-preserving encryption (FPE), Voltage, TripleDES, Blowfish, AES, MD5, HMAC, IDEA, RC6, and symmetric and asymmetric cryptosystems. The systems and methods can also incorporate the SHA series of encryption methods, Elliptic Curve Cryptography (e.g., ECC, ECDH, ECDSA, etc.), and / or other post-quantum cryptography algorithms in development.

[0045]

[0065] A firewall may include any hardware and / or software appropriately configured to protect the CMS components and / or enterprise computing resources from users of other networks. Additionally, a firewall may be configured to limit or restrict access to various systems and components behind the firewall for web clients connecting through the web server. Firewalls may exist in a variety of configurations including stateful inspection, proxy-based access control lists, and packet filtering, among others. A firewall may be integrated into the web server or any other CMS component, or may exist as a separate entity. A firewall may implement network address translation ("NAT") and / or network address port translation ("NAPT"). A firewall may employ various tunneling protocols to facilitate secure communications, such as those used in virtual private networking. A firewall may implement a demilitarized zone ("DMZ") to facilitate communications with public networks, such as the Internet. A firewall may be integrated as software within an Internet server or any other application server component, may reside within a separate computing device, or may take the form of a stand-alone hardware component.

[0046]

[0066] Any of the databases discussed herein may include relational, hierarchical, graphical, blockchain, object-oriented structures, and / or any other database configuration. Any of the databases may also include a flat file structure. In a flat file structure, data may be stored in a single file in the form of rows and columns, with no indexing structure and no structural relationships between records. For example, a flat file structure may include a delimited text file, a CSV (comma-separated value) file, and / or any other suitable flat file structure. Common database products that can be used to implement the database include DB2® by IBM® (Armonk, NY), various database products available from ORACLE® Corporation (Redwood Shores, CA), MICROSOFT ACCESS® or MICROSOFT SQL SERVER® by MICROSOFT® Corporation (Redmond, WA), MYSQL® by MySQL AB (Uppsala, Sweden), MONGODB®, Radis, APACHE CASSANDRA®, HBASE® by APACH®, MapR-DB by MAPR®, Inc., or any other suitable database product. Furthermore, any of the databases may be organized in any suitable manner, for example, as a data table or a lookup table. Each record may be a file, a series of files, a series of linked data fields, or any other data structure.

[0047]

[0067] The present disclosure and claims do not merely recite a particular outcome of a system for failure detection tools, but also include specific rules for implementing the outcome of the system for failure detection tools, and specific rules for expressing information in a particular format and using and applying this format to produce the desired outcome of the system for failure detection tools, as set forth in McRO,Inc v.Bandai Namco Games America Inc. (Fed. Cir. case number 15-1080, Sept 13, 2016). In other words, the outcome of the system for failure detection tools can be implemented by many different types of rules and combinations of rules, and the present disclosure includes various embodiments with specific rules. While a lack of complete preemption is not necessarily a guarantee that a claim is eligible, the present disclosure does not fully anticipate the field of failure detection tool systems at all. The present disclosure acts to narrow, restrict, or otherwise constrain the disclosure so as not to fall under the entire abstract idea of ​​a mere failure detection tool system. Importantly, because there are other systems and methods for fault detection tool systems, it would be inappropriate to assert that the claimed invention preempts the field of fault detection tool systems or monopolises the basic tools of fault detection tool systems. In other words, the present disclosure does not prevent others from fault detection tool systems because other systems already perform the functions in a manner different from the claimed invention. Moreover, the inventive concept contained in the claimed invention is found in the non-conventional and nongeneric configuration of known conventional pieces, consistent with Bascom v. AT&T Mobility, 2015-1763 (Fed. Cir. 2016).The present disclosure and claims go beyond any conventionality of any of these systems, since the interactions and synergies of the systems result in additional functionality not provided by any of the systems operating independently. The present disclosure and claims can include interactions between multiple different systems, so the present disclosure cannot be considered a comprehensive computer implementation, or an implementation that simply "applies it" to an abstract process. The present disclosure and claims can also cover software improvements through specific implementations of solutions to problems in software technology.

Claims

1. 1. A method comprising: receiving, by a computer-based system, a first video stream from a first device; receiving, by the computer-based system, a second video stream from a second device; receiving, by the computer-based system, a third video stream from a third device; extracting, by the computer-based system, timing data from each of the first video stream, the second video stream, and the third video stream; determining, by the computer based system, a time signature associated with each of the first video stream, the second video stream, and the third video stream based on the timing data; generating, by the computer-based system, post-processing data based on each of the first video stream, the second video stream, and the third video stream; aligning, by the computer-based system, one or more time signatures encoded on the post-processed data to generate a production data file including a plurality of time-synchronized video streams corresponding to each of the first video stream, the second video stream, and the third video stream; A method comprising:

2. 10. The method of claim 1, further comprising the step of performing, by said computer-based system, processing or manipulation of raw image data.

3. 10. The method of claim 1, further comprising the step of enabling, by the computer-based system, active viewing, via a mobile device, of each of the first video stream, the second video stream, and the third video stream.

4. 10. The method of claim 1, further comprising the step of enabling, by the computer-based system, selection and simultaneous display, via a mobile device, of each of the plurality of time-synchronized video streams.

5. 2. The method of claim 1, wherein the timing data is WebRTC timing data.

6. 10. The method of claim 1, further comprising configuring a stream fusion module to synchronize and align data streams from multiple image sensors.

Images