1687 results about "Stream processing" patented technology

A flow processing facility, which uses a set of artificial neurons for pattern recognition, such as a self-organizing map, in order to provide security and protection to a computer or computer system supports unified threat management based at least in part on patterns relevant to a variety of types of threats that relate to computer systems, including computer networks. Flow processing for switching, security, and other network applications, including a facility that processes a data flow to address patterns relevant to a variety of conditions are directed at internal network security, virtualization, and web connection security. A flow processing facility for inspecting payloads of network traffic packets detects security threats and intrusions across accessible layers of the IP-stack by applying content matching and behavioral anomaly detection techniques based on regular expression matching and self-organizing maps. Exposing threats and intrusions within packet payload at or near real-time rates enhances network security from both external and internal sources while ensuring security policy is rigorously applied to data and system resources. Intrusion Detection and Protection (IDP) is provided by a flow processing facility that processes a data flow to address patterns relevant to a variety of types of network and data integrity threats that relate to computer systems, including computer networks.

A high level transformation method and apparatus for converting data formats in the context of network applications, among other places. A flexible transformation mechanism is provided that facilitates generation of translation machine code. A translator is dynamically generated by a translator compiler engine. When fed an input stream, the translator generates an output stream by executing the native object code generated on the fly by the translator compiler engine. In addition, the translator may be configured to perform a bi-directional translation between the two streams as well as translation between two distinct protocol sequences. Further a translator may working in streaming mode, to facilitate streaming processing of documents.

A flow processing facility, which uses a set of artificial neurons for pattern recognition, such as a self-organizing map, in order to provide security and protection to a computer or computer system supports unified threat management based at least in part on patterns relevant to a variety of types of threats that relate to computer systems, including computer networks. Flow processing for switching, security, and other network applications, including a facility that processes a data flow to address patterns relevant to a variety of conditions are directed at internal network security, virtualization, and web connection security. A flow processing facility for inspecting payloads of network traffic packets detects security threats and intrusions across accessible layers of the IP-stack by applying content matching and behavioral anomaly detection techniques based on regular expression matching and self-organizing maps. Exposing threats and intrusions within packet payload at or near real-time rates enhances network security from both external and internal sources while ensuring security policy is rigorously applied to data and system resources. Intrusion Detection and Protection (IDP) is provided by a flow processing facility that processes a data flow to address patterns relevant to a variety of types of network and data integrity threats that relate to computer systems, including computer networks.

A flow processing facility, which uses a set of artificial neurons for pattern recognition, such as a self-organizing map, in order to provide security and protection to a computer or computer system supports unified threat management based at least in part on patterns relevant to a variety of types of threats that relate to computer systems, including computer networks. Flow processing for switching, security, and other network applications, including a facility that processes a data flow to address patterns relevant to a variety of conditions are directed at internal network security, virtualization, and web connection security. A flow processing facility for inspecting payloads of network traffic packets detects security threats and intrusions across accessible layers of the IP-stack by applying content matching and behavioral anomaly detection techniques based on regular expression matching and self-organizing maps. Exposing threats and intrusions within packet payload at or near real-time rates enhances network security from both external and internal sources while ensuring security policy is rigorously applied to data and system resources. Intrusion Detection and Protection (IDP) is provided by a flow processing facility that processes a data flow to address patterns relevant to a variety of types of network and data integrity threats that relate to computer systems, including computer networks.

A flow processing facility, which uses a set of artificial neurons for pattern recognition, such as a self-organizing map, in order to provide security and protection to a computer or computer system supports unified threat management based at least in part on patterns relevant to a variety of types of threats that relate to computer systems, including computer networks. Flow processing for switching, security, and other network applications, including a facility that processes a data flow to address patterns relevant to a variety of conditions are directed at internal network security, virtualization, and web connection security. A flow processing facility for inspecting payloads of network traffic packets detects security threats and intrusions across accessible layers of the IP-stack by applying content matching and behavioral anomaly detection techniques based on regular expression matching and self-organizing maps. Exposing threats and intrusions within packet payload at or near real-time rates enhances network security from both external and internal sources while ensuring security policy is rigorously applied to data and system resources. Intrusion Detection and Protection (IDP) is provided by a flow processing facility that processes a data flow to address patterns relevant to a variety of types of network and data integrity threats that relate to computer systems, including computer networks.

A continuous query parallel engine on data streams provides scalability and increases the throughput by the addition of new nodes. The parallel processing can be applied to data stream processing and complex events processing. The continuous query parallel engine receives the query to be deployed and splits the original query into subqueries, obtaining at least one subquery; each subquery is executed in at least in one node. Tuples produced by each operator of each subquery are labeled with timestamps. A load balancer is interposed at the output of each node that executes each one of the instances of the source subquery and an input merger is interposed in each node that executes each one of the instances of a destination subquery. After checks are performed, further load balancers or input managers may be added.

A flow processing facility, which uses a set of artificial neurons for pattern recognition, such as a self-organizing map, in order to provide security and protection to a computer or computer system supports unified threat management based at least in part on patterns relevant to a variety of types of threats that relate to computer systems, including computer networks. Flow processing for switching, security, and other network applications, including a facility that processes a data flow to address patterns relevant to a variety of conditions are directed at internal network security, virtualization, and web connection security. A flow processing facility for inspecting payloads of network traffic packets detects security threats and intrusions across accessible layers of the IP-stack by applying content matching and behavioral anomaly detection techniques based on regular expression matching and self-organizing maps. Exposing threats and intrusions within packet payload at or near real-time rates enhances network security from both external and internal sources while ensuring security policy is rigorously applied to data and system resources. Intrusion Detection and Protection (IDP) is provided by a flow processing facility that processes a data flow to address patterns relevant to a variety of types of network and data integrity threats that relate to computer systems, including computer networks.

A method of delivering a live stream is implemented within a content delivery network (CDN) and includes the high level functions of recording the stream using a recording tier, and playing the stream using a player tier. The step of recording the stream includes a set of sub-steps that begins when the stream is received at a CDN entry point in a source format. The stream is then converted into an intermediate format (IF), which is an internal format for delivering the stream within the CDN and comprises a stream manifest, a set of one or more fragment indexes (FI), and a set of IF fragments. The player process begins when a requesting client is associated with a CDN HTTP proxy. In response to receipt at the HTTP proxy of a request for the stream or a portion thereof, the HTTP proxy retrieves (either from the archive or the data store) the stream manifest and at least one fragment index. Using the fragment index, the IF fragments are retrieved to the HTTP proxy, converted to a target format, and then served in response to the client request. The source format may be the same or different from the target format. Preferably, all fragments are accessed, cached and served by the HTTP proxy via HTTP.

Disclosed are a method, upstream processing node, and computer readable medium for dynamically stabilizing a stream processing system. The method includes receiving at least one computing resource allocation target. The method further includes determining that an input data flow rate of at least one upstream processing element varies. The computing resource is dynamically allocated to the upstream processing element in response to the input rate of the upstream processing element varying. Data flow is dynamically controlled between the upstream processing element and at least one downstream processing element.

A network monitoring device includes a flow processing element, disposed to receive flow information relating to network flows, and to generate a set of virtual packets, each representing a portion of a network flow. The virtual packets are maintained in a time-sequential order, and read by elements of the network monitoring device to generate information relating to network traffic, such as symptoms affecting the communication network, problems affecting the communication network, and otherwise. The network monitoring device randomly samples virtual packets, with at least one of two effects: (1) flow information from traffic reporting devices that are themselves sampling at differing rates can be equalized, with the effect of standardizing information from all of them; (2) the network monitoring device itself can restrict its attention to a fraction of all virtual packets, with the effect of keeping up with a relatively large number of virtual packets.

A system and methodology for parallel processing of continuous data streams. In one embodiment, a system for parallel processing of data streams comprises: a converter receiving input streams of data in a plurality of formats and transforming the streams into a standardized data stream format comprising rows and columns in which values in a given column are of a homogeneous type; a storage system that continuously maintains a finite interval of each stream subject to specified space limits for the stream; an interface enabling a user to construct parallel stream programs for processing streams in the standardized data stream format, wherein a parallel stream program comprises a linear sequence of program building blocks for performing operations on a data stream; and a runtime computing system running multiple parallel stream programs continuously on the streams as they flow through the storage system.

A system and method for providing rapid rerouting of real-time transport protocol (RTP) multi-media flows is disclosed. Generally, a first endpoint is connected to a second endpoint, wherein the first endpoint comprises a transceiver, software stored within the first endpoint defining functions to be performed by the first endpoint, and a processor configured by the software. The processor is configured to perform the steps of, performing flow processing on a data packet received at a first endpoint, from a second endpoint, removing a multi-protocol label switching (MPLS) tag from the data packet, translating a source address and destination address of the data packet, and determining a forwarding destination if more than one destination address of the data packet is provided.

The present invention provides a system and methods for producing multiple encoded representations of a video input sequence. The multiple representations produced each contain identified synchronization frames that allow a server and a client to switch between streamed representations in real time without interruption. Synchronization frames are frames of encoded video that can be independently decoded. A representation can thus be decoded starting at a synchronization frame. Each synchronization frame in one representation has a corresponding synchronization frame at a substantially similar temporal location in any other generated representation of the same video input sequence. The temporal co-location of synchronization frames in all representations facilitates the dynamic switching between representations during the streaming process. The present invention also provides a video encoder application that shares data during the encoding of multiple representations of a video input sequence by reusing data calculated in the encoding of one representation to encode other representations. The application can also generate the multiple encoded representations simultaneously.

A high level transformation method and apparatus for converting data formats in the context of network applications, among other places. A flexible transformation mechanism is provided that facilitates generation of translation machine code. A translator is dynamically generated by a translator compiler engine. When fed an input stream, the translator generates an output stream by executing the native object code generated on the fly by the translator compiler engine. In addition, the translator may be configured to perform a bi-directional translation between the two streams as well as translation between two distinct protocol sequences. Further a translator may working in streaming mode, to facilitate streaming processing of documents.

A flow processing facility, which uses a set of artificial neurons for pattern recognition, such as a self-organizing map, in order to provide security and protection to a computer or computer system supports unified threat management based at least in part on patterns relevant to a variety of types of threats that relate to computer systems, including computer networks. Flow processing for switching, security, and other network applications, including a facility that processes a data flow to address patterns relevant to a variety of conditions are directed at internal network security, virtualization, and web connection security. A flow processing facility for inspecting payloads of network traffic packets detects security threats and intrusions across accessible layers of the IP-stack by applying content matching and behavioral anomaly detection techniques based on regular expression matching and self-organizing maps. Exposing threats and intrusions within packet payload at or near real-time rates enhances network security from both external and internal sources while ensuring security policy is rigorously applied to data and system resources. Intrusion Detection and Protection (IDP) is provided by a flow processing facility that processes a data flow to address patterns relevant to a variety of types of network and data integrity threats that relate to computer systems, including computer networks.

Systems and methods are provided for analyzing a data stream device at a packet level. In one embodiment, design logic is embedded within a target data stream processing device such as a networking device or video device. The embedded design logic is configured to start and stop the target data stream processing device and to inject data into the target data stream processing device for evaluation. The embedded design may also be configured to detect a user selected data value in a packet within the target data stream processing device before the packet is transmitted through the device's physical layer or passed up the device's networking stack. In addition, or in another embodiment, at least a portion of the design logic, including the ability to detect the user selected value in the packet, may be located external to the data stream processing device. In one embodiment, a controlled client / server interfaces with the physical layer of the target data stream processing device and provides packet-level analysis of the data stream.

Methods and systems for flow processing and switching, security, and other network applications, including a facility that processes a data flow to address patterns relevant to a variety of conditions are directed at internal network security, virtualization, and web connection security are described. Such flow processing facilities may be used for inspecting network traffic packet payloads to detect security threats and intrusions across accessible layers of the network IP stack by applying content matching and behavioral anomaly detection techniques based on regular expression matching and self-organizing maps. Exposing threats and intrusions within packet payload at or near real-time rates enhances network security from both external and internal sources while ensuring security policy is rigorously applied to data and system resources.

Disclosed are a method, upstream processing node, and computer readable medium for dynamically stabilizing a stream processing system. The method includes receiving at least one computing resource allocation target. The further includes determining that an input data flow rate of at least one upstream processing element varies. The computing resource is dynamically allocated to the upstream processing element in response to the input rate of the upstream processing element varying. Data flow is dynamically controlled between the upstream processing element and at least one downstream processing element.

A system, method, and computer readable medium for optimizing throughput of a stream processing system are disclosed. The method comprises analyzing a set of input streams and creating, based on the analyzing, an input profile for at least one input stream in the set of input streams. The input profile comprises at least a set of processing requirements associated with the input stream. The method also comprises generating a search space, based on an initial configuration, comprising a plurality of configurations associated with the input stream. A configuration in the plurality of configurations is identified that increases throughput more than the other configurations in the plurality of configurations based on at least one of the input profile and system resources.

A failure recovery framework to be used in cooperative data stream processing is provided that can be used in a large-scale stream data analysis environment. Failure recovery supports a plurality of independent distributed sites, each having its own local administration and goals. The distributed sites cooperate in an inter-site back-up mechanism to provide for system recovery from a variety of failures within the system. Failure recovery is both automatic and timely through cooperation among sites. Back-up sites associated with a given primary site are identified. These sites are used to identify failures within the primary site including failures of applications running on the nodes of the primary site. The failed applications are reinstated on one or more nodes within the back-up sites using job management instances local to the back-up sites in combination with previously stored state information and data values for the failed applications. In additions to inter-site mechanisms, each one of the plurality of sites employs an intra-site back-up mechanism to handle failure recoveries within the site.

An event stream processing device capable of processing larger numbers of events while simultaneously responding to queries. This is achieved through sequential storage of data, the maintenance in memory of information pertaining to the most recent events for each entity monitored and the aggregation of file read / write requests in a single thread which is capable of optimising the execution of those requests.

A stream processing platform that provides fast execution of stream processing applications within a safe runtime environment. The platform includes a stream compiler that converts a representation of a stream processing application into executable program modules for a safe environment. The platform allows users to specify aspects of the program that contribute to generation of modules that execute as intended. A user may specify aspects to control a type of implementation for loops, order of execution for parallel paths, whether multiple instances of an operation can be performed in parallel or whether certain operations should be executed in separate threads. In addition, the stream compiler may generate executable modules in a way that cause a safe runtime environment to allocate memory or otherwise operate efficiently.

A system optimizes two or more stream processing programs based upon the data exchanged between the stream processing programs. The system alternately processes each stream processing program to identify and remove dead program code, thereby improving execution performance. Dead program code is identified by propagating constants received as inputs from other stream processing programs and by analyzing a first stream processing program and determining the outputs of a second stream processing program that are unused by the first stream processing program. The system may perform multiple iterations of this optimization is previous iterations introduce additional constants used as inputs to a stream processing program. Following optimization of the stream processing programs, the optimized stream processing programs are compiled to a format adapted to be executed by a stream processing system.

According to the disclosure hereof, the functionality of a server can be extended to collect data on content streams that the server is delivering to clients, and to beacon certain data back an analytics system to facilitate monitoring of, reporting on, and analysis of the delivery of content streams. At various stages of the streaming process, a server can read and update state information (for example cookie data) on the requesting client reflecting, for example, status in playing a particular stream. Based on the client's requests and the state information at each stage, the server can beacon appropriate information about the stream and its playback status back to the analytics system. The teachings hereof are particularly useful, without limitation, in streaming media analytics and for segment-based streaming approaches, including over HTTP.

Disclosed is a method, upstream processing node, and computer readable medium for processing data packets. The method includes receiving an allocation for at least one computing resource. At least one data packet at one or more upstream processing elements in an upstream processing node is received. The method also includes determining if at least one output queue of the upstream processing elements includes available space for processing of the data packet. The data packet is processed in response to the output queue including available space to form a resulting data packet. The method further includes determining that an input queue of at least one downstream processing element of a plurality of downstream processing elements includes available space for receiving the resulting data packet In response to the determining, the resulting data packet is transmitted from the upstream processing element to the downstream processing element.

A reconfigurable, protocol indifferent bit stream-processing engine, and related systems and data communication methodologies, are adapted to achieve the goal of providing inter-fabric interoperability among high-speed networks operating a speeds of at least 10 gigabits per second. The bit-stream processing engine operates as an omni-protocol, multi-stage processor that can be configured with appropriate switches and related network elements to create a seamless network fabric that permits interoperability not only among existing communication protocols, but also with the ability to accommodate future communication protocols. The method and systems of the present invention are applicable to networks that include storage networks, communication networks and processor networks.

A mechanism for managing ranking information using a sign of a stream tuple generated when stream data is inserted into, or deleted from, a window is provided. A mechanism for generating only the differential information of ranking calculation results, a mechanism for adding ranking information according to a request, an interface for generating and outputting all ranking information from the differential information, a mechanism for generating all ranking calculation results, and an interface for using these mechanisms are provided.

Disclosed herein is a method and system for hardware-accelerating various data processing operations in a rule-based decision-making system such as a business rules engine, an event stream processor, and a complex event stream processor. Preferably, incoming data streams are checked against a plurality of rule conditions. Among the data processing operations that are hardware-accelerated include rule condition check operations, filtering operations, and path merging operations. The rule condition check operations generate rule condition check results for the processed data streams, wherein the rule condition check results are indicative of any rule conditions which have been satisfied by the data streams. The generation of such results with a low degree of latency provides enterprises with the ability to perform timely decision-making based on the data present in received data streams.

A transport stream recording device, method thereof, transport stream reproduction device, method thereof, and recording medium for efficiently recording and reproducing transport streams. A source packet having a four byte TP_extra_Header attached to a 188 byte transport packet is generated, 32 source packets of 192 bytes each are collected, and aligned units generated equivalent to a three sector portion of data (6144 bytes) are recorded or reproduced on the recording medium.

A “Query Optimizer” provides a cost estimation metric referred to as “Maximum Accumulated Overload” (MAO). MAO is approximately equivalent to maximum system latency in a data stream management system (DSMS). Consequently, MAO is directly relevant for use in optimizing latencies in real-time streaming applications running multiple continuous queries (CQs) over high data-rate event sources. In various embodiments, the Query Optimizer computes MAO given knowledge of original operator statistics, including “operator selectivity” and “cycles / event” in combination with an expected event arrival workload. Beyond use in query optimization to minimize worst-case latency, MAO is useful for addressing problems including admission control, system provisioning, user latency reporting, operator placements (in a multi-node environment), etc. In addition, MAO, as a surrogate for worst-case latency, is generally applicable beyond streaming systems, to any queue-based workflow system with control over the scheduling strategy.