654 results about "Data stream processing" patented technology

An apparatus and method to distribute applications and services in and throughout a network and to secure the network includes the functionality of a switch with the ability to apply applications and services to received data according to respective subscriber profiles. Front-end processors, or Network Processor Modules (NPMs), receive and recognize data flows from subscribers, extract profile information for the respective subscribers, utilize flow scheduling techniques to forward the data to applications processors, or Flow Processor Modules (FPMs). The FPMs utilize resident applications to process data received from the NPMs. A Control Processor Module (CPM) facilitates applications processing and maintains connections to the NPMs, FPMs, local and remote storage devices, and a Management Server (MS) module that can monitor the health and maintenance of the various modules.

This invention provides a network management equipment and a packet switching equipment which eliminate a connection setup delay time, reduce a delay and a delay variation involved in data transfer, and effectively perform connectionless data flow processing in a large data network. The network is divided into a connection-oriented core network and a plurality of connectionless access networks connected to the core network where a plurality of connections (called permanent virtual route (PVR)) are set up among a plurality of edge nodes. The network management equipment selects one route from a plurality of PVRs for connectionless data flow received from one of the access networks and transfers data along the PVR.

An apparatus and method to distribute applications and services in and throughout a network and to secure the network includes the functionality of a switch with the ability to apply applications and services to received data according to respective subscriber profiles. Front-end processors, or Network Processor Modules (NPMs), receive and recognize data flows from subscribers, extract profile information for the respective subscribers, utilize flow scheduling techniques to forward the data to applications processors, or Flow Processor Modules (FPMs). The FPMs utilize resident applications to process data received from the NPMs. A Control Processor Module (CPM) facilitates applications processing and maintains connections to the NPMs, FPMs, local and remote storage devices, and a Management Server (MS) module that can monitor the health and maintenance of the various modules.

A continuous query parallel engine on data streams provides scalability and increases the throughput by the addition of new nodes. The parallel processing can be applied to data stream processing and complex events processing. The continuous query parallel engine receives the query to be deployed and splits the original query into subqueries, obtaining at least one subquery; each subquery is executed in at least in one node. Tuples produced by each operator of each subquery are labeled with timestamps. A load balancer is interposed at the output of each node that executes each one of the instances of the source subquery and an input merger is interposed in each node that executes each one of the instances of a destination subquery. After checks are performed, further load balancers or input managers may be added.

An apparatus and method to distribute applications and services in and throughout a network and to secure the network includes the functionality of a switch with the ability to apply applications and services to received data according to respective subscriber profiles. Front-end processors, or Network Processor Modules (NPMs), receive and recognize data flows from subscribers, extract profile information for the respective subscribers, utilize flow scheduling techniques to forward the data to applications processors, or Flow Processor Modules (FPMs). The FPMs utilize resident applications to process data received from the NPMs. A Control Processor Module (CPM) facilitates applications processing and maintains connections to the NPMs, FPMs, local and remote storage devices, and a Management Server (MS) module that can monitor the health and maintenance of the various modules.

An apparatus and method to distribute applications and services in and throughout a network and to secure the network includes the functionality of a switch with the ability to apply applications and services to received data according to respective subscriber profiles. Front-end processors, or Network Processor Modules (NPMs), receive and recognize data flows from subscribers, extract profile information for the respective subscribers, utilize flow scheduling techniques to forward the data to applications processors, or Flow Processor Modules (FPMs). The FPMs utilize resident applications to process data received from the NPMs. A Control Processor Module (CPM) facilitates applications processing and maintains connections to the NPMs, FPMs, local and remote storage devices, and a Management Server (MS) module that can monitor the health and maintenance of the various modules.

An apparatus and method to distribute applications and services in and throughout a network and to secure the network includes the functionality of a switch with the ability to apply applications and services to received data according to respective subscriber profiles. Front-end processors, or Network Processor Modules (NPMs), receive and recognize data flows from subscribers, extract profile information for the respective subscribers, utilize flow scheduling techniques to forward the data to applications processors, or Flow Processor Modules (FPMs). The FPMs utilize resident applications to process data received from the NPMs. A Control Processor Module (CPM) facilitates applications processing and maintains connections to the NPMs, FPMs, local and remote storage devices, and a Management Server (MS) module that can monitor the health and maintenance of the various modules.

Systems and methods are provided for analyzing a data stream device at a packet level. In one embodiment, design logic is embedded within a target data stream processing device such as a networking device or video device. The embedded design logic is configured to start and stop the target data stream processing device and to inject data into the target data stream processing device for evaluation. The embedded design may also be configured to detect a user selected data value in a packet within the target data stream processing device before the packet is transmitted through the device's physical layer or passed up the device's networking stack. In addition, or in another embodiment, at least a portion of the design logic, including the ability to detect the user selected value in the packet, may be located external to the data stream processing device. In one embodiment, a controlled client / server interfaces with the physical layer of the target data stream processing device and provides packet-level analysis of the data stream.

An apparatus and method to distribute applications and services in and throughout a network and to secure the network includes the functionality of a switch with the ability to apply applications and services to received data according to respective subscriber profiles. Front-end processors, or Network Processor Modules (NPMs), receive and recognize data flows from subscribers, extract profile information for the respective subscribers, utilize flow scheduling techniques to forward the data to applications processors, or Flow Processor Modules (FPMs). The FPMs utilize resident applications to process data received from the NPMs. A Control Processor Module (CPM) facilitates applications processing and maintains connections to the NPMs, FPMs, local and remote storage devices, and a Management Server (MS) module that can monitor the health and maintenance of the various modules.

A method of caching commands in microprocessors having a plurality of arithmetic units and in modules having a two- or multidimensional cell arrangement is provided. The method includes combining a plurality of cells and arithmetic units to form a plurality of groups, assigning a cache unit to a group, and connecting the cache unit to a higher level unit via a tree structure. The cache unit may send requests for required commands to the higher level cache unit, which may return a command sequence including the required command, if the higher level cache unit holds the first command sequence including the required command in the higher level cache unit's local memory.

A failure recovery framework to be used in cooperative data stream processing is provided that can be used in a large-scale stream data analysis environment. Failure recovery supports a plurality of independent distributed sites, each having its own local administration and goals. The distributed sites cooperate in an inter-site back-up mechanism to provide for system recovery from a variety of failures within the system. Failure recovery is both automatic and timely through cooperation among sites. Back-up sites associated with a given primary site are identified. These sites are used to identify failures within the primary site including failures of applications running on the nodes of the primary site. The failed applications are reinstated on one or more nodes within the back-up sites using job management instances local to the back-up sites in combination with previously stored state information and data values for the failed applications. In additions to inter-site mechanisms, each one of the plurality of sites employs an intra-site back-up mechanism to handle failure recoveries within the site.

A method is provided for the recovery of an instance of a job manager running on one of a plurality of nodes used to execute the processing elements associated with jobs that are executed within a cooperative data stream processing system. The states of the processing elements are checkpointed to a persistence mechanism in communication with the job manager. From the checkpointed processing element states, the state of each distributed job is determined and checkpointed. Processing element states are also checkpointed locally to the nodes one which the processing elements are running. Following a failure of the job manager, the job manager is reinstantiated on one of the nodes. The recovery instance of the job manger obtains state data for processing elements and jobs from the persistence mechanism and constructs an initial state for jobs and processing elements. These initial states are reconciled against the current states of the processing elements and adjustments are made accordingly. Once the job and processing element states are reconciled, the system is returned to normal operation.

Techniques and mechanisms for exchanging control signals in a data path module of a data stream processing engine. In an embodiment, the data path module may be configured to form a set of one or more data paths corresponding to an instruction which is to be executed. In another embodiment, data processing units of the data path module may be configured to exchange one or more control signals for elastic execution of the instruction.

An apparatus and method to distribute applications and services in and throughout a network and to secure the network includes the functionality of a switch with the ability to apply applications and services to received data according to respective subscriber profiles. Front-end processors, or Network Processor Modules (NPMs), receive and recognize data flows from subscribers, extract profile information for the respective subscribers, utilize flow scheduling techniques to forward the data to applications processors, or Flow Processor Modules (FPMs). The FPMs utilize resident applications to process data received from the NPMs. A Control Processor Module (CPM) facilitates applications processing and maintains connections to the NPMs, FPMs, local and remote storage devices, and a Management Server (MS) module that can monitor the health and maintenance of the various modules.

The invention discloses a distributed network control method. A plurality of controller nodes form a distributed controller network, and each controller node manages one or more pieces of switch equipment. The method comprises the following steps that: the controller node receives information reported by a switch, synchronizes network topology change information in the information among all the controller nodes, and stores host position information in the information into a distributed control network; and when receiving a data stream processing request, the controller node queries the destination address information of a data stream from the distributed controller network, and establishes a data exchange path for the data stream according to locally stored global network topology information. The invention also provides a distributed network control device. According to the technical scheme, the problem of performance bottleneck of a network controller can be solved.

A method of deadlock-free, automatic configuration and reconfiguration of modules having a two- or multidimensional cell arrangement, in which a unit for controlling the configuration and reconfiguration manages a set of associated configurable elements, the set being a subset or the total set of all configurable elements, and the management takes place as follows: reconfiguration requests from the associated configurable elements are sent to the unit; the unit processes the requests; the unit processes the configuration data of the command sequence; and after the configuration data has been fully processed, new requests are accepted again, the configuration data still to be loaded of the existing previous requests being loaded from a buffer memory (FILMO) into the configurable elements until a new request occurs.

Techniques for compiling a data stream processing application are provided. The techniques include receiving, by a compiler executing on a computer system, source code for a data stream processing application, wherein the source code comprises source code for a plurality of operators, each of which performs a data processing function, determining, by the compiler, one or more characteristics of operators within the data stream processing application, grouping, by the compiler, the operators into one or more execution containers based on the one or more characteristics, and compiling, by the compiler, the source code for the data stream processing application into executable code, wherein the executable code comprises a plurality of execution units, wherein each execution unit contains one or more of the operators, wherein each operator is assigned to an execution unit based on the grouping, and wherein each execution unit is to be executed in a partition.

Embodiments of the present application relate to a distributed data stream processing method, a distributed data stream processing device, a computer program product for processing a raw data stream and a distributed data stream processing system. A distributed data stream processing method is provided. The method includes dividing a raw data stream into a real-time data stream and historical data streams, processing the real-time data stream and the historical data streams in parallel, separately generating respective results of the processing of the real-time data stream and the historical data streams, and integrating the generated processing results.

An expanded arithmetic and logic unit (EALU) with special extra functions is integrated into a configurable unit for performing data processing operations. The EALU is configured by a function register, which greatly reduces the volume of data required for configuration. The cell can be cascaded freely over a bus system, the EALU being decoupled from the bus system over input and output registers. The output registers are connected to the input of the EALU to permit serial operations. A bus control unit is responsible for the connection to the bus, which it connects according to the bus register. The unit is designed so that distribution of data to multiple receivers (broadcasting) is possible. A synchronization circuit controls the data exchange between multiple cells over the bus system. The EALU, the synchronization circuit, the bus control unit, and registers are designed so that a cell can be reconfigured on site independently of the cells surrounding it. A power-saving mode which shuts down the cell can be configured through the function register; clock rate dividers which reduce the working frequency can also be set.

Techniques for optimizing data stream processing are provided. The techniques include employing a pattern, wherein the pattern facilitates splitting of one or more incoming streams and distributing processing across one or more operators, obtaining one or mote operators, wherein the one or more operators support at least one group-independent aggregation and join operation on one or more streams, generating code, wherein the code facilitates mapping of the application onto a computational infrastructure to enable workload partitioning, using the one or more operators to decompose each of the application into one or more granular components, and using the code to reassemble the one or more granular components into one or more deployable blocks to map the application to a computational infrastructure, wherein reassembling the one or more granular components to map the application to the computational infrastructure optimizes data stream processing of the application.

A system, method, and computer readable medium adapted to provide software and hardware partitioning for multi-standard video compression and decompression comprises a master-slave bus, a peer-to-peer bus, and an inter-processor communications bus, a prediction engine, a filter engine, and a transform engine, and a video encode control processor, and a video decode control processor adapted to utilize the master-slave bus to interact with the video hardware engines for control flow processing, the peer-to-peer bus for data flow processing, and the inter-processor communications bus for inter-processor communications, and a system data bus adapted to permit data exchange between system resources, the busses, the engines, and the processors.

A cooperative data stream processing system is provided that utilizes a plurality of independent, autonomous and possibly heterogeneous sites in a cooperative arrangement to execute jobs derived from user-define inquires over dynamic, continuous streams of data. A method is provided for cooperative planning for the execution of the jobs across the distributed plurality of sites. An identification of the resources available for sharing from each one of the plurality of sites is communicated to one or more planners disposed on the distributed sites. These planners use the resource information to generate planning domains in which the jobs can be processed. Upon receipt of an inquiry at one of the sites, the inquiry is communicated to one of the planners that uses the planning domain to create at least one distributed plan for the inquiry. Processing of the inquiry is conducted in accordance with the distributed plan. Planning can take advantage of the structure of virtual organizations including cooperative and federated virtual organizations. The distributed plans can make use of the resources within a single virtual organization or across multiple organizations.

A method and a system process a stream of data in parallel across a plurality of nodes. The log processing system has a log module, a query language module, and a query processing module. The log module receives and organizes the stream of data into a sequential and nested data structure. The query language operator module defines operators that operate on the sequential and nested data structure. The query processing module processes in parallel across a plurality of nodes a query based on an operator on the stream of data.

A method including receiving, at a computer system, debugging configuration information specifying a functional area of a data stream processing server to be debugged, is described. Furthermore, the method includes identifying, by the computer system, an object associated with the functional area that has been instantiated by the data stream processing server, determining, by the computer system, that tracing for the object is enabled to perform the debugging, and instantiating, by the computer system, a tracelet associated with the object. Further, the method includes stepping, by the computer system, through the tracelet associated with the object to debug the object, and displaying, by the computer system, a visual representation of debugging results associated with the object.

The embodiment of the invention provides a data stream processing method, a data stream processing system, a controller and switching equipment. The data stream processing method comprises the following steps of: receiving the first characteristic information of a data stream transmitted by any piece of switching equipment in a software defined network (SDN), and judging whether the data stream corresponding to the first characteristic information of the data stream is safe or not according to preset safety rules to obtain a detection result; if the detection result indicates that the data stream is safe, generating a data stream forwarding strategy for the data stream, and transmitting the data stream forwarding strategy to the switching equipment, so that the switching equipment forwards the data stream according to the forwarding strategy; and if the detection result indicates that the data stream is unsafe, instructing the switching equipment to discard the data stream. According to the data stream processing method, the data stream processing system, the controller and the switching equipment, the safety of the data stream of any piece of switching equipment in the SDN can be detected.

A data processing system, wherein a data flow processor (DFP) integrated circuit chip is provided which comprises a plurality of orthogonally arranged homogeneously structured cells, each cell having a plurality of logically same and structurally identically arranged modules. The cells are combined and facultatively grouped using lines and columns and connected to the input / output ports of the DFP. A compiler programs and configures the cells, each by itself and facultatively-grouped, such that random logic functions and / or linkages among the cells can be realized. The manipulation of the DFP configuration is performed during DFP operation such that modification of function parts (MACROs) of the DFP can take place without requiring other function parts to be deactivated or being impaired.

Disclosed herein is a data stream processing apparatus and method using query partitioning, which allow data stream processing apparatuses to perform partitioned processing / parallel processing on partitioned sub-queries. The proposed data stream processing apparatus using query partitioning receives a query from a user, partitions the query into a plurality of sub-queries, transmits the partitioned sub-queries to another data stream processing apparatus or a sub-query processing unit, integrates the results of the processing of sub-queries processed by the other data stream processing apparatus and the sub-query processing unit with each other, generates a response to the query, and transmits the generated response to the user.

Techniques for generating a distributed stream processing application are provided. The techniques include obtaining a declarative description of one or more data stream processing tasks from a graph of operators, wherein the declarative description expresses at least one stream processing task, generating one or more containers that encompass a combination of one or more stream processing operators, and generating one or more execution units from the declarative description of one or more data stream processing tasks, wherein the one or more execution units are deployable across one or more distributed computing nodes, and comprise a distributed data stream processing application binary.