647 results about "Data stream processing" patented technology

This invention provides a network management equipment and a packet switching equipment which eliminate a connection setup delay time, reduce a delay and a delay variation involved in data transfer, and effectively perform connectionless data flow processing in a large data network. The network is divided into a connection-oriented core network and a plurality of connectionless access networks connected to the core network where a plurality of connections (called permanent virtual route (PVR)) are set up among a plurality of edge nodes. The network management equipment selects one route from a plurality of PVRs for connectionless data flow received from one of the access networks and transfers data along the PVR.

An apparatus and method to distribute applications and services in and throughout a network and to secure the network includes the functionality of a switch with the ability to apply applications and services to received data according to respective subscriber profiles. Front-end processors, or Network Processor Modules (NPMs), receive and recognize data flows from subscribers, extract profile information for the respective subscribers, utilize flow scheduling techniques to forward the data to applications processors, or Flow Processor Modules (FPMs). The FPMs utilize resident applications to process data received from the NPMs. A Control Processor Module (CPM) facilitates applications processing and maintains connections to the NPMs, FPMs, local and remote storage devices, and a Management Server (MS) module that can monitor the health and maintenance of the various modules.

An apparatus and method to distribute applications and services in and throughout a network and to secure the network includes the functionality of a switch with the ability to apply applications and services to received data according to respective subscriber profiles. Front-end processors, or Network Processor Modules (NPMs), receive and recognize data flows from subscribers, extract profile information for the respective subscribers, utilize flow scheduling techniques to forward the data to applications processors, or Flow Processor Modules (FPMs). The FPMs utilize resident applications to process data received from the NPMs. A Control Processor Module (CPM) facilitates applications processing and maintains connections to the NPMs, FPMs, local and remote storage devices, and a Management Server (MS) module that can monitor the health and maintenance of the various modules.

An apparatus and method to distribute applications and services in and throughout a network and to secure the network includes the functionality of a switch with the ability to apply applications and services to received data according to respective subscriber profiles. Front-end processors, or Network Processor Modules (NPMs), receive and recognize data flows from subscribers, extract profile information for the respective subscribers, utilize flow scheduling techniques to forward the data to applications processors, or Flow Processor Modules (FPMs). The FPMs utilize resident applications to process data received from the NPMs. A Control Processor Module (CPM) facilitates applications processing and maintains connections to the NPMs, FPMs, local and remote storage devices, and a Management Server (MS) module that can monitor the health and maintenance of the various modules.

An apparatus and method to distribute applications and services in and throughout a network and to secure the network includes the functionality of a switch with the ability to apply applications and services to received data according to respective subscriber profiles. Front-end processors, or Network Processor Modules (NPMs), receive and recognize data flows from subscribers, extract profile information for the respective subscribers, utilize flow scheduling techniques to forward the data to applications processors, or Flow Processor Modules (FPMs). The FPMs utilize resident applications to process data received from the NPMs. A Control Processor Module (CPM) facilitates applications processing and maintains connections to the NPMs, FPMs, local and remote storage devices, and a Management Server (MS) module that can monitor the health and maintenance of the various modules.

Systems and methods are provided for analyzing a data stream device at a packet level. In one embodiment, design logic is embedded within a target data stream processing device such as a networking device or video device. The embedded design logic is configured to start and stop the target data stream processing device and to inject data into the target data stream processing device for evaluation. The embedded design may also be configured to detect a user selected data value in a packet within the target data stream processing device before the packet is transmitted through the device's physical layer or passed up the device's networking stack. In addition, or in another embodiment, at least a portion of the design logic, including the ability to detect the user selected value in the packet, may be located external to the data stream processing device. In one embodiment, a controlled client/server interfaces with the physical layer of the target data stream processing device and provides packet-level analysis of the data stream.

An apparatus and method to distribute applications and services in and throughout a network and to secure the network includes the functionality of a switch with the ability to apply applications and services to received data according to respective subscriber profiles. Front-end processors, or Network Processor Modules (NPMs), receive and recognize data flows from subscribers, extract profile information for the respective subscribers, utilize flow scheduling techniques to forward the data to applications processors, or Flow Processor Modules (FPMs). The FPMs utilize resident applications to process data received from the NPMs. A Control Processor Module (CPM) facilitates applications processing and maintains connections to the NPMs, FPMs, local and remote storage devices, and a Management Server (MS) module that can monitor the health and maintenance of the various modules.

A method of caching commands in microprocessors having a plurality of arithmetic units and in modules having a two- or multidimensional cell arrangement is provided. The method includes combining a plurality of cells and arithmetic units to form a plurality of groups, assigning a cache unit to a group, and connecting the cache unit to a higher level unit via a tree structure. The cache unit may send requests for required commands to the higher level cache unit, which may return a command sequence including the required command, if the higher level cache unit holds the first command sequence including the required command in the higher level cache unit's local memory.

A failure recovery framework to be used in cooperative data stream processing is provided that can be used in a large-scale stream data analysis environment. Failure recovery supports a plurality of independent distributed sites, each having its own local administration and goals. The distributed sites cooperate in an inter-site back-up mechanism to provide for system recovery from a variety of failures within the system. Failure recovery is both automatic and timely through cooperation among sites. Back-up sites associated with a given primary site are identified. These sites are used to identify failures within the primary site including failures of applications running on the nodes of the primary site. The failed applications are reinstated on one or more nodes within the back-up sites using job management instances local to the back-up sites in combination with previously stored state information and data values for the failed applications. In additions to inter-site mechanisms, each one of the plurality of sites employs an intra-site back-up mechanism to handle failure recoveries within the site.

A method is provided for the recovery of an instance of a job manager running on one of a plurality of nodes used to execute the processing elements associated with jobs that are executed within a cooperative data stream processing system. The states of the processing elements are checkpointed to a persistence mechanism in communication with the job manager. From the checkpointed processing element states, the state of each distributed job is determined and checkpointed. Processing element states are also checkpointed locally to the nodes one which the processing elements are running. Following a failure of the job manager, the job manager is reinstantiated on one of the nodes. The recovery instance of the job manger obtains state data for processing elements and jobs from the persistence mechanism and constructs an initial state for jobs and processing elements. These initial states are reconciled against the current states of the processing elements and adjustments are made accordingly. Once the job and processing element states are reconciled, the system is returned to normal operation.

Techniques and mechanisms for exchanging control signals in a data path module of a data stream processing engine. In an embodiment, the data path module may be configured to form a set of one or more data paths corresponding to an instruction which is to be executed. In another embodiment, data processing units of the data path module may be configured to exchange one or more control signals for elastic execution of the instruction.

The invention discloses a distributed network control method. A plurality of controller nodes form a distributed controller network, and each controller node manages one or more pieces of switch equipment. The method comprises the following steps that: the controller node receives information reported by a switch, synchronizes network topology change information in the information among all the controller nodes, and stores host position information in the information into a distributed control network; and when receiving a data stream processing request, the controller node queries the destination address information of a data stream from the distributed controller network, and establishes a data exchange path for the data stream according to locally stored global network topology information. The invention also provides a distributed network control device. According to the technical scheme, the problem of performance bottleneck of a network controller can be solved.

Techniques for compiling a data stream processing application are provided. The techniques include receiving, by a compiler executing on a computer system, source code for a data stream processing application, wherein the source code comprises source code for a plurality of operators, each of which performs a data processing function, determining, by the compiler, one or more characteristics of operators within the data stream processing application, grouping, by the compiler, the operators into one or more execution containers based on the one or more characteristics, and compiling, by the compiler, the source code for the data stream processing application into executable code, wherein the executable code comprises a plurality of execution units, wherein each execution unit contains one or more of the operators, wherein each operator is assigned to an execution unit based on the grouping, and wherein each execution unit is to be executed in a partition.

Embodiments of the present application relate to a distributed data stream processing method, a distributed data stream processing device, a computer program product for processing a raw data stream and a distributed data stream processing system. A distributed data stream processing method is provided. The method includes dividing a raw data stream into a real-time data stream and historical data streams, processing the real-time data stream and the historical data streams in parallel, separately generating respective results of the processing of the real-time data stream and the historical data streams, and integrating the generated processing results.

Techniques for optimizing data stream processing are provided. The techniques include employing a pattern, wherein the pattern facilitates splitting of one or more incoming streams and distributing processing across one or more operators, obtaining one or mote operators, wherein the one or more operators support at least one group-independent aggregation and join operation on one or more streams, generating code, wherein the code facilitates mapping of the application onto a computational infrastructure to enable workload partitioning, using the one or more operators to decompose each of the application into one or more granular components, and using the code to reassemble the one or more granular components into one or more deployable blocks to map the application to a computational infrastructure, wherein reassembling the one or more granular components to map the application to the computational infrastructure optimizes data stream processing of the application.

A cooperative data stream processing system is provided that utilizes a plurality of independent, autonomous and possibly heterogeneous sites in a cooperative arrangement to execute jobs derived from user-define inquires over dynamic, continuous streams of data. A method is provided for cooperative planning for the execution of the jobs across the distributed plurality of sites. An identification of the resources available for sharing from each one of the plurality of sites is communicated to one or more planners disposed on the distributed sites. These planners use the resource information to generate planning domains in which the jobs can be processed. Upon receipt of an inquiry at one of the sites, the inquiry is communicated to one of the planners that uses the planning domain to create at least one distributed plan for the inquiry. Processing of the inquiry is conducted in accordance with the distributed plan. Planning can take advantage of the structure of virtual organizations including cooperative and federated virtual organizations. The distributed plans can make use of the resources within a single virtual organization or across multiple organizations.

A novel and useful apparatus for and method of software based phase locked loop (PLL). The software based PLL incorporates a reconfigurable calculation unit (RCU) that is optimized and programmed to sequentially perform all the atomic operations of a PLL or any other desired task in a time sharing manner. An application specific instruction-set processor (ASIP) incorporating the RCU includes an instruction set whose instructions are optimized to perform the atomic operations of a PLL. A multi-stage data stream based processor incorporates a parallel/pipelined architecture optimized to perform data stream processing efficiently. The multi-stage parallel/pipelined processor provides significantly higher processing speeds by combining multiple RCUs wherein input data samples are input in parallel to all RCUs while computation results from one RCU are used by adjacent downstream RCUs. A register file provides storage for historical values while local storage in each RCU provides storage for temporary results.

The embodiment of the invention provides a data stream processing method, a data stream processing system, a controller and switching equipment. The data stream processing method comprises the following steps of: receiving the first characteristic information of a data stream transmitted by any piece of switching equipment in a software defined network (SDN), and judging whether the data stream corresponding to the first characteristic information of the data stream is safe or not according to preset safety rules to obtain a detection result; if the detection result indicates that the data stream is safe, generating a data stream forwarding strategy for the data stream, and transmitting the data stream forwarding strategy to the switching equipment, so that the switching equipment forwards the data stream according to the forwarding strategy; and if the detection result indicates that the data stream is unsafe, instructing the switching equipment to discard the data stream. According to the data stream processing method, the data stream processing system, the controller and the switching equipment, the safety of the data stream of any piece of switching equipment in the SDN can be detected.

A data processing system, wherein a data flow processor (DFP) integrated circuit chip is provided which comprises a plurality of orthogonally arranged homogeneously structured cells, each cell having a plurality of logically same and structurally identically arranged modules. The cells are combined and facultatively grouped using lines and columns and connected to the input/output ports of the DFP. A compiler programs and configures the cells, each by itself and facultatively-grouped, such that random logic functions and/or linkages among the cells can be realized. The manipulation of the DFP configuration is performed during DFP operation such that modification of function parts (MACROs) of the DFP can take place without requiring other function parts to be deactivated or being impaired.

Disclosed herein is a data stream processing apparatus and method using query partitioning, which allow data stream processing apparatuses to perform partitioned processing/parallel processing on partitioned sub-queries. The proposed data stream processing apparatus using query partitioning receives a query from a user, partitions the query into a plurality of sub-queries, transmits the partitioned sub-queries to another data stream processing apparatus or a sub-query processing unit, integrates the results of the processing of sub-queries processed by the other data stream processing apparatus and the sub-query processing unit with each other, generates a response to the query, and transmits the generated response to the user.

A method for processing a data stream including receiving a transport stream (TS), storing at least a portion of the TS, the at least a portion of the TS having a beginning and including a plurality of TS packets, determining, from among the plurality of TS packets, at least one TS packet including a candidate access point, storing in an index store a byte offset of the candidate access point; and storing in the index store a plurality of indications of byte offset, each byte offset being associated with an arrival time of a received TS packet.

A system for communication between a plurality of functional elements in a cell arrangement and a higher-level unit is described. The system may include, for example, a configuration memory arranged between the functional elements and the higher-level unit; and a control unit configured to move at least one position pointer to a configuration memory location in response to at least one event reported by a functional element. At run time, a configuration word in the configuration memory pointed to by at least one of the position pointers is transferred to the functional element in order to perform reconfiguration without the configuration word being managed by a central logic.