240 results about "Streaming computation" patented technology

The invention specifically discloses a power distribution network simulation research and analysis system and method based on network-wide data. The system comprises a three-layer structure: a data layer, a platform layer and an application layer. The data layer mainly collects basic and operation data of a distribution network; the platform layer mainly gathers and summarizes various data of the distribution network and provides corresponding services to the application layer; and the application layer mainly, under the support of the platform layer and based on network-wide equipment and operation data, carries out a series of correlated application researches related to the power distribution network on scientific modeling, load flow calculation, fault analysis, closed-loop calculation, network reconstruction and self-healing simulation modules of the power distribution network comprising a lot of distributed generation, so as to realize comprehensive digital simulation of the power distribution network, and to provide decision reference for the construction and operation of the power distribution network.

The invention relates to an operation data processing method and system, in particular to an operation data processing method and system based on stream computation and batch computation for a double-engine business analysis system. According to the operation data processing method and system, a stream computation engine (22) is set in a system framework, so that data real-time processing capacity is improved, and a distributed file system (21) and a batch computation engine (24) are integrated based on a data warehouse (23), so that storage and computation of unstructured data and mass data are achieved, big data storage capacity and big data analysis capacity are improved, and the capacity for storing TB-level data and even PB-level data and the capacity for processing big data in a batched mode can be achieved. Meanwhile, due to the fact the data warehouse (23) is adopted for processing structural data, a micro-batch real-time data can be processed with a micro-batch processing method, and thus a real-time data processing mechanism is further perfected.

Techniques are described for transmitting predicted output data on a processing element in a stream computing application instead of processing currently received input data. The stream computing application monitors the output of a processing element and determines whether its output is predictable, for example, if the previously transmitted output values are within a predefined range or if one or more input values correlate with the same one or more output values. The application may then generate a predicted output value to transmit from the processing element instead of transmitting a processed output value based on current input values. The predicted output value may be, for example, an average of the previously transmitted output values or a previously transmitted output value that was transmitted in response to a previously received input value that is similar to a currently received input value. Moreover, the processing element or elements that transmit the predicted output data may be upstream from the processing element with the predictable output.

The invention discloses a service data real-time monitoring system and method based on a Java byte code enhancement technology. The system comprises the components: a monitored application side, Agentbeing deployed in a JVM, when the application program is loaded into the JVM, the Agent modifying and loading the class of the JVM by using ASM, codes of various events and measurement values being dynamically inserted and collected when the application program runs, the newly-added codes executing and obtaining various data and then sending the data to the monitoring platform side, and the Agentperiodically obtaining and executing instructions such as starting and stopping of the Agent and thread analysis issued by the monitoring platform side; a monitoring platform side, used for acquiringthe data of the monitored application side, processing the acquired data and putting the processed data into message queues with different themes of the message middleware according to data types, aconsumer acquiring the data processing from the message queues according to the different themes and performing pre-computation on the data of the themes according to time windows and dimensions by utilizing stream computation; and a user side obtaining the calculation result and outputting the calculation result.

The invention discloses a processing system and a processing method of a large-scale concurrent data stream, relates to the technical field of data and aims to solve the technical problem of low processing efficiency of a stream processor. The system comprises a data stream unit (DSU) buffer area, a DSU cluster queue pool, a DSU mapping table, a stream processor pool, a data stream reading part, a DSU cluster allocation part, a task scheduling part and a computation post-processing part, wherein the stream processor tool consists of a plurality of graphics processing units (GPU); the data stream reading part is used for writing the concurrent data stream into the DSU buffer area; the DSU cluster allocation part is used for classifying data stream units which are currently processed in the DSU buffer area; the task scheduling part is used for loading a ready queue in the DSU cluster queue pool to the GPUs in the stream processor pool and carrying out stream computation; and the computation post-processing part is used for returning a computation result of the GPUs to the data stream. The system provided by the invention can improve the processing efficiency of the stream processor.

A method for determining and enforcing rate constraints on process flows in a network, the method comprising receiving at least one process flow. A flow rate vulnerability (FRV) factor and a required bandwidth (RBW) parameter are determined for said at least one process flow. A rate constraint is calculated for said at least one process flow and the rate constraint is enforced on said at least one process flow.

The invention discloses a Matlab-based rectangular coordinate newton power flow calculation method. Matrix operation and complex operation are adopted, a Jacobi matrix is formed according to the programming characteristic of Matlab, and the power of a node is calculated. The method comprises the following steps: calculating an initial Jacobi calculation matrix; calculating the complex power and the injection current phasor of the node; calculating partitioned sub-matrixes of the initial Jacobi matrix; correcting diagonal elements of the sub-matrixes of the Jacobi matrix by virtue of the injection current phasor of the node and a partial derivative of a voltage deviation to a voltage; forming the Jacobi matrix by virtue of the sub-matrixes of the Jacobi matrix, and performing regulation. The method is implemented on a Matlab platform, so that a researcher can conveniently test and analyze a calculation result by virtue of various tools and functions provided by Matlab. Practices show that convenience is brought to program compiling, modification and debugging of the researcher, the calculation speed is substantially approximate to the speed achieved on a C language platform and an excellent analysis tool is provided for the researcher.

The invention relates to an OPC data real-time acquisition monitoring intelligent system and method based on stream computing. The system comprises an OPC data acquisition server, an OPC centralized control management server, a Kafka data processing module and an OPC client. The OPC data acquisition server receives data from industrial workshop field equipment through a standard OPC interface, stores the received real-time data into a whole-plant real-time database, and transmits the data to the centralized control management server according to task requirements; and the Kafka data processingmodule receives the monitoring data and puts the monitoring data in a distributed message queue for caching, and constructs a processing architecture based on stream computing to perform unified processing and storage on the data. According to the invention, the data acquisition process is simplified, and the integration capability of the field equipment layer, the workshop monitoring layer and the manager decision-making layer is improved; and the real-time performance of monitoring result pushing and data processing access is improved, the problem that the response speed is low is solved, and meanwhile the reliability of data is guaranteed.

The invention discloses a microgrid flow model suitable for various operation modes, and a calculation method. The invention discloses a flow calculation method suitable for microgrids in various operation modes, and the method comprises the steps: forming five types of microgrid operation modes and four types of nodes according to two types of basic operation modes and three types of control strategies of a microgrid; enabling the microgrid flow calculation in five types of different operation modes to be divided into two types of flow calculation; building a unified flow model; and proposing an LM-TR (Levenberg Marquardt method combined with trust region technique, LM-TR) flow algorithm for solving. The method does not need the jacobian matrix of a flow equation to be nonsingular, has global convergence, is small in dependence on initial values of unknown variables, effectively improves the convergence of flow calculation, and can solve a problem that the convergence of flow calculation of a microgrid system is difficult because that the jacobian matrix is singular or there is no balance point. Moreover, the method can process various types of nodes conveniently, and can solve a problem of the diversity of microgrid flow calculation in different operation modes. The method is suitable for the flow calculation of the microgrid in various operation modes, and is good in engineering application value.

The present disclosure relates to a method for implementing a stream processing computer architecture, including creating a stream computer processing (SCP) system by forming a super node cluster of processors representing physical computation nodes (''nodes''), communicatively coupling the processors via a local interconnection means (''interconnect''), and communicatively coupling the cluster to an optical circuit switch (OCS), via optical external links (''links''). The OCS is communicatively coupled to another cluster of processors via the links. The method also includes generating a stream computation graph including kernels and data streams, and mapping the graph to the SCP system, which includes assigning the kernels to the clusters and respective nodes, assigning data stream traffic between the kernels to the interconnection when the data stream is between nodes in the same cluster, and assigning traffic between the kernels to the links when the data stream is between nodes in different clusters. The method also includes configuring the OCSs to provide connectivity between mapped clusters.

The invention discloses a polar coordinate Newton method load flow calculation method based on a Matlab sparse matrix, adopting matrix operation and complex operation and using a sparse matrix technology provided by Matlab. The method is implemented on a Matlab platform, so that researchers are facilitated to use various tools and functions provided by Matlab to test and analyze a calculation method. The main part of the method adopts both the matrix operation and the complex operation, reduces program codes and simplifies a programming process, so that the program is clearer, and the researchers are facilitated to modify the program, debug the program, and improve and add new functions for the program. The method provided by the invention adopts the sparse matrix technology of Matlab, so that calculation speed is greatly improved, meanwhile the sparse matrix of Matlab is very convenient to use, elements of the sparse matrix can be directly used by using a column number like a full matrix, design of a sparse storage structure is not needed, and an excellent analysis tool is provided for scientific research of the researchers.

The invention discloses a block chain-based asset monitoring method. The method comprises the following steps of: associating a stream computation engine for each participator node in a block chain network; realizing an intelligent contract according to a business and deploying the intelligent contract onto the block chain network; adding metadata analysis components on the participator nodes; defining rules and corresponding response actions in the stream computation engines; submitting lamination proposals to the block chain networks by participators through corresponding nodes; verifying and executing the proposals by the block chain network; recording an execution result of the intelligent contract into an account book copy by each participator node and then sending an event to the associated stream computation engine; and processing the events by the stream computation engine according to the predefined rules. According to the method, the block chain main networks do not need to be changed and the working processes of the block chain networks are not changed; and authorization mechanisms of the original block chain networks are perfectly compatible, and each stream computationengine only needs to read a data set belonging to the stream computation engine, so that asset problems can be recognized in real time or quasi real time.

The invention discloses an automatic adjustment method for load flow calculation convergence. The method comprises the steps of designing a state, an action space and a reward of a deep reinforcementlearning network for load flow calculation convergence; constructing a deep reinforcement learning network for load flow calculation convergence according to the state, the action space and the reward; knowledge experience is added into the deep reinforcement learning network, and the process of manually adjusting the power flow is simulated, so that a power flow adjustment strategy is constructed, the power flow convergence of the power grid is adjusted by using the power flow adjustment strategy, and the problems of low convergence working efficiency, inaccuracy and overlarge manpower cost consumption of the current adjustment of the large power grid are solved.

The invention discloses a method for achieving large-range dynamic wave efficient real-time rendering on a stream processor. According to the method, a novel grid model based on the LOD (Level of Detail) technology is involved, dynamic wave simulation is achieved based on a Gerstner model, a concave and convex texture mapping method accelerates remoter wave rendering, and animation concave and convex textures are utilized for improving the wave rendering effect. The method is characterized in that the reflecting effect on the sky and the sun is achieved through the cube textures, a Phong illumination model is applied to achieve wave illumination and achieve Fresnel reflection, the animation textures are adopted for simulating foam in waves, and the special effects such as shadows, projected onto the sea surface, of clouds are achieved. The method can carry out real-time interactive rendering on waves within the hundred-kilometer-level range on a computer platform with the stream processor, and has the advantages that the speed is high, the simulation degree is high, and platform compatibility is good.

The invention discloses a signaling big data-based user travel trajectory extraction method and system, which are used for mining a user travel trajectory sequence from massive signaling data, obtaining a user position switching data sequence in an offline trajectory extraction process, detecting a user residence point by utilizing residence time and extracting a user offline trajectory. In real-time trajectory extraction, streaming computation and Geohash computation are utilized to record a Geohash grid where each user is currently located, the Geohash grid is used as a resident point, and anew resident point is generated, so that real-time trajectory data is acquired. By adopting the technical scheme provided by the invention, the real-time track and the offline track of the user can be extracted, and the flexibility is high.

The invention discloses a task-flow computational model applicable to a reconfigurable high-performance computer system. A form of Five-Tuple Array (T, R, SC, EC, TS) represents the way for solving a problem under the mode of task-flow, wherein, T is a finite set of tasks, R is a relation set among tasks, SC is a starting condition set of tasks, EC is a terminal condition set of tasks and TS is a finite sequence set of tasks. On a premise of no restriction by factors such as resources, the preferred plan for solving problems is possibly used as a drag, the grain size of soft and hard task are reasonably divided, the system is restructured according to problems and requirements of task and high cost-effectiveness and high utilization rate are achieved. The task-flow computational model processes tasks as a basic unit, provides the definition of task-flow, the division principles of tasks as well as various forms of task-flow. The model can be used for the parallelism depiction of different grain sizes and lays a foundation for accurately and clearly dividing tasks and depicting task-flow oriented reconfigurable high-performance computation.

The invention discloses a tax terminal system log real-time analysis method and device, a medium and a system. The method comprises the following steps: carrying out multi-thread distributed real-timeacquisition on log data generated by tax terminal system business operation; caching the acquired log data into a message queue cluster, and performing distributed storage by taking the acquired logdata as original log data; extracting log data in the message queue cluster, performing real-time analysis processing on the log data based on a stream computing tool, and storing an analysis result in real time; and providing an operation interface, and performing query, query result display and offline analysis processing on the original log data based on a big data calculation engine through the operation interface, or performing real-time query and query result display on an analysis result stored in real time through the operation interface according to a query condition. Related information can be rapidly and accurately obtained, and system problems can be rapidly positioned.

The invention relates to a kinetic energy data processing system based on a Spark computing framework. The kinetic energy data processing system comprises a data acquisition module used for monitoringkinetic energy data of an industrial workshop, a data processing and analyzing module used for carrying out data computing on data streams issued to a Kafka message queue, and a distributed data storage module. Considering the variety of the data acquisition terminals, the Kafka cluster is provided with functional interfaces capable of being accessed to various data acquisition terminals, so thata flexible and extensible data acquisition system is established, and the problem of access of a plurality of data terminals is avoided. And secondly, as the amount of detection data needing to be processed is large, the stream data from the Kafka message queue is processed by adopting stream calculation, and the calculation performance can be greatly improved. And thirdly, the real-time performance of processing and analyzing the data according to the decision requirement of the industrial workshop on equipment operation is high, so that a distributed storage mode is adopted for the data.

The invention relates to the technical field of flow calculation and provides a method for management of application programming interface (API) call records; the method comprises the steps: acquiring a plurality of API call records; determining the respective affiliated charging period of each call record in the plurality of the API call records according to the Called time of each call record in the plurality of the API call records; superposing the resource usage amount of each of more than two API call records, which are located in the same charging period, have the same caller mark and have the same charging unit price, in the plurality of the API call records according to the preset charging unit price, and obtaining one API call record subjected to superposition. With adoption of the technical scheme, the quantity of the API call records to be processed by an API charging gateway can be reduced and partial processing resources can be released.

The embodiment of the invention discloses a stream computing platform and method based on visualization, equipment and a storage medium. The stream computing platform comprises a data development component, a stream computing component and an intelligent scheduling component, wherein the data development component is used for visually developing a corresponding stream computing application througha dragging operator in stream computing, and determining a stream computing engine corresponding to the stream computing application; the stream computing component is used for analyzing scheduling parameters carried in the stream computing application and calling the stream computing engine to execute the stream computing application; and the intelligent scheduling component is used for scheduling execution of the stream computing application according to the scheduling parameters. The technical scheme provided by the embodiment of the invention does not need to be realized only by developers who deeply understand a stream computing application technology, so that the application range of the stream computing platform is expanded, and the expandability of different stream computing engines is supported, and the flexibility of stream computing development is improved.

The invention relates to the technical field of networks, in particular to a data center network flow scheduling method based on load balancing, which comprises the following steps: when a data flow reaches a switch, judging whether a source host and a destination host of flow are connected with a same edge switch or not, and if so, directly forwarding; judging whether the data stream transmission rate exceeds a threshold value or not, and if the data stream transmission rate is lower than the threshold value, calculating a forwarding path for the stream by adopting an equivalent multi-path routing algorithm; if the message is higher than the threshold value, sending the message to a controller, enabling the controller to calculate an optimal forwarding path for the flow by using the proposed particle swarm optimization algorithm based on simulated annealing, and issue a flow table to complete data flow scheduling. For the problems that the current network link load is unbalanced, thenetwork flow completion time is relatively long and the like, the advantages of an SDN network and a heuristic algorithm are combined, so that the effect of balancing the network flow transmission load of the data center is achieved, and the network transmission performance is improved.

The invention provides a streaming computation task distribution method and a control server. The streaming computation task distribution method is applied to the control server connected with streaming computation center server clusters and streaming computation unit server clusters. The method comprises the steps of distributing streaming computation tasks to the target streaming computation center server cluster or the target streaming computation unit server cluster; and judging whether the target streaming computation center server cluster or the target streaming computation unit server cluster is abnormal, and if yes, distributing incompletely executed tasks in the streaming computation tasks to the candidate streaming computation center server clusters. By using the method providedby the embodiment of the invention, when one streaming computation center server cluster or the streaming computation unit server cluster is abnormal, the incompletely executed tasks can be continuously executed on the other normal streaming computation center server clusters, and thus smooth implementation of the streaming computation tasks can be ensured.

The invention relates to a data processing method and a data processing system based on STORM streaming computing. The method comprises the steps that an STORM input node sends a state data backup instruction to an STORM target working node; when the STORM target working node receives the state data backup instruction, the node state data of the STORM target working node is backed up and stored; when STORM streaming computation is interrupted, the STORM input node recovers the message offset state data backed up for the last time in the STORM input node, and sends a state data recovery instruction to the STORM target working node; when the STORM target working node receives a state data recovery instruction, recovering node state data of the STORM target working node from the backup storage; wherein the message offset state data and the node state data are used for STORM streaming computation. According to the method, data stream recovery can be carried out when the data processing system executes stream-oriented computation, and the problem of data loss in the STORM stream-oriented computation process in the data processing system is solved.

The invention discloses a management execution platform constructed based on a network video streaming computation method, and the management execution platform relates to the field of new media. The constructed management execution platform supports management and execution of network video applications, is mainly used for a private network of a telecommunication company on tri-networks integration, creates an executable platform which is directly installed on a client for telecommunication users, can enjoy the basic interactive network TV video service, and can run streaming media network TV video applications such as video-assisted e-commerce, e-learning, electronic medical care and the like; and the platform provides a management platform for operators to collect data for analysis, and comprises cloud composed of computers, a server, data storage equipment and the client.