Patents
Literature
Hiro is an intelligent assistant for R&D personnel, combined with Patent DNA, to facilitate innovative research.
Hiro

246 results about "Streaming computation" patented technology

Power distribution network simulation research and analysis system and method based on network-wide data

The invention specifically discloses a power distribution network simulation research and analysis system and method based on network-wide data. The system comprises a three-layer structure: a data layer, a platform layer and an application layer. The data layer mainly collects basic and operation data of a distribution network; the platform layer mainly gathers and summarizes various data of the distribution network and provides corresponding services to the application layer; and the application layer mainly, under the support of the platform layer and based on network-wide equipment and operation data, carries out a series of correlated application researches related to the power distribution network on scientific modeling, load flow calculation, fault analysis, closed-loop calculation, network reconstruction and self-healing simulation modules of the power distribution network comprising a lot of distributed generation, so as to realize comprehensive digital simulation of the power distribution network, and to provide decision reference for the construction and operation of the power distribution network.
Owner:STATE GRID CORP OF CHINA +1

Operation data processing method and system

The invention relates to an operation data processing method and system, in particular to an operation data processing method and system based on stream computation and batch computation for a double-engine business analysis system. According to the operation data processing method and system, a stream computation engine (22) is set in a system framework, so that data real-time processing capacity is improved, and a distributed file system (21) and a batch computation engine (24) are integrated based on a data warehouse (23), so that storage and computation of unstructured data and mass data are achieved, big data storage capacity and big data analysis capacity are improved, and the capacity for storing TB-level data and even PB-level data and the capacity for processing big data in a batched mode can be achieved. Meanwhile, due to the fact the data warehouse (23) is adopted for processing structural data, a micro-batch real-time data can be processed with a micro-batch processing method, and thus a real-time data processing mechanism is further perfected.
Owner:中移动信息技术有限公司 +1

Using predictive determinism within a streaming environment

Techniques are described for transmitting predicted output data on a processing element in a stream computing application instead of processing currently received input data. The stream computing application monitors the output of a processing element and determines whether its output is predictable, for example, if the previously transmitted output values are within a predefined range or if one or more input values correlate with the same one or more output values. The application may then generate a predicted output value to transmit from the processing element instead of transmitting a processed output value based on current input values. The predicted output value may be, for example, an average of the previously transmitted output values or a previously transmitted output value that was transmitted in response to a previously received input value that is similar to a currently received input value. Moreover, the processing element or elements that transmit the predicted output data may be upstream from the processing element with the predictable output.
Owner:IBM CORP

Low-coupling distributed streaming computation framework with master/slave architecture

The embodiment of the invention provides a low-coupling distributed streaming computation framework with a master / slave architecture. The low-coupling distributed streaming computation framework comprises a distributed service environment, a cluster management master node, cluster work slave nodes and cluster service process nodes; the distributed service environment is used for storing the operation states, the load states and the task execution states of various distributed nodes; the cluster management master node is used for managing the operation states of cluster nodes and distributing tasks to the various cluster nodes; the cluster work slave nodes are used for executing tasks distributed by the cluster management master node and caching intermediate results of task execution in a Kafka; and the cluster service process nodes are used for consuming messages generated in the Kafka and obtaining service results. The low-coupling distributed streaming computation framework can be applied to streaming data process under multiple conditions, modules are fully decoupled, dynamic shrinkage and expansion can be achieved, redundant backup of data is achieved, and the service reliability can be achieved by means of the backup mechanism of the master node.
Owner:武汉魅瞳科技有限公司

Service data real-time monitoring system and method based on Java byte code enhancement technology

The invention discloses a service data real-time monitoring system and method based on a Java byte code enhancement technology. The system comprises the components: a monitored application side, Agentbeing deployed in a JVM, when the application program is loaded into the JVM, the Agent modifying and loading the class of the JVM by using ASM, codes of various events and measurement values being dynamically inserted and collected when the application program runs, the newly-added codes executing and obtaining various data and then sending the data to the monitoring platform side, and the Agentperiodically obtaining and executing instructions such as starting and stopping of the Agent and thread analysis issued by the monitoring platform side; a monitoring platform side, used for acquiringthe data of the monitored application side, processing the acquired data and putting the processed data into message queues with different themes of the message middleware according to data types, aconsumer acquiring the data processing from the message queues according to the different themes and performing pre-computation on the data of the themes according to time windows and dimensions by utilizing stream computation; and a user side obtaining the calculation result and outputting the calculation result.
Owner:CHINA TELECOM SHANGHAI IDEAL INFORMATION IND GRP

Processing system and processing method of large-scale concurrent data stream

The invention discloses a processing system and a processing method of a large-scale concurrent data stream, relates to the technical field of data and aims to solve the technical problem of low processing efficiency of a stream processor. The system comprises a data stream unit (DSU) buffer area, a DSU cluster queue pool, a DSU mapping table, a stream processor pool, a data stream reading part, a DSU cluster allocation part, a task scheduling part and a computation post-processing part, wherein the stream processor tool consists of a plurality of graphics processing units (GPU); the data stream reading part is used for writing the concurrent data stream into the DSU buffer area; the DSU cluster allocation part is used for classifying data stream units which are currently processed in the DSU buffer area; the task scheduling part is used for loading a ready queue in the DSU cluster queue pool to the GPUs in the stream processor pool and carrying out stream computation; and the computation post-processing part is used for returning a computation result of the GPUs to the data stream. The system provided by the invention can improve the processing efficiency of the stream processor.
Owner:UNIV OF SHANGHAI FOR SCI & TECH

Method and software for calculating rate constraints

A method for determining and enforcing rate constraints on process flows in a network, the method comprising receiving at least one process flow. A flow rate vulnerability (FRV) factor and a required bandwidth (RBW) parameter are determined for said at least one process flow. A rate constraint is calculated for said at least one process flow and the rate constraint is enforced on said at least one process flow.
Owner:CISCO SYST ISRAEL

Network attack tracing method and device

The invention provides a network attack tracing method and device. The method comprises the following steps: when an attack event reported by security equipment in a network is received, acquiring anattack log which is generated by the security equipment and corresponds to the attack event, and extracting network address information of an attack source and an attack target in the attack log basedon streaming computation, storing the network address information in a memory, and then matching the network address information of the attack source and the attack target with a basic information database pre-established in the memory to obtain network position information of the attack source and the attack target; and finally, a network attack path diagram can be quickly generated according tothe network position information of the attack source and the attack target, and the whole network attack topology can be accurately restored in real time, so that a user can quickly alarm and repairattacked network assets and user terminals, and corresponding loss is reduced.
Owner:NEW H3C SECURITY TECH CO LTD

Matlab-based rectangular coordinate newton power flow calculation method

The invention discloses a Matlab-based rectangular coordinate newton power flow calculation method. Matrix operation and complex operation are adopted, a Jacobi matrix is formed according to the programming characteristic of Matlab, and the power of a node is calculated. The method comprises the following steps: calculating an initial Jacobi calculation matrix; calculating the complex power and the injection current phasor of the node; calculating partitioned sub-matrixes of the initial Jacobi matrix; correcting diagonal elements of the sub-matrixes of the Jacobi matrix by virtue of the injection current phasor of the node and a partial derivative of a voltage deviation to a voltage; forming the Jacobi matrix by virtue of the sub-matrixes of the Jacobi matrix, and performing regulation. The method is implemented on a Matlab platform, so that a researcher can conveniently test and analyze a calculation result by virtue of various tools and functions provided by Matlab. Practices show that convenience is brought to program compiling, modification and debugging of the researcher, the calculation speed is substantially approximate to the speed achieved on a C language platform and an excellent analysis tool is provided for the researcher.
Owner:DALIAN MARITIME UNIVERSITY

OPC data real-time acquisition monitoring intelligent system and method based on stream computing

The invention relates to an OPC data real-time acquisition monitoring intelligent system and method based on stream computing. The system comprises an OPC data acquisition server, an OPC centralized control management server, a Kafka data processing module and an OPC client. The OPC data acquisition server receives data from industrial workshop field equipment through a standard OPC interface, stores the received real-time data into a whole-plant real-time database, and transmits the data to the centralized control management server according to task requirements; and the Kafka data processingmodule receives the monitoring data and puts the monitoring data in a distributed message queue for caching, and constructs a processing architecture based on stream computing to perform unified processing and storage on the data. According to the invention, the data acquisition process is simplified, and the integration capability of the field equipment layer, the workshop monitoring layer and the manager decision-making layer is improved; and the real-time performance of monitoring result pushing and data processing access is improved, the problem that the response speed is low is solved, and meanwhile the reliability of data is guaranteed.
Owner:CHINA TOBACCO ZHEJIANG IND

Flow calculation method suitable for microgrids in various operation modes

The invention discloses a microgrid flow model suitable for various operation modes, and a calculation method. The invention discloses a flow calculation method suitable for microgrids in various operation modes, and the method comprises the steps: forming five types of microgrid operation modes and four types of nodes according to two types of basic operation modes and three types of control strategies of a microgrid; enabling the microgrid flow calculation in five types of different operation modes to be divided into two types of flow calculation; building a unified flow model; and proposing an LM-TR (Levenberg Marquardt method combined with trust region technique, LM-TR) flow algorithm for solving. The method does not need the jacobian matrix of a flow equation to be nonsingular, has global convergence, is small in dependence on initial values of unknown variables, effectively improves the convergence of flow calculation, and can solve a problem that the convergence of flow calculation of a microgrid system is difficult because that the jacobian matrix is singular or there is no balance point. Moreover, the method can process various types of nodes conveniently, and can solve a problem of the diversity of microgrid flow calculation in different operation modes. The method is suitable for the flow calculation of the microgrid in various operation modes, and is good in engineering application value.
Owner:XIANGTAN UNIV

Storm based stream computing frame text index method and system

The invention discloses a Storm based stream computing frame text index method and system. The Storm based stream computing frame text index method includes: implementing topology of a storm, designing a Storm real-time data processing frame, and completing a webpage automatic extraction program of a web spider; automatically extracting key words; and classifying texts: classifying texts into one or more classifications according to content or attributes of the texts. The method and the system can allow backup data back in the conditions of data corruption or data loss, and recovers data; the function of centralized operation and maintenance of the system can be provided; an interface is beautiful and practical, and a convenient and visual graphical user management interface is achieved; function expansion can meet the demands of a user for later system expansion and use range expansion; and fault tolerance allows the system to have a certain fault tolerance mechanism when illegal data generates due to user input or wrong operations.
Owner:YUNNAN UNIV +2

Method and system for implementing a stream processing computer architecture

The present disclosure relates to a method for implementing a stream processing computer architecture, including creating a stream computer processing (SCP) system by forming a super node cluster of processors representing physical computation nodes (''nodes''), communicatively coupling the processors via a local interconnection means (''interconnect''), and communicatively coupling the cluster to an optical circuit switch (OCS), via optical external links (''links''). The OCS is communicatively coupled to another cluster of processors via the links. The method also includes generating a stream computation graph including kernels and data streams, and mapping the graph to the SCP system, which includes assigning the kernels to the clusters and respective nodes, assigning data stream traffic between the kernels to the interconnection when the data stream is between nodes in the same cluster, and assigning traffic between the kernels to the links when the data stream is between nodes in different clusters. The method also includes configuring the OCSs to provide connectivity between mapped clusters.
Owner:IBM CORP

Polar coordinate Newton method load flow calculation method based on Matlab sparse matrix

The invention discloses a polar coordinate Newton method load flow calculation method based on a Matlab sparse matrix, adopting matrix operation and complex operation and using a sparse matrix technology provided by Matlab. The method is implemented on a Matlab platform, so that researchers are facilitated to use various tools and functions provided by Matlab to test and analyze a calculation method. The main part of the method adopts both the matrix operation and the complex operation, reduces program codes and simplifies a programming process, so that the program is clearer, and the researchers are facilitated to modify the program, debug the program, and improve and add new functions for the program. The method provided by the invention adopts the sparse matrix technology of Matlab, so that calculation speed is greatly improved, meanwhile the sparse matrix of Matlab is very convenient to use, elements of the sparse matrix can be directly used by using a column number like a full matrix, design of a sparse storage structure is not needed, and an excellent analysis tool is provided for scientific research of the researchers.
Owner:DALIAN MARITIME UNIVERSITY

Block chain-based asset monitoring method

InactiveCN108694589ANo changes to workflowProtocol authorisationData setStreaming computation
The invention discloses a block chain-based asset monitoring method. The method comprises the following steps of: associating a stream computation engine for each participator node in a block chain network; realizing an intelligent contract according to a business and deploying the intelligent contract onto the block chain network; adding metadata analysis components on the participator nodes; defining rules and corresponding response actions in the stream computation engines; submitting lamination proposals to the block chain networks by participators through corresponding nodes; verifying and executing the proposals by the block chain network; recording an execution result of the intelligent contract into an account book copy by each participator node and then sending an event to the associated stream computation engine; and processing the events by the stream computation engine according to the predefined rules. According to the method, the block chain main networks do not need to be changed and the working processes of the block chain networks are not changed; and authorization mechanisms of the original block chain networks are perfectly compatible, and each stream computationengine only needs to read a data set belonging to the stream computation engine, so that asset problems can be recognized in real time or quasi real time.
Owner:杭州佑他科技有限公司

Automatic adjustment method and device for load flow calculation convergence

The invention discloses an automatic adjustment method for load flow calculation convergence. The method comprises the steps of designing a state, an action space and a reward of a deep reinforcementlearning network for load flow calculation convergence; constructing a deep reinforcement learning network for load flow calculation convergence according to the state, the action space and the reward; knowledge experience is added into the deep reinforcement learning network, and the process of manually adjusting the power flow is simulated, so that a power flow adjustment strategy is constructed, the power flow convergence of the power grid is adjusted by using the power flow adjustment strategy, and the problems of low convergence working efficiency, inaccuracy and overlarge manpower cost consumption of the current adjustment of the large power grid are solved.
Owner:CHINA ELECTRIC POWER RES INST +2

System and method for medical image processing

An embodiment of the present invention provides a system and method for medical image processing. The proposed system includes a grid computing framework adapted for receiving patient data including one or more patient-scan images from an end-user application, and for scheduling image processing tasks to a plurality of nodes of a grid computing network. Each of the nodes includes a central processing unit and at least one of the nodes includes programmable graphics processing unit hardware. The proposed system further includes a second framework for image processing using graphics processing unit that is operative on each node of the network. The second framework operative on any node is adapted to execute the image processing task scheduled to that node based upon the availability of graphics processing unit hardware in that node. When graphics processing unit hardware is available in the node, the second framework is adapted to execute the task on the graphics processing unit of the node using stream computation. When graphics processing unit hardware is not available in the node, the second framework is adapted to execute the task on the central processing unit of the node.
Owner:SIEMENS AG

Large-range dynamic wave real-time rendering method based on stream computing

InactiveCN104143207AInteractive realization producesInteractive implementation disappears3D-image renderingSkyLevel of detail
The invention discloses a method for achieving large-range dynamic wave efficient real-time rendering on a stream processor. According to the method, a novel grid model based on the LOD (Level of Detail) technology is involved, dynamic wave simulation is achieved based on a Gerstner model, a concave and convex texture mapping method accelerates remoter wave rendering, and animation concave and convex textures are utilized for improving the wave rendering effect. The method is characterized in that the reflecting effect on the sky and the sun is achieved through the cube textures, a Phong illumination model is applied to achieve wave illumination and achieve Fresnel reflection, the animation textures are adopted for simulating foam in waves, and the special effects such as shadows, projected onto the sea surface, of clouds are achieved. The method can carry out real-time interactive rendering on waves within the hundred-kilometer-level range on a computer platform with the stream processor, and has the advantages that the speed is high, the simulation degree is high, and platform compatibility is good.
Owner:HARBIN DIANSHI SIMULATION SCI & TECH

User travel track extraction method and system based on signaling big data

ActiveCN111209261AFast fuzzy querySolve the problem of missing real-time trajectory dataDatabase queryingMulti-dimensional databasesEngineeringStreaming computation
The invention discloses a signaling big data-based user travel trajectory extraction method and system, which are used for mining a user travel trajectory sequence from massive signaling data, obtaining a user position switching data sequence in an offline trajectory extraction process, detecting a user residence point by utilizing residence time and extracting a user offline trajectory. In real-time trajectory extraction, streaming computation and Geohash computation are utilized to record a Geohash grid where each user is currently located, the Geohash grid is used as a resident point, and anew resident point is generated, so that real-time trajectory data is acquired. By adopting the technical scheme provided by the invention, the real-time track and the offline track of the user can be extracted, and the flexibility is high.
Owner:邑客得(上海)信息技术有限公司

Task flow computation model

The invention discloses a task-flow computational model applicable to a reconfigurable high-performance computer system. A form of Five-Tuple Array (T, R, SC, EC, TS) represents the way for solving a problem under the mode of task-flow, wherein, T is a finite set of tasks, R is a relation set among tasks, SC is a starting condition set of tasks, EC is a terminal condition set of tasks and TS is a finite sequence set of tasks. On a premise of no restriction by factors such as resources, the preferred plan for solving problems is possibly used as a drag, the grain size of soft and hard task are reasonably divided, the system is restructured according to problems and requirements of task and high cost-effectiveness and high utilization rate are achieved. The task-flow computational model processes tasks as a basic unit, provides the definition of task-flow, the division principles of tasks as well as various forms of task-flow. The model can be used for the parallelism depiction of different grain sizes and lays a foundation for accurately and clearly dividing tasks and depicting task-flow oriented reconfigurable high-performance computation.
Owner:THE PLA INFORMATION ENG UNIV

Method for calculating probabilistic load flow of power system based on diverse random variables

The invention discloses a method for calculating probabilistic load flow of a power system based on diverse random variables. The method comprises the steps: S100. calculating load flow response generated by random variables of continuous type by using a semi-invariant method; S200. calculating load flow response generated by random variables of discrete type by using repeatedly-determined load flow; S300. carrying out convolution processing on calculated results obtained in the above two steps, so as to obtain load flow response during combined action of the random variables of the continuous and discrete types. According to the method, operations of the random variables of the continuous and discrete types in the calculation of the probabilistic load flow of the power system can be taken into account simultaneously, so that the calculation accuracy is high, and the calculating speed is high.
Owner:SOUTHWEST JIAOTONG UNIV

Tax terminal system log real-time analysis method, device, medium and system

The invention discloses a tax terminal system log real-time analysis method and device, a medium and a system. The method comprises the following steps: carrying out multi-thread distributed real-timeacquisition on log data generated by tax terminal system business operation; caching the acquired log data into a message queue cluster, and performing distributed storage by taking the acquired logdata as original log data; extracting log data in the message queue cluster, performing real-time analysis processing on the log data based on a stream computing tool, and storing an analysis result in real time; and providing an operation interface, and performing query, query result display and offline analysis processing on the original log data based on a big data calculation engine through the operation interface, or performing real-time query and query result display on an analysis result stored in real time through the operation interface according to a query condition. Related information can be rapidly and accurately obtained, and system problems can be rapidly positioned.
Owner:AEROSPACE INFORMATION

Kinetic energy data processing system and method based on Spark computing framework

The invention relates to a kinetic energy data processing system based on a Spark computing framework. The kinetic energy data processing system comprises a data acquisition module used for monitoringkinetic energy data of an industrial workshop, a data processing and analyzing module used for carrying out data computing on data streams issued to a Kafka message queue, and a distributed data storage module. Considering the variety of the data acquisition terminals, the Kafka cluster is provided with functional interfaces capable of being accessed to various data acquisition terminals, so thata flexible and extensible data acquisition system is established, and the problem of access of a plurality of data terminals is avoided. And secondly, as the amount of detection data needing to be processed is large, the stream data from the Kafka message queue is processed by adopting stream calculation, and the calculation performance can be greatly improved. And thirdly, the real-time performance of processing and analyzing the data according to the decision requirement of the industrial workshop on equipment operation is high, so that a distributed storage mode is adopted for the data.
Owner:CHINA TOBACCO ZHEJIANG IND

Method and device for management of application programming interface (API) call records

The invention relates to the technical field of flow calculation and provides a method for management of application programming interface (API) call records; the method comprises the steps: acquiring a plurality of API call records; determining the respective affiliated charging period of each call record in the plurality of the API call records according to the Called time of each call record in the plurality of the API call records; superposing the resource usage amount of each of more than two API call records, which are located in the same charging period, have the same caller mark and have the same charging unit price, in the plurality of the API call records according to the preset charging unit price, and obtaining one API call record subjected to superposition. With adoption of the technical scheme, the quantity of the API call records to be processed by an API charging gateway can be reduced and partial processing resources can be released.
Owner:HUAWEI TECH CO LTD

Method, computer program product, and system for a self-throttled computing task

A throttling specification is received by a computer task to direct the computer task's usage of critical computer resources, such as CPU cycles. The computer task comprises units of work, and the elapsed time of each unit of work is calculated as each unit of work completes. Upon the completion of one unit of work, a suspension time is calculated based at least partially on the throttling specification and the elapsed time corresponding to the completed unit of work. Prior to initiating the next unit of work for the computer task, the computer task is suspended for the calculated suspension time. In this manner, other important computing tasks operating in the computer system have access to critical computer resources during the suspension period. Even though the computer task may be given the highest possible priority to ensure a timely and predictable completion, total monopolization of critical computer resources is avoided.
Owner:IBM CORP

Efficient FPGA technology mapping algorithm

The invention discloses an efficient FPGA technology mapping algorithm. Technical mapping is divided into logic optimization and structural optimization. The logic optimization part adopts an AIG model and technologies of balance, rewrite, refactor and the like to optimize a circuit. The structural optimization part adopts a DAG model and comprises three steps of partition generation, partition selection and LUT mapping. For the partition generation, a dynamic planning thought is adopted for quickly generating all k- feasible partitions for each node. For the partition selection, a partition set with a characteristic that a time delay and an area are optimized at the same time is finally selected through multi-time forward traversal and backward traversal iteration based on an iterative heuristic thought with an adaptively changeable iterative frequency. Meanwhile, a node area stream calculation formula is corrected and the randomness of the partition selection is improved. Through the LUT mapping, partition combination generated by the partition selection is mapped into an LUT network.
Owner:XIDIAN UNIV

Maximized memory throughput on parallel processing devices

In parallel processing devices, for streaming computations, processing of each data element of the stream may not be computationally intensive and thus processing may take relatively small amounts of time to compute as compared to memory accesses times required to read the stream and write the results. Therefore, memory throughput often limits the performance of the streaming computation. Generally stated, provided are methods for achieving improved, optimized, or ultimately, maximized memory throughput in such memory-throughput-limited streaming computations. Streaming computation performance is maximized by improving the aggregate memory throughput across the plurality of processing elements and threads. High aggregate memory throughput is achieved by balancing processing loads between threads and groups of threads and a hardware memory interface coupled to the parallel processing devices.
Owner:NVIDIA CORP

Stream computing platform and method based on visualization, equipment and storage medium

The embodiment of the invention discloses a stream computing platform and method based on visualization, equipment and a storage medium. The stream computing platform comprises a data development component, a stream computing component and an intelligent scheduling component, wherein the data development component is used for visually developing a corresponding stream computing application througha dragging operator in stream computing, and determining a stream computing engine corresponding to the stream computing application; the stream computing component is used for analyzing scheduling parameters carried in the stream computing application and calling the stream computing engine to execute the stream computing application; and the intelligent scheduling component is used for scheduling execution of the stream computing application according to the scheduling parameters. The technical scheme provided by the embodiment of the invention does not need to be realized only by developers who deeply understand a stream computing application technology, so that the application range of the stream computing platform is expanded, and the expandability of different stream computing engines is supported, and the flexibility of stream computing development is improved.
Owner:CHINA CONSTRUCTION BANK
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products