Side-end collaborative concurrent real-time streaming data analysis task processing method and device
A technology for data analysis and task processing. It is used in digital data processing, resource allocation, program control design, etc. It can solve problems such as high latency, processing queues, and long processing time, so as to improve analysis accuracy and optimize accuracy. Effect
Pending Publication Date: 2021-08-24
NANJING UNIV
View PDF0 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
When the sampling period of a real-time streaming data analysis task and the sampling rate in each sampling period are high, it requires more processing time, which may cause queues for processing other real-time streaming data analysis tasks, resulting in a relatively high high latency
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0060] The technical solutions of the present invention will be further described below in conjunction with the accompanying drawings and embodiments.
[0061] The present invention aims at the profiling process of concurrent real-time streaming data analysis tasks and resource allocation in the execution process under the constraints of computing resources of the edge computing system, by coordinating the profiling process of concurrent real-time streaming data analysis tasks and the execution process for the edge The use of computing resources of the computing system improves the analysis accuracy of concurrent real-time streaming data analysis tasks in the edge computing system. The optimization concept is that by allocating less resources at the server site for the profiling process of each configuration of the real-time streaming data analysis task, and at the same time using the saved resources for the execution process of the real-time streaming data analysis task, it ca...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a side-end collaborative concurrent real-time streaming data analysis task processing method and device. The method comprises the following steps of performing preliminary screening on configurations of real-time streaming data analysis tasks running on a server site based on Pareto optimum, and determining an optional configuration set of the real-time streaming data analysis tasks; according to the selectable configuration set of the real-time streaming data analysis tasks, selecting a side writing strategy and an execution strategy for all the real-time streaming data analysis tasks executed on the site based on the principle that the total utility value of the tasks executed by the server site is maximized; and executing the real-time streaming data analysis task according to the determined side writing strategy and the execution strategy. According to the method, few resources are distributed at a server site to be used for the side writing process of each configuration of the real-time streaming data analysis task, and the saved resources are used for the execution process of the task, so that long-term optimization in the aspect of accuracy is realized under the condition that the execution accuracy in the current period is slightly influenced.
Description
technical field [0001] The invention belongs to the field of edge computing, and specifically relates to a concurrent real-time streaming data analysis task processing method and device for edge-end collaboration. Background technique [0002] With the development of artificial intelligence technology, real-time streaming data processing has become a key means of information acquisition. On the one hand, many applications require low latency for real-time streaming data processing. On the other hand, in order to reduce the pressure on the WAN caused by transmitting a large amount of metadata on the WAN, more and more real-time streaming data analysis tasks are performed near the data generation. Execute on the edge site at the end. For real-time streaming data analysis tasks, the higher the sampling period of data processing, the higher the sampling rate of each sampling, and the higher the accuracy of data processing. Due to the limited computing resources available on th...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More IPC IPC(8): G06F9/50H04L29/08
CPCG06F9/5038G06F9/5072H04L67/10Y02D10/00
Inventor 钱柱中吴昆陆桑璐
Owner NANJING UNIV