A Multi-objective Optimization Method for Data Stream Processing System Based on Uncertainty

A multi-objective optimization and processing system technology, applied in the field of multi-objective optimization of data flow processing systems, can solve the problems of Palito optimal solution randomness, without considering user response delay and throughput rate, etc., to achieve easy promotion and practicability Strong, efficiency-enhancing effect

Active Publication Date: 2019-12-06
BEIJING INSTITUTE OF TECHNOLOGYGY
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] For the existing multi-objective optimization method based on weight summation in the prior art, the user's trade-off between response delay and throughput rate is not considered during deployment, which results in the defect of Palito optimal solution randomness

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A Multi-objective Optimization Method for Data Stream Processing System Based on Uncertainty
  • A Multi-objective Optimization Method for Data Stream Processing System Based on Uncertainty
  • A Multi-objective Optimization Method for Data Stream Processing System Based on Uncertainty

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0061] This embodiment describes the process of applying the present invention "a multi-objective optimization method for data stream processing system based on uncertainty" to a specific real-time big data analysis system Apache Spark Streaming.

[0062] figure 1 It is an algorithm flow chart of this method and a flow chart of this embodiment. As can be seen from the figure, the method includes the following steps:

[0063] Step 1: Upper bound on the current response latency is initialized as , the lower bound of the current response delay is initialized as , the threshold of the area of ​​uncertainty is initialized as .

[0064] Step 2: According to the upper bound of the current response delay and the Nether , respectively calculate the upper bound and lower bound of the current throughput.

[0065] Step 2.1: According to the upper bound of the current response delay , calculate the upper bound of the current throughput rate for ,Calculated as follow...

Embodiment 2

[0090] This embodiment specifically elaborates the recursive iterative detection described in step 5 of the present invention and the recursive iterative detection in step 5 in embodiment 1, and the algorithm flow is shown in Figure 2. As can be seen from Figure 2, the specific steps of recursive iterative detection are:

[0091] Step 5: Determine whether the area of ​​the uncertain region of the current left half and right half is less than or equal to the threshold of the area of ​​the uncertain region , to decide whether to perform recursive iteration detection.

[0092] Step 5.1: Determine the area of ​​the uncertainty region in the left half Is it less than or equal to the threshold of the area of ​​uncertainty , so as to decide whether to perform recursive iteration detection, the specific process is as follows:

[0093] Step 5.1.1: If the area of ​​the uncertain region in the left half Less than or equal to the threshold for the area of ​​uncertainty , then th...

Embodiment 3

[0109] Change the specific real-time big data analysis system Apache Spark Streaming in embodiment 1 into other real-time big data analysis systems such as Apache Storm, Google Dataflow, etc., that is, the multi-objective optimization method proposed by the present invention is not limited to the source of data, Suitable for data processing in all engineering applications.

[0110] Relevant technical contents not mentioned in the above embodiments can be realized by adopting or referring to existing technologies.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data stream processing system multi-objective optimization method based on uncertainty, relates to a multi-objective optimization method for a data stream processing system, and belongs to the field of the computer application technology and real-time big data analysis. An uncertainty region area is given according to the upper and lower bounds of response delay specified by a user and the upper and lower bounds of the throughput rate; based on the objective of reducing the uncertainty region area, a set of Pareto optimality solutions with typical representative significance are obtained through a recursive binary detection method, and selection space is provided for the user on response delay and the throughput rate. The method is suitable for real-time big data analysis system multi-objective optimization occasions, and is wide in application range, high in practicality and easy to popularize. In addition, the method only aims at processing of data itself without being limited by the source of the data, and is suitable for processing of data in all engineering application.

Description

technical field [0001] The invention relates to a multi-objective optimization method for a data flow processing system based on uncertainty, in particular to a multi-objective optimization method for a data flow processing system, which belongs to the field of computer application technology and real-time big data analysis. Background technique [0002] In recent years, a large number of real-time big data analysis applications have emerged, such as social network dynamic analysis, intelligent traffic data analysis, large-scale data center monitoring, genetic data analysis, etc. Such applications not only have a large amount of data but also continuously and rapidly generate or update the data, requiring the data analysis system to continuously and real-time return or update the analysis results, which we call real-time big data (Big & fast data) analysis. Such applications have an urgent need for real-time big data analysis systems, and the system needs to provide quantita...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F17/00G06Q10/04
CPCG06Q10/04
Inventor 曹朝盛伟曲大成
Owner BEIJING INSTITUTE OF TECHNOLOGYGY
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap