Data stream processing system multi-objective optimization method based on uncertainty

A multi-objective optimization and processing system technology, applied in the field of multi-objective optimization of data flow processing systems, can solve problems such as user response delay and throughput, Palito optimal solution randomness, etc.

Active Publication Date: 2017-06-13
BEIJING INSTITUTE OF TECHNOLOGYGY
View PDF4 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] For the existing multi-objective optimization method based on weight summation in the prior art, the user's trade-off between response delay and throughput rate is not considered during deployment, which results in the defect of Palito optimal solution randomness

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data stream processing system multi-objective optimization method based on uncertainty
  • Data stream processing system multi-objective optimization method based on uncertainty
  • Data stream processing system multi-objective optimization method based on uncertainty

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0061] This embodiment illustrates the process of applying the present invention "a method for multi-objective optimization of a data stream processing system based on uncertainty" to a specific real-time big data analysis system Apache Spark Streaming scenario.

[0062] figure 1 This is the algorithm flowchart of this method and the flowchart of this embodiment. As can be seen from the figure, this method includes the following steps:

[0063] Step 1: The upper bound L of the current response delay upper Initialized to 10.0, the lower bound of the current response delay L lower Is initialized to 0.5, and the threshold UA of the area of ​​the uncertain area is initialized to 10000.

[0064] Step 2: Calculate the upper and lower bounds of the current throughput rate according to the upper bound of 10.0 and the lower bound of 0.5 of the current response delay.

[0065] Step 2.1: Calculate the upper bound T of the current throughput rate according to the upper bound 10.0 of the current...

Embodiment 2

[0090] This embodiment specifically explains the recursive iterative detection described in step 5 of the present invention and the recursive iterative detection of step 5 in embodiment 1. The algorithm flow is as follows figure 2 Shown. From figure 2 It can be seen that the specific steps of recursive iterative detection are:

[0091] Step 5: Determine whether the current uncertain area area of ​​the left half and the right half is less than or equal to the threshold value of 10000 of the uncertain area area, and decide whether to perform recursive iterative detection.

[0092] Step 5.1: Determine whether the area of ​​the uncertainty region 183600.2723725793 in the left half is less than or equal to the threshold value of 10000 for the uncertainty region area, so as to decide whether to perform recursive iterative detection. The specific process is as follows:

[0093] Step 5.1.1: If the area of ​​the uncertainty region in the left half of 183600.2723725793 is less than or equal ...

Embodiment 3

[0109] The specific real-time big data analysis system Apache Spark Streaming in Example 1 is changed to other real-time big data analysis systems such as Apache Storm, Google Dataflow, etc., that is, the multi-objective optimization method proposed by the present invention is not limited to the source of the data. It is suitable for data processing in all engineering applications.

[0110] Relevant technical content not mentioned in the above embodiments can be realized by adopting or learning from existing technologies.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data stream processing system multi-objective optimization method based on uncertainty, relates to a multi-objective optimization method for a data stream processing system, and belongs to the field of the computer application technology and real-time big data analysis. An uncertainty region area is given according to the upper and lower bounds of response delay specified by a user and the upper and lower bounds of the throughput rate; based on the objective of reducing the uncertainty region area, a set of Pareto optimality solutions with typical representative significance are obtained through a recursive binary detection method, and selection space is provided for the user on response delay and the throughput rate. The method is suitable for real-time big data analysis system multi-objective optimization occasions, and is wide in application range, high in practicality and easy to popularize. In addition, the method only aims at processing of data itself without being limited by the source of the data, and is suitable for processing of data in all engineering application.

Description

Technical field [0001] The invention relates to a multi-objective optimization method for a data stream processing system based on uncertainty, in particular to a multi-objective optimization method for a data stream processing system, and belongs to the field of computer application technology and real-time big data analysis. Background technique [0002] In recent years, a large number of real-time big data analysis applications have emerged, such as social network dynamic analysis, intelligent transportation data analysis, large-scale data center monitoring, and genetic data analysis. This type of application not only has a large amount of data, but also the data is continuously and quickly generated or updated. It requires the data analysis system to return or update the analysis results continuously and in real time. We call it real-time big data (Big&fast data) analysis. Such applications have an urgent need for real-time big data analysis systems, and require the system to...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06Q10/04
CPCG06Q10/04
Inventor 曹朝盛伟曲大成
Owner BEIJING INSTITUTE OF TECHNOLOGYGY
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap