Network flow analysis method based on GPU, Hadoop/Spark hybrid computing framework

A technology of network traffic and hybrid computing, applied in the field of network communication, it can solve the problems such as the impossibility of long-term fine-grained backtracking of traffic and the inability of real-time data processing speed to reach the second level.

Active Publication Date: 2017-03-15
中国人民解放军91655部队
View PDF4 Cites 14 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The disadvantages caused by this method include: on the one hand, the real-time data processing speed cannot reach the second level; on the other hand, it is almost impossible to trace back the long-term fine-grained traffic

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Network flow analysis method based on GPU, Hadoop/Spark hybrid computing framework
  • Network flow analysis method based on GPU, Hadoop/Spark hybrid computing framework
  • Network flow analysis method based on GPU, Hadoop/Spark hybrid computing framework

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0060] Embodiments of the present invention are described in detail below, examples of which are shown in the drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary only for explaining the present invention and should not be construed as limiting the present invention.

[0061] Those skilled in the art will understand that unless otherwise stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the word "comprising" used in the description of the present invention refers to the presence of said features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, Integers, steps, operations, elements, components, and / or groups thereof. It will be understoo...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a network flow analysis method based on a GPU, Hadoop / Spark hybrid computing framework. The method mainly comprises the following steps: constructing a GPU computing analysis framework and a Hadoop / Spark computing analysis framework, selecting the GPU or Hadoop / Spark computing analysis framework to process real time or offline network flow, wherein the GPU computing analysis framework is deployed on a stand-alone node installed with the GPU, the Hadoop / Spark computing framework is a distributed processing system and deployed in a server cluster, preferentially adopting the GPU computing analysis framework to process the real-time or offline network flow when the size of the available memory of the GPU is greater than or equal to twice network flow data. Through the construction of the GPU computing analysis framework and the Hadoop / Spark computing analysis framework, the GPU or Hadoop / Spark computing analysis framework is selected for processing the real-time or offline network flow, thereby effectively responding the real-time or offline statistical analysis processing of the high-speed network flow; and an operator, a maintainer and a manager can conveniently backtrack the analysis data.

Description

technical field [0001] The invention relates to the technical field of network communication, in particular to a network traffic analysis method based on GPU and Hadoop / Spark hybrid computing framework. Background technique [0002] Statistical analysis of network traffic is mainly divided into real-time and offline computing. Real-time computing is mainly aimed at real-time analysis scenarios such as clustering statistics, sorting, ranking, filtering, and abnormal monitoring of traffic by unit and information system; offline computing is mainly aimed at offline scenarios such as retrospective analysis and performance evaluation of information system network traffic. The sensitivity and value of network traffic analysis depend on the speed of statistical processing of traffic data. Ideally, data analysis and processing should provide analysis results within a second-level time frame. [0003] Currently, network monitoring generally stores data in relational databases. The ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): H04L12/26
CPCH04L43/026H04L43/028H04L43/0876
Inventor 王璐唐威强
Owner 中国人民解放军91655部队
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap