Ultra-large-scale dpi data processing system and processing method based on edge computing

Abstract

The present invention reveals a large -scale DPI data processing system and corresponding data processing methods based on edge -based computing. The system includes: DPI plug -in unit, deployed in the gateway, for data information collection, cleaning and analysis, generating reports, and reporting reports.Information is uploaded; data transmission unit is used to receive report information, encrypted and forwarded the report information; cloud processing unit is used to receive report information, decrypt, store and process the report information, and finally output data processing results.The present invention fully applies the ideas of marginal computing into the business of network operators' DPI data processing, solving various problems existing in the cloud computing model during the process of processing of traditional business, and enables the response performance and stability of the system as a whole.It has been significantly improved.

Description

technical field [0001] The present invention relates to a data processing system and a corresponding data processing method, in particular to a super-large-scale DPI data processing system and processing method based on edge computing, belonging to the technical field of DPI. Background technique [0002] DPI (Deep Packet Inspection) is a packet-based in-depth inspection technology that conducts in-depth inspections on different network application layer loads (such as HTTP, DNS, etc.), and determines its legality by inspecting the payload of the message. [0003] The DPI technology detects and analyzes the traffic and message content at key points in the network, and filters and controls the detected traffic according to the pre-defined policy, so as to realize the fine identification of services on the link, the analysis of business traffic flow, and the proportion of business traffic. Various functions such as ratio statistics, business ratio shaping, application layer de...

AI Technical Summary

Problems solved by technology

[0005] 1. It is difficult to realize the processing of multi-source heterogeneous data

Since the data in the perception layer of the system is massive, there are frequent conflicts and cooperation between the data, which has strong redundancy, correlation, real-time and multi-source heterogeneous characteristics, and the fusion of multi-source heterogeneous data Both cloud computing and real-time processing requirements have brought huge challenges that cannot be solved

[0006] 2. Large bandwidth load and waste of resources

Since cloud service is a centralized service computing model with a high degree of aggregation, users will consume a large amount of network bandwidth and computing resources when sending data to the cloud for storage and processing, resulting in problems such as excessive bandwidth load and waste of resources.

At the same time, a large number of user visits will further increase network traffic, which will cause service interruption, network delay and other problems

[0007] 3. Resource constraints are serious

Since in the current Internet of Everything model, network edge devices are usually resource-constrained (storage, computing power, and battery capacity, etc.), the energy consumption of long-distance transmission of data between edge devices and cloud computing centers in the current model particularly prominent

[0008] 4. Low level of security and privacy protection

In the current cloud computing processing mode, it is necessary to upload data information related to personal privacy such as network edge data to the cloud computing center for processing, and the user's private data is easily intercepted by a third party, resulting in the leakage of user private data

Images