Method and system for rapidly counting frequencies
A frequency and fast technology, applied in the field of data statistics, can solve problems such as difficult parallelism, deterioration of system throughput, and system throughput limited by CPU processing power.
Active Publication Date: 2016-05-11
BEIJING UNIV OF POSTS & TELECOMM
View PDF2 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
When the query is too frequent, the system throughput will be seriously deteriorated
[0009] And some other methods, such as the frequency statistics system based on the filtering method, it is difficult to carry out further parallelism, so the throughput of this type of system is limited by the CPU processing capacity; and the parallel system of differential transmission can be more effective for the original The serial method is parallel, but this method does not fully maintain some good characteristics of the original serial frequency statistics method
[0010] To sum up, in the existing frequency statistics system, its final throughput is limited by the hardware performance
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0075] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.
[0076] Below in conjunction with accompanying drawing and embodiment the present invention will be further described:
[0077] Such as figure 1 As shown, the present invention proposes a system for fast statistical frequency, including more than two parallel data processing subunits and data summary units;
[0078] The data processing subunit is used for performing frequency statistics in parallel;
[0079] The data summarization unit is used to count the increme...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The embodiment of the invention discloses a method and system for rapidly counting frequencies belonging to the data counting field. According to the method and the system, frequencies are counted by a data processing subunit; the data of the data processing subunit are summarized by a data merging unit; increments are counted by a data summarizing unit; therefore, the current restriction that the throughput capacity of an algorithm is ensured depending on hardware performances is eliminated, thus copying with the increasing network flow velocities; meanwhile the extensibility of the system is ensured, thus copying with the rapidly increased demands of the traffic for the throughput capacities.
Description
technical field [0001] The invention belongs to the field of data statistics, in particular to a method and system for fast frequency statistics. Background technique [0002] In the current era of big data, more and more data streams need to be further processed to obtain more useful information. Counting the frequency of data streams is one of the important processing methods, and there are corresponding applications in many fields. For example, statistics on the frequency of clicks on IP addresses, statistics on the number of times advertisements are played in videos, and so on. In the field of flow counting, some frequency counting methods have been applied to various frequency counting systems. These methods all strive to use relatively less memory to obtain high precision and recall, while ensuring that the statistical system can handle as high a data flow as possible. [0003] However, as the speed of the network continues to increase, the existing frequency counti...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/24568
Inventor 刘军杨迅周文莉刘芳何大中
Owner BEIJING UNIV OF POSTS & TELECOMM