Web click counting method based on web crawler behavior identification and buffering updating strategies

Abstract

The invention belongs to the technical field of web design, and particularly relates to a web click counting method based on web crawler behavior identification and buffering updating strategies. According to the method, firstly, a logic expression of the page organization structure of a web site is built, crawler behavior identification is conducted by comprehensively using client identity identification, web page logic structure matching and time attributes, and then on the basis of setting a buffering counting structure, click counting updating is conducted on the basis of identification results. By means of the method, crawler click behaviors can be identified correctly, the phenomenon that the counted number is artificially high is prevented, and meanwhile web real click counting updating can be conducted timely. The method is suitable for various applications needing linking or page click counting.

Description

technical field [0001] The invention belongs to the technical field of Web design and relates to a novel method for counting Web clicks, in particular to a novel calculation method designed on the basis of network crawler behavior pattern analysis combined with a buffer update strategy. Background technique [0002] On many websites, Web hit counts play an increasing role. For example, on the Oriental Fortune Stock Bar website, each post lists the number of clicks on the post on the left side of the post. The number of these clicks has a certain guiding value for investors to obtain information, and wrong counts are likely to cause misleading. [0003] Most of the Web page counters on the Internet currently use a simple counting method, that is, one click will increase the count value by one. This counting method is constantly being affected by web crawlers, and the resulting count makes the counting of surfing clicks of Internet users more and more unreliable. With the de...

AI Technical Summary

Problems solved by technology

[0005] User identification is simply based on the IP address and the Agent string of the client. Since the crawler program cannot be forced to identify itself, this method cannot distinguish crawlers from network users.

The click sequence model uses a statistical model to describe the conditional probability of two clicks before and after, and can describe the jump relationship between different links in user browsing behavior [3], although it describes the selectivity of network users when browsing websites from the statistical characteristics. behavior, but when it is actually applied to distinguish crawlers and network users, it will encounter the following problems: (1) For a network forum like the stock bar website, due to the huge number of posts, when using a statistical model to describe the jump relationship, the required The state space processed is very large, resulting in a large amount of sparse information in the model

However, since the similarity value is a likelihood value, its value range is not easy to estimate, which makes it difficult to determine the similarity threshold, so there are still big problems in practicality

Images