Log-based user behavior data processing method, medium, device and device

Abstract

The invention discloses a log-based user behavior data processing method, a medium, a device and a device. The method comprises: 1, collecting log information; 2, determining a plurality of access characteristics, and extracting access behavior values of different users for different access characteristics accord to log information; 3, dividing that access characteristic into N groups, and determine the cluster number of each group; N is an integer greater than or equal to 1; step 4, clustering each grouping according to the number of clusters corresponding to the grouping to obtain clusteringresults. The invention solves the problem of poor high-dimensional clustering effect by constructing high-dimensional data as low-dimensional features composed of a plurality of similar or related features and clustering. And by constructing the high-dimensional data into a plurality of similar or related features of the low-dimensional features after clustering, the number of clusters per sub-clustering process is reduced, saving computational time and saving computational memory.

Description

Technical field [0001] The present invention relates to the field of Internet technology, in particular to log-based user behavior data processing methods, media, equipment and devices. Background technique [0002] With the development of Internet services, massive web server (webserver) system logs are generated every day due to user access on the Internet. The web server system log mainly includes: client IP address, client user name, access time, request uri, request status, file size, page link source, client browser and other information. In the prior art, web server system logs are used to classify user behaviors, but the existing classification methods mainly have the following difficulties: [0003] First, when the correspondence between the Uniform Resource Identifier (URI) and its content cannot be known, and the correspondence between the content and the content category cannot be known, it is difficult to use such logs to classify user interests. [0004] Second, the f...

AI Technical Summary

Problems solved by technology

[0003] First, it is difficult to use such logs to classify user interests without knowing the correspondence between Uniform Resource Identifier (URI) and its content, or the correspondence between content and content category

[0004] Second, the feature extraction of the web server is mainly statistical features, such as count, average value, standard deviation, etc., which are access behavior features but do not include access targets, so the access targets of the clustered clusters will be inconsistent, which will lead to Misjudgment

Taking KMeans, the most common clustering method, as an example, the space complexity is O(n*m), where n is the number of data items and m is the number of features. The increase of n and m makes clustering consume a lot of system m

Images