A cloud storage log data analysis method

Abstract

The invention relates to a cloud storage log data analysis method. The method includes the steps that firstly, preanalysis is conducted on cloud storage log data; secondly, the cloud storage log data obtained after preanalysis are calculated, so that a frequent item set required for generating a relation maxim is obtained; thirdly, according to the frequent item set obtained in the second step, the relation maxim of cloud storage logs is generated; fourthly, the relation maxim obtained in the third step is output. According to the method, the frequent item set is simplified, so that the scale of a generated candidate item set matrix is reduced, and the number of candidate item sets generated in the following iterative computation process is effectively reduced; in addition, by the improvement of the technical scheme, the candidate item set matrix is computed through user-defined matrix operations, the whole computation process is relatively simple, the computation amount of the data analysis process can be reduced, and excavation time is shortened.

Description

technical field [0001] The invention belongs to the technical field of data analysis, and in particular relates to a cloud storage log data analysis method, which can be used for data analysis of cloud storage system logs. Background technique [0002] During the operation of the cloud storage system, a large number of log files will be generated. These log files record the operation of the system by the system administrator, the access of the user to the system, and various original information such as system server reception, analysis requests, and runtime errors. Data analysis of system administrator operation logs can standardize administrator operations; data analysis of user access logs can discover user behavior habits, which is conducive to querying and analyzing user operations and improving user satisfaction; cloud storage Data analysis of server logs can detect system status, troubleshoot network faults, implement intrusion detection, and discover design defects,...

AI Technical Summary

Problems solved by technology

[0005] However, the existing matrix-based Apriori algorithm has the following problems: firstly, the calculation amount of the algorithm is relatively large, and when the analyzed database contains many data items, the time spent by the algorithm will increase exponentially, so when dealing with a large amount of data It will take more time to analyze; secondly, the algorithm will generate too many candidate item sets during the iteration process, storing these candidate item sets will occupy memory space, and increase the amount of calculation when performing subsequent iterative calculations

These shortcomings are not conducive to the rapid extraction of association criteria from cloud storage logs, resulting in the data analysis process of the entire cloud storage log taking a long time, the efficiency is not high, and the running status of the cloud storage system cannot be reflected in time, which is not conducive to the system. Optimizations and performance improvements

Images