45 results about "Log mining" patented technology

The invention discloses a method for automatically acquiring a user behavior log of a network. The invention is characterized in that a proxy server is added between a client and a Web server; the text of a response returned from the Web server and received by the proxy server for the first time is a user response in an HTML (hypertext markup language) format, that is, an HTTP (hyper text transport protocol) response; and a user behavior log acquisition module is added to the HTTP response and sent to the client, wherein, the user behavior log acquisition module serves as a data acquisition terminal for the user behavior log at the client, whereas the proxy server serves as a data processing terminal for the user behavior log. Based on the intermediate strategy of the 'proxy server', the invention focuses on the key technologies, such as log acquisition of clients, strategy deployment of proxy servers, visual configuration of log acquisition and the like. Compared with the conventionallogging scheme, the invention can better the user association and ensure the good expansibility; and by conveniently and flexibly customizing the user behavior log to be acquired by a target system on a visual UI (user interface), the invention can further provide a data foundation for log mining.

The invention provides a topic venation digging method and system based on massive searching logs. The method includes: counting the web searching logs to produce statistic data of a first searching word; counting the statistic data of the first searching word into statistic data of a second searching word; extracting key words of a first statistic record; counting total searching number of the keywords to obtain total searching statistic data of the keywords; calculating a searching heat value of the keywords in unit time; determining a comprehensive searching heat value of a topic in unit time; determining key time nodes of the topic. By the method and the system, topic shift caused by topic correlation can be avoided effectively, and a clear and complete topic venation can be dug out.

The invention discloses an information push method based on internet-surfing log mining and user activity recognition.The method includes a data preprocessing stage, a potential pushing user screening stage and a timely push stage.The data preprocessing stage comprises a webpage portrait, webpage level classification and a user portrait.In the potential pushing user screening stage, similarity between new information and historical web pages which users have access to is solved through a method based on matrix decomposition, and potential pushing users who may be interested in the new information are screened out.The timely push stage includes situational information acquisition and activity recognition based on a mobile intelligent device, and information timely pushing adaptive to the activity situation.By mining internet-surfing log data, potential users interested in information are found, when new information is generated, the new information is pushed to the potential users in good time, and therefore information push efficiency is improved.

The invention provides a Web log data preprocessing method for Web log mining systems, which comprises the following steps of: firstly, deleting useless information (namely, needless records) by using a default rule library, and completing the cleaning of Web log data through correcting the rule library; then, solving user identification problems caused by agencies and firewalls by using a heuristic rule, and completing the session identification on users through comprehensively considering home pages and navigation pages; and finally, based on a web page reference relation, supplementing access paths so as to obtain a final page access sequence of each user accessing Web pages, thereby completing the preprocessing of Web log data. Compared with the traditional session identification implemented according to simple time thresholds, by using the method provided by the invention, the accuracy and reliability of access behavior mining of users are obviously improved.

The invention discloses a family member mix identification method and system based on television watching log mining. The method comprises the steps of acquiring log data of television programs watched by a user, wherein the log data comprises following data content: user operation time, program type, and watching time of the program type; dividing the log data by time period; extracting the program type and the proportion of the watching time of the program type from each period of log data and taking the program type and the proportion of the watching time of the program type as a feature of a family member; clustering according to features of family members; and determining the mix of family members by use of a clustering result. Through adoption of the method, the problem in the prior art that user preferences cannot be analyzed through a single account in the intelligent television field is solved. The invention also discloses a family member mix identification system based on television watching log mining.

A system for performing data mining on log record data includes a distributed processing system comprising a plurality of distributed nodes, each node having at least one of a storage device and a data processing device and a distributed data analytics processor configured to perform queries on a set of log records to select at least a portion of the log records and store the selected log records in a plurality of the distributed nodes, wherein the distributed data analytics processor is further configured to perform data analysis on the selected log records stored in the plurality of distributed nodes. The distributed nodes may be arranged in a cluster, with each node being in communication with each other node through a distributed file system. The processor may perform analysis on data stored across the distributed nodes with the processing being performed by a plurality of the distributed nodes in parallel.

The invention discloses a website classification catalogue optimization analysis method based on log mining. According to the method, website log data is firstly preprocessed, wherein the log data refers to a series of webpage access data sets recorded on a server; through preprocessing, a catalogue path through which a user obtains information via a specific website is extracted from the log data; then, a method (VOB) based on the browsing path sequence is used for calculating the similarity between any two catalogue paths until a catalogue path similarity matrix is constructed; then, a divisive hierachical clustering (NHC) algorithm based on matrix transformation is used for performing clustering on the catalogue path similarity matrix, so that users corresponding to the catalogue paths are clustered into different categories; and finally, expected website classification catalogue systems of each category of users are mined out, and are subjected to comparison analysis on the original classification catalogue system. Through the steps, the website classification catalogue systems conforming to the expectation of the users can be mined out, and the quantitative decision support is provided for the website optimization.

The invention discloses a multi-source database real-time fusion system and a fusion method thereof, and belongs to the technical field of database management. The system comprises a source database,a rule definition module, a rule execution module, an operation monitoring module and a big data center; the source database adopts Oracle, and performs memory log mining by utilizing LogMiner in theOracle; the rule definition module comprises source database configuration information, target database configuration information, a rule conversion plug-in and a rule definition result; the rule execution module comprises a rule analysis component, a message queue and a rule execution component; the operation monitoring module is used for counting the number of messages in a message queue by adopting a graph; and the big data center uses an HDFS to store historical data. The problem of low timelines and customizability of database fusion is solved; and by means of the scheme, under the condition that the business operation is not affected, production data can be customized in real time according to the requirements of the data center, so that the timelines and the customizability are achieved.

The invention provides an asset automatic discovery method and device, and relates to the technical field of information security, and the method comprises the steps: configuring an asset discovery execution engine based on a preset engine configuration table, the asset discovery execution engine comprising one or more of a flow detection engine, a port scanning engine, and a log mining engine; and discovering asset information in a preset range by using the configured asset discovering execution engine. According to the method, the network assets are automatically discovered in a mode of combining flow detection, port scanning and log mining, so that monitoring is realized, and a user can conveniently manage and maintain the network assets in the jurisdiction range of the user.

The invention provides a log mining method based on a Petri network behavior profile, and belongs to the field of process mining. The method aims to improve a business progress model by means of implementation of a mining technology, so that procedure operation efficiency and service quality are improved. The method includes the steps that firstly, log sequences are extracted from a track generated in program running and pre-processing is carried out; the behavior relation between two adjacent actions is analyzed, the behavior profile is built, and an initial model is built according to direct dependency; then the log sequence with the largest occurrence frequency is replayed, and two procedure models superior to the initial model are found according to indexes; the consistency degrees of the two models and the initial model are respectively compared by means of behavior profile consistency so as to choose the more superior model; the last two steps are repeated until log replay is finished so as to obtain the optimal business procedure model. The log mining method can guarantee that the business procedure meets the requirement of functionality, enables procedure behaviors to be concise, and shortens program running time.

The invention discloses a method for log mining based on a physical level database. The method comprises the following steps: (1) obtaining the size of a database system data block, an on-line log full path name and a log storage file system format; (2) obtaining current log group information from a database system view, and obtaining operating information; (3) mining log file header information, and obtaining current log series from a log file header; (4) mining a data block after the log file header, mining data of a fixed size each time and recording currently mined cut-off position indexes, and writing a data block content into a newly-built temporary file; (5) when an end mark of the file header is met, writing the end mark into a data block temporary file, ending current file mining, and staring next log file mining. According to the method, consistency and integrity of mined log information and a production library can be achieved fundamentally, and the performance influence on the production library caused by database log mining can be reduced.

The invention discloses an abnormal access log mining method based on a website picture and an apparatus thereof. The method comprises the following steps of collecting an access log of a target website from a website server or a CDN node, cleaning the access log and acquiring a normal access log; analyzing the normal access log and constructing a website picture of the target website; and using the constructed website picture to analyze a website access log which is analyzed, and filtering an access log which is not in a website picture scope, wherein the access log is an abnormal access log. By using the method and the apparatus of the invention, processing efficiency is high; and accuracy of filtering the abnormal log is high and a unknown loophole can be covered.

A computer readable storage medium includes executable instructions to receive a specification for requested data in a log based replication system where data changes are retrieved from both a transaction log and a database. It is determined that the requested data is compromised. A search for a data image containing the requested data is initiated. The data image is accessed to secure the requested data. The requested data is returned.

The invention relates to the OLAP recommending technology, in particular to an online analytical processing (OLAP) query log mining and recommending method based on efficient mining of frequent closed sequences (BIDE). In the method, the possible next query is recommended to OLAP users, so that the process of browsing and analyzing multi-dimensional data by the users is simplified. The method has the advantages that: based on the characteristic of query operation in the field of OLAP, fields for expressing OLAP operation in log files are extracted, and the log files are abstracted to form a query sequence, so that the expression method of the log files is simplified; a query pattern is mined in the query sequence by a BIDE algorithm, so that the efficiency of subsequent recommending is improved on the premise of ensuring that the recommending accuracy is not reduced; a suffix tree is established on the query mode, so that a starting point of query matching is not needed to be searched for by a search algorithm in subsequent pattern matching to improve the speed of the pattern matching; and a matching algorithm of a fuzzy query pattern is provided to improve the recommending accuracy.

The invention discloses a commodity recommendation method based on log mining, a device, a server and a storage medium. The present invention introduces a transaction database, Mining frequent itemsets from transaction databases, and designs a new calculation method of commodity similarity, Then the similarity calculation results and Pearson correlation coefficients are weighted synthesized, The problem of inaccurate similarity measurement caused by sparse data is alleviated, and the nearest neighbor set is found and recommended more accurately, which overcomes the problem that the collaborative filtering algorithm overly relies on the user's scoring data when calculating the similarity of items.

The invention discloses an APT attack identification and defense method. The method comprises the following steps: 1, acquiring network and system logs, and identifying APT attack behaviors; 2, miningan attack subject based on the weblog, and determining attack subjects of different targets; 3, counting the attack timelines of the attack subjects of different targets, and predicting the next attack time of the attack subjects of different targets; and 4, acquiring attack file types expected by the attack subjects of different targets, manufacturing false files of the same type, and providingthe files to an APT attacker according to the predicted next attack time of the attack subjects of the different targets so as to perform APT attack defense. According to the method, the APT attack can be identified, the attack subjects of the APT attack can be mined, the attack timelines can be predicted, a large number of error files can be automatically generated and tampered for mixed audiovisual operation, the error files can be automatically deleted after the attack timelines, the file content information can be recovered, and the problem of the APT attack can be fundamentally solved.

The invention discloses a push system of personal information and is related to the information technology field. The system includes a personal interest acquisition system, an information push system and an information acquisition system; a user end includes a data receiving module, a user registration module, a user searching module and a user modification module; the personal interest acquisition system includes a user attribute static analysis module, a word frequency statistics module, a time analysis module, a log mining analysis module, an analysis module and a personal interest set; the information push system includes an information retrieval module, an interest acquisition module and an information sending module; the information acquisition system includes an information acquisition module, an information classification module and a data base. The system provides an information push system according to the personal interest and is of high accuracy, high learnability and strong pertinence.

A method for context-aware data mining of a text document includes receiving a list of words parsed and preprocessed from an input query; computing a related distributed embedding representation for each word in the list of words using a word embedding model of the text document being queried; aggregating the related distributed embedding representations of all words in the list of words to represent the input query with a single embedding, by using one of an average of all the related distributed embedding representations or a maximum of all the related distributed embedding representations; retrieving a ranked list of document segments of N lines that are similar to the aggregated word embedding representation of the query, where N is a positive integer provided by the user; and returning the list of retrieved segments to a user.

The invention discloses a binlog mining dictionary implementation method which is characterized by comprising the following steps: step 1, before mining, carrying out initialization preparation action on a dictionary database, the initialization preparation action comprising deploying the dictionary database, checking a source database and a database table structure of the dictionary database; step 2, capturing a ddl record in a replay binlog mining process; and step 3, performing backtracking mining at a specified position of the library table. According to the invention, the dictionary information in the mining process is obtained at extremely low cost, the dictionary database is additionally introduced to serve as a persistent layer, and the ddl operation is replayed to the dictionary database, so that the dictionary state of the database at the mining time point can be restored, and the problem that the dictionary and the mining log progress are consistent is guaranteed. The problems that log mining dictionary information cannot be obtained before the existing mysql8.0 and the overhead of binlog recording dictionary information after the existing mysql8.0 is too large are solved.

The invention discloses a non-artificial access log filtering method and device. The method comprises the steps that access logs matched with a preset condition are filtered away, and first standard logs are obtained; based on the standard logs, the access logs in a preset time period are filtered away, and second standard logs are obtained; URL prefixes in the logs are obtained from the first standard logs, and a prefix set is obtained; according to the prefix set, the second standard logs are filtered, and a filtering result log is obtained. By means of the non-artificial access log filtering method and device, high-frequency non-artificial access can be quickly and effectively filtered, and it is of great significance in improving the log mining efficiency, analyzing user behaviors and detecting internal security threats.

The invention discloses a cross-organization business process model mining and compliance checking method and system. The method comprises the following steps: 1) acquiring a cross-organization business process event log; 2) mining a business process model of each organization and a cooperation mode existing between any two organizations by using the obtained cross-organization business process event logs; 3) integrating the mined business process models of different organizations and cooperation modes between the organizations to obtain a global cross-organization business process model; 4) performing compliance check on the business process model of each organization and the cooperation mode between the organizations in the mined cross-organization business process event log; and 5) performing compliance check on the global cross-organization business process model and obtaining a conclusion. The cross-organization business process models in various scenes can be mined, and the problem that the mined cross-organization business process models cannot be effectively measured through an existing compliance checking method is solved.

The invention relates to a key Cookies identification method for Web session merging. Session merging by use of Cookie identification is a key step of Web log preprocessing and directly affects subsequent Web log mining. The key Cookies identification method for Web session merging is provided for solving the problems of low efficiency and low accuracy rate of traditional session merging. The method is used for identifying user-Cookies associated with users and terminal-Cookies of a terminal by which a user browses websites. The method realizes the identification of the user-Cookies by using a CookiePicker system proposed by a predecessor, and also realizes the identification of the terminal-Cookies by virtue of the top-k thought. At last, the user-Cookies and the terminal-Cookies are combined to form final key-Cookies. The method has the advantage of quickly identifying the user associated Cookie information and thus can be well applied to session merging.

The invention discloses a distance-based parallel complete log mining method. Two algorithms are applied in a realization process of the method; the algorithm I is a potential causal relationship discovery algorithm; according to the algorithm I, through distance definition, a new implicit relationship discovery algorithm is established to solve a process model comprising selections; and the algorithm II is an AlphaParallelUseDistance algorithm, and can process a blocky model comprising a selection structure. Through the steps, a parallel complete log comprising fewer logs can be processed, mining is performed to obtain the process model with relatively high accuracy and relatively high conciseness, and the model structure is no longer only limited to the parallel blocky model. Compared with a mining result of an IM algorithm, a result of the method is higher in accuracy and more concise; and compared with an alpha|| algorithm, logs unminable for the algorithm can be processed, correctresults unobtainable for the algorithm can be obtained, and the application range is wider.

The invention discloses a cross-organization multi-instance sub-process model mining method and system, and the method comprises the following steps: 1) obtaining basic data, including a life cycle event log of a cross-organization multi-instance sub-process; 2) utilizing the obtained life cycle event log to mine a nesting relation between life cycle event log activities, and representing the nesting relation by a nesting relation tree; 3) constructing a hierarchical event log by using the nested relationship obtained by mining; 4) performing multi-instance identification and reconstruction on the sub-logs by utilizing the layered event log obtained by mining to obtain a reconstructed layered event log; and 5) mining a cross-organization multi-instance sub-process model from the reconstructed hierarchical event log. According to the invention, the limitation that a traditional process mining technology cannot support mining of multiple calling sub-processes in a cross-organization business scene is broken through, and the efficiency and effect of designing and modeling cross-organization multi-instance sub-processes in process mining are effectively improved.

The invention relates to a micro-service fault prediction method based on log mining. According to functions defined on the system tracking logs, automatically executing original and error injection target programs, obtaining corresponding system tracking logs, and training a fault prediction model on the system tracking log and micro-service levels based on machine learning. The predictive model is applied to the production environment to predict potential errors at runtime, detect faulty microservices, and analyze fault types.

Business personnel are helped to find frequent processes in a business log by means of a sequence mining method, and a process may be further improved in the aspects of execution time and efficiency by improving frequent execution actions; besides, according to abnormal actions occurring at low frequency, an abnormal point mining method is designed to help the business personnel to find the processes. A log mining system framework graph is shown in the figure.

The invention relates to the technical field of internet information, in particular to a method for providing the individual needs of a search engine user based on log mining. The method comprises the steps as follows: firstly, creating models for user search behaviors by adopting a Markov model, wherein the models comprise a user satisfaction behavior model and a user dissatisfaction model; secondly, calculating the user search behaviors by using maximum likelihood estimations generated in the satisfaction behavior model and the dissatisfaction model respectively; and lastly, evaluating the user satisfaction degree by using a maximum likelihood estimation ratio, evaluating the user satisfaction degree in a complicated and fuzzy search engine user behavior information log, analyzing the individual needs of the user, and providing the results of the individual needs to the user through a search engine so as to promote the development of the search engine technology and improve the service quality of the search engine.