Al technical title is built by PatSnap Al team. It summarizes the technical point description of the patent document.
A data mining and data technology, applied in the field of network data, can solve problems such as lack, and achieve the effect of efficient collection and security
Active Publication Date: 2015-11-18
SHANGHAI CTRIP COMMERCE CO LTD
View PDF7 Cites 7 Cited by
Summary
Abstract
Description
Claims
Application Information
AI Technical Summary
This helps you quickly interpret patents by identifying the three key elements:
Problems solved by technology
Method used
Benefits of technology
Problems solved by technology
[0003] The technical problem to be solved by the present invention is to overcome the defect in the prior art that lacks efficient means for collecting, associating, gathering,
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more
Image
Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
Click on the blue label to locate the original text in one second.
Reading with bidirectional positioning of images and text.
Smart Image
Examples
Experimental program
Comparison scheme
Effect test
Embodiment 1
[0041] refer to figure 1 As shown, the data mining method of the present embodiment includes the following steps:
[0042] Step 1. Obtain original data packets from multiple sources of network data, and put the original data packets from different sources into different message queues in the distributed message queue;
[0043] Step 2, extract the original data packet from the message queue, preprocess the extracted data to convert the data format into JSON format, find out the data to be associated and send it back to the corresponding message queue;
[0044] Step 3. Create different distributed processing tasks according to the data type, including the packet original data packet task, and the packet original data packet task generates index information and description information for parsing the original data packet;
[0045] Step 4, store the data in the original data packet and the index information obtained by parsing in HBASE and elasticSearch respectively for data rest...
Embodiment 2
[0050] refer to figure 2 As shown, the data mining system of this embodiment includes:
[0051] The distributed message module 1 is used to obtain original data packets from multiple network data sources, and put the original data packets from different sources into different message queues in the distributed message queue;
[0052] The preprocessing module 2 is used to extract the original data packet from the message queue, preprocess the extracted data to convert the data format into JSON format, and find out the data to be associated and resend the corresponding message in the queue;
[0053]The distributed processing module 3 is used to create different distributed processing tasks according to data types, including the packet original data packet task, and the packet original data packet task generates index information and description information for parsing the original data packet;
[0054] The storage module 4 is used to store the data in the original data packet ...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more
PUM
Login to view more
Abstract
The invention discloses a data mining method and system. The data mining method comprises the following steps of: obtaining original data packets from a plurality of data sources and putting the original data packets into different distributed message queues; extracting the original data packets from the message queues and preprocessing extracted data; creating different distributed processing tasks according to data type, wherein the tasks include an original data packet task for analyzing the original data packets to generate index information and description information; storing the data in the original data packets and the index information obtained by analysis to HBASE and a search server respectively, and storing the data in the original data packets and the description information to a database; and extracting the data from the database and performing data mining. The data mining method and system, based on network data, can efficiently perform information collection, linkage, aggregation, storage and mining, so that network threats and source-tracing attacks can be discovered in time and the security of the network data is ensured.
Description
technical field [0001] The invention relates to network data, in particular to a data mining method and system. Background technique [0002] In recent years, the security situation in cyberspace has undergone tremendous changes, and the growth trend of cyber attacks has developed exponentially, and has gradually evolved into a comprehensive attack of various social engineering attacks and various 0day exploits, becoming the most threatening cyber attack The advanced nature, complexity, concealment and persistence of new security threat technology means have exceeded the response ability of traditional network security technology. In order to adapt to the new security situation, it is necessary to build an information collection, linkage, aggregation, storage, and mining system based on network data to detect network threats in a timely manner, trace the source of attacks, and ensure enterprise security. Contents of the invention [0003] The technical problem to be solve...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more
Application Information
Patent Timeline
Application Date:The date an application was filed.
Publication Date:The date a patent or application was officially published.
First Publication Date:The earliest publication date of a patent with the same application number.
Issue Date:Publication date of the patent grant document.
PCT Entry Date:The Entry date of PCT National Phase.
Estimated Expiry Date:The statutory expiry date of a patent right according to the Patent Law, and it is the longest term of protection that the patent right can achieve without the termination of the patent right due to other reasons(Term extension factor has been taken into account ).
Invalid Date:Actual expiry date is based on effective date or publication date of legal transaction data of invalid patent.
Login to view more 
Login to view more