A method and device for constructing a network sentiment knowledge base based on multi-model data modeling

Abstract

The present invention discloses a method and device for constructing a network sentiment knowledge base based on multi-model data modeling, which relates to the field of Internet technology, including: dividing real-time network traffic data into real-time network sentiment data according to the content of real-time network traffic data; performing data modeling on the real-time network sentiment data based on multiple models to obtain a real-time network sentiment database and store it in a multi-model database; obtaining the network metadata and the application layer protocol parsing data from the multi-model database for association to obtain accumulated network sentiment data; performing data modeling on the accumulated network sentiment data based on multiple models to obtain an accumulated network sentiment database and store it in a multi-model database; associating the real-time network sentiment database and the accumulated network sentiment database stored in the multi-model database to obtain a network sentiment knowledge base. This method not only realizes the comprehensive storage of network sentiment information, but also realizes the creation of multi-dimensional knowledge.

Description

Technical Field

[0001] The present invention relates to the field of Internet technology, and more specifically to a method and device for constructing an Internet sentiment knowledge base based on multi-model data modeling. Background Art

[0002] As the fifth space, the security of cyberspace has become a global research topic, and cybersecurity knowledge is the information foundation for building a network security system. The characteristics of cybersecurity data include: 1) huge data volume, and the access capacity of a small or medium-sized urban area will reach hundreds of 10GE; 2) complex data types. The content extracted from the network mirror traffic contains both network session event information based on time flow, and network target annotation information (such as country, operator, etc.) and network asset information (such as device type and service type corresponding to IP) extracted for long-term accumulation. It can extract structured IP addresses, domain names and other information, and can also capture unstructured software samples, traffic packets and other data; in addition, based on the information extracted from network traffic, it is also possible to mine relevant knowledge of cyberspace mapping through analysis, such as discovering network topology through the OSPF protocol, which is usually represented by graph computing data.

[0003] Therefore, the traditional Internet sentiment knowledge base based on a single data model can only store one data model, either structured relational data or key-value data, and cannot achieve comprehensive storage of Internet sentiment information. Summary of the invention

[0004] The embodiment of the present invention provides a method and device for constructing an Internet sentiment knowledge base based on multi-model data modeling, breaking through the limitations of traditional single-mode databases. Based on the multi-model data modeling design, it not only realizes the comprehensive storage of Internet sentiment information, but also realizes multi-dimensional knowledge creation, providing a new design concept for the construction of an Internet sentiment knowledge base.

[0005] The embodiment of the present invention provides a method for constructing an Internet sentiment knowledge base based on multi-model data modeling, comprising:

[0006] According to the content of the real-time network traffic data, the real-time network traffic data is divided into real-time network condition data including original message data, network metadata and application layer protocol parsing data; data modeling is performed on the real-time network condition data based on a multi-model to obtain a real-time network condition database and store it in the multi-model database;

[0007] Acquire the network metadata and the application layer protocol parsing data from the multi-model database, associate the network metadata and the application layer protocol parsing data, and obtain accumulated network sentiment data including basic knowledge and graph knowledge; perform data modeling on the accumulated network sentiment data based on the multi-model to obtain an accumulated network sentiment database and store it in the multi-model database;

[0008] The real-time network sentiment database and the accumulated network sentiment database stored in the multi-model database are associated to obtain a network sentiment knowledge base.

[0009] Preferably, the real-time network traffic data is divided into real-time network condition data including original message data, network metadata and application layer protocol parsing data according to the content of the real-time network traffic data, specifically including:

[0010] Intercepting message data in network traffic from the real-time network information data, and generating original message data including Pcap file format;

[0011] Intercepting information used to describe network communications and network sessions from network traffic in the real-time network status data to obtain network metadata;

[0012] The session flow is restored from the real-time network information through group reporting, and the application layer information is extracted through a parsing program to obtain application layer protocol parsing data.

[0013] Preferably, the data modeling of the real-time network sentiment data based on multiple models specifically includes:

[0014] The original message data is modeled using document-type data;

[0015] The network metadata is modeled using relational data;

[0016] The application layer protocol parsing data is modeled using a key-value key / value pair model.

[0017] Preferably, the associating the network metadata with the application layer protocol parsing data specifically includes:

[0018] The network metadata and the application layer protocol analysis data are processed offline at regular intervals to obtain accumulated network sentiment knowledge, which includes basic knowledge and graph knowledge.

[0019] Preferably, the data modeling of the accumulated network sentiment data based on multiple models specifically includes:

[0020] For basic knowledge, relational data modeling or key-value modeling is used according to the number of knowledge attributes and whether it contains levels;

[0021] For graph-type knowledge, graph database is used to model entity and relationship models.

[0022] Preferably, associating the real-time network sentiment database and the accumulated network sentiment database stored in the multi-model database specifically includes:

[0023] Based on the multi-model query and analysis engine, the unified SQL-like syntax allows data operations on relational database mode, graph database mode, key-value mode, and document mode data storage;

[0024] In graph computing, event flow information and knowledge base annotation information are associated.

[0025] The embodiment of the present invention further provides a network sentiment knowledge base construction device based on multi-model data modeling, characterized in that it includes:

[0026] The first obtaining unit is used to divide the real-time network traffic data into real-time network condition data including original message data, network metadata and application layer protocol parsing data according to the content of the real-time network traffic data; perform data modeling on the real-time network condition data based on multiple models to obtain a real-time network condition database and store it in the multiple model database;

[0027] The second obtaining unit is used to obtain the network metadata and the application layer protocol parsing data from the multi-model database, associate the network metadata and the application layer protocol parsing data, and obtain accumulated network sentiment data including basic knowledge and graph knowledge; perform data modeling on the accumulated network sentiment data based on the multi-model to obtain an accumulated network sentiment database and store it in the multi-model database;

[0028] The third obtaining unit is used to associate the real-time network sentiment database and the accumulated network sentiment database stored in the multi-model database to obtain a network sentiment knowledge base.

[0029] Preferably, the first obtaining unit is specifically used for:

[0030] Intercepting message data in network traffic from the real-time network information data, and generating original message data including Pcap file format;

[0031] Intercepting information used to describe network communications and network sessions from network traffic in the real-time network status data to obtain network metadata;

[0032] Restore the session flow from the real-time network data through group reporting, and extract application layer information through a parsing program to obtain application layer protocol parsing data;

[0033] The original message data is modeled using document-type data;

[0034] The network metadata is modeled using relational data;

[0035] The application layer protocol parsing data is modeled using a key-value key / value pair model.

[0036] Preferably, the second obtaining unit is specifically used to: perform regular offline processing on the network metadata and the application layer protocol parsing data to obtain accumulated network sentiment knowledge, wherein the accumulated network sentiment knowledge includes basic knowledge and graph knowledge;

[0037] For basic knowledge, relational data modeling or key-value modeling is used according to the number of knowledge attributes and whether it contains levels;

[0038] For graph-type knowledge, graph database is used to model entity and relationship models.

[0039] Preferably, the third obtaining unit is specifically used for:

[0040] Based on the multi-model query and analysis engine, the unified SQL-like syntax allows data operations on relational database mode, graph database mode, key-value mode, and document mode data storage;

[0041] In graph computing, event flow information and knowledge base annotation information are associated.

[0042] The embodiment of the present invention provides a method and method for constructing a network sentiment knowledge base based on multi-model data modeling, the method comprising: according to the content of real-time network traffic data, dividing the real-time network traffic data into real-time network sentiment data including original message data, network metadata and application layer protocol parsing data; performing data modeling on the real-time network sentiment data based on multiple models to obtain a real-time network sentiment database and store it in a multi-model database; obtaining the network metadata and the application layer protocol parsing data from the multi-model database, associating the network metadata and the application layer protocol parsing data to obtain accumulated network sentiment data including basic knowledge and graph knowledge; performing data modeling on the accumulated network sentiment data based on multiple models to obtain an accumulated network sentiment database and store it in a multi-model database; associating the real-time network sentiment database and the accumulated network sentiment database stored in the multi-model database to obtain a network sentiment knowledge base. The present invention breaks through the limitations of traditional single-mode databases, and based on multi-model data modeling design, not only realizes the comprehensive storage of network sentiment information, but also realizes multi-dimensional knowledge creation, providing a new design concept for the construction of a network sentiment knowledge base. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 A schematic diagram of a method for constructing an Internet sentiment knowledge base based on multi-model data modeling provided by an embodiment of the present invention;

[0045] Figure 2 A schematic diagram of the structure of a network sentiment knowledge base construction device based on multi-model data modeling provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] Figure 1 A schematic diagram of a method for constructing a network sentiment knowledge base based on multi-model data modeling provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the method mainly includes the following steps:

[0048] Step 101, according to the content of the real-time network traffic data, the real-time network traffic data is divided into real-time network condition data including original message data, network metadata and application layer protocol parsing data; data modeling is performed on the real-time network condition data based on a multi-model to obtain a real-time network condition database and store it in a multi-model database;

[0049] Step 102, obtaining the network metadata and the application layer protocol parsing data from the multi-model database, associating the network metadata and the application layer protocol parsing data to obtain accumulated network sentiment data including basic knowledge and graph knowledge; performing data modeling on the accumulated network sentiment data based on the multi-model to obtain an accumulated network sentiment database and storing it in the multi-model database;

[0050] Step 103, associating the real-time network sentiment database and the accumulated network sentiment database stored in the multi-model database to obtain a network sentiment knowledge base.

[0051] The embodiment of the present invention provides a method for constructing an Internet sentiment knowledge base based on multi-model data modeling, which can solve the problem that the current Internet sentiment data cannot be fully stored and fully analyzed. It should be noted that the method divides and partitions the Internet sentiment data based on the multi-model design concept to ensure that the full amount of data is stored according to the multi-model data mode; at the same time, the multi-model technology is used to achieve unified collision and analysis of multi-model stored data.

[0052] Specifically, step 101 mainly describes the extraction, division and multi-model modeling of real-time network data. In practical applications, one way is to intercept the message data in the network traffic from the real-time network data, and then generate the original message data including the Pcap file format, which is unstructured data; one way is to intercept the information used to describe network communication and network session from the network traffic from the real-time network data, including time, IP five-tuple, etc., to obtain network metadata, which is structured data; another way is that the real-time streaming processing program realizes the restoration of the session flow through the group report, and extracts the application layer information through the parsing program, including the attribute values ​​of each protocol such as Http, Snmp, FTP, etc., and the attachments carried. This type of data contains both structured data and unstructured data.

[0053] Furthermore, for the divided real-time network sentiment data, the network sentiment knowledge base is modeled based on multiple models, thereby realizing the storage of real-time network sentiment data in a multi-model database: specifically, the original message data is modeled using document-type data; the network metadata is modeled using relational data; and the application layer protocol parsing data is modeled using a key-value pair model.

[0054] In step 102, based on the real-time network sentiment data, the network metadata and application layer protocol parsing data in the real-time network sentiment data stored in the multi-model database are processed offline at regular intervals to extract and associate the accumulated network sentiment knowledge, which includes basic knowledge and graph knowledge.

[0055] In an embodiment of the present invention, basic knowledge includes IP-address mapping, IP-domain name mapping, etc. This type of knowledge is suitable for data modeling using a relational model or a key-value model; graph-type knowledge includes network asset profiling, network event deduction, etc. This type of knowledge is suitable for data modeling using a graph model.

[0056] Furthermore, for the accumulated network sentiment data after division, data modeling is performed on the network sentiment knowledge base based on multiple models, so as to realize the storage of the accumulated network sentiment data in the multi-model database, among which, basic knowledge: according to the number of knowledge attributes and whether it contains levels, choose to use relational data modeling or key-value key / value pair model for modeling; graph knowledge: use graph database to model with entity and relationship model.

[0057] In step 103, the real-time network sentiment database and the accumulated network sentiment database stored in the multi-model database are associated to obtain a network sentiment knowledge base. That is, based on the use of multi-model databases to store real-time and accumulated network sentiment data, based on multi-model query and analysis engines, data operations on relational database mode, graph database mode, key-value mode, and document mode data storage can be connected through unified SQL-like syntax to achieve cross-model data association, associate event flow information, knowledge base annotation information, etc. in graph calculations, and achieve multi-dimensional information analysis.

[0058] The embodiment of the present invention provides a method for constructing an Internet sentiment knowledge base based on multi-model data modeling. The method overcomes the problem that a single data model cannot completely construct an Internet sentiment knowledge base, and realizes: 1) comprehensive storage of complex Internet sentiment information; 2) creation of a multi-dimensional Internet sentiment knowledge base; 3) cross-model data association and analysis.

[0059] Based on the same inventive concept, an embodiment of the present invention provides a device for constructing an Internet sentiment knowledge base based on multi-model data modeling. Since the principle of the device for solving technical problems is similar to a method for constructing an Internet sentiment knowledge base based on multi-model data modeling, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0060] Figure 2 A schematic diagram of a network sentiment knowledge base construction device based on multi-model data modeling provided by an embodiment of the present invention is shown in FIG. Figure 2 As shown, the device includes: a first obtaining unit 201, a second obtaining unit 202 and a third obtaining unit 203.

[0061] The first obtaining unit 201 is used to divide the real-time network traffic data into real-time network condition data including original message data, network metadata and application layer protocol parsing data according to the content of the real-time network traffic data; perform data modeling on the real-time network condition data based on a multi-model to obtain a real-time network condition database and store it in the multi-model database;

[0062] The second obtaining unit 202 is used to obtain the network metadata and the application layer protocol parsing data from the multi-model database, associate the network metadata and the application layer protocol parsing data, and obtain the accumulated network sentiment data including basic knowledge and graph knowledge; perform data modeling on the accumulated network sentiment data based on the multi-model to obtain the accumulated network sentiment database and store it in the multi-model database;

[0063] The third obtaining unit 203 is used to associate the real-time network sentiment database and the accumulated network sentiment database stored in the multi-model database to obtain a network sentiment knowledge base.

[0064] Preferably, the first obtaining unit 201 is specifically used for:

[0065] Intercepting message data in network traffic from the real-time network information data, and generating original message data including Pcap file format;

[0066] Intercepting information used to describe network communications and network sessions from network traffic in the real-time network status data to obtain network metadata;

[0067] Restore the session flow from the real-time network data through group reporting, and extract application layer information through a parsing program to obtain application layer protocol parsing data;

[0068] The original message data is modeled using document-type data;

[0069] The network metadata is modeled using relational data;

[0070] The application layer protocol parsing data is modeled using a key-value key / value pair model.

[0071] Preferably, the second obtaining unit 202 is specifically used to: perform regular offline processing on the network metadata and the application layer protocol parsing data to obtain accumulated network sentiment knowledge, wherein the accumulated network sentiment knowledge includes basic knowledge and graph knowledge;

[0072] For basic knowledge, relational data modeling or key-value modeling is used according to the number of knowledge attributes and whether it contains levels;

[0073] For graph-type knowledge, graph database is used to model entity and relationship models.

[0074] Preferably, the third obtaining unit 203 is specifically used for:

[0075] Based on the multi-model query and analysis engine, the unified SQL-like syntax allows data operations on relational database mode, graph database mode, key-value mode, and document mode data storage;

[0076] In graph computing, event flow information and knowledge base annotation information are associated.

[0077] It should be understood that the units included in the above-mentioned network sentiment knowledge base construction device based on multi-model data modeling are only logical divisions based on the functions implemented by the device. In actual applications, the above-mentioned units can be superimposed or split. And the functions implemented by the network sentiment knowledge base construction device based on multi-model data modeling provided in this embodiment correspond one-to-one to the network sentiment knowledge base construction method based on multi-model data modeling provided in the above-mentioned embodiment. The more detailed processing flow implemented by the device has been described in detail in the above-mentioned method embodiment 1, and will not be described in detail here.

[0078] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0079] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for constructing an Internet sentiment knowledge base based on multi-model data modeling, characterized in that: include: Intercepting the message data in the network traffic from the real-time network data, and generating the original message data including the Pcap file format; The network traffic is intercepted from the real-time network data to intercept information used to describe network communication and network conversations, and obtain network metadata; the real-time network data is grouped to restore the conversation flow, and the application layer information is extracted through a parsing program to obtain application layer protocol parsing data; the original message data is modeled using document data; the network metadata is modeled using relational data; The application layer protocol parsing data is modeled using a key-value key / value pair model to obtain a real-time network information database and store it in a multi-model database; Acquire the network metadata and the application layer protocol parsing data from the multi-model database, perform regular offline processing on the network metadata and the application layer protocol parsing data, and obtain accumulated network sentiment knowledge, wherein the accumulated network sentiment knowledge includes basic knowledge and graph knowledge; For basic knowledge, relational data modeling is used based on the number of knowledge attributes and whether it contains levels; Or key-value modeling; For graph-based knowledge, a graph database is used to model entity and relationship models, and the accumulated network sentiment database is obtained and stored in a multi-model database; The real-time network sentiment database and the accumulated network sentiment database stored in the multi-model database are associated to obtain a network sentiment knowledge base.

2. The method according to claim 1, characterized in that The associating the real-time network sentiment database and the accumulated network sentiment database stored in the multi-model database specifically includes: Based on the multi-model query and analysis engine, the unified SQL-like syntax allows data operations on relational database mode, graph database mode, key-value mode, and document mode data storage; In graph computing, event flow information and knowledge base annotation information are associated.

3. A network sentiment knowledge base construction device based on multi-model data modeling, characterized in that: include: The first obtaining unit is used to intercept the message data in the network traffic from the real-time network information data, and then generate the original message data including the Pcap file format; The network traffic is intercepted from the real-time network data to intercept information used to describe network communication and network conversations, and obtain network metadata; the real-time network data is grouped to restore the conversation flow, and the application layer information is extracted through a parsing program to obtain application layer protocol parsing data; the original message data is modeled using document data; the network metadata is modeled using relational data; The application layer protocol parsing data is modeled using a key-value key / value pair model to obtain a real-time network information database and store it in a multi-model database; A second obtaining unit is used to obtain the network metadata and the application layer protocol parsing data from the multi-model database, and perform regular offline processing on the network metadata and the application layer protocol parsing data to obtain accumulated network sentiment knowledge, wherein the accumulated network sentiment knowledge includes basic knowledge and graph knowledge; For basic knowledge, relational data modeling is used based on the number of knowledge attributes and whether it contains levels; Or key-value modeling; For graph-based knowledge, a graph database is used to model entity and relationship models, and the accumulated network sentiment database is obtained and stored in a multi-model database; The third obtaining unit is used to associate the real-time network sentiment database and the accumulated network sentiment database stored in the multi-model database to obtain a network sentiment knowledge base.

4. The device according to claim 3, characterized in that The third obtaining unit is specifically used for: Based on the multi-model query and analysis engine, the unified SQL-like syntax allows data operations on relational database mode, graph database mode, key-value mode, and document mode data storage; In graph computing, event flow information and knowledge base annotation information are associated.

Images