Network traffic expansion analysis method and system, intelligent terminal and storage medium

Abstract

The invention discloses a network flow expansion analysis system. A service management center manages registration of various service modules and unified control of task states; the main service module obtains a to-be-analyzed task and sends an instruction for starting the analysis task to the three-party analysis service module; the three-party analysis service module starts an analysis engine according to the instruction and sends a receipt of successful starting of the analysis engine to the main service module; the main service module starts a data forwarding function after receiving the receipt, and forwards the task data packet to be analyzed to the three-party analysis service module; the analysis engine analyzes the to-be-analyzed task to obtain a three-party analysis result data packet and sends the three-party analysis result data packet to the main service module; and the main service module combines the three-party analysis result with the analysis result of the main service module to obtain a final analysis result. According to the system, different third-party analysis tools can be well fused to carry out network flow analysis, various analysis results can be uniformly output, and service requirements can be well met.

Description

Technical Field

[0001] The present invention relates to the technical field of data analysis, and in particular to a network traffic expansion analysis method, a system intelligent terminal and a storage medium. Background Art

[0002] With the development of information technology, the analysis of network traffic is becoming more and more important, and the existing traffic analysis products are mainly focused on their own analysis business. However, in actual business, a proprietary product may not be able to complete all analysis business, because each analysis product or tool has its own characteristics, or because the data itself cannot be provided due to privacy and confidentiality, resulting in the inability of proprietary products to analyze these special data. Therefore, it is difficult to directly migrate functions to proprietary products, and it is also impossible to migrate functions in special data scenarios. In other scenarios, migration will also face problems such as long development cycle and difficulty. Therefore, the solution of function migration is not suitable. In the end, it only needs to be integrated between products, that is, there is a lack of a method for traffic expansion analysis.

[0003] Under the existing technical framework, the most common method for performing extended network traffic analysis is script parsing extension, such as Wireshark's Lua script, which writes Lua scripts for protocol analysis. However, this method is limited to the scope of protocol analysis and it is difficult to expand the analysis of more businesses, such as other business analysis based on analysis, such as alarm detection, etc. Moreover, this method also requires the redevelopment of extended parsing scripts, and it is impossible to quickly carry out analysis business with the help of existing analysis engines.

[0004] Another more conventional approach is to filter and store the required data frames, and then import them into extended products or analysis tools for analysis. However, these processes usually require human intervention, the processing process is not automated enough, and there are not many connections between the products. The results generated by the products cannot be uniformly managed and output, and ultimately cannot meet business needs well. Summary of the invention

[0005] In view of the defects of the prior art, the embodiments of the present invention provide a network traffic expansion analysis method, system, intelligent terminal and storage medium, which can well integrate different third-party analysis tools to perform network traffic analysis, and combine various analysis results for unified output, which can well meet business needs.

[0006] The present invention is achieved through the following technical solutions:

[0007] In a first aspect, an embodiment of the present invention provides a network traffic expansion analysis system, comprising: a service management center, a main service module, a three-party analysis service module and an analysis engine;

[0008] The service management center is used to manage the registration of various service modules and the unified control of task status;

[0009] The main service module is used to obtain the task to be analyzed, and send an instruction to start the analysis task to the third-party analysis service module through the service management center;

[0010] The third-party analysis service module is used to manage and schedule any analysis engine, start the analysis engine according to the start analysis task instruction, and send a receipt of the successful start of the analysis engine to the main service module;

[0011] The main service module is also used to enable the data forwarding function after receiving the receipt, and forward the task data packet to be analyzed to the third-party analysis service module;

[0012] The analysis engine analyzes the task to be analyzed to obtain a three-party analysis result data packet;

[0013] The three-party analysis service module sends the three-party analysis result data packet to the main service module;

[0014] The main service module receives the three-party analysis result data packet, combines the three-party analysis result with the analysis result of the main service module, and obtains the final analysis result.

[0015] In a second aspect, an embodiment of this aspect provides a network traffic expansion analysis method, including:

[0016] The main service module and each third-party analysis service module are registered with the service management center respectively;

[0017] The main service module obtains the task to be analyzed, and sends an instruction to start the analysis task to the third-party analysis service module through the service management center;

[0018] The third-party analysis service module receives the instruction to start the analysis task, starts the analysis engine, and sends a receipt of successful start of the analysis engine to the main service module;

[0019] The main service module starts the data forwarding function after receiving the receipt;

[0020] The main service module forwards the task data packet to be analyzed to the third-party analysis service module;

[0021] The three-party analysis service module analyzes the task to be analyzed to obtain the three-party analysis result, and sends the three-party analysis result data packet to the main service module;

[0022] The main service module receives the three-party analysis result data packet, combines the three-party analysis result with the analysis result of the main service module, and obtains the final analysis result.

[0023] In a third aspect, an embodiment of the present invention provides an intelligent terminal, comprising a processor, an input device, an output device and a memory, wherein the processor is connected to the input device, the output device and the memory respectively, the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the method described in the above embodiment.

[0024] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the method described in the above embodiment.

[0025] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0026] The embodiments of the present invention provide a network traffic expansion analysis method, system, intelligent terminal and storage medium, which can well integrate different third-party analysis tools to perform network traffic analysis, and can uniformly output various analysis results, which can well meet business needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without creative work. In the drawings:

[0028] Figure 1 A structural block diagram of a network traffic expansion analysis system provided by the first embodiment of the present invention;

[0029] Figure 2 A service scheduling relationship diagram of each module of a network traffic extension analysis system provided by the first embodiment of the present invention;

[0030] Figure 3 A flowchart of a network traffic expansion analysis method provided by another embodiment of the present invention;

[0031] Figure 4 A flowchart of a process of starting an analysis task in a network traffic extension analysis method provided by another embodiment of the present invention;

[0032] Figure 5 A flowchart of a data analysis process in a network traffic expansion analysis method provided by another embodiment of the present invention;

[0033] Figure 6A structural block diagram of an intelligent terminal provided in another embodiment of the present invention. DETAILED DESCRIPTION

[0034] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments and drawings. The exemplary embodiments of the present invention and their description are only used to explain the present invention and are not intended to limit the present invention.

[0035] Example 1

[0036] like Figure 1-2 As shown, an embodiment of the present invention provides a network traffic expansion analysis system. This system adopts a non-intrusive architecture to expand third-party analysis, and mainly includes: a service management center, a main service module, a third-party analysis service module and an analysis engine. Each module is usually executed by an independent sub-process. Since the third-party analysis tool is expanded in a non-intrusive manner, and the main service module provides a service monitoring mechanism, even if the process where the analysis engine is located is abnormally hung or the process crashes, it will not affect the normal operation of the main service module. The main service module will only give an abnormal alarm for the process of the third-party analysis service module.

[0037] Among them, the service management center is used to manage the registration of various service modules and the unified control of task status. Task status includes task start, task stop, etc. Task start means that a data transmission channel is about to be established. The main service module is used to obtain the task to be analyzed. It is the only entry service for data analysis and a module for various practical businesses. The main service module sends the instruction to start the analysis task to the three-party analysis service module through the service management center. The main service module notifies the three-party analysis service module of some scheduling states, such as analysis start, analysis stop, etc., by executing the scheduling instruction. The three-party analysis service module is used to manage and schedule any analysis engine, start the analysis engine according to the start analysis task instruction, and send the receipt of the successful start of the analysis engine to the main service module. The three-party analysis module is a general scheduling service. As the parent process of the analysis engine, it manages the process management of the analysis engine and is responsible for the process startup, monitoring, and recycling of child processes. The main service module is also used to enable the data forwarding function after receiving the receipt, and forward the task data packet to be analyzed to the three-party analysis service module. The analysis engine analyzes the task to be analyzed to obtain the three-party analysis result data packet. The three-party analysis service module analyzes the task to be analyzed to obtain the three-party analysis results, and sends the three-party analysis result data packet to the main service module. The main service module receives the three-party analysis result data packet, combines the three-party analysis results with the analysis results of the main service module, and obtains the final analysis results.

[0038] The third-party analysis service module is universal and can manage the start and stop of any third-party analysis service process, data import and export, etc., and can be easily connected to the analysis engine. The third-party analysis service module also includes a third-party analysis SDK, which mainly provides data definition, data receiving interface and analysis result return interface, as well as channel creation and channel destruction. The third-party analysis service module includes a startup unit, which creates a data channel and calls the data receiving interface through the third-party analysis SDK to prepare to receive data. Based on these SDKs, the analysis engine can easily implement its data reception and result output business.

[0039] Data channels can have many forms. If the third-party analysis service module is deployed on the same device, IPC can be used to build the channel. If it is deployed on different devices, it needs to cross the network. In this case, sockets are used to build the channel. No matter which channel form is used, it is transparent to the analysis engine. The essence of each channel form is similar. If the channel is viewed as a highway, then the data is transported through containers one by one. These containers are elements of the data queue on the channel. Therefore, both IPC and sockets provide queues to transport data elements, but IPC uses lock-free queues to ensure efficiency, while sockets are local buffer queues, which can still ensure efficiency. By providing a high-speed data channel, third-party analysis tools can be quickly integrated into the main analysis product. The high speed of the channel is built and completed based on IPC or socket. Through the receiving and sending interfaces provided by the SDK, third-party analysis tools can be run at a minimum cost, and the analysis tool can quickly fill in logs and data packets and other information according to the return data format requirements.

[0040] The main service module forwards the task data packet to be analyzed to the third-party analysis service module for analysis. The task data packet to be analyzed is a network traffic data packet. The network traffic data packet includes the session information and data packet information of the data packet, as well as the information parsed when the main service module receives the task to be analyzed, including: IP, port, timestamp, packet length, etc.

[0041] The specific method for the main service module to combine the three-party analysis results with the analysis results of the main service module includes: the returned data packet is directly executed according to the data packet output logic, and the detection log in the three-party analysis results is integrated into the detection log of the main service module. The detection log includes public fields, and the public fields include rule name, rule ID, detection result description and session information. The returned data packet also includes IP, port, timestamp, packet length, session information, etc., which need to be carried in the returned result data to facilitate the main service module to make associations and queries.

[0042] The main service module includes a data processing unit, which is used to process the tasks to be analyzed. The processing methods include: IP packet reassembly, TCP packet reassembly, message identification and diversion processing, which is also an extension method that distinguishes other network traffic analysis. Submitting the processed message data to the third-party analysis service module in sequence can simplify a certain amount of work of the third-party analysis tool, such as the message reassembly, and focus on its core analysis business.

[0043] The present invention provides a system for network traffic extended analysis, which can well integrate different extended analysis products or tools, and can integrate multiple products at the same time. Each product can customize its own data and uniformly define the input and output structure. The main service module uniformly manages and outputs the analysis results generated by each extended product or analysis tool, and these data mainly include logs and data packets.

[0044] Example 2

[0045] like Figure 3-5 As shown, another embodiment of the present invention provides a network traffic expansion analysis method, which is applicable to the network traffic expansion analysis system described in the first embodiment above, and the method includes the following steps:

[0046] The main service module and each third-party analysis service module are registered with the service management center respectively;

[0047] The main service module obtains the task to be analyzed, and sends an instruction to start the analysis task to the third-party analysis service module through the service management center;

[0048] The third-party analysis service module receives the instruction to start the analysis task, starts the analysis engine, and sends a receipt of successful start of the analysis engine to the main service module;

[0049] The main service module starts the data forwarding function after receiving the receipt;

[0050] The main service module forwards the task data packet to be analyzed to the third-party analysis service module;

[0051] The three-party analysis service module sends the generated three-party analysis result data packet to the main service module;

[0052] The main service module receives the three-party analysis result data packet, combines the three-party analysis result with the analysis result of the main service module, and obtains the final analysis result.

[0053] When the main service module obtains the task to be analyzed,

[0054] The method includes: the main service module processes the task to be analyzed, and the processing method includes: IP packet reassembly, TCP packet reassembly, message identification and diversion processing.

[0055] The specific method of starting the analysis engine includes: creating a data channel through a third-party analysis SDK and calling a data receiving interface to prepare to receive data.

[0056] The specific method of combining the third-party analysis results with the analysis results of the main service module includes: integrating the detection log in the third-party analysis results into the detection log of the main service module, the detection log includes public fields, and the public fields include rule name, rule ID, detection result description and session information.

[0057] The task data packet to be analyzed is a network traffic data packet, which includes the session information of the data packet, data packet information, and the information parsed when the main service module receives the task to be analyzed, including: IP, port, timestamp, packet length, etc. The returned data packet also includes IP, port, timestamp, packet length, session information, etc., which need to be carried in the returned result data to facilitate the main service module to make associations and inquiries.

[0058] The following are some code examples:

[0059] Send data interface definition example:

[0060]

[0061]

[0062] The returned analysis results are usually detection logs and related data packets, as well as some received session information, etc. The returned data interface definition example is as follows:

[0063]

[0064] Some main SDK interfaces:

[0065]

[0066]

[0067] A method for extended analysis of network traffic provided by an embodiment of the present invention can well integrate different extended analysis products or tools, and can integrate multiple products at the same time. Each product can customize its own data and uniformly define the input and output structure. The main service module uniformly manages and outputs the analysis results generated by each extended product or analysis tool. These data mainly include logs and data packets.

[0068] Example 3

[0069] like Figure 6 As shown, a structural block diagram of an intelligent terminal according to another embodiment of the present invention is shown, the terminal includes a processor, an input device, an output device and a memory, the processor is connected to the input device, the output device and the memory respectively, the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method described in the above embodiment.

[0070] It should be understood that in the embodiments of the present invention, the processor referred to may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0071] Input devices may include a touchpad, a fingerprint sensor (for collecting the user's fingerprint information and fingerprint direction information), a microphone, etc., and output devices may include a display (LCD, etc.), a speaker, etc.

[0072] The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0073] In a specific implementation, the processor, input device, and output device described in the embodiments of the present invention may execute the implementation described in the method embodiment provided in the embodiments of the present invention, or may execute the implementation described in the system embodiment described in the embodiments of the present invention, which will not be described in detail here.

[0074] Example 4

[0075] In another embodiment of the present invention, an embodiment of a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program. The computer program includes program instructions. When the program instructions are executed by a processor, the processor executes the method described in the above embodiment.

[0076] The computer-readable storage medium may be an internal storage unit of the terminal described in the foregoing embodiments, such as a hard disk or memory of the terminal. The computer-readable storage medium may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Further, the computer-readable storage medium may also include both an internal storage unit of the terminal and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the terminal. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0077] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0078] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the terminals and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0079] In the several embodiments provided in the present application, it should be understood that the disclosed terminal and method can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or it can be an electrical, mechanical or other form of connection.

[0080] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

Claims

1. A network traffic expansion analysis system, characterized in that: include: Service management center, main service module, third-party analysis service module and analysis engine; The service management center is used to manage the registration of various service modules and the unified control of task status; The main service module is used to obtain the task to be analyzed, and send an instruction to start the analysis task to the third-party analysis service module through the service management center; The third-party analysis service module is used to manage and schedule any analysis engine, start the analysis engine according to the start analysis task instruction, and send a receipt of the successful start of the analysis engine to the main service module; The main service module is also used to enable the data forwarding function after receiving the receipt, and forward the task data packet to be analyzed to the third-party analysis service module; The analysis engine analyzes the task to be analyzed to obtain a three-party analysis result data packet; The three-party analysis service module sends the three-party analysis result data packet to the main service module; The main service module receives the three-party analysis result data packet, combines the three-party analysis result with the analysis result of the main service module, and obtains the final analysis result.

2. The system according to claim 1, characterized in that The main service module includes a data processing unit, which is used to process the analysis task, and the processing methods include: IP packet reassembly, TCP packet reassembly, message identification and diversion processing. The third-party analysis service module includes a startup unit, which creates a data channel through a third-party analysis SDK and calls a data receiving interface to prepare for receiving data.

3. The system according to claim 1, characterized in that The main service module also includes a result processing unit, which is used to merge the detection log in the three-party analysis result into the detection log of the main service module. The detection log includes public fields, and the public fields include rule name, rule ID, detection result description and session information.

4. A network traffic expansion analysis method, characterized in that: include: The main service module and each third-party analysis service module are registered with the service management center respectively; The main service module obtains the task to be analyzed, and sends an instruction to start the analysis task to the third-party analysis service module through the service management center; The third-party analysis service module receives the instruction to start the analysis task, starts the analysis engine, and sends a receipt of successful start of the analysis engine to the main service module; The main service module starts the data forwarding function after receiving the receipt; The main service module forwards the task data packet to be analyzed to the third-party analysis service module; The three-party analysis service module analyzes the task to be analyzed to obtain the three-party analysis result, and sends the three-party analysis result data packet to the main service module; The main service module receives the three-party analysis result data packet, combines the three-party analysis result with the analysis result of the main service module, and obtains the final analysis result.

5. The method according to claim 4, characterized in that The method includes: the main service module processes the task to be analyzed, and the processing method includes: IP packet reassembly, TCP packet reassembly, message identification and diversion processing.

6. The method according to claim 4, characterized in that The specific method of starting the analysis engine includes: creating a data channel through a third-party analysis SDK and calling a data receiving interface to prepare to receive data.

7. The method according to claim 4, characterized in that The task data packet to be analyzed is a network traffic data packet, and the network traffic data packet includes session information and data packet information where the data packet is located.

8. The method according to claim 4, characterized in that The specific method of combining the three-party analysis results with the analysis results of the main service module includes: integrating the detection log in the three-party analysis results into the detection log of the main service module, the detection log includes public fields, and the public fields include rule name, rule ID, detection result description and session information.

9. An intelligent terminal, comprising a processor, an input device, an output device and a memory, wherein the processor is connected to the input device, the output device and the memory respectively, and the memory is used to store a computer program, wherein the computer program comprises program instructions, and wherein: The processor is configured to call the program instructions to execute the method according to any one of claims 4 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 4 to 8.

Images