Network traffic analysis method, system and equipment based on low-code design and medium

Abstract

The invention discloses a network traffic analysis method based on low code design. The method comprises the following steps: acquiring network traffic data and information extraction behavior data; performing network flow analysis and information extraction behavior decomposition according to the network flow data and the information extraction behavior data to obtain a basic operator; based on the basic operator, performing engine implementation and deployment to obtain a processing engine supporting the basic operator; compiling a processing flow based on the basic operator to obtain a processing flow file; according to the processing flow file, the processing flow file is uploaded to the processing engine, and the processing engine reads and executes the basic operator in the processing flow file to complete network flow analysis and information extraction. The method can effectively meet the traffic information extraction requirements of a large number of network protocols, and improves the efficiency and flexibility of information collection of a large number of network protocols.

Description

Technical Field

[0001] The present invention relates to the field of network protocol analysis, and in particular to a network traffic analysis method, system, device and medium based on low-code design. Background Art

[0002] In today's information society, the analysis of network traffic information is crucial for network management, performance optimization, security protection and network supervision. It is an urgent need to quickly support new network applications and network protocols and to promptly discover problems and risks in the network. Especially with the rapid development of mobile Internet, the number of applications has exploded, making it a major challenge to support a large number of network application protocols.

[0003] In the prior art, the method to support new network protocols is to analyze the protocol and then write parsing and extraction code, compile the code into machine code and execute it on the server. This method requires developers to be proficient in network protocol programming and high-performance programming technology, and the implementation cycle is long. Minor upgrades to the network protocol also require re-modification and debugging. In addition, the regular expression method directly identifies the network protocol and extracts information, but adding a new protocol requires recompiling the regular library, and the regular expression itself has many limitations, which seriously affects protocol parsing.

[0004] In summary, existing technologies are difficult to efficiently and flexibly support the ever-growing and changing needs of network traffic parsing and information extraction. Summary of the invention

[0005] The present invention provides a network traffic analysis method, system, device and medium based on low-code design, so as to effectively meet the traffic information extraction needs of a large number of network protocols and improve the efficiency and flexibility of collecting a large amount of network protocol information.

[0006] In the first aspect, in order to solve the above technical problems, the present invention provides a network traffic parsing method based on low-code design, comprising: Obtain network traffic data and information extraction behavior data; According to the network traffic data and the information extraction behavior data, network traffic parsing and information extraction behavior decomposition are performed to obtain basic operators; Based on the basic operators, engine implementation and deployment are performed to obtain a processing engine that supports the basic operators; Based on the basic operator, a processing flow is written to obtain a processing flow file; According to the processing flow file, the processing flow file is uploaded to the processing engine, and the processing engine reads and executes the basic operators in the processing flow file to complete network traffic analysis and information extraction.

[0007] As an optional implementation, network traffic parsing and information extraction behavior decomposition are performed according to the network traffic data and the information extraction behavior data to obtain basic operators, including: Performing a network traffic parsing operation according to the network traffic data and the information extraction behavior data to obtain identifiable data packets; According to the data packet, an execution information extraction behavior decomposition operation is performed to obtain the basic operator.

[0008] As an optional implementation, the engine is implemented and deployed based on the basic operator to obtain a processing engine supporting the basic operator, including: Based on the basic operators and engine development environment, a processing engine that supports the basic operators is obtained; Based on the processing engine, the processing engine supporting the basic operator is deployed on the network device and the configuration of the network device is completed.

[0009] As an optional implementation, the processing flow file is uploaded to the processing engine according to the processing flow file, and the processing engine reads and executes the basic operators in the processing flow file to complete network traffic parsing and information extraction, including: Obtaining the processing flow file; According to the processing flow file, upload it to the processing engine and perform parsing processing to obtain the execution sequence of the basic operator; According to the execution sequence, the processing engine performs corresponding processing operations to obtain information in the network traffic; According to the information, the information in the obtained network traffic is output to a preset designated location.

[0010] As an optional implementation, the basic operators include control operators, network message processing operators, judgment operators, calculation operators, state operators, content output operators and time operators.

[0011] As an optional implementation, when a supported network protocol changes, the corresponding basic operator processing flow is modified, and the processing flow file is reloaded by the processing engine.

[0012] As an optional implementation, the processing engine implements the functions of each of the basic operators; provides an operating environment for the basic operators, including storage space, variable space, message space and session space; and also provides the basic operators with the functions of operating network messages and protocol information, storing intermediate states during the processing process, and storing log information that needs to be output.

[0013] Second, a network traffic parsing system based on low-code design includes: Data collection module, used to obtain network traffic data and information extraction behavior data; A data decomposition module, used to perform network traffic analysis and information extraction behavior decomposition according to the network traffic data and the information extraction behavior data to obtain basic operators; An engine deployment module, used to implement and deploy the engine based on the basic operator to obtain a processing engine that supports the basic operator; A process writing module, used to write a processing process based on the basic operator to obtain a processing process file; The traffic analysis module is used to upload the processing flow file to the processing engine according to the processing flow file, and the processing engine reads and executes the basic operators in the processing flow file to complete network traffic analysis and information extraction.

[0014] In the third aspect, the present invention also provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned network traffic parsing methods based on low-code design.

[0015] Compared with the prior art, the present invention has the following beneficial effects: The present invention provides a network traffic analysis method based on low-code design, including: obtaining network traffic data and information extraction behavior data; performing network traffic analysis and information extraction behavior decomposition according to the network traffic data and the information extraction behavior data to obtain basic operators; based on the basic operators, implementing and deploying engines to obtain processing engines that support basic operators; based on the basic operators, writing processing flows to obtain processing flow files; based on the processing flow files, uploading the processing flow files to the processing engine, and the processing engine reading and executing the basic operators in the processing flow files to complete network traffic analysis and information extraction.

[0016] In the present invention, by decomposing the network traffic parsing and information extraction process into several basic operators, the development process is simplified, and developers only need to write the processing flow files of the basic operators without manually writing complex parsing codes. In addition, the present invention controls the execution of the basic operators through the processing engine. For new protocols or protocol changes, only the processing flow files need to be modified, and there is no need to rewrite and debug the parsing code, so that the method can effectively cope with the traffic information extraction needs of a large number of network protocols, and improve the efficiency and flexibility of a large number of network protocol information collection. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1It is a flowchart of a network traffic analysis method based on low-code design provided by an embodiment of the present invention; Figure 2 It is a structural diagram of a network traffic analysis system based on low-code design provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] In today's information society, the analysis of network traffic information is crucial for network management, performance optimization, security protection and network supervision. It is an urgent need to quickly support new network applications and network protocols and to promptly discover problems and risks in the network. Especially with the rapid development of mobile Internet, the number of applications has exploded, making it a major challenge to support a large number of network application protocols.

[0020] In the prior art, the method to support new network protocols is to analyze the protocol and then write parsing and extraction code, compile the code into machine code and execute it on the server. This method requires developers to be proficient in network protocol programming and high-performance programming technology, and the implementation cycle is long. Minor upgrades to the network protocol also require re-modification and debugging. In addition, the regular expression method directly identifies the network protocol and extracts information, but adding a new protocol requires recompiling the regular library, and the regular expression itself has many limitations, which seriously affects protocol parsing.

[0021] In summary, existing technologies are difficult to efficiently and flexibly support the ever-growing and changing needs of network traffic parsing and information extraction.

[0022] In order to solve the above problems, a network traffic parsing method based on low-code design provided in an embodiment of the present application will be introduced and explained in detail through the following specific embodiments.

[0023] Reference Figure 1 The first embodiment of the present invention provides a network traffic parsing method based on low-code design, comprising the following steps: S11, obtaining network traffic data and information extraction behavior data; S12, performing network traffic analysis and information extraction behavior decomposition according to the network traffic data and the information extraction behavior data to obtain basic operators; S13, based on the basic operator, implement and deploy the engine to obtain a processing engine that supports the basic operator; S14, writing a processing flow based on the basic operator to obtain a processing flow file; S15, according to the processing flow file, uploading the processing flow file to the processing engine, the processing engine reads and executes the basic operators in the processing flow file to complete network traffic analysis and information extraction.

[0024] In step S12, the network traffic parsing and information extraction behavior decomposition are performed according to the network traffic data and the information extraction behavior data to obtain basic operators, including: Performing a network traffic parsing operation according to the network traffic data and the information extraction behavior data to obtain identifiable data packets; According to the data packet, an execution information extraction behavior decomposition operation is performed to obtain the basic operator.

[0025] It is worth mentioning that in this step, the process of parsing the protocol can be decomposed into basic operators one by one by summarizing the experience of parsing the protocol; the basic operators required can be inferred based on the data to be output, or the basic operators can be gradually decomposed one by one according to the characteristics of different protocols. In short, basic operators can be analyzed and screened through various different ways until various different protocol parsing processes can be combined using these basic operators.

[0026] In step S13, based on the basic operator, an engine is implemented and deployed to obtain a processing engine supporting the basic operator, including: Based on the basic operators and engine development environment, a processing engine that supports the basic operators is obtained; Based on the processing engine, the processing engine supporting the basic operator is deployed on the network device and the configuration of the network device is completed.

[0027] The processing engine provides the operating environment of the basic operators, and provides storage space, variable space, message space, and session space for the operators, so that the basic operators can conveniently operate network messages and protocol information, store intermediate states during processing, and store log information that needs to be output. At the same time, these spaces are applied for and released according to the needs of network protocol analysis, and the use of these spaces is managed well.

[0028] It is worth noting that the processing engine needs to implement the function of each basic operator obtained in step S12. Exemplarily, the processing engine also needs to control the execution of the processing flow composed of basic operators. Among them, the network protocol parsing process generally includes checking and judging message processing, reading message content, various calculations of data in the message, output of extracted content, etc. These processes can be decomposed into basic operators. By combining these operators according to a specific processing flow, a network protocol parsing process can be defined.

[0029] In one implementation, the processing engine provides the ability to interpret and compile processing flow files. The parsing process of various network protocols can be defined by writing processing flow files. The processing engine reads the processing flow files written by these network protocol parsing processes and compiles these interpretations into execution sequences of internal basic operators. When the processing engine processes network data, it can efficiently run the network protocol parsing process.

[0030] In another implementation, the processing engine provides basic interfaces with other modules at the same time. In a specific implementation, the basic interfaces include file interfaces, database interfaces, log server interfaces, and data encoding interfaces. Of course, the basic interfaces may also include other types of interfaces, which are not limited by the present invention.

[0031] In step S14, a processing flow is written based on the basic operator to obtain a processing flow file.

[0032] It should be noted that the processing flow writing process includes: analyzing the network protocol that needs to be parsed and information extracted, determining the processing flow for parsing the network protocol and extracting information from the network data, continuously decomposing the processing flow into basic operators one by one, until all processing processes are refined into basic operators one by one, and then writing these combined basic operators into a file.

[0033] In step S15, according to the processing flow file, the processing flow file is uploaded to the processing engine, and the processing engine reads and executes the basic operators in the processing flow file to complete network traffic analysis and information extraction, including: Obtaining the processing flow file; According to the processing flow file, upload it to the processing engine and perform parsing processing to obtain the execution sequence of the basic operator; According to the execution sequence, the processing engine performs corresponding processing operations to obtain information in the network traffic; According to the information, the information in the obtained network traffic is output to a preset designated location.

[0034] It should be noted that the processing flow file contains a series of processing flows of basic operators. The processing engine executes basic operators one by one in accordance with the processing flow for the received network messages, and completes the processing of each network message. Exemplarily, during the processing, if a basic operator for log output is executed, the processed information is output to the log server in a specific format.

[0035] After step S15, when a supported network protocol changes, the corresponding basic operator processing flow is modified, and the processing flow file is reloaded by the processing engine.

[0036] It should be noted that the modification of the basic operator processing flow can be carried out through low-code design, without the need for large-scale code rewriting and system reconstruction. This approach not only improves the efficiency of development and maintenance, but also ensures the flexibility and scalability of the system. In this way, it is possible to quickly adapt to new protocols or protocol changes, and maintain the accuracy and real-time performance of system parsing and information extraction.

[0037] Among them, according to the needs of network protocol parsing and information extraction, the basic operators include control basic operators, network message processing basic operators, judgment basic operators, calculation basic operators, status basic operators, content output basic operators, and time basic operators.

[0038] In an optional implementation scheme, the basic control operators are mainly used to control the network protocol parsing process, including jump operators, ifelse condition judgment operators, loop operators, end current message processing operators, next message processing operators, and end current network protocol processing operators; the basic network message processing operators mainly implement data processing in network messages, including read message length operators, read value operators, read network message data segment operators, move network message current processing position operators, and string processing operators.

[0039] Among them, the read value operators include the read network ordinal value operator, the read host ordinal value operator, the read decimal and hexadecimal string value operator, and the read value operators corresponding to various numerical encoding methods used in various network protocols.

[0040] In another optional implementation scheme, the basic operators of the judgment class are mainly used to compare and check the content of the message, which can be used to determine whether the content of the network message is as expected, and can be used for the judgment of the iselse operator and the loop operator in the control class operator, including an operator for checking the content of a certain position of the network message, an operator for checking the content of a certain position of the network message with a mask, an operator for checking the content of a section of the network message, an operator for checking the content of a section of the network message with a mask, and an operator for finding specific content in the message; the basic operators of the operation class are used for the operation of related variables in the network protocol, including assignment operators, addition, subtraction, multiplication and division operators, greater than or less than or equal to judgment operators, AND / NOT logic operators, byte order conversion operators, shift, XOR and NAND bit operation operators; the basic operators of the status class are used to obtain the status related to the network message and the flow table, such as obtaining the direction of the current message and whether there is currently a packet loss situation; the basic operators of the content output class are used to output the extracted information to a specified location, including network operators, file operators, syslog operators, and database operators.

[0041] Among them, network operators include protobuff encoding operators, tlv encoding operators, and raw data operators, and file operators include raw data operators and json format operators; time basic operators are used to obtain time operations, including obtaining absolute time operators and calculating relative time operators. Of course, the type of operator is not fixed, and may need to be adjusted accordingly according to the characteristics of specific problems and the needs of actual applications. For example, in some specific application scenarios, it may be necessary to add security detection operators, specific protocol operators, etc., which are not limited in the present invention.

[0042] Reference Figure 2 The second embodiment of the present invention provides a network traffic parsing system based on low-code design, including: Data collection module, used to obtain network traffic data and information extraction behavior data; A data decomposition module, used to perform network traffic analysis and information extraction behavior decomposition according to the network traffic data and the information extraction behavior data to obtain basic operators; An engine deployment module, used to implement and deploy the engine based on the basic operator to obtain a processing engine that supports the basic operator; A process writing module, used to write a processing process based on the basic operator to obtain a processing process file; The traffic analysis module is used to upload the processing flow file to the processing engine according to the processing flow file, and the processing engine reads and executes the basic operators in the processing flow file to complete network traffic analysis and information extraction.

[0043] It should be noted that the network traffic analysis system based on low-code design provided in an embodiment of the present invention is used to execute all the process steps of the network traffic analysis method based on low-code design in the above-mentioned embodiment. The working principles and beneficial effects of the two correspond one to one, and therefore will not be repeated here.

[0044] In summary, the present invention provides a network traffic parsing method based on low-code design, including: obtaining network traffic data and information extraction behavior data; performing network traffic parsing and information extraction behavior decomposition according to the network traffic data and the information extraction behavior data to obtain basic operators; based on the basic operators, implementing and deploying the engine to obtain a processing engine that supports the basic operators; based on the basic operators, writing the processing flow to obtain a processing flow file; according to the processing flow file, uploading the processing flow file to the processing engine, and the processing engine reads and executes the basic operators in the processing flow file to complete network traffic parsing and information extraction.

[0045] In the present invention, by decomposing the network traffic parsing and information extraction process into several basic operators, the development process is simplified, and developers only need to write the processing flow files of the basic operators without manually writing complex parsing codes. In addition, the present invention controls the execution of the basic operators through the processing engine. For new protocols or protocol changes, only the processing flow files need to be modified, and there is no need to rewrite and debug the parsing code, so that the method can effectively cope with the traffic information extraction needs of a large number of network protocols, and improve the efficiency and flexibility of a large number of network protocol information collection.

[0046] The embodiment of the present invention further provides a terminal device. The terminal device includes: a processor, a memory, and a computer program stored in the memory and executable on the processor, such as an authorization program for a projector. When the processor executes the computer program, the steps in the above-mentioned embodiments of the network traffic parsing method based on low-code design are implemented, such as Figure 1 Alternatively, the processor implements the functions of each module / unit in the above-mentioned system embodiments when executing the computer program.

[0047] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program in the terminal device.

[0048] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, and a smart tablet. The terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the above components are merely examples of terminal devices and do not constitute a limitation on the terminal device. The terminal device may include more or fewer components than the above components, or may combine certain components, or different components. For example, the terminal device may also include input and output devices, network access devices, buses, etc.

[0049] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the terminal device, and uses various interfaces and lines to connect various parts of the entire terminal device.

[0050] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the terminal device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device or other volatile solid-state storage device.

[0051] Wherein, if the module / unit integrated in the terminal device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or system capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0052] It should be noted that the system embodiment described above is merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the system embodiment provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.

[0053] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A network traffic analysis method based on low-code design, It is characterized in that Performed by network devices, including: Obtain network traffic data and information extraction behavior data; According to the network traffic data and the information extraction behavior data, network traffic parsing and information extraction behavior decomposition are performed to obtain basic operators; Based on the basic operators, engine implementation and deployment are performed to obtain a processing engine that supports the basic operators; Based on the basic operator, a processing flow is written to obtain a processing flow file; According to the processing flow file, the processing flow file is uploaded to the processing engine, and the processing engine reads and executes the basic operators in the processing flow file to complete network traffic analysis and information extraction.

2. According to the network traffic parsing method based on low-code design according to claim 1, It is characterized in that The method of performing network traffic parsing and information extraction behavior decomposition according to the network traffic data and the information extraction behavior data to obtain basic operators includes: Performing a network traffic parsing operation according to the network traffic data and the information extraction behavior data to obtain identifiable data packets; According to the data packet, an execution information extraction behavior decomposition operation is performed to obtain the basic operator.

3. According to the network traffic parsing method based on low-code design according to claim 1, It is characterized in that Based on the basic operator, the engine is implemented and deployed to obtain a processing engine supporting the basic operator, including: Based on the basic operators and engine development environment, a processing engine that supports the basic operators is obtained; Based on the processing engine, the processing engine supporting the basic operator is deployed on the network device and the configuration of the network device is completed.

4. According to the network traffic parsing method based on low-code design according to claim 1, It is characterized in that According to the processing flow file, the processing flow file is uploaded to the processing engine, and the processing engine reads and executes the basic operators in the processing flow file to complete network traffic analysis and information extraction, including: Obtaining the processing flow file; According to the processing flow file, upload it to the processing engine and perform parsing processing to obtain the execution sequence of the basic operator; According to the execution sequence, the processing engine performs corresponding processing operations to obtain information in the network traffic; According to the information, the information in the obtained network traffic is output to a preset designated location.

5. According to the network traffic parsing method based on low-code design according to claim 1, It is characterized in that The basic operators include control operators, network message processing operators, judgment operators, calculation operators, state operators, content output operators and time operators.

6. According to the network traffic analysis method based on low-code design according to claim 1, It is characterized in that When a supported network protocol changes, the corresponding basic operator processing flow is modified, and the processing flow file is reloaded by the processing engine.

7. According to the network traffic parsing method based on low-code design according to claim 3, It is characterized in that The processing engine implements the functions of each of the basic operators; provides an operating environment for the basic operators, including storage space, variable space, message space and session space; It also provides the basic operator with the function of operating network messages and protocol information, storing the intermediate states in the processing process, and storing the log information that needs to be output.

8. A network traffic analysis system based on low-code design, It is characterized in that Configured on network devices, including: Data collection module, used to obtain network traffic data and information extraction behavior data; A data decomposition module, used to perform network traffic analysis and information extraction behavior decomposition according to the network traffic data and the information extraction behavior data to obtain basic operators; An engine deployment module, used to implement and deploy the engine based on the basic operator to obtain a processing engine that supports the basic operator; A process writing module, used to write a processing process based on the basic operator to obtain a processing process file; The traffic analysis module is used to upload the processing flow file to the processing engine according to the processing flow file, and the processing engine reads and executes the basic operators in the processing flow file to complete network traffic analysis and information extraction.

9. An electronic device, It is characterized in that include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the network traffic parsing method based on low-code design as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium includes a stored computer program, wherein, when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the network traffic parsing method based on low-code design as described in any one of claims 1 to 7.

Images