Micro-service traffic analysis method and device and computing equipment

Abstract

The invention discloses a micro-service traffic analysis method and device and computing equipment, and the method comprises the steps: pulling traffic data from a network card of a micro-service, carrying out the real-time analysis processing of the traffic data, obtaining a real-time analysis result, and outputting the real-time analysis result to a target file; reading data from the target file, and when a complete first data frame is read, writing the first data frame into the data channel; and reading the second data frame from the data channel, and performing key information analysis processing on the second data frame to obtain traffic key data. Through the above mode, each piece of flow data of micro-service access is captured in real time for analysis, full-amount flow data can be obtained, the integrity of the flow data can be ensured, the analysis success rate of the flow data can be improved, the concurrent security of data transmission can be ensured based on the data channel, and the data transmission efficiency can be improved. And the possibility of errors in the flow analysis process is reduced.

Description

Technical Field

[0001] This application relates to the field of computer technologies, and in particular, to a microservice traffic parsing method, apparatus, computing device, computer storage medium, and computer program product. Background Art

[0002] The microservice architecture has become the mainstream architecture of modern distributed systems due to its high scalability and flexibility. However, with the increase in system complexity, the traffic interaction and communication between microservices have become more and more frequent, and traffic management and analysis have become the key links to ensure system stability and performance optimization.

[0003] In the existing traffic data parsing methods, traffic files are cut into individual files for parsing, resulting in problems such as traffic data loss, incomplete parsing results of traffic data, and even inability to complete the parsing of traffic data. Summary of the Invention

[0004] In view of the above problems, this application is proposed to provide a microservice traffic parsing method, apparatus, computing device, computer storage medium, and computer program product that overcome the above problems or at least partially solve the above problems.

[0005] According to one aspect of this application, a microservice traffic parsing method is provided, including:

[0006] Step 1: Pull traffic data from the network card of the microservice, perform real-time parsing processing on the traffic data, obtain a real-time parsing result, and output it to a target file;

[0007] Step 2: Read data from the target file, and when a complete first data frame is read, write the first data frame into a data channel;

[0008] Step 3: Read a second data frame from the data channel, perform key information parsing processing on the second data frame, and obtain traffic key data.

[0009] Optionally, the method is applied to an associated container of the business main container corresponding to the microservice, and the business main container and the associated container share the network card.

[0010] Optionally, before step 1, the method further includes:

[0011] Step 0: Create a first execution unit, a second execution unit, and a third execution unit respectively;

[0012] Among them, step 1 is executed by the first execution unit, step 2 is executed by the second execution unit, and step 3 is executed by the third execution unit.

[0013] Optionally, the method further includes:

[0014] If it is monitored that the size of the target file reaches the preset file size threshold, delete the target file, end the current first execution unit, second execution unit, and third execution unit, and jump to execute step 0.

[0015] Optionally, the method further includes:

[0016] If it is monitored that the parsing service restart instruction is received, delete the target file, end the current first execution unit, second execution unit, and third execution unit, and jump to execute step 0.

[0017] Optionally, pulling traffic data from the network card of the microservice further includes:

[0018] Pull traffic data of a specified protocol from the network card of the microservice.

[0019] Optionally, pulling traffic data of a specified protocol from the network card of the microservice further includes:

[0020] Parse the traffic data pulled from the network card of the microservice to determine the port information of the traffic data;

[0021] If the port information of the traffic data matches the port information recorded in the configuration file, it is determined that the traffic data of the specified protocol is pulled; wherein, the configuration file is used to record the port information corresponding to the specified protocol.

[0022] Optionally, before step 1, the method further includes:

[0023] Read the environment variables of the associated container to determine the port information corresponding to the specified protocol;

[0024] Generate a configuration file according to the port information corresponding to the specified protocol.

[0025] Optionally, the traffic key data includes: request path, request body, and / or return body.

[0026] According to another aspect of the present application, there is provided a microservice traffic parsing device, including:

[0027] A first processing module, configured to pull traffic data from the network card of the microservice, perform real-time parsing processing on the traffic data, obtain a real-time parsing result, and output it to the target file;

[0028] A second processing module, configured to read data from the target file, and when a complete first data frame is read, write the first data frame into the data channel;

[0029] A third processing module, configured to read a second data frame from the data channel, perform key information parsing processing on the second data frame, and obtain traffic key data.

[0030] Optionally, the device is applied to an accompanying container of a business main container corresponding to a microservice, and the business main container and the accompanying container share a network card.

[0031] Optionally, the device includes:

[0032] A creation module, configured to respectively create a first execution unit, a second execution unit, and a third execution unit;

[0033] Wherein, the first processing module executes an operation by using the first execution unit, the second processing module executes an operation by using the second execution unit, and the third processing module executes an operation by using the third execution unit.

[0034] Optionally, the device further includes:

[0035] A restart module, configured to, if it is monitored that the size of a target file reaches a preset file size threshold, delete the target file and end the current first execution unit, second execution unit, and third execution unit;

[0036] The creation module is further configured to: after the restart module ends the current first execution unit, second execution unit, and third execution unit, respectively create a new first execution unit, second execution unit, and third execution unit.

[0037] Optionally, the restart module is further configured to: if it is monitored that a parsing service restart instruction is received, delete the target file and end the current first execution unit, second execution unit, and third execution unit.

[0038] Optionally, the first processing module is further configured to: pull traffic data of a specified protocol from the network card of the microservice.

[0039] Optionally, the first processing module is further configured to:

[0040] Parse the traffic data pulled from the network card of the microservice to determine the port information of the traffic data; if the port information of the traffic data matches the port information recorded in the configuration file, determine that the traffic data of the specified protocol is pulled; wherein, the configuration file is used to record the port information corresponding to the specified protocol.

[0041] Optionally, the device further includes:

[0042] A configuration module, configured to read the environment variables of the accompanying container to determine the port information corresponding to the specified protocol; generate a configuration file according to the port information corresponding to the specified protocol.

[0043] Optionally, the traffic key data includes: a request path, a request body, and / or a return body.

[0044] According to another aspect of the present application, a computing device is provided, including: a processor, a memory, a communication interface, and a communication bus, and the processor, the memory, and the communication interface complete communication with each other through the communication bus;

[0045] The memory is used to store at least one executable instruction, and the executable instruction causes the processor to perform operations corresponding to the above-mentioned microservice traffic parsing method.

[0046] According to still another aspect of the present application, a computer storage medium is provided, and at least one executable instruction is stored in the storage medium, and the executable instruction causes the processor to perform operations corresponding to the above-mentioned microservice traffic parsing method.

[0047] According to yet another aspect of the present application, a computer program product is provided, including at least one executable instruction, and the executable instruction causes the processor to perform operations corresponding to the above-mentioned microservice traffic parsing method.

[0048] According to the microservice traffic parsing method, device, computing device, computer storage medium, and computer program product provided by the embodiments of the present application, traffic data is pulled from the network card of the microservice, the traffic data is subjected to real-time parsing processing, a real-time parsing result is obtained and output to a target file; data is read from the target file, and when a complete first data frame is read, the first data frame is written into a data channel; a second data frame is read from the data channel, and key information parsing processing is performed on the second data frame to obtain traffic key data. Through the above method, each piece of traffic data entering and leaving the microservice is captured and parsed in real time, and all traffic data can be obtained, the integrity of the traffic data can be guaranteed, so that the parsing success rate of the traffic data can be improved, and the concurrent security of data transmission can be guaranteed based on the data channel, reducing the possibility of errors during the traffic parsing process.

[0049] The above description is only an overview of the technical solution of the present application. In order to be able to understand the technical means of the present application more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features, and advantages of the present application more obvious and understandable, the specific embodiments of the present application are specifically given below. Description of the Drawings

[0050] By reading the following detailed description of the preferred embodiments, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present application. And throughout the drawings, the same reference numerals are used to represent the same components. In the drawings:

[0051] Figure 1 The flowchart of the microservice traffic parsing method provided by an embodiment of the present application is shown;

[0052] Figure 2 The flowchart of the microservice traffic parsing method provided by another embodiment of the present application is shown;

[0053] Figure 3 The flowchart of the microservice traffic parsing method provided by yet another embodiment of the present application is shown;

[0054] Figure 4 The schematic diagram of the functional structure of the microservice traffic parsing device provided by the embodiment of the present application is shown;

[0055] Figure 5 The schematic diagram of the structure of the computing device provided by the embodiment of the present application is shown. Detailed implementation manners

[0056] Hereinafter, the exemplary embodiments of the present application will be described in more detail with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present application can be more thoroughly understood and the scope of the present application can be fully conveyed to those skilled in the art.

[0057] First, the noun terms related to one or more embodiments of the present application are explained.

[0058] Microservice: A software development technology, a variant of the service-oriented architecture (SOA) architectural style, which advocates dividing a single application into a group of small services that coordinate and cooperate with each other to provide the ultimate value for users. Each service runs in its independent process, and lightweight communication mechanisms are used to communicate between services, usually using the HTTP protocol or the GRPC protocol.

[0059] HPACK algorithm: A header compression algorithm for HTTP / 2, which reduces the redundant data in the HTTP header through static or dynamic dictionaries and efficient encoding, thereby improving the network transmission efficiency.

[0060] eBPF technology: A high-performance and programmable kernel technology that allows user-defined code to be safely run without modifying the kernel source code to achieve various purposes such as network analysis, security monitoring, and performance tuning.

[0061] Channel: A pipeline for communication between goroutines, which supports safe and efficient data transfer and synchronization to achieve lock-free concurrent programming.

[0062] Proto file: An efficient and cross-platform serialization data format developed by Google, which is used for the definition, serialization, and deserialization of structured data and is often used for microservice communication and storage.

[0063] Figure 1 The flowchart of the microservice traffic parsing method provided by an embodiment of the present application is shown. As Figure 1 shown, the method includes the following steps:

[0064] Step S110: Pull traffic data from the network card of the microservice, perform real-time parsing processing on the traffic data, obtain the real-time parsing result, and output it to the target file.

[0065] The network card is a hardware device for the microservice to connect to the network. When microservices communicate with each other, network data packets (i.e., traffic data) are sent and received through the network card. Pull the traffic data received and sent by the network card, and perform real-time parsing processing on the pulled traffic data.

[0066] It should be noted that the real-time parsing processing can parse out comprehensive and complex data. The real-time parsing result obtained by the real-time parsing processing is temporarily stored in the target file, and the data is read from the target file for secondary parsing (i.e., key information parsing) in the subsequent process.

[0067] Step S120: Read data from the target file. When a complete first data frame is read, write the first data frame into the data channel.

[0068] A complete data frame refers to: in data communication, it is encapsulated according to specific protocol rules, contains all necessary fields and information, and is the basic unit that can transmit data completely and accurately in the network.

[0069] Read the data line by line in the target file, determine whether the read data constitutes a complete data frame. Any complete data frame read from the target file is called the first data frame. Specifically, determine whether the read data constitutes a complete data frame through the identifier of the data.

[0070] Whenever a complete first data frame is read, the read first data frame is written into the data channel as a whole, so that the complete data frame can be read from the data channel for secondary parsing in the subsequent process. Among them, the data channel can adopt a channel.

[0071] Step S130: Read a second data frame from the data channel, perform key information parsing processing on the second data frame, and obtain traffic key data.

[0072] Any complete data frame read from the data channel is the second data frame. Parse the second data frame according to the communication protocol adopted by the second data frame, and extract the key information therein, including the request path, request body, and / or return body, etc., to obtain the traffic key data. By writing the read complete data frame into the data channel and waiting for secondary parsing, and reading the complete data frame from the data channel for secondary parsing, the concurrent security of data transfer can be ensured, and the possibility of errors in the traffic parsing process can be reduced.

[0073] In the existing traffic parsing methods, due to the defect of being unable to ensure the integrity of traffic data, there are many limitations in actual applications. For example, first, it cannot meet the requirements of scenarios with high requirements for data integrity. For example, in test scenarios with high requirements for traffic recording accuracy, such as full-link service scenario testing, the loss of traffic data often leads to the incompleteness of the entire scenario and the inability to conduct tests; second, when parsing protocols that use the HPACK algorithm of HTTP / 2 such as GRPC, the lack of context will cause the parsing tool to be unable to restore the complete request header data, resulting in the inability to parse the traffic data.

[0074] In summary, according to the microservice traffic parsing method provided in this embodiment, pull traffic data from the network card of the microservice, perform real-time parsing processing on the traffic data, obtain the real-time parsing result and output it to the target file; read data from the target file, and when a complete first data frame is read, write the first data frame into the data channel; read the second data frame from the data channel, and perform key information parsing processing on the second data frame to obtain the traffic key data. Through the above method, each piece of traffic data entering and leaving the microservice is captured and parsed in real time, and the full amount of traffic data can be obtained, the integrity of the traffic data can be ensured, thereby improving the parsing success rate of the traffic data, meeting the requirements of scenarios with high requirements for data integrity, and for protocols that require a complete request context due to the header compression algorithm, since the complete context can be obtained, the traffic data of such protocols can also be successfully parsed; based on the data channel, the concurrent security of data transfer can also be ensured, and the possibility of errors in the traffic parsing process can be reduced.

[0075] Figure 2 The flowchart of the microservice traffic parsing method provided in another embodiment of the present application is shown. The method of the embodiment of the present application is applied to the companion container of the business main container corresponding to the microservice. The business main container and the companion container share the network card. The microservice is deployed in the business main container. The business main container provides a running environment for the microservice. The companion container is an auxiliary container that works in cooperation with the business main container, and the traffic parsing method of the embodiment of the present application is implemented in the companion container.

[0076] As Figure 2 shown, the method includes the following steps:

[0077] Step S210: Create a first execution unit, a second execution unit, and a third execution unit respectively.

[0078] Create three different execution units in the associated container. The three execution units are respectively used to execute multiple different subsequent steps. Specifically, step S220 is executed by the first execution unit, step S230 is executed by the second execution unit, and step S240 is executed by the third execution unit.

[0079] Specifically, step S210 is executed in response to an instruction for starting the parsing service (including the parsing service restart instruction).

[0080] In an alternative manner, the execution unit is specifically a coroutine. A coroutine is a lightweight thread in user space, also known as a micro-thread. Different from traditional threads, the scheduling of coroutines is completely controlled by the user program rather than by the operating system kernel. This means that the creation, destruction, and switching of coroutines do not require frequent switching between the kernel space and the user space, thus reducing system overhead and improving the execution efficiency of the program.

[0081] Step S220: Use the first execution unit to pull traffic data of a specified protocol from the network card of the microservice, perform real-time parsing processing on the traffic data, and obtain a real-time parsing result and output it to a target file.

[0082] Among them, data is transmitted between microservices through a link. The link will be disconnected after being idle for a period of time and then rebuilt when there is a communication requirement, so as to avoid wasting resources due to the long-term retention of idle links. Complete communication traffic data can be transmitted through the link.

[0083] Since all communication traffic data will enter and exit through the network card, but not all traffic data entering and exiting the network card needs to be parsed. In the method of the embodiment of the present application, traffic data of a specified protocol is pulled from the network card of the microservice, where the specified protocol is the protocol used for communication between microservices.

[0084] In an alternative approach, pulling traffic data of a specified protocol from the network card of a microservice specifically includes: parsing the traffic data pulled from the network card of the microservice to determine the port information of the traffic data; if the port information of the traffic data matches the port information recorded in the configuration file, it is determined that the traffic data of the specified protocol has been pulled; wherein, the configuration file is used to record the port information corresponding to the specified protocol. The traffic data is specifically transmitted according to the ports of the application program. After pulling the traffic data from the network card, the corresponding port information is parsed. If the port information of the pulled traffic data is consistent with the port information recorded in the configuration file, then the traffic data is the traffic data of the specified protocol that needs to be parsed and processed; otherwise, if the port information of the pulled traffic data is inconsistent with the port information recorded in the configuration file, there is no need to parse and process the traffic data. Through this method, business traffic can be accurately parsed, avoiding parsing of irrelevant traffic data and reducing the computational pressure of parsing.

[0085] Among them, the configuration file is pre-generated. The specific implementation method for generating the configuration file is: reading the environment variables of the associated container to determine the port information corresponding to the specified protocol; generating a configuration file according to the port information corresponding to the specified protocol. Read the environment variables in the associated container to obtain the ports corresponding to the specified protocol. For example, the port information of the HTTP protocol is 8000, and the port information of the GRPC protocol is 9000, and a configuration file is automatically generated to record the port information corresponding to the specified protocol.

[0086] Specifically, the first execution unit uses a specified tool to pull traffic data from the network card and perform real-time parsing and processing on the traffic data.

[0087] The specified tool can use the tshark tool. Tshark is the command-line version of Wireshark, which is used to capture and analyze network packets, supports parsing of multiple protocols, and is suitable for automated network diagnosis and data processing. Among them, the tshark tool can parse and obtain multi-level traffic information, that is, the real-time parsing result contains a large amount of information, specifically including: basic packet information (such as timestamp, address information, protocol information, etc.), detailed packet content (such as header information, data payload, etc.), network traffic-related information, protocol analysis results, etc. Subsequently, key traffic data of interest needs to be extracted through secondary parsing (i.e., key information parsing).

[0088] After completing the real-time parsing and processing of the traffic data pulled from the network card, the obtained real-time parsing result is written into a local target file. Optionally, the real-time parsing result is converted into a specified form and then written into the target file. The specified form can be the json form, converting the real-time parsing result into a form with less parsing difficulty, facilitating secondary parsing to extract key information and improving the efficiency of secondary parsing.

[0089] Step S230: Use the second execution unit to read data from the target file. When a complete first data frame is read, write the first data frame into the data channel.

[0090] The second execution unit reads data from the target file line by line and determines whether the multiple lines of data read constitute a complete data frame. Whenever it is determined that a complete data frame is read, the complete data frame read is written into the data channel for secondary parsing.

[0091] To avoid the problem of OOM (out of memory) in the associated container, monitor the size of the target file. If it is detected that the size of the target file reaches the preset file size threshold, delete the target file, end the current first execution unit, second execution unit, and third execution unit, and jump to execute step S210. This method can realize automatic restart of the parsing service. When the associated container detects that the size of the target file reaches the threshold, end the current three execution units, delete the traffic file, and jump to execute step S210 to reconstruct the three execution units for traffic parsing processing, ensuring that the size of the target file in the associated container is in a controllable state and avoiding the problem of memory overflow in the associated container caused by long-term real-time parsing.

[0092] In addition, if a parsing service restart instruction is detected, delete the target file, end the current first execution unit, second execution unit, and third execution unit, and jump to execute step S210. Among them, the parsing service restart instruction can be manually triggered by relevant personnel. For example, when the restart switch of the parsing service is turned on, triggering the parsing service restart instruction. When the parsing service restart instruction is detected, end the current three execution units and delete the target file, and then reconstruct the three execution units for traffic parsing processing. In this way, it is convenient to control the state of the parsing service.

[0093] Step S240: Use the third execution unit to read the second data frame from the data channel and perform key information parsing processing on the second data frame to obtain traffic key data.

[0094] Using the second execution unit to write the data frame into the data channel and using the third execution unit to read the data frame from the data channel for parsing makes the data frame reading and data frame parsing execute independently, which can ensure the concurrent security of data transfer and reduce the possibility of errors in the traffic parsing process.

[0095] Among them, the traffic key data includes various key information, specifically including the request path, request body, and return body, etc. After obtaining the traffic key data, persistently store it in the storage device.

[0096] In an alternative approach, various key information contained in traffic critical data is reassembled into an intuitive and highly readable piece of data for persistent storage.

[0097] In another existing traffic data parsing method, the eBPF technology provided by the Linux kernel is used to directly track the plaintext data from the program memory, and then the relevant SDKs relied on by the program are modified for data instrumentation. Through means such as collecting instrumentation data and log analysis, the traffic data of the service is restored and then persistently stored. However, this method requires separate modification for each SDK relied on by the application, and if the upstream code changes, it may be necessary to re-maintain, which is highly invasive to the code, cannot achieve out-of-the-box usability, is not conducive to the promotion after the solution is implemented, and also requires irregular maintenance, consuming a large amount of manpower.

[0098] In the method of the embodiment of the present application, by creating an accompanying container that shares a gateway with the business main container of the microservice, the gateway traffic data is captured and parsed within the accompanying container. Utilizing the feature that the accompanying container and the business main container share a network card, by starting the traffic capture tool in the accompanying container, the network card traffic is directly captured and analyzed item by item. This method does not depend on the implementation method of the service, can ensure no loss of traffic data, avoid invasiveness to the code, and can also achieve the effect of out-of-the-box usability; in addition, the parsing of traffic data of a specified protocol depends on the dependency files required by the microservice interface, such as proto files. In the method of the embodiment of the present application, by adopting an accompanying container, when the dependency file is updated, the accompanying container can be ensured to obtain the latest dependency file by enabling the parsing service; in summary, the method of the embodiment of the present application avoids the problems existing in the traffic parsing method based on the eBPF technology, such as the need to follow upstream changes for update and maintenance, invasiveness to the code, and high maintenance costs.

[0099] In summary, according to the microservice traffic parsing method provided in this embodiment, by capturing each piece of traffic data entering and leaving the microservice in real time at the network card for parsing, all traffic data can be obtained, ensuring the integrity of the traffic data, thereby improving the success rate of traffic data parsing; by constructing three different coroutines to respectively execute traffic data capture and real-time parsing, data reading and writing of complete data frames, and data frame reading and secondary parsing, it is possible to avoid interference between different processes; by writing the read complete data frames into the data channel and waiting for secondary parsing, and reading the complete data frames from the data channel for secondary parsing, it is possible to ensure the concurrent security of data transfer and reduce the possibility of errors during the traffic parsing process; taking advantage of the characteristic that the companion container and the main container share the network card, by starting the traffic capture tool in the companion container to directly capture and analyze the network card traffic data item by item, it is possible to avoid code invasiveness while ensuring the integrity of the traffic data. At the same time, by configuring one companion container, the parsing of traffic data can be achieved, achieving the effect of being ready to use out of the box; by monitoring the size of the target file storing the real-time parsing results and deleting the target file when the size of the target file reaches the threshold, it is possible to avoid the problem of memory overflow in the companion container; by pulling traffic files of a specified protocol using the configuration file, it is possible to accurately parse business traffic, avoid parsing irrelevant traffic data, and reduce the computational pressure of parsing.

[0100] Figure 3 The flowchart of the microservice traffic parsing method provided in another embodiment of the present application is shown. As Figure 3 shown, the service container 30 corresponding to the microservice includes a service main container 350 and its companion container 320. The traffic data of external requests is transmitted through the network card 310. The real-time parsing tool 3201 in the companion container 320 pulls the traffic data from the network card 310 and performs real-time parsing. The real-time parsing results are then processed by the secondary parsing tool 3202 for key information parsing to obtain traffic key data. The traffic key data is assembled into intuitive and highly readable data and sent to the message queue 330, and then the data in the message queue 330 is persistently stored in the search engine 340.

[0101] Figure 4 The functional structure diagram of the microservice traffic parsing device provided in the embodiment of the present application is shown. As Figure 4 shown, the device includes:

[0102] A first processing module 410, configured to pull traffic data from the network card of the microservice, perform real-time parsing processing on the traffic data, obtain real-time parsing results, and output them to a target file;

[0103] A second processing module 420, configured to read data from the target file, and when a complete first data frame is read, write the first data frame into the data channel;

[0104] A third processing module 430, configured to read a second data frame from a data channel, perform key information parsing and processing on the second data frame, and obtain traffic key data.

[0105] In an optional manner, the device is applied to an associated container of a business main container corresponding to a microservice, and the business main container and the associated container share a network card.

[0106] In an optional manner, the device includes:

[0107] A creation module, configured to respectively create a first execution unit, a second execution unit, and a third execution unit;

[0108] Among them, the first processing module 410 executes operations using the first execution unit, the second processing module 420 executes operations using the second execution unit, and the third processing module 430 executes operations through the third execution unit.

[0109] In an optional manner, the device further includes: a restart module, configured to, if it is monitored that the size of a target file reaches a preset file size threshold, delete the target file, and end the current first execution unit, second execution unit, and third execution unit;

[0110] The creation module is further configured to: after the restart module ends the current first execution unit, second execution unit, and third execution unit, respectively create new first execution unit, second execution unit, and third execution unit.

[0111] In an optional manner, the restart module is further configured to: if it is monitored that a parsing service restart instruction is received, delete the target file, and end the current first execution unit, second execution unit, and third execution unit.

[0112] In an optional manner, the first processing module 410 is further configured to: pull traffic data of a specified protocol from the network card of the microservice.

[0113] In an optional manner, the first processing module 410 is further configured to:

[0114] Parse the traffic data pulled from the network card of the microservice to determine the port information of the traffic data; if the port information of the traffic data matches the port information recorded in the configuration file, determine that the traffic data of the specified protocol is pulled; where the configuration file is used to record the port information corresponding to the specified protocol.

[0115] In an optional manner, the device further includes: a configuration module, configured to read the environment variables of the associated container, determine the port information corresponding to the specified protocol; and generate a configuration file according to the port information corresponding to the specified protocol.

[0116] In an alternative approach, the traffic critical data includes: request path, request body, and / or response body.

[0117] In summary, according to the microservice traffic parsing device provided in this embodiment, by capturing and parsing each piece of traffic data in and out of the microservice in real time, all traffic data can be obtained, the integrity of the traffic data can be ensured, thereby improving the parsing success rate of the traffic data. Based on the data channel, the concurrent security of data transmission can also be ensured, reducing the possibility of errors during the traffic parsing process.

[0118] The embodiment of the present application provides a non-volatile computer storage medium, and the computer storage medium stores at least one executable instruction or computer program, and the executable instruction or computer program can enable a processor to perform operations corresponding to the microservice traffic parsing method in any of the above method embodiments.

[0119] The embodiment of the present application provides a computer program product, and the computer program product includes at least one executable instruction or computer program, and the executable instruction or computer program can enable a processor to perform operations corresponding to the microservice traffic parsing method in any of the above method embodiments.

[0120] Figure 5 The structure diagram of the computing device embodiment of the present application is shown, and the specific implementation of the computing device is not limited in the specific embodiments of the present application.

[0121] As Figure 5 shown, the computing device may include: a processor 502, a communication interface 504, a memory 506, and a communication bus 508.

[0122] Among them: the processor 502, the communication interface 504, and the memory 506 communicate with each other through the communication bus 508. The communication interface 504 is used to communicate with network elements of other devices such as clients or other servers. The processor 502 is used to execute the program 510, and specifically can execute relevant steps in the above microservice traffic parsing method embodiment for the computing device.

[0123] Specifically, the program 510 may include program code, and the program code includes computer operation instructions.

[0124] The processor 502 may be a central processing unit (CPU), or a specific application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. One or more processors included in the computing device may be of the same type, such as one or more CPUs; or may be of different types, such as one or more CPUs and one or more ASICs.

[0125] A memory 506 for storing a program 510. The memory 506 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk memory.

[0126] Specifically, the program 510 may be used to cause the processor 502 to execute the microservice traffic parsing method in any of the above method embodiments. For the specific implementation of each step in the program 510, reference may be made to the corresponding steps and descriptions in the microservice traffic parsing embodiments, which will not be elaborated herein. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described devices and modules may refer to the corresponding process descriptions in the foregoing method embodiments, which will not be repeated herein.

[0127] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems may also be used in conjunction with the teachings provided herein. The structure required to construct such a system will be apparent from the above description. In addition, the embodiments of the present application are not directed to any specific programming language. It should be understood that the content of the present application described herein can be implemented using various programming languages, and the descriptions of specific languages above are for disclosing the best mode of the present application.

[0128] In the specification provided herein, a large number of specific details are set forth. However, it can be understood that the embodiments of the present application may be practiced without these specific details. In some instances, well-known methods, structures, and technologies have not been shown in detail so as not to obscure the understanding of this specification.

[0129] Similarly, it should be understood that, for the sake of streamlining this application and assisting in understanding one or more of the various inventive aspects, in the above description of the exemplary embodiments of this application, the various features of the embodiments of this application are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed method should not be construed as reflecting an intention that the claimed application requires more features than are expressly recited in each claim. Rather, as reflected in the claims, the inventive aspects lie in less than all the features of the single embodiments disclosed previously. Thus, the claims following the detailed description hereby expressly incorporate the detailed description, where each claim itself serves as a separate embodiment of this application.

[0130] Those skilled in the art can understand that the modules in the devices in the embodiments can be adaptively changed and disposed in one or more devices different from the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and in addition, they can be divided into multiple sub-modules or sub-units or sub-components. Except that at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all the features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all the processes or units of any method or device so disclosed. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) can be replaced by an alternative feature that provides the same, equivalent, or similar purpose.

[0131] In addition, those skilled in the art can understand that although some of the embodiments herein include certain features included in other embodiments rather than other features, the combination of the features of different embodiments means that it is within the scope of this application and forms different embodiments. For example, in the claims, any one of the claimed embodiments can be used in any combination.

[0132] The various component embodiments of this application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) can be used in practice to implement some or all of the functions of some or all of the components according to the embodiments of this application. This application can also be implemented as a device or apparatus program (such as a computer program and a computer program product) for executing part or all of the methods described herein. Such a program for implementing this application can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0133] It should be noted that the above embodiments are illustrative of the present application rather than restrictive thereof, and those skilled in the art can design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in the claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present application can be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In a unit claim listing several devices, several of these devices may be embodied by the same item of hardware. The use of the words first, second, and third, etc. does not denote any order. These words may be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the order of execution.

Claims

1. A microservice traffic parsing method, comprising: Step 1: Pull traffic data from the network card of the microservice, perform real-time analysis on the traffic data, obtain real-time analysis results and output them to the target file; Step 2: Read data from the target file, and when a complete first data frame is read, write the first data frame into the data channel; Step 3: Read a second data frame from the data channel, perform key information analysis on the second data frame, and obtain traffic key data.

2. The method according to claim 1, wherein: The method is applied to a companion container of a business main container corresponding to the microservice, and the business main container and the companion container share a network card.

3. The method according to claim 1 or 2, wherein: Prior to step 1, the method further comprises: Step 0: Create a first execution unit, a second execution unit, and a third execution unit respectively; Wherein, step 1 is executed by the first execution unit, step 2 is executed by the second execution unit, and step 3 is executed by the third execution unit.

4. The method according to claim 3, wherein: The method further comprises: If it is monitored that the size of the target file reaches a preset file size threshold, the target file is deleted, the current first execution unit, the second execution unit and the third execution unit are terminated, and the execution of step 0 is jumped.

5. The method according to claim 3, wherein: The method further comprises: If a parsing service restart instruction is detected, the target file is deleted, the current first execution unit, the second execution unit and the third execution unit are terminated, and the execution jumps to step 0.

6. The method according to any one of claims 1 to 5, wherein: Pulling traffic data from the network card of the microservice further includes: Pull the traffic data of the specified protocol from the network card of the microservice.

7. The method according to claim 6, wherein: Pulling the traffic data of the specified protocol from the network card of the microservice further includes: Parse the traffic data pulled from the network card of the microservice to determine the port information of the traffic data; If the port information of the traffic data matches the port information recorded in the configuration file, it is determined that the traffic data of the specified protocol is pulled; wherein the configuration file is used to record the port information corresponding to the specified protocol.

8. The method according to claim 7, wherein: Prior to step 1, the method further comprises: Read the environment variables of the companion container to determine the port information corresponding to the specified protocol; The configuration file is generated according to the port information corresponding to the specified protocol.

9. The method according to any one of claims 1 to 8, wherein: The traffic key data includes: request path, request body and / or return body.

10. A microservice traffic analysis device, comprising: The first processing module is used to pull traffic data from the network card of the microservice, perform real-time analysis on the traffic data, obtain real-time analysis results and output them to the target file; A second processing module is used to read data from the target file, and when a complete first data frame is read, write the first data frame into a data channel; The third processing module is used to read the second data frame from the data channel, perform key information analysis on the second data frame, and obtain traffic key data.

11. A computing device comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the microservice traffic parsing method according to any one of claims 1 to 9.

12. A computer storage medium, wherein at least one executable instruction is stored in the storage medium, and the executable instruction enables a processor to perform operations corresponding to the microservice traffic parsing method according to any one of claims 1 to 9.

13. A computer program product, comprising at least one executable instruction, wherein the executable instruction enables a processor to perform operations corresponding to the microservice traffic parsing method according to any one of claims 1 to 9.

Images