Method, system and device for efficiently capturing and analyzing kubernetes application network data packet

Abstract

The invention relates to the technical field of computer networks and cloud computing, and discloses a kubernetes application network data packet efficient capture and analysis method, system and device, and the key points of the technical scheme are that the system comprises a central controller module which is used for obtaining configuration parameters required for capturing a data packet; the resource information acquisition module is used for communicating with an apiserver of kubernetes and acquiring resource information of a cluster; the task scheduling module is used for generating a temporary scheduling-task label for a specified host machine according to resource information of a cluster in combination with configuration parameters, generating a data packet capture task and distributing the data packet capture task; the analysis module is used for creating a shared storage volume and obtaining an analysis result from the shared storage volume; the data packet capturing module is used for executing a data packet capturing operation on a specified host machine according to the data packet capturing task and storing a captured data packet in a shared storage volume; and the data packet analysis module is used for acquiring the data packet, analyzing the data packet in real time and storing an analysis result to the shared storage volume.

Description

Technical Field

[0001] The present invention relates to the technical fields of computer networks and cloud computing, and more specifically, to a method, system, and device for efficiently capturing and parsing Kubernetes application network data packets. Background Art

[0002] In modern cloud-native applications, Kubernetes has become the mainstream container orchestration platform, which provides capabilities such as flexible management, automated scaling, containerized deployment, and scheduling, and has become a key technology for building and managing large-scale distributed systems. Kubernetes clusters help developers more efficiently manage the lifecycle of applications through abstracted resource management, automated load balancing, and self-healing capabilities.

[0003] However, with the sharp increase in the number of services deployed in Kubernetes clusters and the widespread application of the microservices architecture, the network traffic inside the cluster has become extremely complex. The microservices architecture usually consists of hundreds or thousands of services and containers that communicate with each other, and these services conduct frequent network communications through various protocols such as HTTP, gRPC, TCP, and UDP. Each service may access across multiple networks, multiple regions, or even multiple cloud environments, further exacerbating the complexity and dynamics of the traffic.

[0004] In this context, how to efficiently capture and parse the data packets of each service in the Kubernetes cluster has become a key challenge in the operation and maintenance field. For traditional network data packet capture and parsing methods, the process for operation and maintenance personnel to capture the underlying data packets is usually to log in to the host and then execute the data packet capture command to obtain them, and then transmit the data packets out for unified analysis through analysis tools, which often cannot meet the requirements in a large-scale, highly dynamic, and containerized environment. Especially in a highly dynamic environment like Kubernetes, the number of application pods is numerous and distributed on different hosts, and the frequent changes of services, Pods, networks, and traffic make it more difficult to capture and parse data packets.

[0005] Therefore, how to perform efficient data packet capture and parsing becomes particularly important. Summary of the Invention

[0006] The purpose of the present invention is to provide a method, system, and device for efficiently capturing and parsing Kubernetes application network data packets, which optimize the accuracy and efficiency of data packet capture and the real-time performance of subsequent parsing. It can not only meet the data packet capture requirements in a large-scale Kubernetes environment but also ensure the efficient synchronization of data packet capture and parsing.

[0007] The above technical object of the present invention is achieved by the following technical solutions:

[0008] An efficient method for capturing and parsing Kubernetes application network data packets, comprising the following steps:

[0009] S1. Obtain the configuration parameters required for capturing data packets;

[0010] S2. Through communication with the apiserver of Kubernetes, obtain the resource information of the Kubernetes cluster:

[0011] Combined with the configuration parameters, generate a temporary scheduling-task label for the specified host, generate a data packet capture task and distribute it;

[0012] At the same time, create a shared storage volume;

[0013] S3. After the data packet capture task is distributed, perform the data packet capture operation on the specified host and store the captured data packets in the shared storage volume;

[0014] S4. Obtain the data packets from the shared storage volume and parse them in real time, and store the parsing results in the shared storage volume;

[0015] S5. Obtain the parsing results from the shared storage volume.

[0016] An efficient system for capturing and parsing Kubernetes application network data packets, comprising:

[0017] A central controller module, used to obtain the configuration parameters required for capturing data packets;

[0018] Used to communicate with the apiserver of Kubernetes to obtain the resource information of the Kubernetes cluster;

[0019] Used to generate a temporary scheduling-task label for the specified host according to the resource information of the Kubernetes cluster, combine with the configuration parameters, generate a data packet capture task and distribute it;

[0020] Used to create a shared storage volume and obtain the parsing results from the shared storage volume;

[0021] A data packet capture module, used to perform the data packet capture operation on the specified host according to the data packet capture task and store the captured data packets in the shared storage volume;

[0022] A data packet parsing module, used to obtain the data packets from the shared storage volume and parse them in real time, and store the parsing results in the shared storage volume.

[0023] As a preferred technical solution of the present invention, the central controller module is used to monitor the execution status of the packet capture task in the data capture module;

[0024] It is also used to issue a packet analysis task to the packet parsing module while distributing the packet capture task to the packet capture module;

[0025] It is used to dynamically generate temporary host tags according to the resource information of the kubernetes cluster, combine with configuration parameters, mark the specified host with temporary scheduling-task tags, and then distribute the packet capture task by scheduling the Job resources of Kubernetes.

[0026] As a preferred technical solution of the present invention, the configuration parameters required for capturing packets include: application resource type, packet capture time, address and port of peer communication.

[0027] As a preferred technical solution of the present invention, the resource information of the kubernetes cluster includes: information of the host where the application is located and pod container information.

[0028] As a preferred technical solution of the present invention, after the shared storage volume is created, it is mounted on the Pod generated by the capture task and the packet parsing module.

[0029] As a preferred technical solution of the present invention, both the central controller module and the packet parsing module are mounted with the primary storage path of the shared storage volume, and the data of the dynamically created secondary storage path can be obtained through the primary storage path;

[0030] After the packet parsing module parses the packet content, it stores the parsed packet content in the secondary storage path of the shared storage volume;

[0031] The central controller module is used to access the secondary storage path of the shared storage volume to obtain the parsed packets.

[0032] As a preferred technical solution of the present invention, the packet capture module schedules the task Pod on the target node through the Job resources of Kubernetes and runs in the host network mode to capture packets;

[0033] It is used to store the captured packets in the shared storage volume and create a folder with a unique id for each captured pod.

[0034] As a preferred technical solution of the present invention, the packet capture module uses the built-in logging interface of kubernetes to transmit the logs back to the central controller module in real time, and destroys the job resources by itself after a preset time when the packet capture task is completed.

[0035] An efficient capture and parsing device for Kubernetes application network data packets, comprising: a processor and a memory, the memory stores computer programs executable by the processor, and the processor implements the above method when executing the computer programs.

[0036] In summary, the present invention has the following beneficial effects:

[0037] By interacting with the Kubernetes API server, determine the host, Pod, and peer communication addresses for which data packet capture is required, and precisely allocate the data packet capture tasks to the specified hosts and nodes. Utilize the Job resources of Kubernetes to schedule the Pod to run on the specified nodes, and at the same time use the HostNetwork mode to ensure efficient communication between the Pod and the host network, ensuring the accuracy and real-time nature of data packet capture.

[0038] The combination of shared storage and real-time parsing, through the central controller module communicating with the Kubernetes cluster to create a temporary shared storage volume for each task Pod. This shared storage volume can be used to store the captured data packets and be read and parsed in real time by the data packet parsing module. This design avoids the delay in data upload, ensures efficient data sharing, and guarantees the fast synchronization between data packet capture and parsing, greatly improving the data processing efficiency.

[0039] Through modular design, achieve efficient collaboration of task distribution, data packet capture, and parsing. Utilize the scheduling and storage capabilities of Kubernetes to optimize the accuracy, efficiency of data packet capture, and real-time nature of subsequent parsing. This method can not only meet the data packet capture requirements in a large-scale Kubernetes environment but also ensure the efficient synchronization of data packet capture and parsing. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a schematic diagram of the system execution of the present invention;

[0041] Figure 2 is a flowchart of the method execution of the present invention;

[0042] Figure 3 is a flowchart of the data packet capture of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0043] The following further elaborates on the present invention in conjunction with the accompanying drawings.

[0044] The present invention provides an efficient capture and parsing system for Kubernetes application network data packets, comprising: a central controller module, a shared storage volume, a data packet capture module, and a data packet parsing module;

[0045] The central controller module is used to determine the workload of the application in Kubernetes by obtaining user operations when a network-level failure or problem occurs in the application, obtain the configuration parameters required for capturing data packets, and configure relevant parameters according to the configuration parameters, that is, select and determine the Kubernetes application resource type required for data packet capture and the corresponding parameter passing.

[0046] Specifically, the configuration parameters required for capturing data packets include: application resource type, data packet capture time, and address and port information for communicating with the peer.

[0047] Among them, the application resource type is the workload type used by the application, such as deployment, pod, statefulset, demonset, etc.

[0048] In the case where the cluster involves a multi-network card network, the network card name is declared, and the configuration parameters will involve communication with the peer, so the address and port information for communication with the peer also needs to be configured.

[0049] Through these above configuration parameters, the capture requirements of the application data packets can be clarified, and execution parameters can be provided for other lower-level modules.

[0050] The central controller module is also used to communicate with the apiserver of Kubernetes to obtain information about the relevant resources of the Kubernetes cluster. The information about the relevant resources includes: the network address of the specific workload resources, which includes the address of the host where the application is located and the internal address of the pod container.

[0051] After the information about the relevant resources of the Kubernetes cluster is determined, the central controller module will dynamically generate a temporary host label and distribute the data packet capture task by scheduling the Job resource type of Kubernetes. Specifically, the central controller module communicates with the apiserver of Kubernetes to add a temporary schedulable label to the specified host. For example, if the node name is kubernetes-node1, the corresponding task label job=id value is added. When the central controller module creates a Job task, it configures the nodeselector label for the job resource for fixed-point scheduling. In this way, it can avoid wasting cluster resources by scheduling to nodes that are not the target for data capture and improve the accuracy of the capture task.

[0052] Shared storage volume. The central controller module communicates with the apiserver to create a shared storage volume and use the shared storage volume for subsequent data parsing and reading as well as subsequent parsing result reading. These shared storage volumes will be mounted to the Pods generated by the capture task and the packet parsing module, thus ensuring the real-time performance and efficiency of the captured network data. The central controller module mounts the primary shared storage path by itself and can obtain the data of the dynamically created secondary storage path at any time.

[0053] Packet capture module, which is responsible for performing packet capture operations after determining the packet capture task.

[0054] The specific process of the packet capture module is as follows:

[0055] Task scheduling. According to the task distribution of the central controller module, the packet capture module will start the packet capture task on the specified host to ensure the quality and integrity of the captured network traffic.

[0056] To capture the network traffic inside the host and Pod, the packet capture module schedules a dedicated task Pod on the target node through the Kubernetes Job resource and runs it in the host network mode to ensure that the Pod can access the network card on the host network for packet capture. The captured data will be stored in the allocated shared storage volume, and a folder with a unique ID will be assigned to each captured pod for the packet parsing module to identify. The packet capture module also has a logging function, which is transmitted back to the central controller module in real time through the built-in logging interface of Kubernetes. After a period of time when the packet capture task is completed, the job resource will be self-destroyed to achieve resource recycling.

[0057] Packet parsing module, which is responsible for real-time parsing of the content of the captured packets. By communicating with the shared storage volume allocated by the packet capture module, it obtains the captured data and performs operations such as filtering, screening, and comparison of abnormal packets according to the preset parsing and processing requirements. Among them, the packet parsing module mounts the primary directory of the shared storage volume and can access the secondary shared storage directories generated by different tasks. Finally, the parsing results are output to the specified secondary folder of the shared storage volume, and users can directly access the secondary shared storage path through the central controller module to download the parsing results or directly download the source packets.

[0058] Corresponding to the above system, the present invention also provides a method for efficiently capturing and parsing Kubernetes application network packets;

[0059] S1. Obtain the configuration parameters required for capturing packets;

[0060] S2. Obtain the resource information of the Kubernetes cluster through communication with the Kubernetes apiserver, dynamically generate temporary host tags and packet capture tasks in combination with configuration parameters, and distribute the packet capture tasks;

[0061] At the same time, create a shared storage volume;

[0062] After the packet capture tasks are distributed, perform packet capture operations on the hosts specified by the temporary host tags, and store the captured packets in the shared storage volume;

[0063] S4. Obtain the packets from the shared storage volume and parse them in real time, and store the parsing results in the shared storage volume;

[0064] S5. Obtain the parsing results from the shared storage volume.

[0065] Specifically, the present invention also provides a factual example for the method, and the specific content is as follows:

[0066] According to requirements, configure the application resource types and packet acquisition parameters for which packets need to be captured within the Kubernetes cluster. Part of the requirements come from the requirements provided by users, such as the packet capture time, the addresses and ports for communicating with the peer end, so as to adapt to different business scenarios and network environments. These requirement parameters help the system efficiently determine the execution details and scope of the capture tasks.

[0067] Invoke the network information of the application resource hosts and pods through the Kubernetes apiserver interface. Specifically, according to the configuration parameter information, the central controller module communicates with the Kubernetes cluster through the cluster apiserver certificate. Under the call of the Kubernetes cluster's apiserver interface, it is converted into parameters executable by the program. The central controller module will interact with the apiserver to obtain the actual pod information and host information of the application's workload. Specifically, through communication with the apiserver and querying the status.pod.ip and status.host.ip of the pod metadata for the specified workload position. Workloads generally refer to pod, deploymenet, statefulset, and daemonset resources. All execution parameters will be passed to the upcoming capture module in the form of variables, that is, in the env form of the Kubernetes workload resources. The capture module can execute tasks efficiently and accurately based on these parameters. Among them, the packet capture module is constructed in real time during the execution process, and the packet parsing module can be created during the execution process or run as a default instance in the cluster.

[0068] Furthermore, the central processing unit module interacts with the apiserver of Kubernetes to accurately locate the pod address and host address associated with it according to the application services in the cluster. Specifically, it is located through the label correlation of resources, such as the process of obtaining the pod IP address and the host where the pod is located under the resources of the deploymenet application. First, it interacts with the apiserver, accesses the apiserver through the token or certificate provided by the cluster to obtain the label selector of the required deploymenet. The label selector is usually a set of labels in the form of key-value pairs. For example, app=my-app. Use the Kubernetes API to query all Pods that match this selector. The target of the API request is / api / v1 / namespaces / {namespace} / pods, where {namespace} is the namespace. You can add the query parameter labelSelector in the request to specify the label selector response, which contains a series of Pod objects, and each object contains detailed information about the Pod, including its IP address (status.podIP field) and host IP address (spec.hostIP field). The IP address of the host is used for label-based fixed-point scheduling, and the pod address is used as a variable for tasks. This method also applies to other workloads.

[0069] Dynamically create a packet capture task based on the obtained resource information and capture packets. In the Kubernetes cluster, the central controller module dynamically creates a packet capture task and a packet parsing task according to the complete parameters provided in the previous step. Specifically, it communicates with the Kubernetes cluster through the apiserver certificate provided by the Kubernetes cluster, and creates corresponding temporary node label and job resources according to the obtained relevant resources. The nodelabel label resource will add a temporary label to the node corresponding to the obtained host IP parameter for fixed-point scheduling of the pod generated by the job. Assuming that the total number of obtained host parameters is P, the parallelism value of the concurrent parameter in the job is also P. If the image has been extracted and preheated, and the image pulling time is q seconds, in the actual scenario of the present invention, P packet capture modules are successfully created concurrently, and the packet capture start time is q seconds for all of them. However, in the traditional underlying packet capture method, it increases as the number of pods increases, while there is no obvious change in the present invention. The default task execution completion destruction time is set to 600s, that is, the ttlSecondsAfterFinished value is configured to 600, and automated resource recovery is performed through this method.

[0070] Furthermore, to avoid scheduling multiple pods on a single node, the job resource is configured with its own label to enforce anti-affinity parameters for forced decentralized scheduling. At the same time, in addition to the known passed parameters in the env parameter of the job, a fieldRef parameter is added to obtain the status.host.ip and metadata.name parameters of each pod of the packet capture module itself.

[0071] Furthermore, the job resource of the created packet capture module includes the mounting of shared storage. The central controller module is responsible for the overall allocation and use of the storage volume. The central controller module itself mounts the primary storage path of the shared storage volume, such as nas.com: / data / . When the central controller module creates the packet capture module resource, it dynamically generates a task ID value and uses this ID value as the sub-path for mounting the shared storage volume of the packet capture module for this task. For example, if the ID value is id1. Then, the central controller module will create a folder id1 under the mount point nas.com: / data / , that is, nas.com: / data / id1. This path will communicate with the kubernetes apiserver by the central controller module to create a shared storage volume resource, and at the same time configure the mounting method as ReadWriteMany, that is, all task pods mount this data volume at the same time, and the generated data will also be stored on the shared storage volume.

[0072] Furthermore, the job resource should be configured in hostnetwork mode to obtain the privileges of the host network module. After obtaining the privileges, the packet capture of relevant application pods can be performed through the network card of the host.

[0073] Furthermore, the captured tool open source tools such as Wireshark or tcpdump are encapsulated in the running image of the job resource.

[0074] Furthermore, the job resource includes an application packet capture execution startup script. The script contains the capture tool execution command. Combined with the passed execution environment variables, the application packet capture is completed. The captured data is written to the mounted shared storage volume. To distinguish the data captured by different pods, each pod in the task will generate a folder with the unique metadata.name field on the mounted shared storage path and write the data into it. For example, if the metadata.name of one pod is net-1, then the corresponding shared storage path for storing the data is nas.com: / data / id1 / net-1.

[0075] The number of failure retries and the task timeout can be configured to ensure the robustness of the capture task. Specifically, in the job resource parameters, configure the backoffLimit value to determine the number of task failure retries, and configure the activeDeadlineSeconds parameter to determine the task timeout. If the task is not completed within the specified time, it is regarded as a failed execution.

[0076] According to the captured data packets, the content output of the data packet parsing is generated. The data packet parsing module is default-mounted to the first-level path of the shared storage volume, which is the same as that of the central controller module. It can parse the data packets in the folders named by all task IDs and generate a parsing report based on the parsing results, which is written to the shared storage for the central controller module to extract. Through this method, in the actual application scenario, the data packet parsing module obtains the source data of the data packets in real time. This real-time nature eliminates the common network bandwidth loss and latency of data upload in traditional methods.

[0077] Furthermore, as the number of pods of the data packet parsing module increases, the number of replicas of the data packet parsing module also increases. Specifically, the data packet parsing module is deployed using the deployment resource type in Kubernetes. The default number of replicas of the data packet parsing module is 1, which receives instructions sent by the central controller module in the active state. The central controller module affects the number of its replicas in the following way. The central controller module obtains the concurrent execution number of the data packet capture module and then dynamically adjusts the number of replicas of the data packet parsing module in the apiserver. Assume that the concurrent number of data packet capture tasks is T, and the number of tasks that each replica can handle is C. Then the number of replicas When the number of data packet capture tasks T increases, the number of replicas N will also increase dynamically to keep the load of each replica within an acceptable range. After the parsing task ends, the central controller module sets the number of replicas of the data packet parsing module to 1 through the apiserver to recycle resources.

[0078] The data packet parsing module encapsulates open-source parsing tools and uses open-source parsing tools (such as Wireshark or tcpdump) to parse the protocols of the data packets and extract useful information. For example: extract the HTTP request headers and response bodies; parse information such as the source IP, destination IP, and port number of the TCP / IP protocol; filter abnormal data packets, such as packet loss and retransmission. Finally, generate a detailed parsing result of the data packets (such as IP address, port number, protocol type, etc.) and output an abnormal packet detection report.

[0079] The deployment of the data packet parsing module mounts a shared storage. The mounting method is multi-point mounting, that is, all data packet parsing module pods simultaneously mount the relevant shared storage paths to achieve data volume sharing. The path for mounting the shared storage is the first-level directory of the shared storage nas.com: / data / , which can generate parsed data and access the captured data generated by all data packet capture modules. The output of the parsed content is uniformly written to the shared storage, and its folder path is nas.com: / data / task ID-out / .

[0080] After the task is completed, the central controller module can access different paths in the shared storage to extract the required content. For example, access nas.com: / data / task ID-out / to extract the parsed content, or access the nas.com: / data / task ID / path to extract the source data of the data packet capture.

[0081] In this embodiment, the interaction with the kubenetes cluster is emphasized. The central controller module successively creates resources with temporary labels, storage volumes, and job resources with the cluster apiserver, and dynamically controls the number of replicas, and does not inject additional crd resources into the cluster. At the same time, the characteristics of Kubernetes declarative configuration are also utilized to achieve complex but accurate data packet capture requirements, which greatly simplifies the management and maintenance work.

[0082] This embodiment schedules the data packet capture task through the reasonable utilization of native resources of Kubernetes, and uses a shared storage volume for data storage and access. The close cooperation of each module ensures the efficiency and accuracy of data packet capture and parsing. At the same time, through the automated scheduling resource and resource recycling mechanism, the efficient utilization and management of system resources are ensured. Regarding the resource recycling mechanism, the pod resources are automatically cleaned by configuring the ttlSecondsAfterFinished field in the job, and the active Deadline Seconds parameter is configured to forcibly abort the pods that time out.

[0083] In addition, based on the distributed architecture of the Kubernetes cluster, the system can support large-scale and high-concurrency data packet capture and parsing tasks, while ensuring flexible resource scheduling and efficient task execution, and can meet the requirements of different network environments and application scenarios.

[0084] The present invention designs an efficient and flexible Kubernetes network packet capture and parsing solution by comprehensively utilizing Kubernetes native resources (such as Jobs, Pods, shared storage volumes, etc.). This solution can solve the problems of dynamic changes in network topology, packet capture task scheduling, and real-time and efficiency of data capture and parsing in the Kubernetes environment, greatly enhancing the network diagnosis and problem location capabilities in the Kubernetes cluster.

[0085] Compared with existing Istio or Calico, although both support a certain degree of traffic monitoring and capture. For example, Istio uses Envoy as its sidecar proxy, and Envoy supports detailed logging functions. Calico's flow log provides flow logging functions that can record traffic in and out of nodes or specific Pods. However, these are not strict underlying data-level packet captures. The present invention can directly perform complete capture of underlying application network packets according to requirements. And often complex network problems require analysis of underlying packets to obtain the root cause of the problem. For example, in the case of large-scale application concurrent calls, the client calls applications in the group through load balancing, and there is an interruption once in 10,000 calls. It is necessary to determine whether a specific RST packet or other abnormal packet comes from a certain load balancing pod or application pod, which requires the support of underlying packet data analysis. At the same time, in terms of adaptability, the present invention can make up for the deficiencies of existing Istio and Calico related technologies in the ability of underlying packet capture. In addition, it is also applicable to clusters without relevant technical components, and the specific functions do not depend on existing tools. In terms of resource utilization, there is no need to add proxy plugins to each application, and tasks can be flexibly scheduled on demand, effectively utilizing computing resources and avoiding waste on cluster resources.

[0086] The advantages of the present invention are as follows: First, the present invention uses the interaction between Kubernetes native resources (such as Pods, Jobs) and the API server to obtain relevant resources and create job tasks in this way to efficiently capture packet targets.

[0087] Through interaction with the Kubernetes API server, determine the host machine, Pod, and peer communication address that require packet capture, and accurately allocate the packet capture task to the specified host machine and node. Utilize the Kubernetes Job resource to schedule the Pod to run on the specified node, and at the same time use the HostNetwork mode to ensure efficient communication between the Pod and the host machine network, ensuring the accuracy and real-time of packet capture.

[0088] Second, the combination of shared storage and real-time parsing. By communicating with the Kubernetes cluster through the central controller module, a temporary shared storage volume is created for each task Pod. This shared storage volume can be used to store the captured data packets and is available for the data packet parsing module to read and parse in real time. This design avoids the delay in data upload, ensures efficient data sharing, guarantees the rapid synchronization between data packet capture and parsing, and greatly improves the data processing efficiency.

[0089] Third, through modular design, efficient collaboration of task distribution, data packet capture, and parsing is achieved. Utilizing the scheduling and storage capabilities of Kubernetes, the accuracy, efficiency of data packet capture, and real-time performance of subsequent parsing are optimized. This method can not only meet the data packet capture requirements in a large-scale Kubernetes environment but also ensure the efficient synchronization of data packet capture and parsing. At the same time, whether it is a small-scale Kubernetes cluster or a resource-constrained environment, the performance and resource utilization can be optimized by dynamically adjusting the number of replicas and resource configuration of the analysis tasks. It provides strong technical support for cloud-native network troubleshooting.

[0090] The present invention helps developers and operators quickly capture data packet files in the Kubernetes cluster, identify potential performance bottlenecks, latency issues, or abnormal traffic patterns, and can also optimize network performance to ensure the high availability and low-latency operation of the system. At the same time, accurate data packet parsing is extremely important for problem diagnosis and location. Data packet capture and parsing play an important role in diagnosing application problems and troubleshooting. For example, by parsing the communication data packets between microservices, problems in a certain service or network path can be quickly located, helping to quickly restore the normal operation of the system.

[0091] Generally speaking, the efficient data packet capture and parsing method for the Kubernetes cluster helps to improve the efficiency of cloud application fault diagnosis, thereby effectively enhancing the overall operation and maintenance ability and system reliability of Kubernetes.

[0092] The above are only the preferred embodiments of the present invention. The protection scope of the present invention is not limited to the above embodiments. All technical solutions falling within the idea of the present invention belong to the protection scope of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements should also be regarded as within the protection scope of the present invention.

Claims

1. An efficient method for capturing and parsing Kubernetes application network data packets, characterized by: It includes the following steps: S1. Obtain the configuration parameters required for capturing data packets; S2. Through communication with the apiserver of kubernetes, obtain the resource information of the kubernetes cluster: Combined with the configuration parameters, generate temporary scheduling-task labels for the specified host, generate data packet capture tasks and distribute them; At the same time, create a shared storage volume; S3. After the data packet capture task is distributed, perform data packet capture operations on the specified host and store the captured data packets in the shared storage volume; S4. Obtain the data packets from the shared storage volume and parse them in real time, and store the parsing results in the shared storage volume; S5. Obtain the parsing results from the shared storage volume.

2. An efficient capture and parsing system for Kubernetes application network data packets, characterized by: It includes: A central controller module, which is used to obtain the configuration parameters required for capturing data packets; It is used to communicate with the apiserver of kubernetes to obtain the resource information of the kubernetes cluster; It is used to generate temporary scheduling-task labels for the specified host according to the resource information of the kubernetes cluster, combined with the configuration parameters, generate data packet capture tasks and distribute them; It is used to create a shared storage volume and obtain the parsing results from the shared storage volume; A data packet capture module, which is used to perform data packet capture operations on the specified host according to the data packet capture task and store the captured data packets in the shared storage volume; A data packet parsing module, which is used to obtain the data packets from the shared storage volume and parse them in real time, and store the parsing results in the shared storage volume.

3. The efficient capture and parsing system for Kubernetes application network data packets according to claim 1, characterized in that: The central controller module is used to listen to the execution status of the data packet capture task in the data capture module by accessing the pod status associated with the corresponding job through the apiserver of kubernetes; It is also used to issue data packet analysis tasks to the data packet parsing module while distributing the data packet capture tasks to the data packet capture module; It is used to dynamically generate temporary host labels according to the resource information of the kubernetes cluster, combined with the configuration parameters, mark the specified host with temporary scheduling-task labels, and then distribute the data packet capture tasks by scheduling the Job resources of Kubernetes.

4. The efficient capture and parsing system for Kubernetes application network data packets according to claim 1, characterized in that: The configuration parameters required for capturing data packets include: application resource type, data packet capture time, address and port of peer communication.

5. The high - efficiency capture and parsing system for Kubernetes application network data packets according to claim 1, characterized in that: The resource information of the kubernetes cluster includes: application host information and pod container information.

6. The efficient capture and parsing system for Kubernetes application network data packets according to claim 1, characterized in that: Both the central controller module and the data packet parsing module are mounted with the primary storage path of the shared storage volume, and the data of the dynamically created secondary storage path can be obtained through the primary storage path; After the data packet parsing module parses the data packet content, it stores the parsed data packet content in the secondary storage path of the shared storage volume; The central controller module is used to access the secondary storage path of the shared storage volume to obtain the parsed data packets.

7. An efficient capture and parsing system for Kubernetes application network data packets according to claim 1, characterized in that: The data packet capture module schedules task Pods on the target node through the Job resources of Kubernetes and runs in host network mode to capture data packets; A folder for storing captured data packets in a shared storage volume and assigning a unique ID to each captured pod.

8. An efficient kubernetes application network data packet capture and parsing system according to claim 1, characterized in that: The central controller module accesses the pod logs of the corresponding task through the apiserver interface of Kubernetes. The central controller module displays the task logs in real time and self-destroys the job resources after a preset time when the data packet capture task is completed.

9. An apparatus for efficiently capturing and parsing Kubernetes application network data packets, characterized in that: Comprising: A processor and a memory, the memory stores computer programs executable by the processor, and when the processor executes the computer programs, the method described in claim 1 is implemented.

Images