A data acquisition method, system, device and storage medium
By using eBPF to map, store, and update collected configuration information in cloud-native networks, the problems of high resource consumption and dynamic configuration updates in cloud-native networks are solved, enabling flexible network data collection and monitoring.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SANGFOR TECH INC
- Filing Date
- 2022-11-25
- Publication Date
- 2026-05-29
AI Technical Summary
Traditional network monitoring solutions cannot be effectively migrated to cloud-native networks. In particular, non-intrusive monitoring solutions face challenges such as increased resource consumption, diverse data collection metrics, and dynamic configuration updates in cloud-native networks.
It utilizes the extended Berkeley Packet Filter (eBPF) to map and store collection configuration information, and uses eBPF to control the Map and data Map to achieve dynamic collection configuration updates and data storage. It supports on-demand collection of metrics, adaptively selects eBPF program types and mount points, and adapts to the dynamic nature of cloud-native networks.
It enables dynamic network data collection based on configuration information without restarting the collection program, supports multi-source data collection, reduces resource overhead, and improves the flexibility and applicability of monitoring.
Smart Images

Figure CN115883407B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and in particular to a data acquisition method, system, device and storage medium. Background Technology
[0002] In cloud-native environments, with the widespread use of virtualization and container technologies, the scale of cloud-native networks continues to grow, and network boundaries become blurred. The resources in cloud-native networks are highly dynamic, and the objects of monitoring have expanded from a limited number of hosts to virtual machines and containers. The number of collected metrics in cloud-native networks has exploded, and traditional network monitoring solutions cannot be directly migrated to cloud-native networks. Summary of the Invention
[0003] To enable the collection of network data in cloud-native networks and to conduct monitoring of cloud-native networks based on the collected network data, embodiments of this application provide a data collection method, system, device, and storage medium.
[0004] On one hand, embodiments of this application provide a data acquisition method, the method comprising:
[0005] In response to a data acquisition command for the network to be monitored, the acquisition configuration information stored in the control type mapping of the extended Berkeley Packet Filter (eBPF) is read based on the identification information of the network to be monitored; wherein, the acquisition configuration information includes the acquisition indicator information of the network to be monitored;
[0006] Obtain the first target data in the network to be monitored that corresponds to the collected indicator information.
[0007] On the other hand, embodiments of this application provide a data acquisition system, the data acquisition system comprising:
[0008] The reading module is used to respond to a data acquisition command for the network to be monitored by reading the acquisition configuration information stored in the control type mapping of the extended Berkeley Packet Filter (eBPF) based on the identification information of the network to be monitored; wherein the acquisition configuration information includes the acquisition indicator information of the network to be monitored.
[0009] The acquisition module is used to acquire the first target data in the network to be monitored that corresponds to the collected indicator information.
[0010] On the other hand, this application provides a data acquisition device, which includes a memory and a processor. The memory stores computer-executable instructions, and the processor can implement the data acquisition method described in the first aspect embodiment when it executes the computer-executable instructions in the memory.
[0011] On the other hand, embodiments of this application provide a computer storage medium storing executable instructions, which, when executed by a processor, implement the data acquisition method described in the first aspect of the embodiment.
[0012] The technical solution of this application embodiment, in response to a data acquisition command for the network to be monitored, reads acquisition configuration information stored in the control type mapping of the extended Berkeley Packet Filter (eBPF) based on the identification information of the network to be monitored; wherein, the acquisition configuration information includes acquisition indicator information of the network to be monitored; and obtains first target data in the network to be monitored corresponding to the acquisition indicator information. This application embodiment utilizes the method of storing acquisition configuration information in the control type mapping of eBPF, enabling dynamic acquisition of network data in the network to be monitored based on the acquisition indicator information in the acquisition configuration information without restarting the acquisition program. Attached Figure Description
[0013] Figure 1 Flowchart of the data acquisition method provided in the embodiments of this application Figure 1 ;
[0014] Figure 2 Flowchart of the data acquisition method provided in the embodiments of this application Figure 2 ;
[0015] Figure 3 Flowchart of the data acquisition method provided in the embodiments of this application Figure 3 ;
[0016] Figure 4 A schematic diagram illustrating the storage and output process of collected data provided in an embodiment of this application;
[0017] Figure 5 A schematic diagram of a data acquisition framework provided in an embodiment of this application;
[0018] Figure 6 The workflow of each module in the data acquisition framework provided in the embodiments of this application;
[0019] Figure 7 A schematic diagram of the workflow of the management module provided in the embodiments of this application;
[0020] Figure 8 A schematic diagram of the workflow of the collection module provided in the embodiments of this application. Figure 1 ;
[0021] Figure 9 A schematic diagram of the workflow of the collection module provided in the embodiments of this application. Figure 2 ;
[0022] Figure 10 A schematic diagram of the workflow of the collection module provided in the embodiments of this application. Figure 3 ;
[0023] Figure 11 A schematic diagram of the workflow of the collection module provided in the embodiments of this application. Figure 4 ;
[0024] Figure 12 A schematic diagram illustrating the workflow of the data acquisition module provided in this application embodiment;
[0025] Figure 13 A schematic diagram of the workflow of the output module provided in the embodiments of this application;
[0026] Figure 14 This is a schematic diagram of the structural composition of the data acquisition system provided in the embodiments of this application;
[0027] Figure 15 This is a schematic diagram of the structural composition of the data acquisition device provided in the embodiments of this application. Detailed Implementation
[0028] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.
[0029] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be defined and explained in subsequent figures.
[0030] In this document, the term "at least one" means any combination of at least two of any one or more of a plurality of elements, such as including at least one of A, B, and C, and may mean including any one or more elements selected from the set consisting of A, B, and C.
[0031] Currently, network monitoring solutions for cloud-native environments can be broadly categorized into two types: intrusive monitoring solutions and non-intrusive monitoring solutions.
[0032] Intrusive monitoring solutions require modifying the source code or binaries of user programs. For example, Pythia (a modular deep learning framework for vision and language multimodal research) focuses on automatically selecting when and where to inject code in a distributed application. Seer is an online cloud performance debugging system that includes a lightweight distributed Remote Procedure Call (RPC) level tracing system based on the timing interface of Apache Thrift (a development framework specifically for cross-platform service development) to collect end-to-end traces of request execution and track incomplete requests for each microservice. Seer uses these traces to train deep neural networks to identify impending Quality of Service (QoS) violations and identify microservices causing performance degradation. MicroRank (a spectral analysis-based latency fault location system for microservice environments) introduces a spectral analysis-based latency fault location system for microservice environments. It inserts tracing interfaces into the code of each microservice and matches service pattern information by tracing the process from the Application Program Interface (API) to the microservice application to obtain request information. Intrusive monitoring solutions are closely related to the type of application, the programming language used, and the protocols used, resulting in poor universality, high difficulty, and significant overhead.
[0033] Non-intrusive solutions include MicroScope (a performance diagnostic tool for microservice environments), which monitors changes in service dependency graphs by capturing network connectivity information of the microservice system; and MicroRCA (an application-agnostic root cause diagnostic system for performance problems in microservice environments), which continuously collects application and system-level metrics through a service mesh and detects anomalies in SLO (Service Level Objective) metrics. Once an anomaly is detected, it constructs a property graph containing services and hosts to model the propagation of anomalies between services. MicroRCA correlates the symptoms of communication service anomalies with relevant resource utilization to infer potential anomalous services and rank potential root causes. By correlating service anomalies with resource utilization, MicroRCA can identify anomalous, non-computationally intensive services with subtle service anomaly symptoms and mitigate the impact of false positives on root cause localization. MicroDiag (a fine-grained automatic root cause localization system) primarily infers root causes by constructing causal relationships between metrics based on dependency graphs between components and different anomaly propagation patterns. MicroDiag monitors operating system-level information on server nodes, monitors container resource information through Cadvisor (a tool for analyzing the resource consumption and performance characteristics of running Docker containers), and monitors inter-microservice communication (including response latency information) through a service mesh. Datadog (a website for monitoring website operation) and Sysdig (a system monitoring attack) collect monitoring data based on eBPF, but do not support on-demand metric collection, and require restarting the monitoring program when the monitoring configuration changes. ViperProbe (a microservice data collection architecture based on extended Berkeley Packet Filter (eBPF)) identifies key metrics to be collected through offline analysis and supports real-time dynamic updates, but it only collects some numerical metrics, ignoring other types of information. Kmon (a microservice monitoring system based on eBPF) collects metrics using eBPF technology and can dynamically detect changes in configuration and microservice information. From the above analysis, it can be seen that most non-intrusive monitoring solutions either rely on service meshes or eBPF to collect metrics. However, due to the performance issues brought about by service mesh, this approach is not very practical. Furthermore, for eBPF-based collection solutions, there are still issues to be addressed, such as how to solve the problems of increased resource overhead caused by the increase in the number of collection metrics, the diversity of collection metrics, how to more "elegantly" perceive microservices, and the dynamic updates of configurations.
[0034] As can be seen, for monitoring cloud-native networks, non-intrusive monitoring solutions are more applicable and less complex than intrusive ones, making them more suitable. However, for non-intrusive monitoring solutions, metric collection based on service mesh is not very practical due to the performance limitations inherent in service meshes themselves.
[0035] To enable monitoring of cloud-native networks, this application provides a data acquisition method. This data acquisition method can be applied to data acquisition in various types of networks, including but not limited to cloud-native networks.
[0036] Below, in order to better understand the technical solutions of the embodiments of this application, the data acquisition method provided in the embodiments of this application is described as follows.
[0037] Figure 1 Flowchart of the data acquisition method provided in the embodiments of this application Figure 1 ,like Figure 1 As shown, the data acquisition methods include:
[0038] S101: In response to a data acquisition command for the network to be monitored, read the acquisition configuration information stored in the control type mapping of the extended Berkeley Packet Filter (eBPF) based on the identification information of the network to be monitored.
[0039] In this embodiment of the application, the data collection configuration information includes the data collection indicator information of the network to be monitored.
[0040] In this embodiment of the application, the network to be monitored includes various network data that need to be monitored, such as various data of hosts (e.g., memory usage, system load, disk space, etc.), virtual machines (e.g., memory usage, disk input / output data, etc.), and containers (e.g., memory usage, network input / output metrics) in the network to be monitored; and transmission metric data (e.g., latency, packet loss rate, etc.) of data transmission between devices in the network to be monitored, and operation-related data (e.g., access count) of devices in the network to be monitored.
[0041] The network to be monitored in the embodiments of this application includes, but is not limited to, cloud-native networks.
[0042] In one implementation, the data acquisition command for the network to be monitored can be generated periodically; for example, the data acquisition command can be generated once per hour, and the data acquisition device reads the acquisition configuration information in the eBPF control Map once per hour based on the data acquisition command to collect network data of the network to be monitored. This application does not specifically limit the triggering method of the data acquisition command, which can be set by the user according to their needs.
[0043] eBPF can run sandboxed programs within the kernel of an operating system (such as Linux). It is used to efficiently extend the functionality of the kernel without modifying the operating system kernel source code or loading kernel modules.
[0044] eBPF maps, or eBPF Maps, are a collective term for all different types of data storage areas available in eBPF. They are key-value pairs stored in the kernel and can be accessed by any eBPF program for communication between kernel-mode and user-mode programs. eBPF maps are a general-purpose data structure that allows data to be transferred between the kernel and user space.
[0045] In this embodiment of the application, the eBPF control type mapping, referred to as eBPF control Map, is used to store the acquisition configuration information.
[0046] eBPF mappings include various types, such as hash table mapping (BPF_MAP_TYPE_HASH) and data mapping (BPF_MAP_TYPE_ARRAY). In one embodiment, hash table mapping (BPF_MAP_TYPE_HASH) is used as the mapping for the control type of eBPF in this application embodiment, for storing the acquisition configuration information of the network to be monitored.
[0047] In some implementations, the eBPF control map may also store other information related to the network to be monitored, besides the acquisition configuration information of the network to be monitored, and may also store acquisition configuration information of other networks besides the network to be monitored. In order to accurately obtain the acquisition configuration information of the network to be monitored from the eBPF control map, the acquisition configuration information can be identified with the identification information of the network to be monitored when the acquisition configuration information is stored in the eBPF control map, so that the acquisition program can read the identification information of the network to be monitored in the eBPF control map based on the identification information of the network to be monitored.
[0048] In some implementations, each network to be monitored can correspond to an eBPF control map, which corresponds to the identification information of the network to be monitored. The eBPF control map is used to store the acquisition configuration information of the network to be monitored.
[0049] S102: Obtain the first target data in the network to be monitored that corresponds to the collected indicator information.
[0050] In this embodiment of the application, in order to monitor various network data in the network to be monitored, it is necessary to collect various network data in the network to be monitored, and then perform analysis based on the collected network data. In order to collect various network data in the network to be monitored, the collection configuration information of the network to be monitored can be pre-configured, and the network data in the network to be monitored can be collected to obtain the first target data in the network to be monitored corresponding to the collected indicator information.
[0051] In this embodiment of the application, network data in the network to be monitored is collected by a data acquisition device, specifically by a collection program in the data acquisition device; wherein, the data acquisition device may be one of the devices included in the network to be monitored.
[0052] In this embodiment, a data acquisition program reads the data acquisition configuration information stored in the eBPF control Map. Then, based on the data acquisition indicator information contained in the configuration information, the program acquires network data from the network to be monitored, obtaining the first target data. In one implementation, the data acquisition program is an eBPF data acquisition program.
[0053] In one embodiment, step S102 includes: collecting first target data in the network to be monitored that corresponds to the collected indicator information; or, collecting network data in the network to be monitored and filtering the network data based on the collected indicator information to obtain the first target data corresponding to the collected indicator information.
[0054] After reading the acquisition configuration information from the eBPF control Map, the acquisition program directly acquires network data corresponding to the acquisition indicators in the network to be monitored, based on the acquisition indicator information contained in the acquisition configuration information, to obtain the first target data. Alternatively, the acquisition program can directly acquire network data corresponding to the default acquisition indicators in the network to be monitored according to pre-set default acquisition indicators. Then, it filters the acquired network data corresponding to the default acquisition indicators based on the acquisition indicator information contained in the acquisition configuration information read from the eBPF control Map to obtain the first target data.
[0055] The technical solution of this application embodiment utilizes an eBPF control map to store collection configuration information. When it is necessary to collect network data from the network to be monitored, the collection program reads the collection configuration information stored in the eBPF control map, and then collects network data from the network to be monitored based on the collection indicator information in the read collection configuration information. This method of using an eBPF control map to store collection configuration information enables dynamic collection of network data from the network to be monitored based on the collection configuration information without restarting the collection program.
[0056] Figure 2 Flowchart of the data acquisition method provided in the embodiments of this application Figure 2 ,like Figure 2 As shown, the data acquisition methods include:
[0057] S201: Obtain user configuration information, parse the user configuration information to obtain the collection index information of the network to be monitored and the index information of the target platform.
[0058] In this embodiment, the user sets user configuration information according to their needs. This configuration information includes the metrics that need to be monitored in the network to be monitored, i.e., the collected metric information. Additionally, in some cases, the user also needs to obtain metric information related to certain network platforms, such as Kubernetes (k8s) related metrics. Kubernetes is a lightweight and scalable open-source platform used to manage containerized applications and services.
[0059] In one implementation, the collected indicator information includes at least one of the following: newly added collected indicators, updated collected parameters, and deleted collected indicators; the collected parameters include at least one of the following: collected frequency, collected period, collected times, data collected length, and collected time.
[0060] Because the states of various aspects of the network under monitoring may change—for example, the number of hosts or clients connected to the network may change—the collected metrics information in the network will also change accordingly. For instance, if a new host is added to the network, the collected metrics related to the newly added host need to be increased accordingly. Similarly, if the data collection frequency for a particular device in the network may need to be adjusted, the collection period parameters corresponding to the device's metrics will also be changed accordingly. Furthermore, if a device leaves the designated network area or malfunctions, data collection for that device is no longer necessary, and the collected metrics associated with that device need to be deleted.
[0061] In one implementation, the collected configuration information also includes platform data of the target platform. This platform data, corresponding to the target platform's metrics, can be obtained from the target component using an interface provided by the target platform's client library. After obtaining the target platform data, it is also stored in the eBPF's control type mapping.
[0062] For example, if the information obtained by parsing the user configuration information also includes target platform metrics, such as Kubernetes (k8s) related metrics, such as Pod (the smallest resource management component in Kubernetes) information and node information in the k8s platform, then the API server (one of the key components of Kubernetes) is pulled through the interface provided by client-go (the client that calls the Kubernetes cluster resource object application interface), and the information associated with the relevant k8s metrics is stored in the eBPF control map.
[0063] For example, when the network data to be monitored consists of traffic information of containers, Pods, etc., it is necessary to associate the collected traffic information of containers, Pods, etc. with the Kubernetes platform data.
[0064] S202: Update the mapping of the control type of the eBPF based on the collected index information of the network to be monitored.
[0065] In this embodiment of the application, after parsing the user configuration information, if the parsed collection indicator information includes the collection indicator information of the network to be monitored, the historical collection indicator information in the control Map of eBPF can be directly updated based on the parsed collection indicator information.
[0066] In one embodiment, step S202 includes:
[0067] If the newly added data collection indicator is included in the data collection indicator information, the newly added data collection indicator is stored in the control type mapping of the eBPF;
[0068] If the updated acquisition parameters are included in the acquisition indicator information, the acquisition parameters of the acquisition indicator corresponding to the updated acquisition parameters are updated in the mapping of the control type of the eBPF.
[0069] If the data collection information includes the data collection indicator to be deleted, then the data collection indicator to be deleted is removed from the mapping of the control type in the eBPF.
[0070] When the collected metrics change, whether it is necessary to add, update, or delete the collected metrics, the corresponding operations such as adding, updating, or deleting the collected metrics can be performed in the eBPF control map.
[0071] When the collection parameters of a data collection indicator need to be updated, the collection parameters of the data collection indicator can be changed directly in the eBPF control map; when a data collection indicator needs to be deleted, the data collection indicator to be deleted can also be deleted directly in the eBPF control map.
[0072] S203: Based on the identification information of the network to be monitored, read the collected index information stored in the control type mapping of the eBPF.
[0073] S204: Obtain the first target data in the network to be monitored that corresponds to the collected indicator information.
[0074] In this embodiment, after updating the collection metrics in the eBPF control map, the collection program can collect data from the network based on the updated metrics. For example, if the collection period of a certain collection metric in the eBPF control map changes, the collection program will collect the data corresponding to the new collection metric according to the new collection period. If a certain collection metric is deleted from the eBPF control map, the collection program will no longer collect network data corresponding to the deleted collection metric in the network to be monitored.
[0075] In the technical solution of this application embodiment, the user sets the collection indicator information in the network to be monitored according to their needs, and uses the eBPF control map to store the collection indicator information in the network to be monitored. If the user's needs include updated collection indicator information, the updated collection indicator information is updated in the eBPF control map. Then, the collection program reads the updated collection indicator information stored in the eBPF control map to collect network data in the network to be monitored corresponding to the updated collection indicator information. The technical solution of this application embodiment can dynamically collect network data related to the updated collection indicator information in the network to be monitored according to the user's needs without restarting the collection program.
[0076] Figure 3 Flowchart of the data acquisition method provided in the embodiments of this application Figure 3 ,like Figure 3 As shown, the data acquisition methods include:
[0077] S301: In response to a data acquisition command for the network to be monitored, read the acquisition configuration information stored in the control type mapping of the extended Berkeley Packet Filter (eBPF) based on the identification information of the network to be monitored; the acquisition configuration information includes newly added acquisition metrics.
[0078] S302: Determine the mount point of the eBPF acquisition program of the target type and the target eBPF acquisition program corresponding to the newly added acquisition index.
[0079] In this embodiment of the application, a data acquisition program is used to collect network data in the network to be monitored; the data acquisition program can be an eBPF data acquisition program.
[0080] eBPF procedures include various types, such as network packet filters (BPF_PROG_TYPE_SOCKET_FILTER), network flow control operations (BPF_PROG_TYPE_SCHED_ACT), and procedures for determining whether device operations should be allowed (BPF_PROG_CGROUP_DEVICE). The type of eBPF procedure used generally differs depending on the data collection metrics.
[0081] When the data collection metrics are updated, and the data collection metrics are newly added, it is necessary to determine the mount point of the data collection program corresponding to the newly added data collection metrics and the eBPF data collection program corresponding to the newly added data collection metrics.
[0082] In one implementation, the eBPF acquisition program mount point corresponding to the newly added acquisition index can be determined as follows:
[0083] Determine whether the collectable indicators in the tracking point list can cover the newly added collectable indicators;
[0084] If the collectable metrics in the tracking point list can cover the newly added collectable metrics, determine the eBPF acquisition program mount point of the tracking point type corresponding to the newly added collectable metrics.
[0085] If the collectable metrics in the tracking point list cannot cover the newly added collectable metric, determine the eBPF acquisition program mount point of the kernel probe type corresponding to the newly added collectable metric.
[0086] When adding a new data collection metric, it is necessary to check whether the collectable metrics in the Tracepoint list can cover the new data collection metric. If so, select the corresponding Tracepoint type eBPF program mount point in the Tracepoint list, write the new data collection metric to the eBPF control Map, and then load the specified eBPF data collection program to the selected Tracepoint type eBPF program mount point. If the collectable metrics in the Tracepoint list cannot cover the new data collection metric, select a kprobe type eBPF data collection program mount point, write the new data collection metric to the eBPF control Map, and then load the specified eBPF data collection program to the selected kprobe type eBPF data collection program mount point.
[0087] Kprobe, a kernel probe, allows inserting eBPF programs before executing any instructions. Tracepoint, a static marker in kernel code, can be used to attach code to the running kernel. Tracepoint-type eBPF program mount points are more stable than kprobes, remaining unchanged regardless of operating system kernel version changes; therefore, tracepoint-type eBPF program mount points are preferred, although their number is relatively limited. For kprobe-type mount points, the operating system itself provides a very rich set of mount points. Therefore, kprobe-type mount points can be used when tracepoint-type eBPF program mount points cannot collect newly added metrics.
[0088] S303: Load the target eBPF acquisition program at the mounting point of the target type eBPF acquisition program, and use the target eBPF acquisition program to obtain the first target data in the network to be monitored that corresponds to the newly added acquisition indicator.
[0089] The technical solution of this application embodiment can, based on the characteristics of the newly added data collection indicators, independently select the type of the mounting point of the data collection program and the type of the data collection program, and then use the selected data collection program type at the selected mounting point of the data collection program to collect relevant data of the newly added data collection indicators.
[0090] In this embodiment of the application, after obtaining the first target data corresponding to the collection indicator information contained in the collection configuration information, the first target data or the data after processing the first target data can be further stored and output, so that the user can monitor and analyze the operation status of the network to be monitored based on the output data.
[0091] Figure 4 This is a schematic diagram illustrating the process of storing and outputting collected data, as provided in an embodiment of this application. Figure 4 As shown, the methods for storing and outputting the collected data are as follows:
[0092] S401: Store the first target data in the eBPF data type mapping.
[0093] In this embodiment of the application, the eBPF data type mapping, referred to as eBPF data Map, is used to store the collected data.
[0094] In one implementation, a hash table mapping BPF_MAP_TYPE_HASH is used as the data type mapping of the eBPF in this application embodiment to store the collected data.
[0095] In one implementation, after obtaining the collection data corresponding to the collection indicator information in the collection configuration information in the network using the eBPF collection program, i.e., the first target data, the collection data corresponding to the collection indicator information can be directly stored in the eBPF data map.
[0096] In one implementation, persistence instructions can be generated periodically; for example, for data stored in the eBPF data map, a persistence operation is performed on the collected data in the eBPF data map every hour, with a 1-hour cycle, and the collected data in the eBPF data map is output to the database. This application embodiment does not specifically limit the triggering method for generating persistence instructions, which can be set by the user according to their needs.
[0097] S402: In response to the persistence instruction, store the first target data stored in the mapping of the data type of the eBPF into the database.
[0098] In one embodiment, step S402 includes:
[0099] If the collected configuration information includes platform data, the first target data stored in the eBPF data type mapping is associated with the platform data to obtain the second target data;
[0100] The second target data is stored in the database.
[0101] For example, if the collection configuration information includes Kubernetes platform data, it indicates that the collection data obtained by the eBPF collection program needs to be further associated with the Kubernetes platform data. By matching the platform data in the eBPF control map with the Internet Protocol address (IP, Internet Protocol) information in the collection data, the collection data and Kubernetes platform data are associated, and the associated data is stored in the database.
[0102] In this embodiment of the application, after associating the collected data with the Kubernetes platform data, the associated data can be first stored in the eBPF data map, and then the associated data stored in the eBPF data map can be output to the database; alternatively, after associating the collected data with the Kubernetes platform data, the associated data can be directly stored in the database.
[0103] In this embodiment of the application, after storing the collected data corresponding to the collected indicator information in the eBPF data map, or storing the information after associating the collected data with the k8s platform data in the eBPF data map, the data in the eBPF data map is further output to the database so that users can analyze the network operation status based on the output data.
[0104] In one implementation, before outputting the data in the eBPF data map to the database, the data in the eBPF data map can be processed, such as format conversion, converting the collected hexadecimal ports to decimal, converting IP addresses to a specific format, etc., and then the processed data is output to the database.
[0105] In this embodiment of the application, storing the data stored in the eBPF data map in the database can achieve the purpose of persistently saving the data in the eBPF data map.
[0106] The technical solution of this application embodiment utilizes eBPF data map to store the data corresponding to the collected indicator information in the network to be monitored, which enables on-demand collection and storage of network data in the network to be monitored. Ultimately, the collected and saved data is the data that the user wants to obtain, which facilitates the user's subsequent monitoring and analysis of the data collected in the network to be monitored.
[0107] This application provides a data acquisition framework applicable to cloud-native networks, addressing the challenges of highly dynamic cloud-native network resources, numerous data acquisition metrics, and high acquisition overhead. When collecting data from the network under monitoring, this application supports on-demand metric collection, adaptive selection of eBPF program types, autonomous mount point selection, dynamic awareness of configuration changes, and multi-source (virtual machine, container, physical machine) data acquisition, exhibiting flexibility, scalability, and low overhead.
[0108] Figure 5 This is a schematic diagram of a data acquisition framework provided in an embodiment of this application. Figures 1 to 4 The method steps of each embodiment can be based on Figure 5 The data acquisition framework described above is implemented. Below, we will discuss... Figure 5 The data acquisition framework is described below.
[0109] Figure 5 The data acquisition framework shown is a flexible and scalable data acquisition framework implemented using eBPF, such as... Figure 5 As shown, the data acquisition framework consists of the following modules: collection module 501, acquisition module 502, management module 503, and output module 504.
[0110] Collection module 501: Responsible for receiving user configuration information from management module 503, obtaining collection configuration information based on user configuration information; automatically selecting eBPF collection program type based on collection configuration information; responsible for interacting with collection module 502, loading the eBPF collection program in collection module 502 into the kernel, communicating with collection module 502 through eBPF Map (including eBPF control Map and eBPF data Map); aggregating and transforming the data corresponding to the collection indicators obtained by the eBPF collection program of collection module 502, and integrating it with the corresponding stream information.
[0111] Among them, the data corresponding to the acquisition indicators obtained by the eBPF acquisition program of the aggregation conversion acquisition module 502 refers to the processing of the data corresponding to the acquisition indicators, such as format conversion, conversion of the acquired hexadecimal port to decimal, and conversion of IP address to a specific format; integration with the corresponding flow information refers to associating the acquisition data corresponding to the acquisition indicators with the platform data of the target platform, such as the platform data of k8s.
[0112] Data Acquisition Module 502: Responsible for collecting data from the network to be monitored.
[0113] Management module 503: responsible for collecting user configuration information, such as collection metrics and collection cycle; responsible for obtaining Kubernetes (k8s) related metric information, such as Pod information and node information; responsible for communicating with collection module 501, sending user configuration information and k8s information to collection module 501 for updating monitoring policies.
[0114] Output module 504: Responsible for collecting data from collection module 501 or receiving data sent by collection module 501, storing the data in the database for subsequent display.
[0115] In this embodiment, the eBPF Map is divided into an eBPF control Map and an eBPF data Map. The eBPF control Map is used to transmit the acquisition configuration information from the collection module 501 to the acquisition module 502; the eBPF data Map is used to store the data collected by the acquisition module 502 that corresponds to the acquisition configuration information.
[0116] pass Figure 5The data acquisition framework shown utilizes an eBPF control map to store acquisition configuration information. The eBPF acquisition program then reads this configuration information from the eBPF control map to collect relevant data on the indicators in the network under monitoring. This allows for dynamic data collection based on the acquisition configuration information without restarting the acquisition program. When the acquisition configuration information is updated, the eBPF program reads the updated configuration information from the eBPF control map and collects relevant data on the indicators in the network under monitoring based on this updated information. This dynamic detection of monitoring configuration updates is possible without restarting the acquisition program.
[0117] Figure 6 The workflow of each module in the data acquisition framework provided in the embodiments of this application includes:
[0118] S601: User configuration information is sent to the management module;
[0119] S602: The management module parses the user configuration information and sends it to the collection module;
[0120] S603: The collection module can independently select the eBPF collection program type according to the collection requirements;
[0121] S604: The collection module writes the collection configuration information to the eBPF control Map and loads the specified eBPF collection program to the corresponding hook point;
[0122] S605: The acquisition module reads the information from the eBPF control map, filters the acquired data, and reports the data to the eBPF data map;
[0123] S606: The collection module reads data from the eBPF data map, processes it, and then reports it to the output module;
[0124] S607: The output module writes data to the database.
[0125] In this embodiment, the user issues monitoring configurations according to their needs; the management module parses the user configurations and sends the configuration information to the collection module; the collection module selects the eBPF program type according to the collection requirements, writes the collection configuration information into the eBPF control Map of the collection program, and loads the specified eBPF program into the corresponding hook point; the collection program reads the information in the control Map, filters the collected data, and reports the filtered data to the data Map; the collection module reads data from the eBPF data Map, processes it, and reports it to the output module; the output module finally writes the data into the database for subsequent analysis and display.
[0126] Below, in conjunction with Figures 7 to 12 right Figure 5 The execution steps of each module in the data acquisition framework are described below:
[0127] Figure 7 A schematic diagram of the workflow of the management module provided in the embodiments of this application is shown below. Figure 7 As shown, the workflow of the management module is as follows:
[0128] S701: Obtain user configuration information sent by the user;
[0129] S702: Parse user configuration information;
[0130] S703: Determine whether it is necessary to pull Kubernetes information; if so, proceed to step S704; if not, proceed to step S705.
[0131] S704: Obtain Kubernetes information via the client-go interface;
[0132] S705: Sends the parsed user configuration information to the collection module.
[0133] In this embodiment, the management module is responsible for obtaining and parsing the user configuration information sent by the user. This user configuration information includes: adding a collection metric (or adding a host), updating the collection frequency (period), deleting a collection metric, etc. After parsing the user configuration information, if the parsed user configuration information requires Kubernetes platform data, the management module will pull the Kubernetes platform data from the API server through the interface provided by client-go. Finally, the management module will send the parsed configuration information to the collection module.
[0134] Figures 8 to 11 This is a flowchart illustrating the workflow of the collection module. The collection module is responsible for receiving the parsed collection configuration information issued by the management module and adopting different processing procedures according to the different types of collection configuration information.
[0135] Figure 8 A schematic diagram of the workflow of the collection module provided in the embodiments of this application. Figure 1 , Figure 8 This describes the workflow of the data collection module when new configuration information is included in the collected configuration information. For example... Figure 8 As shown, the workflow of the collection module is as follows:
[0136] S801: Receives data collection configuration information sent by the management module;
[0137] S802: Determine if there is newly added configuration information in the collected configuration information;
[0138] S803: Determine whether a tracepoint type mount point can be used; if yes, proceed to step 804; otherwise, proceed to step 805.
[0139] S804: Select the corresponding tracepoint type eBPF program clipping point;
[0140] S805: Select the corresponding kprobe type eBPF program clipping point;
[0141] S806: Writes the acquisition configuration information to the eBPF control Map and loads the specified eBPF program to the corresponding hook point.
[0142] In this embodiment, when the collection module determines that the collection configuration information includes a newly added collection indicator, the collection module checks whether the collectable indicators in the tracepoint list can cover the newly added collection indicator. If so, it selects the corresponding tracepoint type eBPF program mount point, writes the collection configuration information to the eBPF control Map, and loads the specified eBPF collection program to the selected tracepoint type eBPF program mount point. If the collectable indicators in the tracepoint list cannot cover the newly added collection indicator, it selects the corresponding kprobe type program mount point, writes the collection configuration information to the eBPF control Map, and loads the specified eBPF collection program to the selected kprobe type program mount point.
[0143] Figure 9 A schematic diagram of the workflow of the collection module provided in the embodiments of this application. Figure 2 , Figure 9 This describes the workflow of the data collection module when collecting configuration information including updated parameters. For example... Figure 9 As shown, the workflow of the collection module is as follows:
[0144] S901: Receives data collection configuration information from the management module;
[0145] S902: Determine if there is configuration information in the collected configuration information where parameters have been updated;
[0146] S903: Write the updated acquisition configuration information to the eBPF control Map.
[0147] In this embodiment of the application, when the collection module determines that the collection configuration information includes parameters for updating a certain collection indicator, such as updating the collection frequency or collection cycle of a certain collection indicator, the collection module can directly update the parameters of the collection indicator in the corresponding collection configuration information of the eBPF control Map.
[0148] Figure 10 A schematic diagram of the workflow of the collection module provided in the embodiments of this application. Figure 3 , Figure 10 This describes the workflow of the data collection module when collecting configuration information includes deleted configuration information. For example... Figure 10 As shown, the workflow of the collection module is as follows:
[0149] S1001: Receive the data collection configuration information sent by the management module;
[0150] S1002: Determine if there is deleted configuration information in the collected configuration information;
[0151] S1003: Write the configuration information that needs to be deleted to the eBPF control Map.
[0152] In this embodiment of the application, when the collection module determines that the collection configuration information includes the deletion of a certain collection indicator, the collection module can directly delete the corresponding collection indicator in the collection configuration information in the eBPF control Map.
[0153] Figure 11 A schematic diagram of the workflow of the collection module provided in the embodiments of this application. Figure 4 ,like Figure 11 As shown, the collection module, in addition to performing... Figures 8 to 10 In addition to managing the collection configuration information, it is also responsible for processing the data collected by the eBPF collection module. The specific processing flow includes:
[0154] S1101: Read data from the eBPF data map;
[0155] S1102: Determine whether the data in the eBPF data map needs to be associated with Kubernetes information; if it needs to be associated, execute S1103; otherwise, execute S1104.
[0156] S1103: Associate Kubernetes information with IP information;
[0157] S1104: Report data to the output module.
[0158] In this embodiment of the application, if the collection configuration information received by the collection module from the management module contains Kubernetes platform data, it indicates that the data collected by the collection program needs to be further associated. By matching the collection configuration information and the IP information in the collection data, the collection data and Kubernetes information are associated, the associated information is stored in the eBPF data map, and the subsequently associated data is reported to the output module.
[0159] Figure 12 This is a schematic diagram illustrating the workflow of the data acquisition module provided in an embodiment of this application. Figure 12 As shown, the workflow of the data acquisition module is as follows:
[0160] S1201: Read the acquisition configuration information in the eBPF control Map;
[0161] S1202: Collect data according to the collection configuration information; or, filter the collected data according to the collection configuration information in the eBPF control Map;
[0162] S1203: Report the collected data to the eBPF data map.
[0163] In this embodiment, the acquisition module uses the acquisition program to read the acquisition configuration information in the control Map, filters out the traffic data that does not need to be reported from the acquired data, and reports the filtered data to the eBPF data Map for the collection module to process later.
[0164] Figure 13 This is a schematic diagram illustrating the workflow of the output module provided in an embodiment of this application. Figure 13 As shown, the workflow of the output module is as follows:
[0165] S1301: Read data from the eBPF data map;
[0166] S1302: Store the data in the eBPF Data Map into the database.
[0167] In this embodiment, the output module is responsible for reading data from the eBPF data map of the collection module and writing the data into the database; or, it is responsible for writing the data reported by the collection module into the database for subsequent analysis and display.
[0168] This application embodiment proposes... Figure 5The data acquisition framework shown is specifically an eBPF-based framework. By leveraging the collaborative work of its modules, it enables on-demand collection of metrics from the network under monitoring. During data acquisition, it can autonomously select the mount point of the acquisition program and utilize eBPF to control the Map, achieving dynamic awareness of changes in acquisition configuration information. It collects data on various metrics of virtual machines, containers, and physical machines in the network under monitoring. The data acquisition method implemented in this embodiment using the data acquisition framework is flexible, scalable, and low-overhead, effectively addressing the challenges of highly dynamic resources, numerous metrics, and high data acquisition overhead in the network under monitoring.
[0169] Figure 14 This is a schematic diagram of the structural composition of a data acquisition system provided in an embodiment of this application, as shown below. Figure 14 As shown, the data acquisition system 1400 includes:
[0170] The reading module 1401 is used to read the acquisition configuration information stored in the control type mapping of the extended Berkeley Packet Filter (eBPF) based on the identification information of the network to be monitored in response to the data acquisition command of the network to be monitored; wherein the acquisition configuration information includes the acquisition indicator information of the network to be monitored.
[0171] The acquisition module 1402 is used to acquire the first target data in the network to be monitored that corresponds to the collected indicator information.
[0172] In some embodiments, when the collected indicator information includes newly added collected indicators, the data acquisition system 1400 further includes:
[0173] The determination module is used to determine the mount point of the target type of eBPF acquisition program and the target eBPF acquisition program corresponding to the newly added acquisition index.
[0174] The acquisition module 1402 is specifically used to: load the target eBPF acquisition program at the mounting point of the target type eBPF acquisition program, and use the target eBPF acquisition program to acquire the first target data in the network to be monitored that corresponds to the newly added acquisition indicator.
[0175] In some implementations, the determining module is specifically used to: determine whether the collectable indicators in the trace point list can cover the newly added collectable indicator; if the collectable indicators in the trace point list can cover the newly added collectable indicator, determine the eBPF acquisition program mount point of the trace point type corresponding to the newly added collectable indicator; if the collectable indicators in the trace point list cannot cover the newly added collectable indicator, determine the eBPF acquisition program mount point of the kernel probe type corresponding to the newly added collectable indicator.
[0176] In some embodiments, the data acquisition system 1400 further includes:
[0177] The parsing module is used to obtain user configuration information, parse the user configuration information, and obtain the collection indicator information of the network to be monitored and the indicator information of the target platform.
[0178] The update module is used to update the mapping of the control type of the eBPF based on the collected index information of the network to be monitored.
[0179] In some embodiments, the acquisition configuration information also includes platform data of the target platform, and the data acquisition system 1400 further includes:
[0180] The first storage module is used to obtain platform data corresponding to the indicator information of the target platform from the target component using the interface provided by the client library of the target platform, and store the platform data in the control type mapping of the eBPF.
[0181] In some implementations, the collected indicator information includes at least one of the following: newly added collected indicators, updated collected parameters, and deleted collected indicators; the collected parameters include at least one of the following: collected frequency, collected period, collected times, data collected length, and collected time;
[0182] The update module is specifically configured to: when the newly added acquisition indicator is included in the acquisition indicator information, store the newly added acquisition indicator in the mapping of the control type of the eBPF; when the updated acquisition parameter is included in the acquisition indicator information, update the acquisition parameter of the acquisition indicator corresponding to the updated acquisition parameter in the mapping of the control type of the eBPF; when the deleted acquisition indicator is included in the acquisition indicator information, delete the deleted acquisition indicator in the mapping of the control type of the eBPF.
[0183] In some embodiments, the data acquisition system 1400 further includes:
[0184] The second storage module is used to store the first target data into the eBPF data type mapping;
[0185] The third storage module is used to store the first target data stored in the mapping of the data type of the eBPF into the database in response to the persistence instruction.
[0186] In some implementations, the third storage module is specifically used to: when the acquisition configuration information includes platform data, associate the first target data stored in the data type mapping of the eBPF with the platform data to obtain second target data; and store the second target data in the database.
[0187] Those skilled in the art should understand that Figure 14 The functions of each module in the data acquisition system 1400 shown can be understood by referring to the relevant description of the aforementioned data acquisition method. Figure 14 The functions of each module in the data acquisition system 1400 shown can be implemented by a program running on a processor or by specific logic circuits.
[0188] This application also provides a data acquisition device. Figure 15 This is a schematic diagram of the hardware structure of the data acquisition device 1500 according to an embodiment of this application, as shown below. Figure 15 As shown, the data acquisition device 1500 includes: a communication component 1503 for data transmission, at least one processor 1501, and a memory 1502 for storing computer programs that can run on the processor 1501. The various components in the terminal are coupled together via a bus system 1504. It is understood that the bus system 1504 is used to implement communication between these components. In addition to a data bus, the bus system 1504 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 15 The general labeled all buses as Bus System 1504.
[0189] Wherein, when the processor 1501 executes the computer program, it performs at least the following: Figures 1 to 4 as well as Figures 5 to 13 The steps of the method shown.
[0190] It is understood that memory 1502 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memory 1502 described in the embodiments of this application is intended to include, but is not limited to, these and any other suitable types of memory.
[0191] The methods disclosed in the embodiments of this application can be applied to or implemented by the processor 1501. The processor 1501 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in the processor 1501 or by instructions in the form of software. The processor 1501 may be a general-purpose processor, a DSP, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 1501 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in the memory 1502. The processor 1501 reads the information in the memory 1502 and completes the steps of the aforementioned method in conjunction with its hardware.
[0192] In an exemplary embodiment, the data acquisition device 1500 may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), FPGAs, general-purpose processors, controllers, MCUs, microprocessors, or other electronic components to perform the aforementioned call recording method.
[0193] This application also provides a computer-readable storage medium storing a computer program thereon, characterized in that the program, when executed by a processor, is at least used to perform... Figures 1 to 4 as well as Figures 5 to 13 The steps of the method are shown. The computer-readable storage medium may specifically be a memory. The memory may be, for example... Figure 15 The memory 1502 shown.
[0194] The technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.
[0195] In the several embodiments provided in this application, it should be understood that the disclosed methods and smart devices can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.
[0196] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.
[0197] In addition, each functional unit in the various embodiments of this application can be integrated into a second processing unit, or each unit can be a separate unit, or two or more units can be integrated into a unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.
[0198] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.
Claims
1. A data acquisition method, characterized in that, The method includes: In response to a data acquisition command for the network to be monitored, the acquisition configuration information stored in the control type mapping of the extended Berkeley Packet Filter (eBPF) is read based on the identification information of the network to be monitored; wherein, the acquisition configuration information includes the acquisition indicator information of the network to be monitored; Obtain the first target data in the network to be monitored that corresponds to the collected indicator information; When the collected indicator information includes newly added collected indicators, before acquiring the first target data corresponding to the collected indicator information in the network to be monitored, the method further includes: Determine the mount point of the target type of eBPF acquisition program and the target eBPF acquisition program corresponding to the newly added acquisition index; The step of acquiring the first target data in the network to be monitored, corresponding to the collected indicator information, includes: The target eBPF acquisition program is loaded at the mounting point of the target type eBPF acquisition program, and the first target data corresponding to the newly added acquisition indicator in the network to be monitored is obtained using the target eBPF acquisition program.
2. The method according to claim 1, characterized in that, The process of determining the eBPF data acquisition program mount point corresponding to the newly added data acquisition indicator includes: Determine whether the collectable indicators in the tracking point list can cover the newly added collectable indicators; If the collectable metrics in the tracking point list can cover the newly added collectable metrics, determine the eBPF acquisition program mount point of the tracking point type corresponding to the newly added collectable metrics. If the collectable metrics in the tracking point list cannot cover the newly added collectable metric, determine the eBPF acquisition program mount point of the kernel probe type corresponding to the newly added collectable metric.
3. The method according to claim 1, characterized in that, Before reading the acquisition configuration information stored in the eBPF control type mapping based on the identification information of the network to be monitored, the method further includes: Obtain user configuration information, parse the user configuration information to obtain the collected indicator information of the network to be monitored and the indicator information of the target platform; The mapping of the control type of the eBPF is updated based on the collected index information of the network to be monitored.
4. The method according to claim 3, characterized in that, The collected configuration information also includes platform data of the target platform, and the method further includes: The platform data corresponding to the indicator information of the target platform is obtained from the target component using the interface provided by the client library of the target platform, and the platform data is stored in the control type mapping of the eBPF.
5. The method according to claim 3 or 4, characterized in that, The collected indicator information includes at least one of the following: newly added collected indicators, updated collected parameters, and deleted collected indicators; the collected parameters include at least one of the following: collected frequency, collected period, collected times, data collected length, and collected time; The mapping of the control type of the eBPF based on the collected index information of the network to be monitored includes: If the newly added data collection indicator is included in the data collection indicator information, the newly added data collection indicator is stored in the control type mapping of the eBPF; If the updated acquisition parameters are included in the acquisition indicator information, the acquisition parameters of the acquisition indicator corresponding to the updated acquisition parameters are updated in the mapping of the control type of the eBPF. If the data collection information includes the data collection indicator to be deleted, then the data collection indicator to be deleted is removed from the mapping of the control type in the eBPF.
6. The method according to any one of claims 1 to 4, characterized in that, The method further includes: Store the first target data into the eBPF data type mapping; In response to the persistence instruction, the first target data stored in the mapping of the data type of the eBPF is stored in the database.
7. The method according to claim 6, characterized in that, The step of storing the first target data stored in the mapping of the data type of the eBPF into the database includes: If the collected configuration information includes platform data, the first target data stored in the eBPF data type mapping is associated with the platform data to obtain the second target data; The second target data is stored in the database.
8. A data acquisition system, characterized in that, The data acquisition system includes: The reading module is used to respond to a data acquisition command for the network to be monitored by reading the acquisition configuration information stored in the control type mapping of the extended Berkeley Packet Filter (eBPF) based on the identification information of the network to be monitored; wherein the acquisition configuration information includes the acquisition indicator information of the network to be monitored. The acquisition module is used to acquire the first target data in the network to be monitored that corresponds to the collected indicator information; When the collected indicator information includes newly added collected indicators, the data acquisition system further includes: a determination module, used to determine the mounting point of the target type eBPF acquisition program and the target eBPF acquisition program corresponding to the newly added collected indicator; the acquisition module is specifically used to: load the target eBPF acquisition program at the mounting point of the target type eBPF acquisition program, and use the target eBPF acquisition program to acquire the first target data in the network to be monitored corresponding to the newly added collected indicator.
9. A data acquisition device, characterized in that, The data acquisition device includes a memory and a processor, wherein the memory stores computer-executable instructions, and the processor, when executing the computer-executable instructions in the memory, can implement the method of any one of claims 1 to 7.
10. A computer storage medium, characterized in that, The storage medium stores executable instructions that, when executed by a processor, implement the method of any one of claims 1 to 7.