File data collection method and file data collection device based on cross-container

By deploying independent acquisition processes on the host in the Kubernetes cluster and configuring their cross-container access capabilities, the resource and security problems caused by the acquisition processes in the container are solved, cross-container file data collection is realized, and resource consumption and risks are reduced. Quarantine costs.

CN114077577BActive Publication Date: 2024-12-03ALIPAY (HANGZHOU) INFORMATION TECH CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202111367619.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-18
Publication Date
2024-12-03
Estimated Expiration
2041-11-18

AI Technical Summary

Technical Problem

In a Kubernetes cluster, deploying a collection process in each container results in high resource cost overhead, and the collection process and business are deployed in the same container, which poses security risks and is difficult to isolate resources and risks.

Method used

Deploy the collection process independent of each business container on the host, and configure its cross-container access capabilities, replace the collection process in the container, realize cross-container file data collection, reduce the number of collection processes on the host, and realize business processes and containers Resource isolation between.

Benefits of technology

By reducing the number of collected processes on the host, reducing resource consumption, realizing resource isolation between business processes and containers, improving security, and reducing the isolation cost of resources and risks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114077577B_ABST
    Figure CN114077577B_ABST
Patent Text Reader

Abstract

The embodiments of this specification provide a file data collection method and a file data collection device based on cross-container. In the file data collection method, a collection process independent of each business container on the host is deployed on the host to replace the collection process offline in each business container; the access rights of the deployed collection process in the host are configured so that the collection process has cross-container access capabilities; and the collection process is used to collect target file data through the business container to which the access path of the target file data belongs.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present specification relate to the field of computer technology, and in particular, to a file data collection method and a file data collection device based on cross-container. Background Art

[0002] Kubernetes (K8S) is a portable container orchestration management tool created for container services. In K8S, containerization technology can be used to virtualize a small-scale environment (i.e., container) for technical personnel to develop. Containerization technology is different from virtual machines, which require virtualization of the entire operation, thus solving technical problems in virtual machines such as slow startup, large space occupation, and difficulty in migration. These advantages of containerization technology make K8S widely used.

[0003] A K8S cluster can be composed of multiple hosts as nodes. Multiple Pods can be run on each host, and a container can be run under each Pod. The resources between each container are isolated from each other. The processes started in each container include business processes and collection processes. The business process is used to perform corresponding business operations, and the collection process is a process deployed with the business to collect business-related information. Through this container deployment method, resource isolation between various businesses is achieved. Summary of the invention

[0004] In view of the above, the embodiments of this specification provide a file data collection method and a file data collection device based on cross-container. Through the technical solution of the embodiments of this specification, the collection process is deployed on the host machine independently of each business container, and cross-container access capability is provided. While collecting file data from each business container, the number of collection processes deployed on the host machine is reduced, and resource isolation between the business process and the business container is achieved.

[0005] According to one aspect of an embodiment of the present specification, a cross-container based file data collection method is provided, including: deploying a collection process independent of each business container on the host machine on a host machine to replace the collection process that is offline in each business container; configuring the access rights of the deployed collection process in the host machine so that the collection process has cross-container access capability; and using the collection process to collect the target file data through the business container to which the access path of the target file data belongs.

[0006] According to another aspect of the embodiments of this specification, a cross-container based file data collection device is also provided, including: a collection process deployment unit, which deploys a collection process independent of each business container on the host machine on a host machine to replace the collection process that has gone offline in each business container; an access permission configuration unit, which configures the access permission of the deployed collection process in the host machine so that the collection process has cross-container access capability; and a file data collection unit, which uses the collection process to collect the target file data through the business container to which the access path of the target file data belongs.

[0007] According to another aspect of an embodiment of this specification, an electronic device is also provided, comprising: at least one processor, a memory coupled to the at least one processor, and a computer program stored on the memory, wherein the at least one processor executes the computer program to implement any of the above-described methods for collecting file data based on cross-containers.

[0008] According to another aspect of the embodiments of this specification, a computer-readable storage medium is further provided, which stores a computer program, and when the computer program is executed by a processor, the cross-container-based file data collection method as described above is implemented.

[0009] According to another aspect of the embodiments of this specification, a computer program product is further provided, including a computer program, wherein when the computer program is executed by a processor, the cross-container-based file data collection method described above is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] A further understanding of the nature and advantages of the embodiments of the present specification can be achieved by referring to the following drawings. In the drawings, similar components or features may have the same reference numerals.

[0011] Figure 1 A schematic diagram showing an example of a service container deployed in a host machine.

[0012] Figure 2 A schematic diagram showing an example of the structure of a K8S cluster according to an embodiment of this specification.

[0013] Figure 3 A flowchart of an example of a method for collecting file data based on cross-containers according to an embodiment of this specification is shown.

[0014] Figure 4 A schematic diagram showing an example of separating a service container from a collection process according to an embodiment of this specification.

[0015] Figure 5A flowchart of an example of configuring access rights for a collection process in a host machine according to an embodiment of this specification is shown.

[0016] Figure 6 A flowchart showing an example of a collection process collecting target file data through a business container according to an embodiment of this specification.

[0017] Figure 7 A block diagram of an example of a cross-container-based file data collection device according to an embodiment of this specification is shown.

[0018] Figure 8 A block diagram showing another example of a cross-container-based file data collection device according to an embodiment of this specification.

[0019] Fig. 9 A block diagram showing another example of a cross-container-based file data collection device according to an embodiment of this specification.

[0020] Fig.10 A block diagram of an electronic device for implementing a cross-container-based file data collection method according to an embodiment of the present specification is shown. DETAILED DESCRIPTION

[0021] The subject matter described herein will be discussed below with reference to example implementations. It should be understood that the discussion of these implementations is only to enable those skilled in the art to better understand and implement the subject matter described herein, and is not a limitation of the scope of protection, applicability or examples set forth in the claims. The functions and arrangements of the elements discussed may be changed without departing from the scope of protection of the embodiments of this specification. Various processes or components may be omitted, substituted or added to each example as needed. In addition, the features described relative to some examples may also be combined in other examples.

[0022] As used herein, the term "including" and its variations represent open terms, meaning "including but not limited to". The term "based on" means "based at least in part on". The terms "one embodiment" and "an embodiment" mean "at least one embodiment". The term "another embodiment" means "at least one other embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other definitions may be included below, whether explicit or implicit. Unless the context clearly indicates otherwise, the definition of a term is consistent throughout the specification.

[0023] Kubernetes is a portable container orchestration management tool created for container services. In K8S, containerization technology can be used to virtualize a small-scale environment (i.e., container) for technical personnel to develop. Containerization technology is different from virtual machines, which need to virtualize the entire operation, thus solving technical problems such as slow startup, large space occupation, and difficulty in migration in virtual machines. These advantages of containerization technology make K8S widely used.

[0024] A K8S cluster can be composed of multiple hosts as nodes. Multiple Pods can be run on each host. Each Pod runs a container, and the resources between each container are isolated from each other. Figure 1 A schematic diagram showing an example of a business container deployed in a host machine is shown as follows: Figure 1 As shown, the processes started in each container include business processes and collection processes (agent processes). Business processes are used to perform corresponding business operations, and collection processes are processes deployed along with the business to collect business-related information. Through this container deployment method, resource isolation between various businesses is achieved.

[0025] However, in the above deployment method, a collection process needs to be deployed in each container, which leads to high resource cost overhead, especially when the K8S cluster undertakes more businesses and the scale of containers increases exponentially, the resource cost overhead also increases exponentially. In addition, the collection process and the business are deployed in the same container, without isolating resources and risks, resulting in security risks for the business.

[0026] In view of the above, the embodiments of this specification provide a file data collection method and a file data collection device based on cross-containers. In the file data collection method, a collection process independent of each business container on the host is deployed on the host to replace the collection process offline in each business container; the access rights of the deployed collection process in the host are configured so that the collection process has cross-container access capabilities; and the collection process is used to collect the target file data through the business container to which the access path of the target file data belongs. Through the technical solution of the embodiments of this specification, the collection process is deployed on the host independently of each business container and has cross-container access capabilities. While realizing the collection of file data from each business container, the number of collection processes deployed on the host is reduced, and resource isolation between the business process and the business container is realized.

[0027] The following describes in detail the cross-container based file data collection method and file data collection device provided by the embodiments of this specification in conjunction with the accompanying drawings.

[0028] The cross-container file data collection method and file data collection device provided in the embodiments of this specification can be applied to the application environment of the K8S cluster, so that cross-container file data collection can be implemented in the K8S cluster. Figure 2 A schematic diagram showing an example of the structure of a K8S cluster according to an embodiment of this specification.

[0029] like Figure 2 As shown, the K8S cluster can be composed of Master nodes and Node nodes, and the Master node can communicate with each Node node. In the K8S cluster, the Master node and each Node node can be composed of a host machine, each host machine can run multiple Pods, and a container can be run under each pod through the OCI (Open Container Initiative) standard.

[0030] The Master node can be used as a cluster control node to manage and control the entire K8S cluster. The Master node can send control commands to each Node node. The Master node can include components such as Kubernetes Controller Manager, Kubernetes Scheduler, and Etcd. The Kubernetes Controller Manager can be used as the automation control center for all resource objects in the K8S cluster to maintain and manage the status of the cluster, such as fault detection, automatic expansion, rolling updates, etc. The Kubernetes Scheduler can be used to schedule resources and schedule Pods to the corresponding machines according to the predetermined scheduling strategy. Etcd can be used to save the status of the entire K8S cluster.

[0031] The Node node can be assigned workloads by the Master node. The Node node can include components such as Kubelet, Kube-Proxy, and Docker. Kubelet can be responsible for tasks such as creating, starting, and stopping containers corresponding to the Pod, and can also collaborate with the Master node to implement basic functions of K8S cluster management. Kube-Proxy can be used to implement Service communication and load balancing. Docker can be responsible for the creation and management of containers on this host.

[0032] Figure 3 A flowchart of an example 300 of a method for collecting file data based on a cross-container basis according to an embodiment of this specification is shown.

[0033] like Figure 3As shown, at 310, a collection process independent of each business container on the host is deployed on the host to replace the offline collection process in each business container.

[0034] In the embodiments of this specification, the K8S cluster can be configured with DaemenSet, and DaemenSet can be used as a Pod controller to ensure that a copy of a Pod runs on the Node node.

[0035] In one example, the collection process can be deployed on the host machine by DaemenSet, and DaemenSet can control the deployed collection process. The collection process deployed on the host machine is independent of each business container on the host machine and belongs to a different container from each business container.

[0036] After deploying a collection process independent of each business container, the collection in each business container can be taken offline, and the deployed collection process can replace the offline collection process in each business container, that is, the deployed collection process can implement the functions that the collection process in each business container can perform before it is offline.

[0037] In one example, a new collection process that is independent of each business container on the host can be started on the host. Then, the traffic of the collection process in each business container is switched to the new collection process. Then, the collection process in each business container can be offline. In the offline collection process mode, the collection process in each business container can be offline from the base image, and there will be no collection thread in the newly deployed business container.

[0038] After the collection process in each business container goes offline, only the collection process independent of each business container is deployed in the host machine, and the collection process can collect file data from each business container.

[0039] Figure 4 FIG. 1 is a schematic diagram showing an example of separating a business container from a collection process according to an embodiment of this specification. Figure 4 As shown, before the collection process is separated, one collection process is deployed in each business container in the host machine, and the collection process in each business container can only collect file data in the business container. After the above 310 operation, only one collection process independent of other business containers is saved in the host machine, and there is no collection process in each business container. In this way, only one collection process needs to be run in the host machine, reducing resource consumption in the host machine.

[0040] At 320 , the access rights of the deployed collection process in the host machine are configured so that the collection process has cross-container access capability.

[0041] In the embodiments of this specification, the collection process with cross-container access capability can access the file systems mounted by each business container, so that file data can be collected from each business container. The file systems that the collection process can access may include physical file systems with physical structures (e.g., storage volumes) and virtual file systems presented in virtual structures (e.g., Overlay file systems), and the virtual file systems rely on and are built on the physical file systems.

[0042] Figure 5 A flowchart of an example 500 of configuring access rights for a collection process in a host machine according to an embodiment of this specification is shown.

[0043] like Figure 5 As shown, at 322, the deployed collection process can be configured by switching the Mnt Namespace of the collection process, so that the collection process has the same file system perspective as the host machine.

[0044] In this example, Mnt Namespace can be used in the host machine to implement environment isolation of the file systems mounted by each container. Different Mnt Namespaces can correspond to different file systems. You can switch MntNamespace to point to different file systems.

[0045] The file system perspective means that the file system can be seen. The root directory that the collection process can see with the same file system perspective as the host is the root directory of the host. In an example, the collection process can see all directory files on the host, including the file system root directories mounted by other business containers isolated by Mnt Namespace.

[0046] In one example, when the Pod where the collection process is located is started, the Proc file system of the host machine can be mounted on the Pod.

[0047] In this example, the Proc file system may be a virtual file system, and the Proc file system may be used to view information about system hardware and running processes. Thus, the Proc file system of the host machine may provide information about processes started by the host machine.

[0048] In the embodiment of this specification, the storage directory of the host machine's Proc file system can be / Proc, and the Proc file system can be mounted in a container volume mount manner. In one example, the host machine's Proc file system can be mounted on the Pod where the collection process is located through DaemenSet.

[0049] Then, you can set the mode of the Pid (Process ID) Namespace of the collection process to Host mode.

[0050] Among them, Pid Namespace can be used to virtualize the container of the process Pid and implement isolation between containers from the perspective of Pid. Each business container can only see the Pid in its own container, while the host can see the Pid of all processes.

[0051] Because the collection process is configured as Host mode and the Proc file system is mounted on the Pod where the collection process is located, the collection process can read the host process in the Proc file system.

[0052] Next, the Mnt Namespace of the acquisition process is switched to the process of the read host machine, so that the acquisition process has the same file system perspective as the host machine.

[0053] In one example, the Mnt Namespace of the acquisition process can be switched to any process of the host machine that is read. In another example, the switched host machine process can be the first process of the host machine, that is, the Mnt Namespace of the acquisition process can be switched to / Proc / 1 / ns / Mnt on the host machine, where / Proc / 1 represents the first process of the host machine. By switching the Mnt Namespace to the first process, it can be ensured that the process switched by the Mnt Namespace exists in the host machine.

[0054] In one example, the acquisition process can switch its own MntNamespace to the read process of the host machine through the Linux Setns system call.

[0055] After switching to the host process through Mnt Namespace, since the host process itself has the host's file system perspective, that is, it can see all file systems in the host, the collection process with the same file system perspective as the host can also see all file systems in the host and the root directory under each file system.

[0056] Back to Figure 5 At 324, storage metadata of each business container is obtained through a collection process.

[0057] The storage meta information of the business container can be used to determine the mount point on the business container, and further determine the location of each mount point on the host machine. The location determined by the storage meta information is a physical location on the host machine.

[0058] In an example, each storage meta-information of a business container can be used to represent a mount point on the business container. The storage meta-information of each business container can include the physical file system mounted on the business container and the virtual file system configured in the business container. The physical file system can include storage volumes, etc. The virtual file system can include Overlay, Overlay2, etc. Overlay and Overlay2 can be used as Docker image Graphdriver in K8S. For the virtual file system, Overlay is used as an example to illustrate below.

[0059] In addition, the storage meta-information of the service container may also include a container identifier, an IP address, and a mapping relationship between the container identifier and the IP address corresponding to the service container, etc., which are used to determine the meta-information of the service container.

[0060] Back to Figure 3 After the collection process has the cross-container access capability, at 330, the collection process can be used to collect the target file data through the business container to which the access path of the target file data belongs.

[0061] Figure 6 A flowchart of an example 600 of a collection process collecting target file data through a business container according to an embodiment of this specification is shown.

[0062] like Figure 6 As shown, at 332, the collection process may determine the storage directory mounted by the business container to which the access path of the target file data belongs based on the same file system perspective as the host machine.

[0063] In this example, the target file data may be the file data to be collected, and the access path is the path to access the target file data, which is provided to the collection process so that the collection process can collect the target file data according to the access path. The composition of the access path of the target file data may include a storage directory mounted in the business container, and the storage directory may be a root directory, or may be composed of a root directory and directories at all levels under it. For example, the access path of the target file data is: / home / admin / logs / a.log, where / home / admin / logs may be a storage directory mounted on the business container.

[0064] After determining the access path of the target file data, based on the same file system perspective as the host machine, the acquisition process can see each root directory in each business container and the storage directory under each root directory. The storage directory included in the access path can be compared with the storage directory in each business container to determine the business container to which the access path of the target file data belongs, and the storage directory mounted by the business container. For example, based on the above example, one of the storage directories mounted by business container A in the host machine is / home / admin / logs, which matches the storage directory included in the access path of the target file data, so it can be determined that the access path of the target file data belongs to the storage directory / home / admin / logs mounted under the business container A.

[0065] At 334 , the location of the target file data on the host machine is determined based on the storage meta information corresponding to the access path and the determined storage directory.

[0066] In this example, the storage meta-information corresponding to the storage directory is used to represent the file system to which the storage directory belongs, and the storage directory is one of the directories in the file system represented by the corresponding storage meta-information. For example, a storage volume mounted on a business container includes the storage directory / home / admin / logs, and the storage meta-information corresponding to the storage directory is the storage volume. In addition, the determined location of the target file data on the host machine is a physical location.

[0067] In an example, the physical location of the storage directory on the host machine corresponding to the storage directory to which the access path belongs may be determined according to the storage meta-information corresponding to the storage directory.

[0068] In the embodiments of the present specification, the storage meta information can be used to determine the physical location of the mount point on the host machine, thereby determining the physical location of the file system represented by the storage meta information on the host machine, and further, determining the physical location of each storage directory in the file system on the host machine.

[0069] In this example, the storage directory and the physical location on the host machine can correspond one to one, so that the corresponding relationship between the storage directory and the physical location can be established. For example, taking the storage meta information as a storage volume, a storage directory included in the storage volume mounted under the business container is: / home / admin / logs. The physical location of the storage volume on the host machine can be determined through the storage meta information of the storage volume. Further, it can be determined that the physical location corresponding to the storage directory is: / roofs / PodA / logs, indicating that the actual storage location of the storage directory on the host machine is / roofs / PodA / logs.

[0070] For another example, taking the storage metadata as an Overlay file system, a storage directory in the Overlay file system mounted in the business container is the root directory " / ". The storage metadata of the Overlay file system can be used to determine that the physical location on the host machine is: / roofs / PodA / graph / mergeDir, which means that the actual storage location of the storage directory on the host machine is / roofs / PodA / graph / mergeDir.

[0071] After the physical location corresponding to the storage directory is determined, the access path can be converted into the location of the target file data on the host machine according to the determined physical location and the storage directory.

[0072] The determined physical location corresponds to the storage directory one by one, and the composition of the access path may include the storage directory, that is, the access path is one of the paths under the storage directory. Based on this, the storage directory included in the composition of the access path can be replaced with the determined physical location to obtain a path including the physical location corresponding to the storage directory, and the path is used to represent the physical location of the target file data on the host machine.

[0073] For example, if the storage metadata is a storage volume, a storage directory of the storage volume is / home / admin / logs. The physical location corresponding to the storage directory is / roofs / PodA / logs. The access path of the target file data is / home / admin / logs / a.log. The storage directory included in the composition of the access path is / home / admin / logs. This part of the storage directory can be replaced with the path of the physical location, so that the location of the target file data on the host machine is / roofs / PodA / logs / a.log.

[0074] Taking the Overlay file system as an example, the storage directory of the Overlay file system is the root directory, that is, / . The physical location of the Overlay file system on the host is / roofs / PodA / graph / mergeDir. The access path of the target file data is / etc / localtime. Replace the root directory in the access path with the path of the physical location, and the location of the target file data on the host is / roofs / PodA / graph / mergeDir / etc / localtime.

[0075] In one way of determining the physical location of the target file data, the storage meta information may include a storage volume and a virtual file system. First, it may be determined whether the directory of the access path belongs to the storage directory of the storage volume according to the access path of the target file data.

[0076] When it is determined that the directory to which the access path belongs belongs to the storage directory of the storage volume, the physical location of the directory in the storage volume corresponding to the storage directory on the host machine can be determined based on the storage metadata of the storage volume, and then the access path can be converted into the location of the target file data on the host machine based on the determined physical location, storage directory and access path.

[0077] When it is determined that the directory to which the access path belongs does not belong to the storage directory of the storage volume, it can be determined that the directory to which the access path belongs belongs to the storage directory of the virtual file system, so that the physical location of the directory in the virtual file system corresponding to the storage directory on the host machine can be determined based on the storage metadata of the virtual file system, and then the access path is converted into the location of the target file data on the host machine based on the determined physical location, storage directory and access path.

[0078] Back to Figure 6 At 336, the acquisition process acquires the target file data from the determined location.

[0079] Through the cross-container based file data collection method and file data collection device provided in the embodiments of this specification, the collection process is deployed on the host machine independently of each business container, and has cross-container access capability. While realizing the collection of file data from each business container, the number of collection processes deployed on the host machine is reduced, and resource isolation between the business process and the business container is achieved.

[0080] Figure 7 A block diagram of an example of a cross-container-based file data collection device 700 according to an embodiment of this specification is shown.

[0081] like Figure 7 As described above, the file data collection device 700 includes a collection process deployment unit 710 , an access permission configuration unit 720 and a file data collection unit 730 .

[0082] The collection process deployment unit 710 is configured to deploy, on the host machine, a collection process that is independent of each service container on the host machine, to replace the offline collection process in each service container.

[0083] In an example, the collection process deployment unit 710 can also be configured to: start a new collection process on the host machine that is independent of each business container on the host machine; switch the traffic of the collection process in each business container to the new collection process; and take the collection process in each business container offline.

[0084] The access permission configuration unit 720 is configured to configure the access permission of the deployed collection process in the host machine so that the collection process has cross-container access capability.

[0085] The file data collection unit 730 is configured to use a collection process to collect target file data through a business container to which an access path of the target file data belongs.

[0086] Figure 8 A block diagram of another example of a cross-container-based file data collection device 700 according to an embodiment of this specification is shown.

[0087] The access rights configuration unit 720 may include: a collection process configuration module 722 and a meta information acquisition module 724. The collection process configuration module 722 is configured to configure the deployed collection process by switching the Mnt Namespace of the collection process so that the collection process has the same file system perspective as the host machine.

[0088] In one example, the collection process configuration module 722 is configured to: when the Pod where the collection process is located is started, mount the host machine's Proc file system on the Pod; set the mode of the collection process's Pid Namespace to host mode; read the host machine's process in the Proc file system through the collection process; and switch the collection process's MntNamespace to the read host machine's process, so that the collection process has the same file system perspective as the host machine.

[0089] The meta information acquisition module 724 is configured to acquire storage meta information of each business container through a collection process.

[0090] Fig. 9 A block diagram of another example of a cross-container-based file data collection device 700 according to an embodiment of this specification is shown.

[0091] In an example, the file data collection unit 730 may further include: a business container determination module 732 , a storage location determination module 734 , and a file data collection module 736 .

[0092] The business container determination module 732 is configured to determine the storage directory mounted by the business container to which the access path of the target file data belongs based on the file system perspective during the collection process.

[0093] The storage location determination module 734 is configured to determine the location of the target file data on the host machine based on the access path and the storage meta information corresponding to the storage directory.

[0094] In one example, the storage location determination module 734 is configured to: determine the physical location of the storage directory on the host machine based on the storage metadata corresponding to the storage directory; and convert the access path into the location of the target file data on the host machine based on the physical location and the storage directory.

[0095] The file data collection module 736 is configured to collect target file data from the determined location during the collection process.

[0096] Reference above Figures 1 to 9 , describes an embodiment of a file data collection method and a file data collection device based on a cross-container according to an embodiment of this specification.

[0097] The cross-container file data collection device of the embodiment of this specification can be implemented by hardware, software, or a combination of hardware and software. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of the device in which it is located reading the corresponding computer program instructions in the memory into the memory and running them. In the embodiment of this specification, the cross-container file data collection device can be implemented, for example, using an electronic device.

[0098] Fig.10 A block diagram of an electronic device 1000 for implementing a cross-container-based file data collection method according to an embodiment of the present specification is shown.

[0099] like Fig.10 As shown, the electronic device 1000 may include at least one processor 1010, a memory (e.g., a non-volatile memory) 1020, a memory 1030, and a communication interface 1040, and the at least one processor 1010, the memory 1020, the memory 1030, and the communication interface 1040 are connected together via a bus 1050. At least one processor 1010 executes at least one computer-readable instruction stored or encoded in the memory (i.e., the above-mentioned element implemented in the form of software).

[0100] In one embodiment, computer executable instructions are stored in a memory, which when executed cause at least one processor 1010 to: deploy a collection process on a host machine that is independent of each business container on the host machine to replace the collection process that has gone offline in each business container; configure access rights of the deployed collection process in the host machine so that the collection process has cross-container access capabilities; and use the collection process to collect target file data through the business container to which the access path of the target file data belongs.

[0101] It should be understood that the computer executable instructions stored in the memory, when executed, cause at least one processor 1010 to perform the above combined operations in various embodiments of this specification. Figure 1-9Describes the various operations and functions.

[0102] According to one embodiment, a program product such as a machine-readable medium is provided. The machine-readable medium may have instructions (i.e., the above-mentioned elements implemented in the form of software), which, when executed by a machine, causes the machine to perform the above-mentioned combination of various embodiments of this specification. Figure 1-9 Describes the various operations and functions.

[0103] Specifically, a system or device equipped with a readable storage medium can be provided, on which software program codes that implement the functions of any of the above-mentioned embodiments are stored, and a computer or processor of the system or device can read and execute instructions stored in the readable storage medium.

[0104] In this case, the program code itself read from the machine-readable medium can realize the function of any one of the above-mentioned embodiments, and thus the machine-readable code and the machine-readable storage medium storing the machine-readable code constitute a part of the present invention.

[0105] The computer program code required for the operation of each part of this specification can be written in any one or more programming languages, including object-oriented programming languages, such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB, NET and Python, conventional procedural programming languages ​​such as C language, Visual Basic 2003, Perl, COBOL2002, PHP and ABAP, dynamic programming languages ​​such as Python, Ruby and Groovy, or other programming languages. The program code can be run on the user's computer, or run on the user's computer as an independent software package, or run partly on the user's computer and another part on the remote computer, or all on the remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any network form, such as a local area network (LAN) or a wide area network (WAN), or connected to an external computer (e.g., via the Internet), or in a cloud computing environment, or used as a service, such as software as a service (SaaS).

[0106] Examples of readable storage media include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, the program code may be downloaded from a server computer or a cloud via a communication network.

[0107] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0108] Not all steps and units in the above-mentioned processes and system structure diagrams are necessary, and some steps or units can be ignored according to actual needs. The execution order of each step is not fixed and can be determined according to needs. The device structure described in the above-mentioned embodiments can be a physical structure or a logical structure, that is, some units may be implemented by the same physical entity, or some units may be implemented by multiple physical entities, or some components in multiple independent devices may be implemented together.

[0109] The term "exemplary" as used throughout this specification means "serving as an example, instance, or illustration" and does not imply "preferred" or "advantageous" over other embodiments. The detailed description includes specific details for the purpose of providing an understanding of the described techniques. However, the techniques may be implemented without these specific details. In some instances, in order to avoid obscuring the concepts of the described embodiments, well-known structures and devices are shown in block diagram form.

[0110] The optional implementation modes of the embodiments of the present specification are described in detail above in conjunction with the accompanying drawings. However, the embodiments of the present specification are not limited to the specific details in the above implementation modes. Within the technical concept of the embodiments of the present specification, various simple modifications can be made to the technical solutions of the embodiments of the present specification, and these simple modifications all belong to the protection scope of the embodiments of the present specification.

[0111] The above description of the contents of this specification is provided to enable any person of ordinary skill in the art to implement or use the contents of this specification. Various modifications to the contents of this specification will be apparent to those of ordinary skill in the art, and the general principles defined herein may be applied to other variations without departing from the scope of protection of the contents of this specification. Therefore, the contents of this specification are not limited to the examples and designs described herein, but are consistent with the widest range of principles and novel features disclosed herein.

Claims

1. A file data collection method based on cross-container, include: Deploy a collection process on the host machine that is independent of each business container on the host machine to replace the offline collection process in each business container; Configuring access rights of the deployed acquisition process in the host machine so that the acquisition process has cross-container access capability; as well as Using the acquisition process to acquire the target file data through the business container to which the access path of the target file data belongs, The step of configuring the access rights of the deployed acquisition process in the host machine so that the acquisition process has cross-container access capability includes: Configuring the deployed acquisition process by switching the Mnt Namespace of the acquisition process so that the acquisition process has the same file system perspective as the host machine; and Acquire storage metadata of each business container through the acquisition process, Using the acquisition process to acquire the target file data through the business container to which the access path of the target file data belongs includes: The acquisition process determines, based on the file system perspective, a storage directory mounted by a business container to which the access path of the target file data belongs; Determine the location of the target file data on the host machine based on the access path and the storage meta information corresponding to the storage directory; and The acquisition process acquires the target file data from the determined location.

2. The file data collection method according to claim 1, in, Configuring the deployed collection process by switching the MntNamespace of the collection process so that the collection process has the same file system perspective as the host machine includes: When the Pod where the collection process is located is started, the Proc file system of the host machine is mounted on the Pod; Set the Pid Namespace mode of the acquisition process to Host mode; Reading the process of the host machine in the Proc file system through the acquisition process; and The Mnt Namespace of the acquisition process is switched to the read process of the host machine, so that the acquisition process has the same file system perspective as the host machine.

3. The file data collection method according to claim 2, in, Switching the Mnt Namespace of the acquisition process to the read process of the host machine so that the acquisition process has the same file system perspective as the host machine includes: The Mnt Namespace of the acquisition process is switched to the first process of the host machine, so that the acquisition process has the same file system perspective as the host machine.

4. The file data collection method according to claim 1, in, The storage meta-information includes a storage volume and a virtual file system.

5. The file data collection method according to claim 1, in, Determining the location of the target file data on the host machine based on the storage meta information corresponding to the access path and the storage directory includes: Determining a physical location on the host machine corresponding to the storage directory according to the storage meta-information corresponding to the storage directory; and The access path is converted into the location of the target file data on the host machine according to the physical location and the storage directory.

6. The file data collection method according to claim 1, in, Deploying a collection process independent of each business container on the host machine on the host machine to replace the offline collection process in each business container includes: Starting a new collection process on the host machine that is independent of each business container on the host machine; Switching the traffic of the collection process in each service container to the new collection process; and The collection processes in each of the service containers are taken offline.

7. A file data collection device based on cross-container, include: A collection process deployment unit, which deploys, on a host machine, a collection process that is independent of each service container on the host machine, to replace the offline collection process in each service container; An access rights configuration unit, configured with respect to access rights of the deployed acquisition process in the host machine, so that the acquisition process has cross-container access capability; as well as The file data collection unit uses the collection process to collect the target file data through the business container to which the access path of the target file data belongs, Wherein, the access permission configuration unit includes: A collection process configuration module, which configures the deployed collection process by switching the Mnt Namespace of the collection process so that the collection process has the same file system perspective as the host machine; and A meta information acquisition module acquires storage meta information of each business container through the acquisition process. The file data acquisition unit comprises: A business container determination module, wherein the acquisition process determines the storage directory mounted by the business container to which the access path of the target file data belongs based on the file system perspective; a storage location determination module, which determines the location of the target file data on the host machine based on the access path and the storage meta information corresponding to the storage directory; and The file data collection module collects the target file data from the determined location.

8. The file data collection device according to claim 7, in, The acquisition process configuration module is configured as follows: When the Pod where the collection process is located is started, the Proc file system of the host machine is mounted on the Pod; Set the Pid Namespace mode of the acquisition process to host mode; Reading the process of the host machine in the Proc file system through the acquisition process; as well as The Mnt Namespace of the acquisition process is switched to the read process of the host machine, so that the acquisition process has the same file system perspective as the host machine.

9. The file data collection device according to claim 7, in, The storage location determination module is configured to: Determining a physical location on the host machine corresponding to the storage directory according to the storage meta-information corresponding to the storage directory; and The access path is converted into the location of the target file data on the host machine according to the physical location and the storage directory.

10. The file data collection device according to claim 7, in, The acquisition process deployment unit is configured as follows: Starting a new collection process on the host machine that is independent of each business container on the host machine; Switching the traffic of the collection process in each service container to the new collection process; and The collection processes in each of the service containers are taken offline.

11. An electronic device, include: At least one processor, a memory coupled to the at least one processor, and a computer program stored in the memory, wherein the at least one processor executes the computer program to implement the file data collection method as described in any one of claims 1-6.

12. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the file data collection method according to any one of claims 1 to 6.

13. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the file data collection method according to any one of claims 1 to 6 is implemented.

Citation Information

Patent Citations

  • Cluster log collection method and system, equipment and medium

    CN111258722A

  • Acquisition method and system for text logs in cluster container and storage medium

    CN111309546A

  • Method and device for acquiring monitoring data of virtual machine in Docker container

    CN111651237A

  • Identifying container file events for providing container security

    US20180293394A1

  • File systems constructed of block objects

    WO2021202175A1