Log collection method and device, computer equipment and computer readable storage medium

Abstract

The embodiment of the invention discloses a log collection method and device, computer equipment and a computer readable storage medium. The log collection method comprises the steps that a target physical node establishes communication connection with a container cloud management client according to a verification identifier associated with a cloud service cluster; when the target physical node has a change event for the application container in the target period, generating container change information corresponding to the change event; sending container change information to the container cloud management client through the cloud service cluster; obtaining a second container group state record table, and generating a new flexible connection corresponding to the target application container in the target collection directory according to the second container group state record table; the target application container refers to an application container with a change event in a second container group state record table; the new soft connection is used for acquiring first log data of the target application container under the original directory; the original directory is the initial path of the first log data, so as to collect the log data of the large-scale cloud service cluster and improve the log collection efficiency.

Description

Technical Field

[0001] The present application relates to the field of cloud computing technology, and in particular to a log collection method, device, computer equipment, and computer-readable storage medium. Background Art

[0002] Network devices, systems, and applications will generate relevant event records when they are running. These event records are called log data. Applications usually need to be packaged into a portable application container before they can be published on any popular Linux machine. Cloud computing is increasingly used in financial technology, digital healthcare, and other fields. There are more and more physical nodes deployed in cloud service clusters (such as kubernetes clusters). The number and frequency of each application container in the cloud service cluster being pulled up and destroyed has also increased significantly, which will generate a large amount of log data. The current log collection system usually needs to regularly scan all application containers running on physical nodes globally to collect log data, which leads to low log collection efficiency and large waste of resources in cloud service clusters. Summary of the invention

[0003] The embodiments of the present application provide a log collection method, apparatus, computer device, and computer-readable storage medium, which can improve log collection efficiency.

[0004] The present invention provides a log collection method, which includes:

[0005] The target physical node establishes a communication connection with the container cloud management client according to the verification identifier associated with the cloud service cluster; the target physical node belongs to the cloud service cluster;

[0006] When a change event for an application container occurs at a target physical node within a target period, container change information corresponding to the change event is generated; the target physical node includes multiple application containers;

[0007] Sending container change information to the container cloud management client through the cloud service cluster, so that the container cloud management client calls the event processing function corresponding to the container change information, and performs change processing on the first container group state record table associated with the target physical node according to the event processing function to obtain the second container group state record table;

[0008] The second container group status record table is obtained, and a new soft link corresponding to the target application container is generated in the target collection directory according to the second container group status record table; the target application container refers to the application container in which a change event exists in the second container group status record table; the new soft link is used to obtain the first log data of the target application container in the original directory; the original directory is the initial path of the first log data.

[0009] The present invention provides a log collection method, which includes:

[0010] The container cloud management client obtains the verification identifier associated with the cloud service cluster, and establishes a communication connection with the target physical node in the cloud service cluster according to the verification identifier;

[0011] When receiving container change information sent by the connected target physical node within the target period, calling the event processing function corresponding to the container change information, and performing change processing on the first container group state record table associated with the target physical node according to the event processing function to obtain the second container group state record table;

[0012] Among them, the second container group status record table is used to instruct the target physical node to generate a new soft link corresponding to the target application container in the target collection directory; the target application container refers to the application container for which a change event exists in the second container group status record table; the new soft link is used to obtain the first log data of the target application container in the original directory; the original directory is the initial path of the first log data.

[0013] An embodiment of the present application provides a log collection device, including:

[0014] A cloud service cluster communication module is used to establish a communication connection with a container cloud management client according to a verification identifier associated with the cloud service cluster; the target physical node belongs to the cloud service cluster;

[0015] A change information generation module, used for generating container change information corresponding to a change event when a change event for an application container occurs on a target physical node within a target period; the cloud service cluster includes multiple application containers;

[0016] An information sending module is used to send container change information to a container cloud management client, so that the container cloud management client calls an event processing function corresponding to the container change information, and performs change processing on a first container group state record table associated with a target physical node according to the event processing function to obtain a second container group state record table;

[0017] The soft link generation module is used to obtain the second container group status record table, and generate a new soft link corresponding to the target application container in the target collection directory according to the second container group status record table; the target application container refers to the application container with a change event in the second container group status record table; the new soft link is used to obtain the first log data of the target application container in the original directory; the original directory is the initial path of the first log data.

[0018] The soft link generation module includes:

[0019] a target application container determining unit, configured to traverse the second container group state record table, and determine the target application container according to the first container group state record table and the second container group state record table;

[0020] A log collection unit, configured to obtain first log data corresponding to a target application container in an original directory according to the second container group state record table;

[0021] The soft link generation unit is used to generate a new soft link corresponding to the first log data in the target collection directory.

[0022] The log collection unit includes:

[0023] The log data acquisition subunit is used to acquire the log data matching rule corresponding to the application container group where the target application container is located according to the second container group state record table, and acquire the first log data corresponding to the target application container in the original directory according to the log data matching rule.

[0024] The above device further comprises:

[0025] The storage module is used to store the newly created soft link in the target collection directory;

[0026] A storage module is used to obtain existing soft connections and newly created soft connections from a target collection directory through a log collection container in a log component, collect node log data from an original directory according to the existing soft connections and the newly created soft connections, and store the node log data in a storage device; the node log data includes first log data; the existing soft connection is a soft connection corresponding to an unchanged application container generated based on a first container group status record table; the unchanged application container refers to an application container for which no change event exists in a second container group status record table.

[0027] The above device further comprises:

[0028] A container group list acquisition module is used to acquire an existing container group list and an original container group list, wherein the existing container group list includes application container groups in a working state; and the original container group list includes each application container group in a target physical node;

[0029] A cleaning module, used to determine the application container in an invalid state according to the existing container group list and the original container group list, and determine the application container in the invalid state as a container to be deleted;

[0030] A cleaning module, used to clean up the second log data corresponding to the to-be-deleted container from the original directory, and determine the soft link corresponding to the second log data as an invalid soft link;

[0031] The cleaning module is used to traverse the soft links in the target collection directory and clean up the invalid soft links traversed.

[0032] The container group list acquisition module includes:

[0033] A container group acquisition unit, used to acquire a container group name corresponding to each application container group and a survival status corresponding to each application container group from the second container group status record table; the survival status includes a working status and an invalid status;

[0034] A container group acquisition unit, used to acquire a list of existing container groups through a cloud service cluster according to a container group name corresponding to each application container group and a survival status corresponding to each application container group;

[0035] The container group acquisition unit is used to acquire the original container group list corresponding to the target physical node.

[0036] The storage module includes:

[0037] The mounting unit is used to mount the target collection directory and configuration file to the collection subcomponent in the log collection container;

[0038] The soft link acquisition unit is used to acquire existing soft links and create new soft links from the target collection directory according to the configuration file.

[0039] An embodiment of the present application provides a log collection device, including:

[0040] The client communication module is used to obtain the verification identifier associated with the cloud service cluster and establish a communication connection with the target physical node in the cloud service cluster according to the verification identifier;

[0041] an event processing module, configured to, when receiving container change information sent by the connected target physical node within a target period, call an event processing function corresponding to the container change information, and perform change processing on a first container group state record table associated with the target physical node according to the event processing function to obtain a second container group state record table;

[0042] Among them, the second container group status record table is used to instruct the target physical node to generate a new soft connection corresponding to the target application container in the target collection directory; the target application container refers to the application container for which a change event exists in the second container group status record table; the new soft connection is used to instruct the log component to collect logs for the target application container.

[0043] The event processing module includes:

[0044] an updating unit, configured to, when receiving container change information sent by the connected target physical node within a target period and the container change information is container update information, call an update event processing function corresponding to the container update information, and perform update processing on a state record of a target application container indicated by the container update information in a first container group state record table according to the update event processing function;

[0045] a new adding unit, configured to, when receiving container change information sent by the connected target physical node within a target period and the container change information is container new information, call a new event processing function corresponding to the container new information, and perform new processing on the state record of the target application container indicated by the container new information in the first container group state record table according to the new event processing function;

[0046] The deleting unit is configured to, when receiving container change information sent by the connected target physical node within a target period and the container change information is container deletion information, call a deletion event processing function corresponding to the container deletion information, and perform deletion processing on the state record of the target application container indicated by the container deletion information in the first container group state record table according to the deletion event processing function.

[0047] An embodiment of the present application provides a computer device, including: a processor and a memory;

[0048] The above-mentioned processor is connected to the memory and the network interface, wherein the network interface is used to provide data communication function, the memory is used to store program code, and the processor is used to call the above-mentioned program code to execute the method provided in the first aspect of the embodiment of the present application.

[0049] On the one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. The computer program is suitable for being loaded and executed by a processor so that a computer device having the processor executes the method provided by the embodiment of the present application.

[0050] In one aspect, an embodiment of the present application provides a computer program product, which includes a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the method provided in the embodiment of the present application.

[0051] In an embodiment of the present application, the target physical node and the container cloud management client establish a communication connection, so that the container cloud management client performs a change process on the first container group state record table associated with the target physical node according to the container change information sent by the target physical node through the cloud service cluster, and obtains the second container group state record table. The sensitivity of the target physical node and the container cloud management client to perceive the application container change can be improved through the container group state record table. The target physical node can generate a new soft link corresponding to the target application container in a unified target collection directory. Among them, the target application container refers to the application container with a change event in the second container group state record table. That is to say, when the first container group state record table is updated, the application container with a change event can be determined as the target container by comparing the first container group state record table with the second container group state record table, and a new soft link corresponding to the target application container is generated in the target collection directory. Therefore, when there is a change event in the application container in the target physical node, a new soft link corresponding to the application container with the change event can be generated in the target collection directory, and then the log data corresponding to the application container with the state change is collected according to the new soft link. In addition, the creation of a new soft connection only needs to be determined by comparing the first container group status record table with the second container group status record table, and there is no need to perform a full scan of all application containers in the target physical node. Therefore, the present application can improve the log collection efficiency of the target physical node and has strong applicability. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a structural diagram of a network architecture provided by an embodiment of the present application;

[0053] Figure 2 is a structural diagram of a physical node provided in an embodiment of the present application;

[0054] Figure 3 This is a flow chart of a log collection method provided by an embodiment of the present application;

[0055] Figure 4 It is a data communication flow chart of the log collection method provided by an embodiment of the present application;

[0056] Figure 5 is another data communication flow chart of the log collection method provided by an embodiment of the present application;

[0057] Figure 6 is a structural schematic diagram of a log collection container provided in an embodiment of the present application;

[0058] Figure 7 is another data communication flow chart of the log collection method provided by an embodiment of the present application;

[0059] Figure 8This is another flow chart of the log collection method provided by the embodiment of the present application;

[0060] Fig. 9 is another data communication flow chart of the log collection method provided by an embodiment of the present application;

[0061] Fig.10 It is an interactive schematic diagram of the log collection method provided in an embodiment of the present application;

[0062] Fig.11 It is a structural schematic diagram of a log collection device provided in an embodiment of the present application;

[0063] Fig.12 is another structural schematic diagram of the log collection device provided in an embodiment of the present application;

[0064] Fig.13 is a structural schematic diagram of a computer device provided in an embodiment of the present application;

[0065] Fig.14 It is another structural schematic diagram of the computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0066] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0067] To facilitate understanding, some nouns are briefly explained as follows:

[0068] 1. Container cloud management tool (Kubernetes, abbreviated as k8s): It is an open source container cloud management tool from Google. The Kubernetes cluster includes a master node (Master) and a large number of physical nodes (Node). The physical node is a working machine in the Kubernetes cluster, usually a virtual machine or a physical machine. At the same time, each physical node can run multiple application container groups (pods), and each application container group can include multiple application containers.

[0069] 2. Master node: The Master node is the control node of the cluster and is responsible for the management and control of the entire cluster.

[0070] 3. Physical node: It is the host that runs a specific container and is responsible for providing specific services.

[0071] The implementation of the technical solution of the present application is further described in detail below in conjunction with the accompanying drawings.

[0072] See also Figure 1 , Figure 1 Schematic diagram of a network architecture provided by an embodiment of the present application. Figure 1 As shown, the system architecture may include a cloud service cluster 100a, a cloud server 100b, and a terminal device cluster, and the terminal device cluster may include: terminal devices 200b, terminal devices 200c, ..., terminal devices 200n and the like. Among them, the cloud service cluster 100a may include: a master node 10a, a physical node 10b, a physical node 10c, and a physical node 10d. It can be understood that the cloud service cluster 100a may include one or more physical nodes, and the embodiment of the present application does not limit the number of physical nodes. Among them, each physical node may include multiple application container groups, and each application container group may include multiple application containers. The embodiment of the present application does not limit the number of each application container group, nor does it limit the number of application containers included in each application container group. It can be understood that in the cloud service cluster 100a, the master node 10a, the physical node 10b, and the physical node 10d can all interact with data through the network connection between the physical node 10c. A network connection can also be established between any two application container groups in each physical node in the cloud service cluster 100a for data interaction.

[0073] It is understandable that the above system may include one or more cloud servers, and the number of cloud servers is not limited here. Figure 1 The cloud server 100b shown can be connected to the cloud service cluster 100a through a network so as to be able to exchange data through the network connection between the cloud service cluster 100a. The cloud server 100b can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud database, cloud service, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN, and big data and artificial intelligence platform.

[0074] The storage device 200a may be a device with data storage function, such as a cloud file server or Inter Planetary File System (IPFS) distributed storage, and the storage device 200a may provide data storage function for the terminal device cluster, the cloud server 100b, and the cloud service cluster 100a. For example, the storage device 200a may provide data storage function for any physical node in the cloud service cluster 100a.

[0075] The terminal device (including terminal device 200b, terminal device 200c, terminal device 200d, ..., terminal device 200n) can be a tablet computer, a smart phone, a laptop computer, a desktop computer, a PDA, a wearable device (such as a smart watch, a smart bracelet, etc.), a smart TV, a smart car, and other smart terminals. Among them, the cloud server 100b can establish a communication connection with each terminal device in the terminal device cluster, and a communication connection can also be established between each terminal device in the terminal device cluster. In other words, the cloud server 100b can establish a communication connection with each terminal device in the terminal device 200b, the terminal device 200c, the terminal device 200d, ..., and the terminal device 200n, for example, a communication connection can be established between the terminal device 200b and the cloud server 100b. A communication connection can be established between the terminal device 200b and the terminal device 200c, and a communication connection can also be established between the terminal device 200b and the terminal device 200d. At the same time, any terminal device in the terminal device cluster can have a network connection with the cloud service cluster 100a, for example, there is a network connection between the terminal device 200d and the cloud service cluster 100a. At the same time, any terminal device in the terminal device cluster can have a network connection with the storage device 200a, for example, there is a network connection between the terminal device 200n and the storage device 200a. Among them, the above-mentioned communication connection does not limit the connection method, and can be directly or indirectly connected through a wired communication method, or directly or indirectly connected through a wireless communication method, etc., which can be determined according to the actual application scenario, and this application does not limit it here.

[0076] It should be understood that Figure 1 Each terminal device in the terminal device cluster shown in the figure can be installed with a container cloud management client. When the container cloud management client runs in each terminal device, it can be respectively connected to the above Figure 1 Data interaction is performed between the cloud servers 100b shown. Among them, the container cloud management client is a client with the function of managing physical nodes in the cloud service cluster. The container cloud management client can be an independent client or an embedded sub-client integrated in a client (such as a log collection client, etc.), which is not limited here. Taking the container cloud management client as an example, the cloud server 100b can be a collection of multiple servers including a background server and a data processing server corresponding to the cloud service cluster. Therefore, each terminal device can transmit data with the cloud server 100b through the container cloud management client. For example, after the terminal device establishes a communication connection with any physical node in the cloud service cluster through the container cloud management client, the cloud server 100b can send container change information to the container cloud management client, and the terminal device can change the first container group status record table associated with the above-mentioned physical node according to the change information.

[0077] It is understandable that in the specific implementation of this application, related data such as user information is involved. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0078] It can be understood that the above-mentioned cloud server 100b, terminal device 200b, terminal device 200c, terminal device 200d, ..., terminal device 200n can all be physical nodes in the cloud service cluster 100a, and the log data described in the full text can be stored in the storage device 200a.

[0079] See also Figure 2 , Figure 2 is a schematic diagram of a physical node provided in an embodiment of the present application. To facilitate subsequent understanding and description, the embodiment of the present application can be Figure 1 In the cloud service cluster 100a shown, a physical node is selected as the target physical node, for example, the physical node 10b is used as the target physical node, that is, the cloud server 100b may be the server corresponding to the physical node 10b. Figure 2 As shown in Figure a in the figure, the physical node 10b may include a log component 101, an application container group 102 and an application container group 103. Each application container group may include multiple application containers, such as application container group 102 may include application container 102a, application container 102b, application container 102c and application container 102d. It should be understood that the present application does not limit the number of application container groups included in the physical node and the number of application containers in each application container group. Among them, in the process of cloud service cluster deployment, the log component can be deployed on each physical node in the form of DaemonSet. The DemeonSet is a resource type of the cloud service cluster. Through the deployment method of DaemonSet, the cloud service cluster will run the log component on each physical node that constitutes the cluster to collect the log data generated by all containers on the corresponding physical node. For example, the log component 101 deployed on the physical node 10b can collect the log data of each application container running on the physical node 10b.

[0080] To facilitate subsequent understanding and description, the embodiments of the present application can be Figure 1A terminal device is selected as the target terminal device in the terminal device cluster shown, for example, the terminal device 200c is used as the target terminal device. The cloud server 100b can establish a communication connection with the container cloud management client loaded on the terminal device 200c. When the cloud server 100b senses a change event of any application container through the target physical node (such as the physical node 10b), such as the addition event, deletion event and update event of the application container in the physical node 10b (that is, the addition, deletion and update of the log data corresponding to the application container), the cloud server 100b can generate the corresponding container change information according to the above change time, and send the container change information to the terminal device 200c. When the terminal device 200c receives the container change information sent by the cloud server 100b through the container cloud management client, it can call the event processing function corresponding to the container change information, and perform a change processing on the first container group state record table associated with the physical node 10b according to the event processing function to obtain the second container group state record table. Among them, the time point of the first container group state record table is before the time point of the second container group state record table. It should be understood that the cloud server 100b can obtain the path name of the log data corresponding to each application container in the physical node 10b according to the first container group status record table and the second container group status record table, that is, the log data corresponding to any application container can be obtained through the first container group status record table and the second container group status record table.

[0081] Further, Figure 1 The cloud server 100b shown can generate a new soft link corresponding to the target application container in the target collection directory according to the second container group status record table through the physical node 10b. The target application container refers to the application container whose status has changed in the second container group status record table, and the new soft link is used to obtain the first log data of the target application container in the original directory. The original directory is the initial path of the first log data. It should be understood that the cloud server 100b can obtain the first log data of the target application container in the original directory according to the new soft link through the physical node 10b.

[0082] It is understandable that the log collection method provided in the embodiment of the present application can collect log data corresponding to all application containers in each physical node. At the same time, when an application container change event occurs in the physical node, the log data corresponding to the application container with a state change can also be collected. Therefore, the log collection method provided in the embodiment of the present application can be applied to the situation where a large number of application containers are running on the same physical node, which improves the efficiency of log collection, reduces resource waste, and has stronger applicability.

[0083] The log collection method can be Figure 1 The cloud server in the Figure 1The terminal device in the Figure 1 The cloud server and the terminal device in the embodiment interact and execute. For ease of understanding, the present application embodiment takes the cloud server and the terminal device interacting and executing the method as an example for explanation. Figure 3 , Figure 3 This is a flow chart of the log collection method provided by the embodiment of the present application. Figure 3 In the log collection method shown in FIG. 1 , each step of log collection can be performed by a cloud server, such as Figure 3 As shown, the log collection method may at least include the following steps S101 to S104.

[0084] Step S101, the target physical node establishes a communication connection with the log management client according to the verification identifier associated with the cloud service cluster; the target physical node belongs to the cloud service cluster.

[0085] In some feasible implementations, the cloud server 100b can be determined by the target physical node (for the convenience of description, the target physical node is determined as Figure 1 The physical node 10b shown in the figure obtains the verification identifier associated with the cloud service cluster 100a, and establishes a communication connection with the container cloud management client based on the verification identifier associated with the cloud service cluster 100a. Exemplarily, the terminal device 200c can send a verification request for the cloud service cluster 100a to the cloud server 100b through the container cloud management client, and the cloud server 100b can return the verification identifier associated with the cloud service cluster 100a to the terminal device 200c. Among them, the verification identifier associated with the cloud service cluster 100a can carry the node information of the target physical node. Furthermore, the terminal device 200c can establish a communication connection with the physical node 10b in the cloud service cluster 100a through the container cloud management client based on the verification identifier, so that the cloud server 100b and the terminal device 200c can exchange data.

[0086] Step S102: when a change event for an application container occurs at a target physical node within a target period, container change information corresponding to the change event is generated; the target physical node includes a plurality of application containers.

[0087] In some possible implementations, such as Figure 4 As shown, Figure 4It is a data communication flow chart of the log collection method provided in an embodiment of the present application. After the cloud server 100b establishes a communication connection with the terminal device 200c through the physical node 10b, when the physical node 10b has a change event for the application container within the target period, the cloud server 100b can generate container change information corresponding to the change event through the physical node 10b. Exemplarily, the cloud server 100b can detect the existence of a change event for the application container in the physical node 10b within the target period through the master node 10a in the cloud service cluster 100a. It should be understood that the cloud server 100b can detect whether there is a change event for the application container in the cloud service cluster 100a according to the target period, and the terminal device 200c can also detect whether the change information returned by the cloud server 100b through the physical node 10b is received according to the target period. Among them, the change event can represent a container update event, a container addition event, and a container deletion event for the log data corresponding to the application container in the physical node 10b, and the change information may include container update information, container addition information, and container deletion information. As Figure 2 As shown in Figure a in FIG. 1 , the application container group and application container included in the physical node 10b before the change can be represented. Figure 2 As shown in FIG. b in FIG. 1 , the application container group and application container included in the physical node 10b after the change can be represented. For example, Figure 2 As shown in FIG. 1, when the log data corresponding to the application container 102a in the physical node 10b has a container update event, the cloud server 100b can generate the following through the physical node 10b: Figure 2 Similarly, when there is a container deletion event in the log data corresponding to the application container 103c in the physical node 10b, the application container group 103 includes the application container 103a, the application container 103b, and the application container 103c. Figure 2 For the log data corresponding to the application container 103c shown in FIG. b, the cloud server 100b can generate container deletion information for the application container 103c through the physical node 10b. Similarly, when there is a container addition event in the physical node 10b, such as adding log data corresponding to the application container 104a included in the application container group 104 on the basis of the original application container group 102 and the application container group 103, the cloud server 100b can generate container deletion information for the application container 104a through the physical node 10b.

[0088] Step S103: Send container change information to the container cloud management client through the cloud service cluster, so that the container cloud management client calls an event processing function corresponding to the container change information, and performs change processing on the first container group state record table associated with the target physical node according to the event processing function to obtain a second container group state record table.

[0089] In some feasible implementations, after the cloud server 100b generates container change information through the physical node 10b, the cloud server 100b can send the container change information to the container cloud management client through the cloud service cluster 100a. Specifically, the cloud server 100b can send the container change information to the terminal device 200c through the master node 10a in the cloud service cluster 100a. Further, the terminal device 200c can call the event processing function corresponding to the container change information through the container cloud management client, and perform change processing on the first container group status record table associated with the physical node 10b according to the event processing function to obtain the second container group status record table. Among them, the first container group status record table is used to obtain such as Figure 2 The node log data corresponding to each application container in the physical node 10b shown in FIG. a, the second container group state record table is used to obtain Figure 2 Node log data corresponding to each application container in the physical node 10b shown in Figure b.

[0090] Step S104, obtaining the second container group state record table, and generating a new soft link corresponding to the target application container in the target collection directory according to the second container group state record table; the target application container refers to the application container for which a change event exists in the second container group state record table; the new soft link is used to obtain the first log data of the target application container in the original directory; the original directory is the initial path of the first log data.

[0091] In some possible implementations, such as Figure 2 As shown in Figure a in the figure, the log component 101 may include a log discovery container 101a, a log collection container 101b, and a log cleaning container 101c. The cloud server 100b can traverse the second container group status record table through the log discovery container 101a in the physical node 10b, and determine the target application container according to the first container group status record table and the second container group status record table. Specifically, the cloud server 100b can obtain the node log data corresponding to each application container in the original directory corresponding to the first container group status record table according to the first container group status record table, and the cloud server 100b can obtain the node log data corresponding to each application container in the original directory corresponding to the second container group status record table according to the second container group status record table. Exemplarily, the cloud server 100b can obtain the node log data corresponding to each application container in the original directory corresponding to the second container group status record table according to the first container group status record table. Figure 2The node log data corresponding to each application container in the original directory in the physical node 10b shown in FIG. a, the cloud server 100b can also obtain the node log data according to the second container group status record table. Figure 2 The node log data corresponding to each application container in the original directory in the physical node 10b shown in Figure b. The original directory is the initial path name of the node log data. The cloud server 100b compares the physical node 10b with Figure 2 The node log data corresponding to each application container shown in Figure a is as follows Figure 2 The node log data corresponding to each application container shown in FIG. b is used to determine the application container with a change event in the second container group status record table as the target application container. Figure 2 As shown, the cloud server 100b may determine the application container 102a having a container update event, the application container 103c having a container deletion event, and the application container 104a having a container addition event as target application containers.

[0092] Furthermore, the cloud server 100b can discover the container 101a through the log in the physical node 10b, and obtain the first log data corresponding to the target application container in the original directory according to the second container group status record table. Figure 4 As shown, the cloud server 100b can discover the container 101a through the log in the physical node 10b, obtain the log data matching rule corresponding to the application container group where the target application container is located according to the second container group status record table, and obtain the first log data corresponding to the target application container in the original directory according to the log data matching rule. Among them, the second container group status record table records the status record corresponding to each application container group, and the cloud server 100b can obtain the log data matching rule of each application container group from the status record corresponding to each application container group. The cloud server 100b can collect the log data corresponding to each application container in the application container group from the original directory according to the log data matching rule corresponding to each application container group, that is, obtain the initial path name of the log data corresponding to each application container in the original directory. It should be understood that the present application does not limit the content of the log data matching rule, the number of log data matching rules corresponding to each application container group, and the number of log data corresponding to each application container.

[0093] Exemplarily, the cloud server 100b can discover the container 101a through the log in the physical node 10b, and obtain the log data matching rules corresponding to the application container group 102 where the application container 102a is located, the application container group 103 where the application container 103c is located, and the application container group 104 where the application container 104a is located according to the second container group state record table. For example, the log data matching rule can be a naming rule of the application container group. The cloud server 100b can obtain the first log data corresponding to the application container 102a in the application container group 102 in the original directory according to the naming rule of the application container group 102, obtain the first log data corresponding to the application container 103c in the application container group 103 in the original directory according to the naming rule of the application container group 103, and obtain the first log data corresponding to the application container 104a in the application container group 104 in the original directory according to the naming rule of the application container group 104a. Among them, since the first log data corresponding to the application container 103c in the application container group 103 has been deleted, the first log data corresponding to the application container 103c obtained by the cloud server 100b in the original directory is empty.

[0094] Furthermore, if Figure 4 As shown, cloud server 100b can discover container 101a through the log in physical node 10b, and generate a new soft connection corresponding to the first log data in the target collection directory. The new soft connection is used to obtain the first log data of the target application container under the original directory. Specifically, in the target collection directory, cloud server 100b can discover container 101a through the log in physical node 10b, generate a new soft connection corresponding to the first log data according to the first log data corresponding to application container 102a, generate a new soft connection corresponding to the first log data according to the first log data corresponding to application container 103c, and generate a new soft connection corresponding to the first log data according to the first log data corresponding to application container 104a. The new soft connection for the first log data corresponding to application container 103c is empty.

[0095] Furthermore, if Figure 4 As shown, the cloud server 100b can store the newly created soft connection in the target collection directory through the log discovery container 101a in the log component 101. Exemplarily, the cloud server 100b stores the newly created soft connection corresponding to the first log data 1, the newly created soft connection corresponding to the first log data 2, and the newly created soft connection corresponding to the first log data 3 in the target collection directory through the log discovery container 101a.

[0096] Furthermore, the cloud server 100b can obtain the existing soft connection and the newly created soft connection from the target collection directory through the log collection container 101b in the log component 101, collect the node log data from the original directory according to the existing soft connection and the newly created soft connection, and store the node log data in the memory 200a. Among them, the node log data includes the first log data, and the existing soft connection is the soft connection corresponding to the unchanged application container generated based on the first container group status record table. Among them, the unchanged application container refers to the application container for which there is no change event in the second container group status record table.

[0097] The process of the cloud server 100b generating an existing soft link based on the first container group state record table through the physical node 10b can be seen in Figure 5 , Figure 5 is another data communication flow chart of the log collection method provided by the embodiment of the present application. The cloud server 100b can discover the container 101a through the log in the log component 101, and obtain the status record corresponding to each application container group according to the first container group status record table. For example, Figure 2 As shown in Figure a, cloud server 100b can discover container 101a through logs, and obtain the status records corresponding to application container group 102 and application container group 103 respectively according to the first container group status record table. Further, cloud server 100b can discover container 101a through logs, and obtain the log data matching rules corresponding to each application container group according to the status records corresponding to each application container group. For example, cloud server 100b can discover container 101a through logs, and obtain the log data matching rules corresponding to application container group 102 and application container group 103 respectively according to the status records corresponding to application container group 102 and application container group 103 respectively. For example, the log data matching rule can be a naming rule for application container groups.

[0098] Furthermore, the cloud server 100b can discover the container 101a through the log, and obtain the node log data corresponding to each application container in each application container group from the original directory according to the log data matching rules corresponding to each application container group. Figure 2 As shown in FIG. a, cloud server 100b can obtain node log data corresponding to each application container in application container group 102 from the original directory according to the naming rule of application container group 102, and obtain node log data corresponding to each application container in application container group 103 from the original directory according to the naming rule of application container group 103. It should be understood that the present application does not limit the number of node log data corresponding to the application container.

[0099] Further, the cloud server 100b can discover the container 101a through the log, and generate an existing soft link corresponding to the node log data in the target collection directory. Figure 2 The existing soft links corresponding to the node log data corresponding to each application container shown in Figure a. It should be understood that the target collection directory contains the following Figure 2 At the time point of the second container group status record table, the existing soft connection obtained by the cloud server 100b from the collection directory through the log collection container 101b is the soft connection corresponding to the unchanged application container generated based on the first container group status record table. The unchanged application container refers to the application container for which there is no change event in the second container group status record table. Figure 2 It can be seen that the cloud server 100b obtains from the target collection directory the existing soft links corresponding to the application container 102b, the application container 102c, the application container 102d, the application container 103a and the application container 103b respectively.

[0100] It should be understood that the cloud server 100b finds through the log that the container 101a can process the status records in the first container group status record table one by one in sequence, that is, the cloud server 100b can process the status records in the first container group status record table in forward order, or can process the status records in the first container group status record table in reverse order.

[0101] Furthermore, the cloud server 100b can collect node log data from the original directory according to the existing soft connection and the newly created soft connection through the log collection container 101b in the physical node 10b, and store the node log data in the memory 200a. Figure 2 As shown, cloud server 100b collects application container 102b, application container 102c, application container 102d, application container 103a and node log data corresponding to application container 103b from the original directory through log collection container 101b according to the existing soft connection. Cloud server 100b collects first log data corresponding to application container 102a, application container 103c and application container 104a from the original directory through log collection container 101b according to the newly created soft connection. Among them, node log data includes first log data. Further, as Figure 4 As shown, the cloud server 100b stores the node log data in the memory 200a through the log collection container 101b.

[0102] The process of the log collection container 101b collecting node log data from the original directory can be seen in Figure 6 , Figure 6Schematic diagram of a log collection container provided in an embodiment of the present application. Figure 6 As shown, the cloud server 100b calls the container cloud management client loaded with the terminal device 200c through the diary collection container 101b in the physical node 10b to mount the target collection directory and the configuration file into the collection subcomponent in the log collection container. Among them, the configuration file is a pre-compiled file, and the configuration file is used to specify that the log collection container collects node log data from the target collection directory. For example, the cloud server 100b can mount the target collection directory to the collection subcomponent in the log collection container in the hostPath manner through the container cloud management client, and mount the configuration file to the collection subcomponent in the log collection container in the configMap manner. Exemplarily, the collection subcomponent can be a fluentd container or a fluent-bit container. Furthermore, in the collection subcomponent in the log collection container, the cloud server 100b can obtain existing soft connections and new soft connections from the target collection directory according to the configuration file.

[0103] In some possible implementations, such as Figure 4 As shown, after the cloud server 100b completes the collection task, the cloud server 100b can implement the cleanup task through the log cleanup container 101c in the physical node 10b. Figure 7 , Figure 7 FIG. 1 is another data communication flow chart of the log collection method provided by the embodiment of the present application. Figure 7 As shown, cloud server 100b can obtain the existing container group list and the original container group list through cloud service cluster 100a. The existing container group list includes the application container group in working state, and the original container group list includes each application container group in the target physical node. Specifically, cloud server 100b obtains the container group name corresponding to each application container group and the survival state corresponding to each application container group from the second container group state record table through physical node 10b. The survival state includes the working state and the failure state. For example, Figure 2 As shown in FIG. b, cloud server 100b can obtain container group names corresponding to application container group 102, application container group 103, and application container group 104 and survival status corresponding to application container group 102, application container group 103, and application container group 104 from the second container group status record table through physical node 10b. Figure 2 As shown in Figure b, since application container group 102, application container group 103 and application container group 104 all have application containers, it can be seen that the survival states corresponding to application container group 102, application container group 103 and application container group 104 are all working states.

[0104] Furthermore, if Figure 7As shown, cloud server 100b can obtain the existing container group list through the cloud service cluster according to the container group name corresponding to each application container group and the survival status corresponding to each application container group through physical node 10b. Specifically, cloud server 100b enables the cloud service cluster to filter the application container group whose survival status is working status according to the container group name corresponding to each application container group and the survival status corresponding to each application container group through physical node 10b, and obtain the existing container group list through the cloud service cluster. Among them, the existing container group list includes the application container group whose survival status is working status. Exemplarily, as Figure 2 As shown in FIG. b, the existing container group list includes application container group 102, application container group 103 and application container group 104, namely application container 102a, application container 102b, application container 102c, application container 102d, application container 103a, application container 103b and application container 104a. The original container group list includes Figure 2 All application containers shown, namely application container 102a, application container 102b, application container 102c, application container 102d, application container 103a, application container 103b, application container 103c and application container 104a.

[0105] Furthermore, if Figure 2 and Figure 7 As shown, the cloud server 100b determines the application container in an invalid state according to the existing container group list and the original container group list through the log cleaning container 101c in the physical node 10b, and determines the application container in an invalid state as a container to be deleted. Exemplarily, the cloud server 100b compares the existing container group list and the original container group list through the physical node 10b, determines that the application container 103c is in an invalid state, and determines the application container 103c as a container to be deleted. Further, the cloud server 100b cleans up the second log data corresponding to the container to be deleted from the original directory through the physical node 10b, and determines the soft connection corresponding to the second log data as an invalid soft connection. Exemplarily, the cloud server 100b determines the node log data corresponding to the application container 103c as the second log data through the physical node 10b, and determines the soft connection corresponding to the application container 103c as an invalid soft connection. Further, the cloud server 100b cleans up the second log data corresponding to the application container 103c from the original directory through the physical node 10b, traverses the soft links in the target collection directory, and cleans up the invalid soft links traversed.

[0106] The log collection method provided in this application can be applied to cloud game log collection. It should be understood that each application container in the cloud service cluster where the cloud game is located can run the game process. Therefore, the cloud service cluster has the characteristics of pulling up and destroying the application container many times, high frequency, and short intervals. With the expansion of the cloud service cluster business, the number of application containers included in the cloud service cluster has also increased sharply. Therefore, the log collection of cloud games will face the characteristics of large log collection data and large resource consumption. Please refer to Table 1, which is a cloud game log indicator data table provided by this application. Compared with the prior art, the log collection method provided in this application is suitable for large-scale log collection, that is, the log collection method provided in this application can adapt to the order of tens of billions. As shown in Table 1, the log collection method provided in this application can collect a total file size of about 200Tb, a total number of files of about 3 billion, can upload files about 12 billion times, and can manage about 10,000 physical nodes, among which a single physical node processes about 10,000 files per day on average, and the average file size can reach about 10,000.

[0107] index data Total file size About 200Tb Total number of files About 3 billion Number of file uploads About 12 billion times Manage the number of physical nodes About 10,000 Average number of files processed per day by a single physical node About 10,000 Average file size 70Kb

[0108] Table 1

[0109] It can be understood that the log collection method provided by the present application is suitable for business scenarios where the application container changes quickly and the application container is frequently pulled up and destroyed. By establishing a communication connection with the container cloud management client through the target physical node, the container cloud management client can change the first container group status record table associated with the target physical node through the container change information sent by the target physical node, so that the target physical node obtains the log data corresponding to the application container with the change event according to the second container group status record table. It is not necessary to perform a full scan of all application containers in the cloud service cluster on a regular basis. It is only necessary to obtain the log data corresponding to the unchanged application container through a full scan according to the first container group status record table. Therefore, the log collection method provided by the present application has the characteristics of low resource consumption and high log collection efficiency. In addition, the log collection method provided by the present application can also collect and clean up non-native services in the cloud service cluster on the basis of automatic log discovery, automatic log collection, and automatic log cleaning. At the same time, it is also suitable for the collection and cleaning of custom services, and has stronger applicability.

[0110] Above Figures 3 to 7 The execution process of the log collection method shown in FIG. 1 is based on the cloud server as the execution subject. The following uses the terminal device as the execution subject to specifically illustrate the terminal device side behavior of the log collection method through an embodiment. Figure 8 , Figure 8 FIG. 1 is another flow chart of the log collection method provided in the embodiment of the present application. Figure 8As shown, the method is applicable to a terminal device equipped with a container cloud management client (a client having a cloud service cluster management function, etc.), and includes the following steps:

[0111] Step S201: The container cloud management client obtains a verification identifier associated with the cloud service cluster, and establishes a communication connection with a target physical node in the cloud service cluster according to the verification identifier.

[0112] In some possible implementations, such as Figure 1 As shown, the terminal device 200c can send a verification request for the cloud service cluster 100a to the cloud server 100b through the container cloud management client. Exemplarily, the terminal user can log in to the account on the container cloud management client loaded on the terminal device 200c according to the account information, that is, the terminal device 200c needs to obtain the authorization of the terminal user before sending a verification request for the cloud service cluster 100a to the cloud server 100b through the container cloud management client, thereby enabling the terminal user to implement the log collection process through the container cloud management client. Among them, the above-mentioned verification information can carry the account information of the terminal user. When the cloud server 100b receives the verification request for the cloud service cluster 100a sent by the terminal device 200c through the container cloud management client, it will verify the account information carried by the verification request. If the verification is successful, a verification identifier associated with the cloud service cluster 100a is generated, and the verification identifier associated with the cloud service cluster 100a is returned to the terminal device 200c. If the verification fails, the verification failure information is returned to the terminal device 200c. Among them, the verification identifier associated with the cloud service cluster 100a can carry the target physical node (such as Figure 1 Furthermore, the terminal device 200c can establish a communication connection with the physical node 10b in the cloud service cluster through the container cloud management client according to the verification identifier.

[0113] Step S202: when container change information sent by the connected target physical node is received within the target period, an event processing function corresponding to the container change information is called, and a change processing is performed on the first container group state record table associated with the target physical node according to the event processing function to obtain a second container group state record table; wherein the second container group state record table is used to indicate that the target physical node generates a new soft connection corresponding to the target application container in the target collection directory; the target application container refers to an application container for which a change event exists in the second container group state record table; the new soft connection is used to obtain the first log data of the target application container in the original directory; the original directory is the initial path of the first log data.

[0114] In some possible implementations, such as Figure 1As shown, when the terminal device 200c receives the container change information sent by the connected target physical node through the container cloud management client within the target period, that is, when the terminal device 200c receives the container change information sent by the connected physical node 10b through the container cloud management client within the target period, the terminal device 200c can call the event processing function corresponding to the container change information through the container cloud management client, and perform a change process on the first container group state record table associated with the target physical node according to the event processing function to obtain the second container group state record table. It should be understood that the terminal device 200c can detect whether the container change information sent by the cloud server 100b is received through the container cloud management client according to the target period. In other words, the terminal device 200c can perform a change process on the first container group state record table according to the target period through the container cloud management client to obtain the second container group state record table, that is, the cloud server 100b can obtain the second container group state record table every time the target period passes. It should be understood that when the terminal device 200c does not receive the container change information sent by the cloud server 100b through the container cloud management client within the target period, at this time, the second container group state record table is the same as the first container group state record table.

[0115] Specifically, see Fig. 9 , Fig. 9 FIG. 1 is another data communication flow chart of the log collection method provided by the embodiment of the present application. Fig. 9 As shown, when the terminal device 200c receives the container change information sent by the connected physical node 10b through the container cloud management client within the target period, and the container change information is container update information, the update event processing function corresponding to the container update information is called, and the state record of the target application container indicated by the container update information is updated in the first container group state record table according to the update event processing function. Exemplarily, as Figure 2 As shown in FIG. 1, when the log data corresponding to the application container 102a in the physical node 10b has a container update event, the cloud server 100b can generate the following through the physical node 10b: Figure 2 Container update information for application container 102a as shown in FIG. b in FIG. When terminal device 200c receives container update information for application container 102a through container cloud management client within the target period, terminal device 200c calls update event processing function through container cloud management client according to container update information to update the status record of application container 102a in the first container group status record table, and obtains the second container group status record table.

[0116] Similarly, if Fig. 9As shown, when the terminal device 200c receives the container change information sent by the connected physical node 10b through the container cloud management client within the target period, and the container change information is the container new information, the new event processing function corresponding to the container new information is called, and the state record of the target application container indicated by the container new information is added in the first container group state record table according to the new event processing function. Exemplarily, as Figure 2 As shown, when there is a container addition event in the physical node 10b, such as adding log data corresponding to the application container 104a included in the application container group 104 on the basis of the original application container group 102 and the application container group 103, the cloud server 100b can generate container deletion information for the application container 104a through the physical node 10b. When the terminal device 200c receives the container addition information for the application container 104a included in the application container group 104 through the container cloud management client within the target period, the terminal device 200c calls the new event processing function through the container cloud management client to add the status record of the application container 104a included in the application container group 104 to the first container group status record table according to the container addition information, and obtains the second container group status record table.

[0117] Similarly, if Fig. 9 As shown, when the terminal device 200c receives the container change information sent by the connected physical node 10b through the container cloud management client within the target period, and the container change information is container deletion information, the deletion event processing function corresponding to the container deletion information is called, and the state record of the target application container indicated by the container deletion information is deleted in the first container group state record table according to the deletion event processing function. For example, Figure 2 As shown, when the log data corresponding to the application container 103c in the physical node 10b has a container deletion event, the application container 103a, the application container 103b and the application container 103c included in the application container group 103 are deleted. Figure 2 For the log data corresponding to the application container 103c shown in FIG. b, the cloud server 100b can generate container deletion information for the application container 103c through the physical node 10b. When the terminal device 200c receives the container deletion information of the application container 103c included in the application container group 103 through the container cloud management client within the target period, the terminal device 200c calls the deletion event processing function through the container cloud management client according to the container deletion information to delete the status record of the application container 103c included in the application container group 103 in the first container group status record table, and obtains the second container group status record table.

[0118] It can be understood that the log collection method provided by the present application establishes a communication connection with the container cloud management client through the target physical node. The terminal device can determine the type of change event of any application container in the target physical node associated with the terminal device based on the container change information sent by the cloud server, which can increase the user stickiness between the terminal user and the container cloud management client, and can also improve the terminal device's perception of changes in application containers on the cloud service cluster, thereby increasing the flexibility of log collection and improving the efficiency of log collection.

[0119] Please also see Fig.10 , Fig.10 It is an interactive schematic diagram of the log collection method provided in an embodiment of the present application. Fig.10 Taking the case of a change event in the physical node 10b in chronological order as an example, the present application is described in detail. The system between the terminal device and the cloud server can adopt the client / server (Client / Server, C / S) mode or the browser / server (Browser / Server, B / S) mode. The log collection method includes:

[0120] Step 1. End user login account information.

[0121] In some feasible implementations, the terminal user can log in to the account according to the account information on the container cloud management client loaded on the terminal device 200c. It should be understood that before logging in the account information, the terminal user needs to register the account on the container cloud management client to generate the account information. After the terminal user logs in the account information on the container cloud management client, the terminal device 200c can implement the log collection process through the operation interface of the container cloud management client. Exemplarily, the terminal user can choose to collect logs for the cloud service cluster 100a on the operation interface of the container cloud management client.

[0122] Step 2: The terminal device 200c sends a verification request for the cloud service cluster 100a to the cloud server 100b.

[0123] In some feasible implementations, after the terminal user logs in the account information on the container cloud management client through the terminal device 200c, the terminal device 200c can send a verification request for the cloud service cluster 100a to the cloud server 100b through the container cloud management client. The verification information can carry the account information of the terminal user.

[0124] Step 3: The cloud server 100b verifies the account information carried in the verification request.

[0125] In some feasible implementations, when the cloud server 100b receives the verification request for the cloud service cluster 100a sent by the terminal device 200c through the container cloud management client, it will verify the account information carried in the verification request. If the verification fails, step 4 is executed, and the cloud server 100b returns the verification failure information to the terminal device 200c. If the verification is successful, step 5 is executed, that is, the verification identifier associated with the cloud service cluster 100a is generated, and the verification identifier associated with the cloud service cluster 100a is returned to the terminal device 200c.

[0126] Step 4: The cloud server 100b returns verification failure information to the terminal device 200c.

[0127] In some feasible implementations, when the cloud server 100b fails to verify the account information carried in the verification request, the cloud server 100b returns verification failure information to the terminal device 200c.

[0128] Step 5: The cloud server 100b generates a verification identifier associated with the cloud service cluster 100a.

[0129] In some feasible implementations, when the cloud server 100b successfully verifies the account information carried in the verification request, the cloud server 100b generates a verification identifier associated with the cloud service cluster 100a.

[0130] Step 6: The cloud server 100b returns the verification identifier associated with the cloud service cluster 100a to the terminal device 200c.

[0131] Step 7: The terminal device 200c establishes a communication connection with the physical node 10b in the cloud service cluster through the container cloud management client according to the verification identifier.

[0132] Step 8: The cloud server 100b obtains the status record corresponding to each application container group according to the first container group status record table associated with the physical node 10b.

[0133] In some feasible implementations, the cloud server 100b obtains the status record corresponding to each application container group according to the first container group status record table associated with the physical node 10b. The present application does not limit the number of application container groups included in the physical node 10b and the number of application containers included in each application container group. For example, Figure 2 As shown in Figure a, the cloud server 100b can discover the container 101a through the log in the physical node 10b, and obtain the status records corresponding to the application container group 102 and the application container group 103 according to the first container group status record table.

[0134] Step 9: The cloud server 100b can obtain the log data matching rules corresponding to each application container group through the physical node 10b according to the status records corresponding to each application container group.

[0135] In some possible implementations, such as Figure 2 As shown in Figure a, cloud server 100b can discover container 101a through logs, and obtain log data matching rules corresponding to application container group 102 and application container group 103 respectively according to the status records corresponding to application container group 102 and application container group 103 respectively. For example, the log data matching rule can be a naming rule for the application container group.

[0136] Step 10: The cloud server 100b can obtain the node log data corresponding to each application container in each application container group from the original directory through the physical node 10b according to the log data matching rules corresponding to each application container group.

[0137] In some possible implementations, such as Figure 2 As shown in Figure a, cloud server 100b can obtain node log data corresponding to each application container in application container group 102 from the original directory according to the naming rule of application container group 102, and obtain node log data corresponding to each application container in application container group 103 from the original directory according to the naming rule of application container group 103. It should be understood that the present application does not limit the number of node log data corresponding to application containers. Exemplarily, cloud server 100b can obtain node log data corresponding to application container 102a, application container 102b, application container 102c, and application container 102d in application container group 102 through log discovery container 101a, and obtain node log data corresponding to application container 103a, application container 103b, and application container 103c in application container group 103.

[0138] Step 11: The cloud server 100b may generate an existing soft link corresponding to the node log data in the target collection directory through the physical node 10b.

[0139] In some feasible implementations, the cloud server 100b can discover the container 101a through the log in the physical node 10b, and generate the following in the target collection directory: Figure 2 The existing soft links corresponding to the node log data corresponding to each application container shown in Figure a. It should be understood that the target collection directory contains the following Figure 2At the time point of the second container group status record table, the existing soft connection obtained by the cloud server 100b from the collection directory through the log collection container 101b is the soft connection corresponding to the unchanged application container generated based on the first container group status record table. The unchanged application container refers to the application container for which there is no change event in the second container group status record table. Figure 2 It can be seen that the cloud server 100b obtains from the target collection directory the existing soft links corresponding to the application container 102b, the application container 102c, the application container 102d, the application container 103a and the application container 103b respectively.

[0140] Step 12: The cloud server 100b detects a change event for the application container within the target period through the physical node 10b.

[0141] In some feasible implementations, the cloud server 100b can detect whether there is a change event for the application container in the cloud service cluster 100a according to the target period, that is, the cloud server 100b can detect the existence of a change event for the application container through the physical node 10b within the target period. The change event can represent a container update event, a container addition event, and a container deletion event in which the log data corresponding to the application container exists in the physical node 10b.

[0142] Step 13: The cloud server 100b generates container change information corresponding to the change event through the physical node 10b.

[0143] In some feasible implementations, for example, Figure 2 As shown in Figure a of FIG, when cloud server 100b can detect the presence of a change event for an application container through physical node 10b within a target period, cloud server 100b can generate container change information corresponding to the change event. The change information may include container update information, container addition information, and container deletion information. For example, Figure 2 As shown in FIG. a, the log data corresponding to the application container 102a in the physical node 10b contains a container update event. Similarly, the log data corresponding to the application container 103c in the physical node 10b contains a container deletion event, that is, the application container 103a, the application container 103b, and the application container 103c included in the application container group 103 are deleted. Figure 2 Similarly, when there is a container addition event in the physical node 10b, for example, log data corresponding to the application container 104a included in the application container group 104 is added to the original application container group 102 and the application container group 103.

[0144] Step 14: The cloud server 100b sends container change information to the container cloud management client.

[0145] In some feasible implementations, the cloud server 100b may send container change information to the container cloud management client through the cloud service cluster 100a. Specifically, the cloud server 100b may send container change information to the terminal device 200c through the master node 10a in the cloud service cluster 100a.

[0146] Step 15: The terminal device 200c may call the event processing function corresponding to the container change information through the container cloud management client.

[0147] In some feasible implementations, the terminal device 200c may call an event processing function corresponding to the container change information through the container cloud management client, wherein the event processing function may include an update event processing function, a delete event processing function, and a new event processing function.

[0148] Step 16: The terminal device 200c performs a change process on the first container group state record table associated with the physical node 10b according to the event processing function to obtain a second container group state record table.

[0149] In some possible implementations, such as Fig. 9 As shown, when the terminal device 200c receives the container change information sent by the connected physical node 10b through the container cloud management client within the target period, and the container change information is container update information, the update event processing function corresponding to the container update information is called, and the state record of the target application container indicated by the container update information is updated in the first container group state record table according to the update event processing function. Exemplarily, as Figure 2 As shown in FIG. 1, when the log data corresponding to the application container 102a in the physical node 10b has a container update event, the cloud server 100b can generate the following through the physical node 10b: Figure 2 Container update information for application container 102a as shown in FIG. b in FIG. When terminal device 200c receives container update information for application container 102a through container cloud management client within the target period, terminal device 200c calls update event processing function through container cloud management client according to container update information to update the status record of application container 102a in the first container group status record table, and obtains the second container group status record table.

[0150] Similarly, if Fig. 9As shown, when the terminal device 200c receives the container change information sent by the connected physical node 10b through the container cloud management client within the target period, and the container change information is the container new information, the new event processing function corresponding to the container new information is called, and the state record of the target application container indicated by the container new information is added in the first container group state record table according to the new event processing function. Exemplarily, as Figure 2 As shown, when there is a container addition event in the physical node 10b, such as adding log data corresponding to the application container 104a included in the application container group 104 on the basis of the original application container group 102 and the application container group 103, the cloud server 100b can generate container deletion information for the application container 104a through the physical node 10b. When the terminal device 200c receives the container addition information for the application container 104a included in the application container group 104 through the container cloud management client within the target period, the terminal device 200c calls the new event processing function through the container cloud management client to add the status record of the application container 104a included in the application container group 104 to the first container group status record table according to the container addition information, and obtains the second container group status record table.

[0151] Similarly, if Fig. 9 As shown, when the terminal device 200c receives the container change information sent by the connected physical node 10b through the container cloud management client within the target period, and the container change information is container deletion information, the deletion event processing function corresponding to the container deletion information is called, and the state record of the target application container indicated by the container deletion information is deleted in the first container group state record table according to the deletion event processing function. For example, Figure 2 As shown, when the log data corresponding to the application container 103c in the physical node 10b has a container deletion event, the application container 103a, the application container 103b and the application container 103c included in the application container group 103 are deleted. Figure 2 For the log data corresponding to the application container 103c shown in FIG. b, the cloud server 100b can generate container deletion information for the application container 103c through the physical node 10b. When the terminal device 200c receives the container deletion information of the application container 103c included in the application container group 103 through the container cloud management client within the target period, the terminal device 200c calls the deletion event processing function through the container cloud management client according to the container deletion information to delete the status record of the application container 103c included in the application container group 103 in the first container group status record table, and obtains the second container group status record table.

[0152] Step 17: The cloud server 100b may traverse the second container group state record table through the physical node 10b, and determine the target application container according to the first container group state record table and the second container group state record table.

[0153] In some feasible implementations, the cloud server 100b can find the container 101a through the log in the physical node 10b, and traverse the second container group status record table, and determine the target application container according to the first container group status record table and the second container group status record table. Specifically, the cloud server 100b can obtain the node log data corresponding to each application container in the first container group status record table and the second container group status record table in the original directory according to the first container group status record table and the second container group status record table. Exemplarily, the cloud server 100b can obtain the following according to the first container group status record table: Figure 2 The node log data corresponding to each application container in the original directory in the physical node 10b shown in FIG. a, the cloud server 100b can also obtain the node log data according to the second container group status record table. Figure 2 The node log data corresponding to each application container in the original directory in the physical node 10b shown in Figure b. The original directory is the initial path name of the node log data. The cloud server 100b compares the physical node 10b with Figure 2 The node log data corresponding to each application container shown in Figure a is as follows Figure 2 The node log data corresponding to each application container shown in FIG. b is used to determine the application container with a change event in the second container group status record table as the target application container. Figure 2 As shown, the cloud server 100b may determine the application container 102a having a container update event, the application container 103c having a container deletion event, and the application container 104a having a container addition event as target application containers.

[0154] Step 18: The cloud server 100b obtains the first log data corresponding to the target application container in the original directory through the physical node 10b according to the second container group status record table.

[0155] In some feasible implementations, the cloud server 100b can discover the container 101a through the log in the physical node 10b, and obtain the first log data corresponding to the target application container in the original directory according to the second container group state record table. Figure 4As shown, the cloud server 100b can discover the container 101a through the log in the physical node 10b, obtain the log data matching rule corresponding to the application container group where the target application container is located according to the second container group state record table, and obtain the first log data corresponding to the target application container in the original directory according to the log data matching rule. The cloud server 100b can collect the log data corresponding to each application container in the application container group from the original directory according to the log data matching rule corresponding to each application container group, that is, obtain the initial path name of the log data corresponding to each application container in the original directory.

[0156] Exemplarily, cloud server 100b may obtain the first log data corresponding to application container 102a in application container group 102 in the original directory according to the naming rule of application container group 102, obtain the first log data corresponding to application container 103c in application container group 103 in the original directory according to the naming rule of application container group 103, and obtain the first log data corresponding to application container 104a in application container group 104 in the original directory according to the naming rule of application container group 104a. Since the first log data corresponding to application container 103c in application container group 103 has been deleted, the first log data corresponding to application container 103c obtained by cloud server 100b in the original directory is empty.

[0157] Step 19: The cloud server 100b generates a new soft link corresponding to the first log data in the target collection directory through the physical node 10b.

[0158] In some feasible implementations, the cloud server 100b can discover the container 101a through the log in the physical node 10b, and generate a new soft connection corresponding to the first log data in the target collection directory. The cloud server 100b can generate a new soft connection corresponding to the first log data in the target collection directory through the physical node 10b. Specifically, in the target collection directory, the cloud server 100b can discover the container 101a through the log in the physical node 10b, generate a new soft connection corresponding to the first log data according to the first log data corresponding to the application container 102a, generate a new soft connection corresponding to the first log data according to the first log data corresponding to the application container 103c, and generate a new soft connection corresponding to the first log data according to the first log data corresponding to the application container 104a. Among them, the new soft connection for the first log data corresponding to the application container 103c is empty.

[0159] Step 20: The cloud server 100b may store the newly created soft connection in the target collection directory through the physical node 10b.

[0160] In some feasible implementations, the cloud server 100b can discover the container 101a through the log in the physical node 10b, and store the newly created soft connection in the target collection directory. Exemplarily, the cloud server 100b can discover the container 101a through the log in the physical node 10b, and store the newly created soft connection corresponding to the application container 102a, the newly created soft connection corresponding to the application container 103c, and the newly created soft connection corresponding to the application container 104a in the target collection directory.

[0161] Step 21, the cloud server 100b can obtain existing soft connections and newly created soft connections from the target collection directory through the log collection container 101b in the physical node 10b, collect node log data from the original directory according to the existing soft connections and newly created soft connections, and store the node log data in the memory 200a.

[0162] In some feasible implementations, the cloud server 100b can obtain existing soft connections and newly created soft connections from the target collection directory through the log collection container 101b in the log component 101, collect node log data from the original directory based on the existing soft connections and the newly created soft connections, and store the node log data in the memory 200a.

[0163] For example, Figure 2 As shown, cloud server 100b collects application container 102b, application container 102c, application container 102d, application container 103a and node log data corresponding to application container 103b from the original directory through log collection container 101b according to the existing soft connection. Cloud server 100b collects first log data corresponding to application container 102a, application container 103c and application container 104a from the original directory through log collection container 101b according to the newly created soft connection. Among them, node log data includes first log data. Further, as Figure 4 As shown, the cloud server 100b stores the node log data in the memory 200a through the log collection container 101b.

[0164] Step 22: The cloud server 100b obtains the existing container group list and the original container group list.

[0165] In some feasible implementations, the existing container group list includes application container groups in a working state, and the original container group list includes each application container group in the target physical node. Specifically, the cloud server 100b obtains the container group name corresponding to each application container group and the survival state corresponding to each application container group from the second container group state record table through the physical node 10b. The survival state includes a working state and an invalid state.

[0166] Furthermore, if Figure 7As shown, the cloud server 100b enables the cloud service cluster to filter the application container groups whose survival status is working status according to the container group name corresponding to each application container group and the survival status corresponding to each application container group through the physical node 10b, and obtain the existing container group list through the cloud service cluster. For example, Figure 2 As shown in FIG. b, the existing container group list includes application container group 102, application container group 103 and application container group 104, namely application container 102a, application container 102b, application container 102c, application container 102d, application container 103a, application container 103b and application container 104a. The original container group list includes Figure 2 All application containers shown, namely application container 102a, application container 102b, application container 102c, application container 102d, application container 103a, application container 103b, application container 103c and application container 104a.

[0167] Step 23: The cloud server 100b determines the application container in a failed state through the physical node 10b according to the existing container group list and the original container group list, and determines the application container in the failed state as a container to be deleted.

[0168] In some feasible implementations, the cloud server 100b determines the application container in a failed state according to the existing container group list and the original container group list through the log cleaning container 101c in the physical node 10b, and determines the application container in a failed state as a container to be deleted. Exemplarily, the cloud server 100b compares the existing container group list and the original container group list through the physical node 10b, determines that the application container 103c is in a failed state, and determines the application container 103c as a container to be deleted.

[0169] Step 24: The cloud server 100b cleans up the second log data corresponding to the container to be deleted from the original directory through the physical node 10b, and determines the soft link corresponding to the second log data as an invalid soft link.

[0170] In some possible implementations, such as Figure 2 As shown, the cloud server 100b determines the node log data corresponding to the application container 103c as the second log data through the physical node 10b, and determines the soft link corresponding to the second log data as an invalid soft link. Further, the cloud server 100b cleans up the second log data corresponding to the application container 103c from the original directory through the physical node 10b.

[0171] Step 25: The cloud server 100b traverses the soft links in the target collection directory through the physical node 10b, and clears the invalid soft links traversed.

[0172] In some feasible implementations, the cloud server 100b traverses the soft links in the target collection directory through the physical node 10b and cleans up the invalid soft links corresponding to the application container 103c.

[0173] It can be understood that the log collection method provided by the present application is suitable for business scenarios where the application container changes quickly and the application container is frequently pulled up and destroyed. By establishing a communication connection with the container cloud management client through the target physical node, the container cloud management client can change the first container group status record table associated with the target physical node through the container change information sent by the target physical node, so that the target physical node obtains the log data corresponding to the application container with the change event according to the second container group status record table. It is not necessary to perform a full scan of all application containers in the cloud service cluster on a regular basis. It is only necessary to obtain the log data corresponding to the unchanged application container through a full scan according to the first container group status record table. Therefore, the log collection method provided by the present application has the characteristics of low resource consumption and high log collection efficiency. In addition, the log collection method provided by the present application can also collect and clean up non-native services in the cloud service cluster on the basis of automatic log discovery, automatic log collection, and automatic log cleaning. At the same time, it is also suitable for the collection and cleaning of custom services, and has stronger applicability.

[0174] See also Fig.11 , Fig.11 Schematic diagram of a log collection device provided in an embodiment of the present application. The log collection device may be a computer program (including program code) running in a computer device, for example, the log collection device is an application software; the device may be used to execute the corresponding steps in the method provided in an embodiment of the present application. Fig.11 As shown, the log collection device 1 may include: a cloud service cluster communication module 11 , a change information generation module 12 , an information sending module 13 and a soft connection generation module 14 .

[0175] The cloud service cluster communication module 11 is used to establish a communication connection with the container cloud management client according to the verification identifier associated with the cloud service cluster; the target physical node belongs to the cloud service cluster;

[0176] A change information generating module 12 is used to generate container change information corresponding to a change event when a change event for an application container occurs at a target physical node within a target period; the cloud service cluster includes multiple application containers;

[0177] An information sending module 13 is used to send container change information to the container cloud management client, so that the container cloud management client calls an event processing function corresponding to the container change information, and performs change processing on the first container group state record table associated with the target physical node according to the event processing function to obtain a second container group state record table;

[0178] The soft link generation module 14 is used to obtain the second container group status record table, and generate a new soft link corresponding to the target application container in the target collection directory according to the second container group status record table; the target application container refers to the application container with a change event in the second container group status record table; the new soft link is used to obtain the first log data of the target application container in the original directory; the original directory is the initial path of the first log data.

[0179] In some feasible implementations, the specific functional implementation of the cloud service cluster communication module 11, the change information generation module 12, the information sending module 13 and the soft connection generation module 14 can be referred to above. Figure 3 Steps S101 to S104 in the corresponding embodiment will not be described in detail here.

[0180] The soft connection generation module 14 includes:

[0181] A target application container determining unit 141 is configured to traverse the second container group state record table and determine the target application container according to the first container group state record table and the second container group state record table;

[0182] The log collection unit 142 is used to obtain the first log data corresponding to the target application container in the original directory according to the second container group state record table;

[0183] The soft link generation unit 143 is used to generate a new soft link corresponding to the first log data in the target collection directory.

[0184] The log collection unit 142 includes:

[0185] The log data acquisition subunit 1421 is used to acquire the log data matching rule corresponding to the application container group where the target application container is located according to the second container group state record table, and acquire the first log data corresponding to the target application container in the original directory according to the log data matching rule.

[0186] The above device further comprises:

[0187] Storage module 15, used to store the newly created soft link in the target collection directory;

[0188] The storage module 15 is used to obtain existing soft connections and newly created soft connections from the target collection directory through the log collection container in the log component, collect node log data from the original directory according to the existing soft connections and the newly created soft connections, and store the node log data in the memory; the node log data includes first log data; the existing soft connection is a soft connection corresponding to the unchanged application container generated based on the first container group status record table; the unchanged application container refers to an application container for which no change event exists in the second container group status record table.

[0189] The above device further comprises:

[0190] The container group list acquisition module 16 is used to acquire an existing container group list and an original container group list, wherein the existing container group list includes application container groups in working state; and the original container group list includes each application container group in the target physical node;

[0191] A cleaning module 17 is used to determine the application container in the invalid state according to the existing container group list and the original container group list, and determine the application container in the invalid state as a container to be deleted;

[0192] A cleaning module 17, used to clean up the second log data corresponding to the to-be-deleted container from the original directory, and determine the soft link corresponding to the second log data as an invalid soft link;

[0193] The cleaning module 17 is used to traverse the soft links in the target collection directory and clean up the invalid soft links traversed.

[0194] The container group list acquisition module 16 includes:

[0195] The container group acquisition unit 161 is used to acquire the container group name corresponding to each application container group and the survival status corresponding to each application container group from the second container group status record table; the survival status includes a working status and an invalid status;

[0196] The container group acquisition unit 161 is used to acquire an existing container group list through the cloud service cluster according to the container group name corresponding to each application container group and the survival status corresponding to each application container group;

[0197] The container group acquisition unit 161 is used to acquire the original container group list corresponding to the target physical node.

[0198] Wherein, the storage module 15 includes:

[0199] The mounting unit 151 is used to mount the target collection directory and configuration file to the collection subcomponent in the log collection container;

[0200] The soft link acquisition unit 152 is used to acquire existing soft links and newly create soft links from the target collection directory according to the configuration file.

[0201] In the embodiment of the present application, the functional implementation of each unit in the log collection device 1 can be applied to business scenarios where the application container changes quickly and the application container is frequently pulled up and destroyed. A communication connection is established between the target physical node and the container cloud management client. The container cloud management client can change the first container group status record table associated with the target physical node through the container change information sent by the target physical node, so that the target physical node obtains the log data corresponding to the application container with the change event according to the second container group status record table. It is not necessary to perform a full scan of all application containers in the cloud service cluster on a regular basis. It is only necessary to obtain the log data corresponding to the unchanged application container through a full scan according to the first container group status record table. Therefore, the log collection method provided by the present application has the characteristics of low resource consumption and high log collection efficiency. In addition, the log collection method provided by the present application can also collect and clean up non-native services in the cloud service cluster on the basis of automatic log discovery, automatic log collection, and automatic log cleaning. At the same time, it is also suitable for the collection and cleaning of custom services, and has stronger applicability.

[0202] See also Fig.12 , Fig.12 is another structural diagram of the log collection device provided in the embodiment of the present application. The above-mentioned log collection device can be a computer program (including program code) running in a computer device, for example, the log collection device is an application software; the device can be used to execute the corresponding steps of the method provided in the embodiment of the present application. Fig.12 As shown, the log collection device 2 can be run on a terminal device, which can be the above-mentioned Figure 8 The terminal device in the corresponding embodiment. The log collection device 2 may include: a client communication module 21 and an event processing module 22.

[0203] The client communication module 21 is used to obtain the verification identifier associated with the cloud service cluster and establish a communication connection with the target physical node in the cloud service cluster according to the verification identifier;

[0204] The event processing module 22 is used to call the event processing function corresponding to the container change information when receiving the container change information sent by the connected target physical node within the target period, and perform change processing on the first container group state record table associated with the target physical node according to the event processing function to obtain the second container group state record table;

[0205] Among them, the second container group status record table is used to instruct the target physical node to generate a new soft connection corresponding to the target application container in the target collection directory; the target application container refers to the application container for which a change event exists in the second container group status record table; the new soft connection is used to instruct the log component to collect logs for the target application container.

[0206] The event processing module 22 includes:

[0207] The updating unit 221 is configured to, when receiving container change information sent by the connected target physical node within a target period and the container change information is container update information, call an update event processing function corresponding to the container update information, and perform update processing on the state record of the target application container indicated by the container update information in the first container group state record table according to the update event processing function;

[0208] The adding unit 222 is configured to, when receiving container change information sent by the connected target physical node within the target period and the container change information is container new information, call a new event processing function corresponding to the container new information, and perform new processing on the state record of the target application container indicated by the container new information in the first container group state record table according to the new event processing function;

[0209] The deleting unit 223 is used to call a deletion event processing function corresponding to the container deletion information when the container change information sent by the connected target physical node is received within the target period, and the container change information is container deletion information, and delete the state record of the target application container indicated by the container deletion information in the first container group state record table according to the deletion event processing function.

[0210] In some feasible implementations, the specific functional implementation of the client communication module 21 and the event processing module 22 can be referred to above. Figure 8 The steps S201 and S202 in the corresponding embodiment will not be described in detail here.

[0211] In the embodiment of the present application, the functional implementation of each unit in the log collection device 2 can realize the interaction between the container cloud management client and the cloud service cluster, and the cloud service cluster can display the node data or log data on the cloud service cluster to the terminal user, which enhances the visualization effect of log collection and makes it more applicable. Based on the functional implementation of each unit in the log collection device 2 in the embodiment of the present application, the state perception efficiency of the terminal user for log collection can be improved, and the instant perception of application container changes in the cloud service cluster can be enhanced.

[0212] See also Fig.13 , Fig.13It is a structural diagram of a computer device provided in an embodiment of the present application, and the computer device may be a terminal device or a server. The computer device 1000 may include: a processor 1001, a network interface 1004 and a memory 1005. In addition, the above-mentioned computer device 1000 may also include: a transceiver 1003, and at least one communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. Among them, in some embodiments, the transceiver 1003 may include a display screen (Display), a keyboard (Keyboard), and the optional transceiver 1003 may also include a standard wired interface and a wireless interface. Optionally, the network interface 1004 may include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or it may be a non-volatile memory (non-volatile memory), such as at least one disk storage. Optionally, the memory 1005 may also be at least one storage device located away from the aforementioned processor 1001. As Fig.13 As shown, the memory 1005 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module, and a device control application program.

[0213] In such Fig.13 In the computer device 1000 shown, the network interface 1004 can provide a network communication function; the transceiver 1003 is mainly used to provide an input interface for the user; and the processor 1001 can be used to call the device control application stored in the memory 1005 to achieve:

[0214] The target physical node establishes a communication connection with the log management client according to the verification identifier associated with the cloud service cluster; the target physical node belongs to the cloud service cluster;

[0215] When a change event for an application container occurs at a target physical node within a target period, container change information corresponding to the change event is generated; the target physical node includes multiple application containers;

[0216] Sending container change information to the container cloud management client through the cloud service cluster, so that the container cloud management client calls the event processing function corresponding to the container change information, and performs change processing on the first container group state record table associated with the target physical node according to the event processing function to obtain the second container group state record table;

[0217] The second container group status record table is obtained, and a new soft link corresponding to the target application container is generated in the target collection directory according to the second container group status record table; the target application container refers to the application container in which a change event exists in the second container group status record table; the new soft link is used to obtain the first log data of the target application container in the original directory; the original directory is the initial path of the first log data.

[0218] It should be understood that the computer device 1000 described in the embodiment of the present application can execute the above Figure 3 , Fig.10 The description of the log collection method in the corresponding embodiment can also be performed as described above. Fig.11 The description of the log collection device in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated here.

[0219] See also Fig.14 , Fig.14 is another structural diagram of a computer device provided in an embodiment of the present application. Fig.14 As shown, the computer device 2000 may be the above Figure 1 Corresponding to the terminal device in the embodiment, the above-mentioned computer device 2000 may include: a processor 2001, a network interface 2004 and a memory 2005. In addition, the above-mentioned computer device 2000 may also include: a transceiver 2003, and at least one communication bus 2002. Among them, the communication bus 2002 is used to realize the connection and communication between these components. Among them, the network interface 2004 may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface). The memory 2005 may be a high-speed RAM memory, or it may be a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 2005 may optionally also be at least one storage device located away from the aforementioned processor 2001. Fig.14 As shown, the memory 2005 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module, and a device control application program.

[0220] exist Fig.14 In the computer device 2000 shown, the network interface 2004 can provide a network communication function; and the transceiver 2003 and the processor 2001 can be used to call the device control application stored in the memory 2005 to achieve:

[0221] The container cloud management client obtains the verification identifier associated with the cloud service cluster, and establishes a communication connection with the target physical node in the cloud service cluster according to the verification identifier;

[0222] When receiving container change information sent by the connected target physical node within the target period, calling the event processing function corresponding to the container change information, and performing change processing on the first container group state record table associated with the target physical node according to the event processing function to obtain the second container group state record table;

[0223] Among them, the second container group status record table is used to instruct the target physical node to generate a new soft link corresponding to the target application container in the target collection directory; the target application container refers to the application container for which a change event exists in the second container group status record table; the new soft link is used to obtain the first log data of the target application container in the original directory; the original directory is the initial path of the first log data.

[0224] It should be understood that the computer device 2000 described in the embodiment of the present application can execute the above Figure 8 , Fig.10 The description of the log collection method in the corresponding embodiment can also be performed as described above. Fig.12 The description of the log collection device 2 in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated here either.

[0225] In addition, it should be pointed out here that: the embodiment of the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program executed by the log collection device mentioned above. When the processor executes the computer program, it can execute the above-mentioned Figure 3 , Figure 8 The description of the log collection method in the corresponding embodiment will therefore not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application. As an example, the computer program can be deployed on a computer device for execution, or on multiple computer devices located at one location, or on multiple computer devices distributed at multiple locations and interconnected by a communication network. Multiple computer devices distributed at multiple locations and interconnected by a communication network can form a communication system.

[0226] In addition, it should be noted that: the embodiment of the present application also provides a computer program product, which may include a computer program, and the computer program may be stored in a computer-readable storage medium. The processor of the computer device reads the computer program from the computer-readable storage medium, and the processor may execute the computer program, so that the computer device can execute the above Figure 3 , Figure 8The description of the log collection method in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated. For technical details not disclosed in the computer program product embodiment involved in this application, please refer to the description of the method embodiment of this application.

[0227] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0228] The terms "first", "second", etc. in the description, claims, and drawings of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, device, product, or equipment that includes a series of steps or units is not limited to the listed steps or modules, but optionally includes steps or modules that are not listed, or optionally includes other step units inherent to these processes, methods, devices, products, or equipment.

[0229] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0230] The method and related apparatus provided by the embodiment of the present application are described with reference to the method flow chart and / or structural diagram provided by the embodiment of the present application. Specifically, each process and / or box in the method flow chart and / or structural diagram, as well as the combination of the processes and / or boxes in the flow chart and / or block diagram, can be implemented by a computer program. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the process. Figure 1 Schematic diagram of one or more processes and / or structures Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or structures Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A flow or multiple flows and / or structures illustrate the steps of the functions specified in one block or multiple blocks.

[0231] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A log collection method, characterized in that: The method comprises: The target physical node establishes a communication connection with the container cloud management client according to the verification identifier associated with the cloud service cluster; the target physical node belongs to the cloud service cluster; When a change event for an application container occurs at the target physical node within a target period, container change information corresponding to the change event is generated; the target physical node includes a plurality of application containers; Sending the container change information to the container cloud management client through the cloud service cluster, so that the container cloud management client calls an event processing function corresponding to the container change information, and performs change processing on the first container group state record table associated with the target physical node according to the event processing function to obtain a second container group state record table; The second container group state record table is obtained, and a new soft link corresponding to a target application container is generated in a target collection directory according to the second container group state record table; the target application container refers to an application container in which the change event exists in the second container group state record table; the new soft link is used to obtain first log data of the target application container in an original directory; the original directory is an initial path of the first log data.

2. The log collection method according to claim 1, characterized in that: The generating a new soft link corresponding to the target application container in the target collection directory according to the second container group state record table includes: Traversing the second container group state record table through the log discovery container in the log component, and determining the target application container according to the first container group state record table and the second container group state record table; Discovering a container through the log, and acquiring first log data corresponding to the target application container in the original directory according to the second container group state record table; The new soft link corresponding to the first log data is generated in the target collection directory through the log discovery container.

3. The log collection method according to claim 2, characterized in that: The target physical node includes a plurality of application container groups, each of which includes a plurality of application containers; in the log discovery container in the log component, obtaining first log data corresponding to the target application container in the original directory according to the second container group state record table, including: The container is discovered through the log, a log data matching rule corresponding to the application container group where the target application container is located is obtained according to the second container group status record table, and first log data corresponding to the target application container is obtained in the original directory according to the log data matching rule.

4. The log collection method according to claim 1, characterized in that: The method further comprises: Discovering a container through the log, and storing the newly created soft connection in the target collection directory; Through the log collection container in the log component, the existing soft connection and the newly created soft connection are obtained from the target collection directory, node log data is collected from the original directory according to the existing soft connection and the newly created soft connection, and the node log data is stored in a memory; the node log data includes the first log data; the existing soft connection is a soft connection corresponding to an unchanged application container generated based on the first container group status record table; the unchanged application container refers to an application container for which no change event exists in the second container group status record table.

5. The log collection method according to any one of claims 1 to 4, characterized in that: The target physical node includes a plurality of application container groups, each of the application container groups includes a plurality of application containers, and the method further includes: Obtaining an existing container group list and an original container group list through the cloud service cluster, wherein the existing container group list includes application container groups in a working state; and the original container group list includes each application container group in the target physical node; Determine, through a log cleaning container in a log component, an application container in an invalid state according to the existing container group list and the original container group list, and determine the application container in the invalid state as a container to be deleted; Clean up the second log data corresponding to the to-be-deleted container from the original directory, and determine the soft link corresponding to the second log data as an invalid soft link; The soft links in the target collection directory are traversed, and the invalid soft links traversed are cleared.

6. The log collection method according to claim 5, characterized in that: The target physical node includes a plurality of application container groups, each of which includes a plurality of application containers; and obtaining the existing container group list and the original container group list through the cloud service cluster includes: Acquire the container group name corresponding to each of the application container groups and the survival status corresponding to each of the application container groups from the second container group status record table; the survival status includes a working status and an invalid status; According to the container group name corresponding to each of the application container groups and the survival status corresponding to each of the application container groups, obtaining the existing container group list through the cloud service cluster; The original container group list corresponding to the target physical node is obtained through the cloud service cluster.

7. The log collection method according to claim 4, characterized in that: The acquiring the existing soft link and the newly created soft link from the target collection directory through the log collection container in the log component includes: Mounting the target collection directory and configuration file to the collection subcomponent in the log collection container; In the collection subcomponent in the log collection container, the existing soft link and the newly created soft link are obtained from the target collection directory according to the configuration file.

8. A log collection method, characterized in that: The method comprises: The container cloud management client obtains a verification identifier associated with the cloud service cluster, and establishes a communication connection with a target physical node in the cloud service cluster according to the verification identifier; When receiving container change information sent by the connected target physical node within a target period, calling an event processing function corresponding to the container change information, and performing change processing on a first container group state record table associated with the target physical node according to the event processing function to obtain a second container group state record table; The second container group status record table is used to instruct the target physical node to generate a new soft link corresponding to the target application container in the target collection directory; the target application container refers to the application container in which the change event exists in the second container group status record table; the new soft link is used to obtain the first log data of the target application container in the original directory; the original directory is the initial path of the first log data.

9. The log collection method according to claim 8, characterized in that: When container change information sent by the connected target physical node is received within a target period, an event processing function corresponding to the change information is called, and a change process is performed on a first container group state record table associated with the target physical node according to the event processing function, including: When container change information sent by the connected target physical node is received within a target period, and the container change information is container update information, calling an update event processing function corresponding to the container update information, and performing update processing on a state record of the target application container indicated by the container update information in the first container group state record table according to the update event processing function; When container change information sent by the connected target physical node is received within a target period, and the container change information is container new information, calling a new event processing function corresponding to the container new information, and performing a new processing on the state record of the target application container indicated by the container new information in the first container group state record table according to the new event processing function; When container change information sent by the connected target physical node is received within a target period, and the container change information is container deletion information, a deletion event processing function corresponding to the container deletion information is called, and a status record of the target application container indicated by the container deletion information is deleted in the first container group status record table according to the deletion event processing function.

10. A log collection device, characterized in that: include: A cloud service cluster communication module is used to establish a communication connection with a container cloud management client according to a verification identifier associated with the cloud service cluster; The target physical node belongs to the cloud service cluster; A change information generating module, configured to generate container change information corresponding to a change event when a change event for an application container occurs at the target physical node within a target period; The cloud service cluster includes multiple application containers; an information sending module, configured to send the container change information to the container cloud management client, so that the container cloud management client calls an event processing function corresponding to the container change information, and performs a change process on the first container group state record table associated with the target physical node according to the event processing function to obtain a second container group state record table; a soft link generation module, configured to obtain the second container group state record table, and generate a new soft link corresponding to the target application container in the target collection directory according to the second container group state record table; The target application container refers to the application container in which the change event exists in the second container group state record table; The newly created soft link is used to obtain the first log data of the target application container in the original directory; the original directory is the initial path of the first log data.

11. A log collection device, characterized in that: include: The client communication module is used to obtain the verification identifier associated with the cloud service cluster and establish a communication connection with the target physical node in the cloud service cluster according to the verification identifier; an event processing module, configured to, when receiving container change information sent by the connected target physical node within a target period, call an event processing function corresponding to the container change information, and perform change processing on a first container group state record table associated with the target physical node according to the event processing function to obtain a second container group state record table; The second container group state record table is used to instruct the target physical node to generate a new soft connection corresponding to the target application container in the target collection directory; the target application container refers to the application container in which the change event exists in the second container group state record table; The newly created soft connection is used to instruct the log component to collect logs for the target application container.

12. A computer device, characterized in that: include: Processors, transceivers, memory, and network interfaces; The processor is connected to the transceiver, the memory, and the network interface, wherein the network interface is used to provide a data communication function, the memory is used to store a computer program, and the processor and the transceiver are used to call the computer program so that the computer device executes the method described in any one of claims 1 to 9.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded and executed by a processor, so that a computer device having the processor executes the method according to any one of claims 1 to 9.

14. A computer program product, characterized in that The computer program product comprises computer instructions, which are stored in a computer-readable storage medium. The computer instructions are suitable for being read and executed by a processor, so that a computer device having the processor executes the method according to any one of claims 1 to 9.

Images