Log collection method and device, electronic equipment and storage medium

Abstract

The invention discloses a log acquisition method and device, electronic equipment and a storage medium, and relates to the field of data acquisition, the method comprises the following steps: acquiring acquisition configuration information of a first service node in response to acquiring a first heartbeat message sent by the first service node; wherein the acquisition configuration information comprises a grouping category, an acquisition mode and a transmission mode; and sending the first response message configured with the acquisition configuration information to the first service node, thereby enabling the first service node to acquire the first log information based on the packet category and the acquisition mode, and enabling the first service node to send the first log message based on the transmission mode. According to the technical scheme, automatic issuing of the collection configuration information is achieved, the log information collection efficiency is improved, automatic updating of the collection configuration information is achieved when the network topology where the nodes are located is changed, the management cost is further reduced, and the operation complexity of log collection is simplified.

Description

Technical Field

[0001] The present invention relates to the field of data collection, and particularly to a log collection method, device, electronic device and storage medium. Background Art

[0002] Log collection plays a key role in modern distributed systems and is an important basis for operation and maintenance, fault diagnosis, performance analysis and business decision-making.

[0003] In a server cluster, since the business types carried by each node are different, different types of log information need to be collected. Traditional log collection methods usually rely on static configuration files, and the collection rules of each node need to be deployed and maintained manually so that each node performs targeted log information collection operations.

[0004] However, such a collection method not only requires a large amount of human and time costs and has a low log information collection efficiency, but also requires adaptive manual updates when the network topology where the node is located changes, further increasing the management cost and exacerbating the operation complexity of log collection. Summary of the Invention

[0005] The present invention provides a log collection method, device, electronic device and storage medium to solve the problem of low efficiency in configuring collection configuration information.

[0006] According to one aspect of the present invention, there is provided a log collection method applied to a management node, including:

[0007] In response to obtaining a first heartbeat message sent by a first service node, obtaining the collection configuration information of the first service node; wherein, the collection configuration information includes a grouping category, a collection method, and a transmission method;

[0008] Sending a first response message configured with the collection configuration information to the first service node, so that the first service node collects first log information based on the grouping category and the collection method, and enables the first service node to send the first log message based on the transmission method.

[0009] Before sending the first response message configured with the collection configuration information to the first service node, it includes: performing compression processing on the collection configuration information, and replacing the timestamp information in the first response message with the compression result.

[0010] Compressing the acquired collection configuration information and replacing the timestamp information in the first response message includes: performing compression processing on the collection configuration information through multiple compression methods, and obtaining a target compression result with the least number of data bits and not equal to the first data bits among the respective compression results, so as to replace the timestamp information in the first response message with the target compression result; wherein, the first data bits are the data bits of the timestamp information in the first response message.

[0011] Obtaining a first heartbeat message sent by a first service node specifically includes: obtaining the first heartbeat message sent by the first service node through a first transmission channel; sending the first response message configured with the collection configuration information to the first service node includes: sending the first response message configured with the collection configuration information to the first service node through the first transmission channel, so that the first service node sends the first log information based on the first transmission channel.

[0012] After obtaining the first heartbeat message sent by the first service node, it further includes: adding the first heartbeat message to a message management queue; after sending the first response message configured with the collection configuration information to the first service node, it further includes: adding the obtained first log message to the message management queue; the log collection method further includes: determining the message type of each message in the message management queue according to the reception time or the number of data bits of each message; wherein, the message type includes a heartbeat message and a log message.

[0013] Obtaining the collection configuration information of the first service node includes: obtaining a heartbeat delay threshold and a status retention threshold according to the collection method and the transmission method, and adding the heartbeat delay threshold and the status retention threshold to the collection configuration information; sending the first response message configured with the collection configuration information to the first service node further includes: sending the first response message configured with the collection configuration information to the first service node, so that the first service node does not send a heartbeat message within the heartbeat delay threshold and sends at least one heartbeat message within the status retention threshold.

[0014] According to another aspect of the present invention, a log collection device is provided, including:

[0015] A configuration information acquisition module, configured to acquire the collection configuration information of the first service node in response to obtaining a first heartbeat message sent by the first service node; wherein, the collection configuration information includes a packet category, a collection method, and a transmission method.

[0016] A response message sending module, configured to send a first response message configured with the acquisition configuration information to the first service node, so that the first service node acquires first log information based on the packet category and the acquisition method, and the first service node sends the first log message based on the transmission method.

[0017] According to another aspect of the present invention, there is provided an electronic device, which includes:

[0018] At least one processor; and

[0019] A memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor, so that the at least one processor can execute the log acquisition method according to any embodiment of the present invention.

[0021] According to another aspect of the present invention, there is provided a computer-readable storage medium storing computer instructions for implementing the log acquisition method according to any embodiment of the present invention when executed by a processor.

[0022] According to another aspect of the present invention, there is provided a computer program product including a computer program, which implements the log acquisition method according to any embodiment of the present invention when executed by a processor.

[0023] In the technical solution of the embodiment of the present invention, in response to obtaining a first heartbeat message sent by a first service node, acquisition configuration information of the first service node is obtained; a first response message configured with the acquisition configuration information is sent to the first service node, so that the first service node acquires first log information based on a packet category and an acquisition method, and the first service node sends a first log message based on a transmission method. Thereby, not only the automatic distribution of the acquisition configuration information is realized, the labor cost and time cost consumed in the configuration process are saved, and the log information acquisition efficiency is improved, but also when the network topology where the node is located changes, the automatic update of the acquisition configuration information is realized, further reducing the management cost and simplifying the operation complexity of log acquisition.

[0024] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present invention, nor is it used to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. Description of the Drawings

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0026] Figure 1 is a flowchart of a log collection method provided in Embodiment 1 of the present invention;

[0027] Figure 2 is a flowchart of another log collection method provided in Embodiment 2 of the present invention;

[0028] Figure 3 is a flowchart of yet another log collection method provided in Embodiment 3 of the present invention;

[0029] Figure 4 is a schematic structural diagram of a log collection device provided in Embodiment 4 of the present invention;

[0030] Figure 5 is a schematic structural diagram of an electronic device for implementing the log collection method of the embodiments of the present invention. Detailed Embodiments

[0031] In order to enable those skilled in the art to better understand the solutions of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, rather than all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above accompanying drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described here can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0033] Embodiment 1

[0034] Figure 1The flowchart of a log collection method provided by Embodiment 1 of the present invention. This embodiment is applicable to the management node sending a response message configured with collection configuration information to the service node. This method can be executed by the log collection device in any embodiment of the present invention. The log collection device can be implemented in the form of hardware and / or software, and can be configured in the management node of the server cluster, such as Figure 1 shown. The method includes:

[0035] S101. In response to obtaining a first heartbeat message sent by a first service node, obtain the collection configuration information of the first service node; wherein, the collection configuration information includes a grouping category, a collection method, and a transmission method.

[0036] In a server cluster, one or more servers can be configured as management nodes (i.e., as the Server side), and the remaining servers are service nodes; the management node is used for the operation and maintenance management of one or more service nodes, and the service node is used to process specific service data; the service node is pre-configured with client (i.e., Agent side) software, and communicates with the management node through this client software; the service node sends a heartbeat message to the management node according to its own heartbeat cycle, so as to inform the management node of the current connection state of the service node through the heartbeat message.

[0037] After obtaining the first heartbeat message, the management node obtains the matching collection configuration information according to the identity identifier of the first service node; among them, different service nodes can often be divided into different grouping categories according to the service types they process, that is, different service groups. Each service node in the same service group is often used to process the same type of service, that is, it is necessary to obtain the same type of log information. Therefore, the collection method can refer to the type of log information to be collected by each service node in the group where the current service node belongs.

[0038] In addition, the grouping category can also be divided into different geographical groups or device groups based on the location or device type of the service node. At this time, each service node in the same geographical group or device group may be used to process different types of services, that is, it is necessary to obtain different types of log information. Therefore, the collection method can also refer to the type of log information to be collected by the current service node; the collection method can include the collection tool type, the collection time, and the collection object. The collection tool type indicates which collection tool is used to perform the collection operation, the collection time indicates when to perform the log collection operation, and the collection object indicates the identifier and location of the information to be collected; the transmission method can include the transmission time and the protocol type, etc. The transmission time indicates the specific time to perform the log information transmission operation. For example, it is an immediate transmission or a delayed transmission after a specified time.

[0039] S102. Send the first response message configured with the collection configuration information to the first service node, so that the first service node collects the first log information based on the packet category and the collection method, and the first service node sends the first log message based on the transmission method.

[0040] After the management node obtains the first heartbeat message, it first needs to determine whether there are updated parameters in the collection configuration information of the first service node. If there are no updated parameters, it is not necessary to repeatedly send the collection configuration information to the first service node. At this time, a heartbeat response message (i.e., the first response message) can be directly fed back according to the first heartbeat message; among them, the heartbeat response message is composed of a message type, a status code, a timestamp information, and additional information; the message type indicates that the message type is a heartbeat response message (the length can be 1 byte); the status code indicates the communication status of the heartbeat response message (for example, normal, error, timeout, etc., the length can be 1 byte); the timestamp information indicates the generation time of the current heartbeat response message (the length can be 8 bytes); the additional information is configured as a null value.

[0041] If there are updated parameters, the collection configuration information needs to be configured in the first response message. For example, the collection configuration information is configured in the additional information to send the collection configuration information to the first service node by means of the first response message; in particular, the number of data bits of the collection configuration information is relatively large. At this time, the collection configuration information can be compressed. For example, the management node compresses the collection configuration information through a locally stored dynamic dictionary or mapping table, or other lossless compression algorithms, so as to facilitate configuring the compressed collection configuration information in the first response message. For example, it is configured at the position of the additional information in the first response message; the first service node then decodes through the locally stored dynamic dictionary or mapping table, or based on the same lossless compression algorithm, to restore the collection configuration information.

[0042] After the first service node obtains the first response message, it first parses the first response message to obtain the message type and the status code, then obtains the packet category and the collection method therein by parsing the collection configuration parameters, and realizes the collection of the first log information based on this collection method; then, according to the transmission method obtained by parsing the collection configuration parameters, the first log information after collection is sent in the form of the first log message through this transmission method, so as to realize the collection and transmission of the log information.

[0043] Specifically, in the traditional technical solution, heartbeat messages (e.g., the first heartbeat message) and data messages (e.g., the first log message) are usually configured to be transmitted through different transmission channels, that is, through different service node interfaces and management node interfaces. This is because if the same transmission channel is used for transmission, at the moment when the heartbeat message is sent, there may be a data message being transmitted, which results in a transmission delay phenomenon for the heartbeat message. If the heartbeat message is sent first and then the data message is transmitted continuously, it will not only affect the transmission continuity of the data message, but also the response message (e.g., the first response message) matching the heartbeat message still has a transmission conflict with the data message.

[0044] After the management node obtains the first log message, it parses the first log message to obtain the first log information recorded therein and stores the first log information. For example, the first log information is stored in a local or cloud database; it is also possible to first perform a normalization check on the first log information to fill in the missing data in the first log information and clean the abnormal data to ensure the normalization and integrity of the obtained log information, and then store the cleaned first log information; specifically, the management node can manage each obtained message through the message management queue. Since the heartbeat message and the log message come from different interfaces, they can be managed through different message management queues, that is, the heartbeat message management queue and the log message management queue, to manage the heartbeat message and the log message respectively, and execute each received message in an orderly manner according to the different reception times of each message.

[0045] Optionally, in the embodiment of the present invention, obtaining the first heartbeat message sent by the first service node specifically includes: obtaining the first heartbeat message sent by the first service node through the first transmission channel; sending the first response message configured with the acquisition configuration information to the first service node includes: sending the first response message configured with the acquisition configuration information to the first service node through the first transmission channel, so that the first service node sends the first log information based on the first transmission channel.

[0046] Specifically, in the embodiments of the present invention, the first log message, the first heartbeat message, and the first response message are all configured to be transmitted in the same transmission channel (i.e., the first transmission channel), that is, through the same service node interface and management node interface. This is because in the embodiments of the present invention, the timing of the first service node sending the first log message depends on the acquisition of the first response message, and the acquisition of the first response message depends on the sending of the first heartbeat message. That is, there is a transmission timing sequence of the first heartbeat message, the first response message, and the first log message between the first service node and the management node. Therefore, transmitting the above types of messages through the same transmission channel not only saves transmission resources but also ensures the orderly execution of the transmission timing sequence and avoids data congestion in the transmission channel.

[0047] Optionally, in the embodiments of the present invention, after obtaining the first heartbeat message sent by the first service node, it further includes: adding the first heartbeat message to the message management queue; after sending the first response message configured with the acquisition configuration information to the first service node, it further includes: adding the obtained first log message to the message management queue; the log acquisition method further includes: determining the message type of each message according to the reception time or the number of data bits of each message in the message management queue; where the message type includes a heartbeat message and a log message.

[0048] Specifically, as described in the above technical solution, on the premise of solving the timing problem of the heartbeat message and the log message, the heartbeat message and the log message have been configured to be transmitted in the same transmission channel. After the management node obtains the heartbeat message and the log message through the same interface, it can directly put them into the same management queue to simplify the message classification process; at the same time, in the traditional technical solution, when managing multiple types of messages through a management queue, each message needs to be configured with a message type identifier to enable the management queue to distinguish different types of messages. However, in the embodiments of the present invention, as a lightweight message, the data capacity of the heartbeat message, that is, the number of data bits it contains, is obviously different from that of the log message. The message type can be directly distinguished according to the number of data bits in each message.

[0049] Meanwhile, there is a transmission timing sequence of heartbeat messages, response messages, and log messages between the first service node and the management node. For example, if the sending period of the heartbeat message is one minute, then the messages obtained at 11:01, 11:02, and 11:03 are heartbeat messages. When there is a need to send a log message, its sending time must also be within the transmission gap between two heartbeat messages. For example, a message obtained at 11:01:05 is obviously a log message. Thus, the management node can determine the message type of each message through the reception time or data bit number of each message in the message management queue, and no message type identifier needs to be configured in either the heartbeat message or the log message, thereby simplifying the structure of various types of messages between the service node and the management node and improving the message transmission efficiency.

[0050] In the technical solution of the embodiment of the present invention, in response to obtaining the first heartbeat message sent by the first service node, the acquisition configuration information of the first service node is obtained; the first response message configured with the acquisition configuration information is sent to the first service node, so that the first service node collects the first log information based on the packet category and the acquisition method, and enables the first service node to send the first log message based on the transmission method. This not only realizes the automatic distribution of the acquisition configuration information, saves the labor cost and time cost consumed in the configuration process, and improves the log information acquisition efficiency, but also realizes the automatic update of the acquisition configuration information when the network topology where the node is located changes, further reducing the management cost and simplifying the operation complexity of log collection.

[0051] Embodiment 2

[0052] Figure 2 It is a flowchart of a log collection method provided by the second embodiment of the present invention. The relationship between this embodiment and the above embodiment is that the compressed result of the acquisition configuration information replaces the timestamp information in the first response message. As Figure 2 shown, the method includes:

[0053] S201. In response to obtaining the first heartbeat message sent by the first service node, obtain the acquisition configuration information of the first service node; wherein, the acquisition configuration information includes a packet category, an acquisition method, and a transmission method.

[0054] S202. Perform compression processing on the acquisition configuration information, and replace the timestamp information in the first response message with the compressed result.

[0055] As described in the above technical solution, the response message itself is a lightweight signal. If the acquisition configuration information is directly configured in the response signal, it may cause the data volume of the response signal to be too large, affecting the transmission characteristics of the response message, making the response message unable to be fed back to the first service node in time, resulting in the first service node may mistakenly think that the management node has not received the heartbeat message, and then continuously send heartbeat messages, thereby increasing the transmission compliance of the transmission channel; at this time, the acquisition configuration information can be compressed first.

[0056] At the same time, there is a timestamp information in the response message itself, which is used to represent the response generation time of the management node. The first service node can use it to calculate the transmission delay time; in the embodiment of the present invention, the compression result of the above acquisition configuration information is used to replace the timestamp information, because the acquisition configuration information is not sent at each heartbeat response moment, but only when the acquisition task needs to be executed or there are updated parameters in the acquisition configuration information. Therefore, the acquisition configuration information does not need to be configured in most response messages, and the first service node can still calculate the transmission delay according to the timestamp information in the response messages obtained at other moments. Thus, using the compressed acquisition configuration information to replace the timestamp information in the first response message further improves the transmission speed of the first response message without affecting the calculation of the transmission delay time.

[0057] Optionally, in the embodiment of the present invention, the step of compressing the acquisition configuration information and replacing the timestamp information in the first response message with the compression result includes: respectively compressing the acquisition configuration information by multiple compression methods, and obtaining a target compression result with the least number of data bits and not equal to the first number of data bits among the respective compression results, so as to replace the timestamp information in the first response message with the target compression result; wherein, the first number of data bits is the number of data bits of the timestamp information in the first response message.

[0058] Specifically, taking the number of data bits (i.e., the first number of data bits) of the timestamp information in the first response message in the above technical solution as 8 bytes as an example, if the compression result of the acquisition configuration information happens to be 8 bytes in size and is configured after the first response message, the first service node will mistake this part of the data for the timestamp information, not only causing the loss of the acquisition configuration information, but also resulting in the confusion of the timestamp information; therefore, after respectively compressing the acquisition configuration information by multiple compression methods, it is necessary to obtain a target compression result with the least number of data bits and not equal to 8 bytes among the respective compression results, so that the service node can distinguish the timestamp information and the acquisition configuration information according to the number of data bits of the response message, and at the same time ensure the relatively fast transmission speed of the response message configured with the acquisition configuration information.

[0059] S203. Send the first response message configured with the collection configuration information to the first service node, so that the first service node collects first log information based on the packet category and the collection method, and the first service node sends the first log message based on the transmission method.

[0060] In the technical solution of the embodiment of the present invention, after obtaining the first heartbeat message sent by the first service node, the collection configuration information of the first service node is obtained; then, the collection configuration information is compressed, and the compression result is used to replace the timestamp information in the first response message; finally, the first response message configured with the collection configuration information is sent to the first service node. Thus, the compressed collection configuration information is used to replace the timestamp information in the first response message, which further improves the transmission speed of the first response message without affecting the transmission delay time.

[0061] Embodiment 3

[0062] Figure 3 It is a flowchart of a log collection method provided by Embodiment 3 of the present invention. The relationship between this embodiment and the above embodiments is that a heartbeat delay threshold and a status retention threshold are set for the first service node. As Figure 3 shown, the method includes:

[0063] S301. In response to obtaining the first heartbeat message sent by the first service node, obtain the collection configuration information of the first service node; wherein, the collection configuration information includes a packet category, a collection method, and a transmission method.

[0064] S302. According to the collection method and the transmission method, obtain a heartbeat delay threshold and a status retention threshold, and add the heartbeat delay threshold and the status retention threshold to the collection configuration information; wherein, the status retention threshold is greater than the heartbeat delay threshold.

[0065] When the management node issues the collection configuration information to the first service node, in order to enable the first service node to concentrate node resources to collect log information, the management node can guide the first service node to temporarily close the transmission of the heartbeat message. At the same time, since the management node itself already knows that the first service node is performing the log collection task during this period, even if it does not receive the heartbeat message, it will not regard it as a connection disconnection; the heartbeat delay threshold indicates the specific time when the first service node does not need to send the heartbeat message. According to different collection methods and transmission methods, the collection time required for different types and different amounts of data (i.e., the number of data bits) of log information is different, and the delay time and transmission time required for different transmission methods are also different. Therefore, according to the collection method and the transmission method, a matching heartbeat delay threshold can be obtained.

[0066] Specifically, if the collected configuration information issued this time is to guide the first service node to delay the collection or transmission of log information, that is, it is not real-time collection and real-time transmission, then at this time, the heartbeat delay threshold can actually refer to a specific period in the future; in addition, when the log collection time is too long due to reasons such as excessive node resource occupancy of the first service node itself, it may still not be completed after a long time. If waiting indefinitely, it may cause the communication connection between the management node and the first service node to be disconnected. The status retention threshold indicates that at least one heartbeat message needs to be sent within this period to maintain the communication connection between the two.

[0067] S303. Send the first response message configured with the collected configuration information to the first service node, so that the first service node does not send a heartbeat message within the heartbeat delay threshold, and sends at least one heartbeat message within the status retention threshold.

[0068] After obtaining the collected configuration information, the first service node parses and obtains the heartbeat delay threshold and the status retention threshold. Within the heartbeat delay threshold, the first service node concentrates its own node resources to execute the log collection task and will not send a heartbeat message again until the collected log information is collected and sent. After that, this restriction is lifted; if the log information is still not collected after the heartbeat delay threshold, then at least one heartbeat message needs to be sent within the status retention threshold to maintain the communication connection with the management node.

[0069] The technical solution of the embodiment of the present invention obtains the heartbeat delay threshold and the status retention threshold according to the collection method and the transmission method, and adds the heartbeat delay threshold and the status retention threshold to the collected configuration information; sends the first response message configured with the collected configuration information to the first service node, so that the first service node does not send a heartbeat message within the heartbeat delay threshold, and sends at least one heartbeat message within the status retention threshold. Thereby, the service node can concentrate node resources to execute the log collection task to the greatest extent within the heartbeat delay threshold, avoiding the dispersion of node resources. At the same time, it ensures that at least one communication connection is completed during the long-time-consuming collection process, avoiding the situation that when the service node stops running due to reasons such as downtime, the management node still regards it as a connected state node, ensuring the timely detection of node anomalies.

[0070] Embodiment 4

[0071] Figure 4 It is a structural block diagram of a log collection device provided by Embodiment 4 of the present invention, specifically including:

[0072] A configuration information acquisition module 401, configured to acquire the collection configuration information of the first service node in response to obtaining a first heartbeat message sent by the first service node; wherein, the collection configuration information includes a grouping category, a collection method, and a transmission method;

[0073] A response message sending module 402, configured to send a first response message configured with the collection configuration information to the first service node, so that the first service node collects first log information based on the grouping category and the collection method, and enables the first service node to send the first log message based on the transmission method.

[0074] The technical solution of the embodiment of the present invention acquires the collection configuration information of the first service node in response to obtaining a first heartbeat message sent by the first service node; sends a first response message configured with the collection configuration information to the first service node, so that the first service node collects first log information based on the grouping category and the collection method, and enables the first service node to send the first log message based on the transmission method. Thereby, not only the automatic distribution of the collection configuration information is realized, the labor cost and time cost consumed in the configuration process are saved, and the log information collection efficiency is improved, but also when the network topology where the node is located changes, the automatic update of the collection configuration information is realized, further reducing the management cost and simplifying the operation complexity of log collection.

[0075] Optionally, the log collection device is further configured to perform compression processing on the collection configuration information, and replace the timestamp information in the first response message with the compression result.

[0076] Optionally, the log collection device is specifically further configured to perform compression processing on the collection configuration information by multiple compression methods respectively, and obtain a target compression result with the least number of data bits and not equal to the first data bits in each compression result, so as to replace the timestamp information in the first response message with the target compression result; wherein, the first data bits are the data bits of the timestamp information in the first response message.

[0077] Optionally, the log collection device is specifically configured to obtain the first heartbeat message sent by the first service node through the first transmission channel; send the first response message configured with the collection configuration information to the first service node through the first transmission channel, so that the first service node sends the first log information based on the first transmission channel.

[0078] Optionally, the log collection device is further configured to add the first heartbeat message to a message management queue; add the obtained first log message to the message management queue; determine the message type of each message according to the reception time or the number of data bits of each message in the message management queue; wherein the message type includes a heartbeat message and a log message.

[0079] Optionally, the log collection device is further configured to obtain a heartbeat delay threshold and a status retention threshold according to the collection mode and the transmission mode, and add the heartbeat delay threshold and the status retention threshold to the collection configuration information; wherein the status retention threshold is greater than the heartbeat delay threshold; send a first response message configured with the collection configuration information to the first service node, so that the first service node does not send a heartbeat message within the heartbeat delay threshold, and sends at least one heartbeat message within the status retention threshold.

[0080] The above device can execute the log collection method provided by any embodiment of the present invention, and has corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in this embodiment can be referred to the log collection method provided by any embodiment of the present invention.

[0081] Embodiment 5

[0082] Figure 5 FIG. shows a schematic structural diagram of an electronic device 10 that can be used to implement an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as, laptop computers, desktop computers, workstations, personal digital assistants, electronic devices, blade electronic devices, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0083] As Figure 5As shown, the electronic device 10 includes at least one processor 11 and a memory communicatively connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc. The memory stores a computer program executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0084] Multiple components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disc, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0085] The processor 11 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include but are not limited to a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the log collection method.

[0086] In some embodiments, the log collection method can be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as the storage unit. In some embodiments, part or all of the computer program can be loaded and / or installed onto a heterogeneous hardware accelerator via the ROM and / or the communication unit. When the computer program is loaded into the RAM and executed by the processor, one or more steps of the log collection method described above can be executed. Alternatively, in other embodiments, the processor can be configured to execute the log collection method in any other appropriate way (e.g., by means of firmware).

[0087] The various embodiments of the systems and techniques described above in this specification can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that receives data and instructions from, and transmits data and instructions to, a storage system, at least one input device, and at least one output device.

[0088] The computer programs for implementing the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions / operations specified in the flowchart and / or block diagram to be implemented. The computer programs can be executed entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or electronic device.

[0089] In the context of the present invention, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0090] To provide interaction with a user, the systems and techniques described herein can be implemented on a heterogeneous hardware accelerator that has: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the heterogeneous hardware accelerator. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0091] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data electronic device), or a computing system that includes middleware components (e.g., an application electronic device), or a computing system that includes frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), blockchain network, and the Internet.

[0092] A computing system can include a client and an electronic device. The client and the electronic device are generally remote from each other and typically interact through a communication network. The relationship between the client and the electronic device is created by computer programs that run on respective computers and have a client - electronic device relationship with each other. The electronic device can be a cloud electronic device, also known as a cloud computing electronic device or a cloud host, which is a host product in the cloud computing service system, solving the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services.

[0093] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in the present invention can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved, and no limitation is imposed herein.

[0094] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A log collection method, characterized in that: Applicable to management nodes, including: In response to obtaining a first heartbeat message sent by a first service node, obtaining collection configuration information of the first service node; wherein the collection configuration information includes a grouping category, a collection mode, and a transmission mode; A first response message configured with the collection configuration information is sent to the first service node, so that the first service node collects first log information based on the grouping category and the collection method, and the first service node sends the first log message based on the transmission method.

2. The method according to claim 1, characterized in that Before sending the first response message configured with the collection configuration information to the first service node, the method includes: The collection configuration information is compressed, and the compression result replaces the timestamp information in the first response message.

3. The method according to claim 2, characterized in that The compressing the collection configuration information and replacing the timestamp information in the first response message with the compression result includes: The collection configuration information is compressed respectively by multiple compression methods, and a target compression result with the least number of data bits in each compression result and not equal to the first number of data bits is obtained, so as to replace the timestamp information in the first response message with the target compression result; wherein, the first number of data bits is the number of data bits of the timestamp information in the first response message.

4. The method according to claim 1, characterized in that: Obtaining a first heartbeat message sent by a first service node specifically includes: Acquire a first heartbeat message sent by a first service node through a first transmission channel; The sending the first response message configured with the collection configuration information to the first service node includes: A first response message configured with the collection configuration information is sent to the first service node through the first transmission channel, so that the first service node sends the first log information based on the first transmission channel.

5. The method according to claim 4, characterized in that After obtaining the first heartbeat message sent by the first service node, the method further includes: Adding the first heartbeat message to the message management queue; After sending the first response message configured with the collection configuration information to the first service node, the method further includes: Adding the first log message obtained to the message management queue; The log collection method further includes: The message type of each message in the message management queue is determined according to the receiving time or the number of data bits of each message; wherein the message type includes a heartbeat message and a log message.

6. The method according to claim 1, characterized in that The acquiring the collection configuration information of the first service node includes: According to the acquisition mode and the transmission mode, a heartbeat delay threshold and a state holding threshold are acquired, and the heartbeat delay threshold and the state holding threshold are added to the acquisition configuration information; wherein the state holding threshold is greater than the heartbeat delay threshold; The sending the first response message configured with the collection configuration information to the first service node also includes: A first response message configured with the collection configuration information is sent to the first service node, so that the first service node no longer sends a heartbeat message within the heartbeat delay threshold, and sends at least one heartbeat message within the state maintenance threshold.

7. A log collection device, characterized in that: include: A configuration information acquisition module, configured to acquire, in response to acquiring a first heartbeat message sent by a first service node, collection configuration information of the first service node; wherein the collection configuration information includes a grouping category, a collection mode, and a transmission mode; A response message sending module is used to send a first response message configured with the collection configuration information to the first service node, so that the first service node collects the first log information based on the grouping category and the collection method, and enables the first service node to send the first log message based on the transmission method.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the log collection method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the log collection method according to any one of claims 1 to 6 when executed.

10. A computer program product, comprising a computer program, wherein the computer program implements the log collection method according to any one of claims 1 to 6 when executed by a processor.

Images