Log data management method of cloud-edge collaboration platform and related equipment

Abstract

The application provides a log data management method of a cloud-edge collaborative platform and related equipment, and relates to the field of cloud-edge collaboration. The method comprises the following steps: when a cloud end receives a policy configuration request sent by a user through a front end, it is judged whether the request is a request associated with the cloud end. If yes, the request is analyzed to obtain configuration information, based on which a cloud end log collection policy is configured, so that log data of the cloud end is collected and stored in real time based on the policy. If the request is not a request associated with the cloud end, an edge end corresponding to the request is determined. The request is sent to the corresponding edge end through a cloud-edge channel, so that the edge end analyzes the request to obtain configuration information, thereby configuring an edge end log collection policy, so that log data of the edge end is collected and stored in real time based on the policy. By applying the method, a user can configure log collection policies of the cloud end and the edge end through a front end interface, which is conducive to improving operation and maintenance efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of cloud-edge collaboration, in particular to a log data management method of a cloud-edge collaboration platform and related equipment. BACKGROUND

[0002] With the development of cloud computing technology, in the scene of industrial internet, cloud-edge collaboration mode has been widely used to deploy various business platforms. The business platform deployed based on cloud-edge collaboration can be referred to as a cloud-edge collaboration platform. One of the common application deployment modes in the cloud-edge collaboration platform is to deploy applications through containers. In the container orchestration scene, the Kubernetes (K8s) system is one of the common container orchestration platforms. In the container application scene based on the K8s system, container logs and container cluster events (i.e. K8s system events) and other log data need to be collected in order to facilitate the operation and maintenance of the application environment.

[0003] At present, the collection of log data is usually completed by using component invasive configuration. For example, the collection of container logs usually configures container log collection components such as log-Pilot in the container, and the container logs are collected by the components and stored in the data storage platform.

[0004] However, the cloud-edge collaboration platform usually includes a cloud end and multiple edge ends, that is, there are multiple nodes, and multiple containers are deployed on each node. Based on the existing log data collection method, when it is necessary to configure or update the log data collection rule, the relevant personnel need to perform invasive configuration in the container of each node to set the data collection rule of the corresponding component. The configuration and update process of the log data collection rule is relatively cumbersome, which leads to low operation and maintenance efficiency. SUMMARY

[0005] Therefore, the embodiments of the present application provide a log data management method of a cloud-edge collaboration platform to solve the problem that the existing log collection method of the cloud-edge collaboration platform needs to manually configure each component in each node, the process is relatively cumbersome, and the operation and maintenance efficiency is low.

[0006] The embodiments of the present application also provide a log data management device of a cloud-edge collaboration platform to ensure the implementation and application of the above method in practice.

[0007] To achieve the above object, the embodiments of the present application provide the following technical solutions:

[0008] A log data management method of a cloud-edge collaboration platform, the method is applied to a cloud end of the cloud-edge collaboration platform, and the method comprises:

[0009] When a policy configuration request sent by a user through a front-end interface is received, it is judged whether the policy configuration request is a request associated with the cloud end.

[0010] If the policy configuration request is a request associated with the cloud, the policy configuration request is parsed to obtain first configuration information;

[0011] In the form of a custom resource, a cloud log collection policy corresponding to the first configuration information is configured, so that a preset cloud-native log collection tool in the cloud collects container logs and container cluster events of the cloud in real time based on the cloud log collection policy, and stores the container logs and the container cluster events of the cloud to a data storage platform corresponding to the cloud;

[0012] If the policy configuration request is not a request associated with the cloud, a first edge terminal corresponding to the policy configuration request is determined in each edge terminal of the cloud-edge collaborative platform;

[0013] The policy configuration request is sent to the first edge terminal through a preset cloud-edge channel, so that the first edge terminal parses the policy configuration request to obtain second configuration information, and configures an edge log collection policy corresponding to the second configuration information in the form of a custom resource, so that a preset cloud-native log collection tool in the first edge terminal collects container logs and container cluster events of the first edge terminal in real time based on the edge log collection policy, and stores the container logs and the container cluster events of the first edge terminal to a data storage platform corresponding to the first edge terminal.

[0014] The above method can further include:

[0015] When receiving a data query request sent by a user through a front-end interface, it is judged whether the data query request is a request associated with the cloud;

[0016] If the data query request is a request associated with the cloud, first query information corresponding to the data query request is determined;

[0017] Based on the first query information, a data query interface of a data storage platform corresponding to the cloud is called to query log data corresponding to the first query information in the data storage platform corresponding to the cloud;

[0018] In the front-end interface, the log data corresponding to the first query information is visually displayed.

[0019] The above method can further include:

[0020] The data query request is parsed to obtain parsed data corresponding to the data query request;

[0021] Determining a query object corresponding to the data query request based on the parsed data; the query object is the container log on the cloud or the container cluster event on the cloud;

[0022] If the query object is a container cluster event on the cloud, obtaining event parameters corresponding to the query object from the parsed data, and using the event parameters as the first query information;

[0023] If the query object is the container log of the cloud, determining whether the query mode corresponding to the data query request is a context query;

[0024] If the query mode corresponding to the data query request is context query, the container name, log identifier and query start and end time are obtained from the parsed data, and the container name, log identifier and query start and end time are used as the first query information.

[0025] The above method may optionally further include:

[0026] If the data query request is not a request associated with the cloud, determining a second edge end corresponding to the data query request among each edge end of the cloud-edge collaboration platform;

[0027] Sending the data query request to the second edge end through the cloud-edge channel, so that the second edge end determines second query information corresponding to the data query request, and based on the second query information, calling a data query interface of a data storage platform corresponding to the second edge end to query log data corresponding to the second query information in the data storage platform corresponding to the second edge end, and sending the log data corresponding to the second query information to the cloud;

[0028] When the log data corresponding to the second query information is received, the log data corresponding to the second query information is visually displayed in the front-end interface.

[0029] The above method may optionally further include:

[0030] When receiving a data deletion request sent by the user through the front-end interface, determining whether the data deletion request is a cloud-related request;

[0031] If the data deletion request is a request associated with the cloud, parsing the data deletion request to obtain first data information;

[0032] Based on the first data information, calling a data operation interface of a data storage platform corresponding to the cloud, and deleting log data corresponding to the first data information in the data storage platform corresponding to the cloud;

[0033] If the data deletion request is not a request associated with the cloud, determining a third edge end corresponding to the data deletion request among the edge ends of the cloud-edge collaboration platform;

[0034] The data deletion request is sent to the third edge end through the cloud-edge channel, so that the third edge end parses the data deletion request to obtain second data information, and based on the second data information, calls the data operation interface of the data storage platform corresponding to the third edge end to delete the log data corresponding to the second data information in the data storage platform corresponding to the third edge end.

[0035] The above method may optionally further include:

[0036] When receiving an alarm configuration request sent by a user through the front-end interface, determining whether the alarm configuration request is a request associated with the cloud;

[0037] If the alarm configuration request is a request associated with the cloud, parsing the alarm configuration request to obtain first alarm rule information;

[0038] Setting a cloud alarm rule based on the first alarm rule information, and issuing an abnormal alarm for the cloud based on the cloud alarm rule;

[0039] If the alarm configuration request is not a request associated with the cloud, determining a fourth edge end corresponding to the alarm configuration request among the edge ends of the cloud-edge collaboration platform;

[0040] The alarm configuration request is sent to the fourth edge end through the cloud edge channel, so that the fourth edge end parses the alarm configuration request, obtains the second alarm rule information, sets the edge end alarm rule based on the second alarm rule information, and issues an abnormal alarm to the fourth edge end based on the edge end alarm rule.

[0041] A log data management device for a cloud-edge collaborative platform, the device being applied to the cloud of the cloud-edge collaborative platform, the device comprising:

[0042] a judgment unit, configured to, upon receiving a policy configuration request sent by a user through a front-end interface, judge whether the policy configuration request is a cloud-related request;

[0043] a parsing unit, configured to parse the policy configuration request to obtain first configuration information if the policy configuration request is a request associated with the cloud;

[0044] a configuration unit configured to configure, in the form of custom resources, a cloud log collection policy corresponding to the first configuration information, so that a cloud-native log collection tool preset in the cloud collects container logs and container cluster events in real time on the cloud based on the cloud log collection policy, and stores the container logs and container cluster events on the cloud to a corresponding data storage platform on the cloud;

[0045] a determining unit, configured to determine, if the policy configuration request is not a request associated with the cloud, a first edge end corresponding to the policy configuration request among the edge ends of the cloud-edge collaboration platform;

[0046] A sending unit is used to send the policy configuration request to the first edge end through a preset cloud-edge channel, so that the first edge end parses the policy configuration request, obtains second configuration information, and configures the edge end log collection policy corresponding to the second configuration information in the form of a custom resource, so that the cloud native log collection tool preset in the first edge end collects the container logs and container cluster events of the first edge end in real time based on the edge end log collection policy, and stores the container logs and container cluster events of the first edge end to the data storage platform corresponding to the first edge end.

[0047] A log data management system for a cloud-edge collaborative platform, comprising:

[0048] A cloud module and multiple edge modules; the cloud module is deployed on the cloud of the cloud-edge collaboration platform, the multiple edge modules correspond one-to-one to each edge of the cloud-edge collaboration platform, and each edge module is deployed on the edge corresponding to the edge module;

[0049] The cloud module is configured to: upon receiving a policy configuration request sent by a user through the front-end interface, determine whether the policy configuration request is a request associated with the cloud; if the policy configuration request is a request associated with the cloud, parse the policy configuration request to obtain first configuration information; configure the cloud log collection policy corresponding to the first configuration information in the form of a custom resource, so that the cloud native log collection tool preset in the cloud collects the container logs and container cluster events of the cloud in real time based on the cloud log collection policy, and stores the container logs and container cluster events of the cloud to the corresponding data storage platform of the cloud; if the policy configuration request is not a request associated with the cloud, determine the edge end corresponding to the policy configuration request among the various edge ends of the cloud-edge collaborative platform; and send the policy configuration request to its corresponding edge end through the preset cloud-edge channel;

[0050] The edge module is configured to: when the edge corresponding to the edge module receives the policy configuration request, parse the policy configuration request to obtain second configuration information; configure the edge log collection policy corresponding to the second configuration information in the form of a custom resource, so that the cloud-native log collection tool preset in the edge collects the container logs and container cluster events of the edge in real time based on the edge log collection policy, and stores the container logs and container cluster events of the edge to the data storage platform corresponding to the edge.

[0051] A storage medium includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the log data management method of the cloud-edge collaborative platform as described above.

[0052] An electronic device includes a memory and one or more instructions, wherein the one or more instructions are stored in the memory and are configured to be executed by one or more processors to perform the log data management method of the cloud-edge collaborative platform as described above.

[0053] A log data management method for a cloud-edge collaborative platform provided based on the above-mentioned embodiment of the present invention includes: when the cloud receives a policy configuration request sent by a user through a front-end interface, determining whether the policy configuration request is a cloud-related request; if the policy configuration request is a cloud-related request, parsing the policy configuration request to obtain first configuration information; configuring a cloud log collection policy corresponding to the first configuration information in the form of a custom resource, so that a cloud-native log collection tool preset in the cloud collects container logs and container cluster events in real time on the cloud based on the cloud log collection policy, and stores the container logs and container cluster events on the cloud to the corresponding data storage platform on the cloud; if If the policy configuration request is not a cloud-related request, then the first edge end corresponding to the policy configuration request is determined among the edge ends of the cloud-edge collaboration platform; the policy configuration request is sent to the first edge end through the preset cloud-edge channel, so that the first edge end parses the policy configuration request, obtains the second configuration information, and configures the edge end log collection policy corresponding to the second configuration information in the form of a custom resource, so that the cloud-native log collection tool preset in the first edge end, based on the edge end log collection policy, collects the container logs and container cluster events of the first edge end in real time, and stores the container logs and container cluster events of the first edge end to the data storage platform corresponding to the first edge end. By applying the method provided by the embodiment of the present invention, a user can configure the log collection policy in the cloud or an edge end through the front-end interface. When the cloud receives the policy configuration request sent by the user, it can determine whether the request is a request for configuration for the cloud or a request for configuration for the edge end. If the request is a request for configuration for the cloud, the request can be parsed, and the log collection policy can be configured according to the configuration information obtained by the parsing, so that the cloud native log collection tool collects container logs and container cluster events based on the policy, thereby completing the policy configuration for collecting container logs and container cluster events on the cloud. If the request is a request for configuration for the edge end, the cloud can send the request to the corresponding edge end through the cloud-edge channel, triggering the edge end controller to parse the request, thereby configuring the log collection policy, so that the cloud native log collection tool collects container logs and container cluster events based on the policy, thereby completing the policy configuration for collecting container logs and container cluster events on the edge end. Users can configure the collection policy for log data in the cloud and edge end respectively through a unified front-end interface, without having to manually configure multiple components embedded in each node separately, which is conducive to simplifying the policy configuration process and improving operation and maintenance efficiency. Secondly, the cloud and edge use cloud-native log collection tools to monitor and collect container logs and container cluster events in real time, without the need for invasive configuration within the container or the integration of multiple components. This helps reduce the complexity of the cloud-edge collaboration platform and reduce container security risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0055] Figure 1 A flow chart of a log data management method for a cloud-edge collaborative platform provided by an embodiment of the present invention;

[0056] Figure 2 A signaling diagram of the log data management process of a cloud-edge collaboration platform provided in an embodiment of the present invention;

[0057] Figure 3 A schematic structural diagram of a log data management device for a cloud-edge collaborative platform provided by an embodiment of the present invention;

[0058] Figure 4 A schematic diagram of the architecture of a log data management system for a cloud-edge collaboration platform provided by an embodiment of the present invention;

[0059] Figure 5 A schematic diagram of the component architecture of a log data management system for a cloud-edge collaboration platform provided by an embodiment of the present invention;

[0060] Figure 6 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0061] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0062] In this application, the terms "comprises," "comprising," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0063] The embodiment of the present invention provides a log data management method for a cloud-edge collaborative platform. The method can be applied to the cloud of the cloud-edge collaborative platform. The execution subject can be a server or controller in the cloud. The method flow chart of the method is as follows: Figure 1 As shown, the method includes:

[0064] S101: When receiving a policy configuration request sent by a user through a front-end interface, determining whether the policy configuration request is a cloud-related request;

[0065] The method provided by the embodiment of the present invention can be applied to the cloud side of the cloud-edge collaborative platform, and can be specifically deployed as a log data management system. In the embodiment of the present invention, a visual system front-end interface is designed in advance according to the configuration requirements of the log collection strategy. The front-end interface is provided with a node selection control for configuring the strategy, including selectable nodes, specifically including cloud nodes and various edge nodes. The front-end interface is also provided with input controls for various configuration contents of the log collection strategy, which may include the configuration contents of container log collection and the configuration contents of container cluster events (K8s system events) collection, such as the type of container logs that need to be collected, the name of the container that needs to be collected, the type of container cluster events that need to be collected, etc. The user can select the node that currently needs to configure the log collection strategy through the front-end interface, such as the cloud side or a specified edge side, or can select multiple nodes at the same time for batch configuration. The user can input relevant configuration data through the front-end interface, and then send a policy configuration request to the system by triggering the control.

[0066] When the cloud-based log data management system receives a policy configuration request from a user, it obtains the node ID for the node requiring policy configuration from the request information. Based on the node ID, it identifies whether the node requiring policy configuration is on the cloud or on the edge. If the node requiring policy configuration is on the cloud, the policy configuration request is considered cloud-related. If the node requiring policy configuration is on the edge, the policy configuration request is not considered cloud-related.

[0067] S102: Parse the policy configuration request to obtain first configuration information;

[0068] In the method provided by the embodiment of the present invention, if it is determined that the policy configuration request is a cloud-related request, the cloud will perform data analysis on the policy configuration request, obtain the configuration data for the log configuration policy input by the user, and use the above configuration as the first configuration information.

[0069] S103: Configuring a cloud log collection policy corresponding to the first configuration information in the form of a custom resource, so that a cloud-native log collection tool preset in the cloud collects container logs and container cluster events in real time based on the cloud log collection policy, and stores the container logs and container cluster events in the cloud to a corresponding data storage platform in the cloud;

[0070] In the method provided in this embodiment of the present invention, the cloud defines a log collection configuration policy in the form of a Custom Resource Definition (CRD) based on the configuration data in the first configuration information, and uses the resulting log collection configuration policy as the cloud-side log collection policy. CRD is an extension capability of the Kubernetes container orchestration platform that allows for the customization of new resource types to implement business function logic. This will not be discussed in detail here.

[0071] In the method provided by the embodiment of the present invention, cloud native log collection tools can be deployed in advance in the cloud and each edge end, such as applying the cloud native log collection tool Loggie, and data storage platforms can be deployed in the cloud and each edge end. For example, the data storage platform can be deployed based on the distributed full-text search engine Elasticsearch. The cloud native log collection tool can detect the log collection strategy configured in the system in real time. When a new log collection strategy is configured, the cloud native log collection tool detects that the configuration has changed, and then collects log data based on the new log collection strategy, including collecting container logs and container cluster events in the cloud, and storing the collected log data in the data storage platform corresponding to the cloud.

[0072] S104: Determine, among the edge terminals of the cloud-edge collaboration platform, a first edge terminal corresponding to the policy configuration request;

[0073] In the method provided by an embodiment of the present invention, if it is determined that the policy configuration request is not a cloud-related request, the system will determine the edge end associated with the policy configuration request among the various edge ends of the cloud-edge collaboration platform based on the node identifier, that is, the edge end that currently needs to perform policy configuration, and use this edge end as the first edge end.

[0074] It should be noted that if the user selects multiple nodes for batch configuration, the first edge corresponding to the policy configuration request can have multiple, and each first edge is executed subsequently. And in the batch configuration scenario, if the node identifier corresponding to the policy configuration request includes the identifier of the cloud, that is, the policy configuration request is the request associated with the cloud, in addition to executing the steps of S102 and S103, steps S104 and subsequent step S105 also need to be executed, that is, in addition to configuring the log collection policy in the cloud, the policy configuration request also needs to be sent to the corresponding edge for configuration operation.

[0075] S105: Send the policy configuration request to the first edge through the preset cloud-edge channel, so that the first edge analyzes the policy configuration request, obtains second configuration information, and configures the edge log collection policy corresponding to the second configuration information in the form of a custom resource, so that the preset cloud-native log collection tool in the first edge collects the container log and container cluster event of the first edge in real time based on the edge log collection policy, and stores the container log and container cluster event of the first edge to the data storage platform corresponding to the first edge.

[0076] In the method provided by the embodiment of the application, the system can send the policy configuration request to the first edge through the pre-established cloud-edge channel, trigger the edge to analyze the policy configuration request, and perform corresponding configuration operation. That is, when the edge receives the policy configuration request, it can analyze the policy configuration request, obtain corresponding configuration data, define the log collection configuration policy according to the configuration data in the second configuration information in the form of a custom resource CRD, obtain the edge log collection policy of the current edge, and the cloud-native log collection tool in the edge can collect the container log and container cluster event in the current edge in real time according to the edge log collection policy, and store them in the data storage platform corresponding to the edge.

[0077] As shown in Figure 2 The log data management process provided by the embodiment of the application mainly includes:

[0078] When the cloud platform receives the configuration request associated with the cloud, the configuration data carried in the configuration request is used to configure the collection strategy of the log data, so that the cloud-native log collection tool of the cloud collects the log data such as the container log and the container cluster event of the cloud in real time according to the collection strategy, and stores the log data in the storage platform of the cloud.

[0079] When the cloud platform receives a configuration request associated with the edge, it sends it to the cloud-edge channel, which forwards it to the designated edge. The edge controller configures a log data collection policy based on the configuration data contained in the request. This allows the edge's cloud-native log collection tool to collect container logs, container cluster events, and other logs in real time based on this policy, and store the log data on the edge's storage platform.

[0080] Based on the method provided by the embodiment of the present invention, the user can configure the log collection policy in the cloud or a certain edge end through the front-end interface. When the cloud receives the policy configuration request sent by the user, it can determine whether the request is a request for configuration for the cloud or a request for configuration for the edge end. If the request is a request for configuration for the cloud, the request can be parsed, and the log collection policy can be configured according to the configuration information obtained by the parsing, so that the cloud native log collection tool collects container logs and container cluster events based on the policy, thereby completing the policy configuration for collecting container logs and container cluster events on the cloud. If the request is a request for configuration for the edge end, the cloud can send the request to the corresponding edge end through the cloud-edge channel, triggering the edge end controller to parse the request, thereby configuring the log collection policy, so that the cloud native log collection tool collects container logs and container cluster events based on the policy, thereby completing the policy configuration for collecting container logs and container cluster events on the edge end. Users can configure the collection policy for log data in the cloud and edge end respectively through a unified front-end interface, without having to manually configure multiple components embedded in each node separately, which is conducive to simplifying the policy configuration process and improving operation and maintenance efficiency. Secondly, the cloud and edge use cloud-native log collection tools to monitor and collect container logs and container cluster events in real time, without the need for invasive configuration within the container or the integration of multiple components. This helps reduce the complexity of the cloud-edge collaboration platform and reduce container security risks.

[0081] exist Figure 1 On the basis of the method shown, the method provided in the embodiment of the present invention further includes:

[0082] When receiving a data query request sent by a user through the front-end interface, determining whether the data query request is a cloud-related request;

[0083] In the method provided by the embodiment of the present invention, users can query log data in the cloud or edge through the front-end interface of the log data management system. Specifically, users can select the node where the log data to be queried is located through the front-end interface, that is, whether to query log data in the cloud or log data of a certain edge, and set corresponding query conditions. For example, if querying container logs, the container name, log identifier or log type to be queried can be set. If querying container cluster events, the event type to be queried, the time when the event occurred, etc. can be set. Based on the set query conditions, the corresponding data query request is sent to the system.

[0084] When the cloud receives a data query request from a user, it can retrieve the node ID of the node where the currently queried log data resides from the request information. Based on this node ID, it can determine whether the currently queried log data resides on the cloud or the edge. If the currently queried log data is cloud-based, the data query request is considered a cloud-related request. If the currently queried log data is edge-based, the data query request is not considered a cloud-related request.

[0085] If the data query request is a request associated with the cloud, determining first query information corresponding to the data query request;

[0086] In the method provided by the embodiment of the present invention, if it is determined that the current data query request is a cloud-related request, the data query request is parsed, the current query condition is determined based on the parsed data, and the corresponding query condition is used as the first query information.

[0087] Based on the first query information, calling a data query interface of the data storage platform corresponding to the cloud, and querying the log data corresponding to the first query information in the data storage platform corresponding to the cloud;

[0088] In the front-end interface, the log data corresponding to the first query information is visually displayed.

[0089] In the method provided by an embodiment of the present invention, the cloud can generate a corresponding interface call request based on the first query information, thereby calling the data query interface of the data storage platform, so that the data storage platform performs data query in the stored log data according to the corresponding query conditions to obtain log data that matches the query conditions, that is, the log data corresponding to the first query information. After the cloud obtains the log data corresponding to the first query information, the corresponding log data can be visualized in the front-end interface according to a predetermined display method to complete the query process corresponding to the data query request.

[0090] Based on the method provided in the embodiment of the present invention, users can query container logs through the log data management system to obtain visual data, which is helpful for users to carry out operation and maintenance work.

[0091] Based on the method provided in the above embodiment, in the method provided in the embodiment of the present invention, determining the first query information corresponding to the data query request includes:

[0092] Parsing the data query request to obtain parsed data corresponding to the data query request;

[0093] Determining a query object corresponding to the data query request based on the parsed data; the query object is the container log on the cloud or the container cluster event on the cloud;

[0094] In the method provided by the embodiment of the present invention, data parsing can be performed on the data query request to obtain the parsed data of the request, and the user-specified query object type, that is, the type of log data currently required to be queried, is obtained from the parsed data, thereby determining whether the query object is a container log in the cloud or a container cluster event.

[0095] If the query object is a container cluster event on the cloud, obtaining event parameters corresponding to the query object from the parsed data, and using the event parameters as the first query information;

[0096] In the method provided by an embodiment of the present invention, if the current query object is a container cluster event in the cloud, the parsed data will include event parameters of the container cluster event to be queried, such as event type, event occurrence time, etc., set by the user, and the corresponding event parameters in the parsed data will be used as the first query information.

[0097] If the query object is the container log of the cloud, determining whether the query mode corresponding to the data query request is a context query;

[0098] In the method provided by the embodiments of the present invention, when querying container logs through the front-end interface, users can choose whether to perform a contextual query. The selected query method is carried as a request parameter in the data query request. If the current query target is a container log in the cloud, the parsed data is used to determine whether the user selected a contextual query or a standalone query.

[0099] If the query mode corresponding to the data query request is context query, the container name, log identifier and query start and end time are obtained from the parsed data, and the container name, log identifier and query start and end time are used as the first query information.

[0100] In the method provided by an embodiment of the present invention, if the query method selected by the user is a context query, the parsed data will include parameters such as the container name of the query selected by the user, the log identifier, and the start and end time of the context query (i.e., the query start and end time). The above parameters can be obtained from the parsed data as the first query information, so as to query the specified container log of the specified container based on the above parameters, as well as the container logs generated within a certain time before and after the container log.

[0101] Based on the method provided by the embodiment of the present invention, contextual query of container logs can be provided to users. That is, when users query a specific container log, they can obtain log entries before and after the container log on demand, which helps users better understand the log generation situation.

[0102] exist Figure 1 On the basis of the method shown, the method provided in the embodiment of the present invention further includes:

[0103] If the data query request is not a request associated with the cloud, determining a second edge end corresponding to the data query request among each edge end of the cloud-edge collaboration platform;

[0104] In the method provided by an embodiment of the present invention, if it is determined that the current data query request is not a cloud-related request, the edge end corresponding to the node identifier carried in the data query request can be determined in each edge end, and the edge end can be used as the second edge end.

[0105] Sending the data query request to the second edge end through the cloud-edge channel, so that the second edge end determines second query information corresponding to the data query request, and based on the second query information, calling a data query interface of a data storage platform corresponding to the second edge end to query log data corresponding to the second query information in the data storage platform corresponding to the second edge end, and sending the log data corresponding to the second query information to the cloud;

[0106] In the method provided by the embodiment of the application, the cloud end can send a data query request to a pre-constructed cloud-edge channel, the cloud-edge channel forwards the data query request to a second edge end, triggers the edge end to analyze the data query request, and performs a corresponding data query operation. When the edge end receives the data query request for analysis, the corresponding query condition can be determined based on the analysis data, the obtained query condition is taken as second query information, and a corresponding interface calling request is generated based on the second query information, so that the data query interface of the data storage platform of the edge end is called, so that the data query platform of the edge end performs data query in the stored log data according to the corresponding query condition, and obtains the log data matched with the second query information. After the second edge end obtains the log data corresponding to the second query information, the log data can be fed back to the cloud end through the cloud-edge channel.

[0107] When the log data corresponding to the second query information is received, the log data corresponding to the second query information is visually displayed in the front-end interface.

[0108] In the method provided by the embodiment of the application, when the cloud end receives the log data corresponding to the second query information sent by the second edge end, the log data can be visually displayed in a predetermined display mode.

[0109] Based on the method provided by the embodiment of the application, the log data of the cloud end and the log data of the edge end can be uniformly queried, and the queried log data is visually displayed to the user. The user can query the log data of the cloud end or the edge end through a unified front-end interface, obtain visual data, and facilitate the user to carry out operation and maintenance work.

[0110] In Figure 1 Based on the method shown in the method, the method provided by the embodiment of the application further comprises:

[0111] When receiving the data deletion request sent by the user through the front-end interface, it is judged whether the data deletion request is a request associated with the cloud end;

[0112] In the method provided by the embodiment of the application, the user can clean the log data in the cloud end or the edge end through the front-end interface of the log data management system. Specifically, the user can select the node where the log data to be deleted is located through the front-end interface, that is, whether to delete the log data in the cloud end or the log data in a certain edge end, and input the related information of the log data to be deleted, such as the type or event identifier of the container cluster event to be deleted, the event generation time, the container name of the container log to be deleted, and the container log identifier.

[0113] When the cloud receives a data deletion request sent by a user, it can obtain the node identifier of the node where the log data currently to be deleted is located from the request information, and determine based on the node identifier whether the current data deletion request is a request to delete the log data in the cloud. If the data deletion request is a request to delete the log data in the cloud, then the data deletion request is considered to be a cloud-related request. If the data deletion request is a request to delete the log data in the edge, then the data deletion request is considered not to be a cloud-related request.

[0114] If the data deletion request is a request associated with the cloud, parsing the data deletion request to obtain first data information;

[0115] In the method provided by an embodiment of the present invention, if it is determined that the current data deletion request is a cloud-related request, the cloud will parse the data deletion request, determine the relevant information of the data that needs to be deleted based on the parsed data, and use this information as the first data information.

[0116] Based on the first data information, calling a data operation interface of a data storage platform corresponding to the cloud, and deleting log data corresponding to the first data information in the data storage platform corresponding to the cloud;

[0117] In the method provided by an embodiment of the present invention, the cloud can generate a corresponding interface call request based on the first data information, thereby calling the data operation interface of the cloud data storage platform, so that the data storage platform can filter out matching log data according to the data information and perform a deletion operation on the above log data.

[0118] If the data deletion request is not a request associated with the cloud, determining a third edge end corresponding to the data deletion request among the edge ends of the cloud-edge collaboration platform;

[0119] In the method provided by an embodiment of the present invention, if it is determined that the current data deletion request is not a cloud-related request, then in each edge end, the edge end corresponding to the node identifier carried in the data deletion request is determined, and the edge end is used as the third edge end.

[0120] The data deletion request is sent to the third edge end through the cloud-edge channel, so that the third edge end parses the data deletion request to obtain second data information, and based on the second data information, calls the data operation interface of the data storage platform corresponding to the third edge end to delete the log data corresponding to the second data information in the data storage platform corresponding to the third edge end.

[0121] In the method provided by an embodiment of the present invention, the cloud can send a data deletion request to a cloud-edge channel, which forwards the data deletion request to a third edge end, triggering the edge end to parse the data deletion request, determine the relevant information of the data that currently needs to be deleted based on the parsed data, use this information as the second data information, and generate a corresponding interface call request based on the second data information, thereby calling the data operation interface of the data storage platform of the edge end, so that the data storage platform filters out matching log data according to the data information and performs corresponding deletion operations.

[0122] Based on the method provided in the embodiment of the present invention, users can uniformly clean up log data on the cloud and edge through the front-end interface, which is conducive to improving the convenience of users in managing log data.

[0123] exist Figure 1 On the basis of the method shown, the method provided in the embodiment of the present invention further includes:

[0124] When receiving an alarm configuration request sent by a user through the front-end interface, determining whether the alarm configuration request is a request associated with the cloud;

[0125] In the method provided by the embodiment of the present invention, the user can configure the alarm rules in the cloud or edge through the front-end interface of the log data management system, that is, the user can select the node for which the alarm rules currently need to be configured, that is, whether the alarm rules of the cloud or the edge need to be configured, and input the alarm rules to be configured, and then send a response alarm configuration request to the system. It is understandable that in the cloud-edge collaborative platform, the cloud and edge will perform anomaly detection based on the log data they collect. When anomalies or certain events are detected, alarm processing must be performed according to certain alarm rules.

[0126] When the cloud receives an alarm configuration request sent by a user, it can obtain the node identifier of the node that currently needs to configure the alarm rules from the request information, and determine whether the current alarm configuration request is a request to configure the alarm rules of the cloud based on the node identifier. If the alarm configuration request is a request to configure the alarm rules of the cloud, then the alarm configuration request is considered to be a cloud-related request. If the alarm configuration request is a request to configure the alarm rules of the edge, then the alarm configuration request is considered not to be a cloud-related request.

[0127] If the alarm configuration request is a request associated with the cloud, parsing the alarm configuration request to obtain first alarm rule information;

[0128] In the method provided by the embodiment of the present invention, if it is determined that the current alarm configuration request is a cloud-related request, the cloud will parse the alarm configuration request, determine the rule content of the user-configured alarm rule based on the parsed data, and use the above rule content as the first alarm rule information.

[0129] Setting a cloud alarm rule based on the first alarm rule information, and issuing an abnormal alarm for the cloud based on the cloud alarm rule;

[0130] In the method provided by the embodiment of the present invention, the cloud can configure the cloud alarm rules based on the first alarm rule information to obtain cloud alarm rules, and alarm the monitored anomalies according to the cloud alarm rules.

[0131] If the alarm configuration request is not a request associated with the cloud, determining a fourth edge end corresponding to the alarm configuration request among the edge ends of the cloud-edge collaboration platform;

[0132] In the method provided by an embodiment of the present invention, if it is determined that the current alarm configuration request is not a cloud-associated request, then based on the node identifier carried in the alarm configuration request, an edge end that matches the node identifier is determined among each edge end, and the edge end is used as the fourth edge end.

[0133] The alarm configuration request is sent to the fourth edge end through the cloud edge channel, so that the fourth edge end parses the alarm configuration request, obtains the second alarm rule information, sets the edge end alarm rule based on the second alarm rule information, and issues an abnormal alarm to the fourth edge end based on the edge end alarm rule.

[0134] In the method provided by the embodiment of the present invention, the cloud can send the current alarm configuration request to the cloud-edge channel, and the cloud-edge channel forwards the request to the fourth edge end, triggering the edge end to parse the alarm configuration request, obtain the rule content of the alarm rule configured by the user, thereby setting the alarm rules of the edge end, and issuing an abnormal alarm according to the alarm rules.

[0135] Based on the method provided in the embodiment of the present invention, users can customize the alarm rules of the cloud and edge through the front-end interface of the log data management system, which is conducive to users to uniformly configure the alarm rules of the cloud-edge collaborative platform and improve the convenience of operation and maintenance.

[0136] and Figure 1 Corresponding to the log data management method of a cloud-edge collaborative platform shown in FIG, an embodiment of the present invention further provides a log data management device of a cloud-edge collaborative platform for Figure 1 The specific implementation of the method shown in , the device is applied to the cloud side of the cloud-edge collaborative platform, and its structural diagram is as follows Figure 3 Shown, including:

[0137] The judging unit 301 is configured to, upon receiving a policy configuration request sent by a user through a front-end interface, judge whether the policy configuration request is a cloud-related request;

[0138] a parsing unit 302 configured to parse the policy configuration request to obtain first configuration information if the policy configuration request is a request associated with the cloud;

[0139] A configuration unit 303 is configured to configure a cloud log collection policy corresponding to the first configuration information in the form of a custom resource, so that a cloud native log collection tool preset in the cloud collects container logs and container cluster events in real time based on the cloud log collection policy, and stores the container logs and container cluster events in the cloud to a corresponding data storage platform in the cloud;

[0140] A determining unit 304 is configured to determine, if the policy configuration request is not a request associated with the cloud, a first edge end corresponding to the policy configuration request among the edge ends of the cloud-edge collaboration platform;

[0141] The sending unit 305 is used to send the policy configuration request to the first edge end through a preset cloud-edge channel, so that the first edge end parses the policy configuration request, obtains second configuration information, and configures the edge end log collection policy corresponding to the second configuration information in the form of a custom resource, so that the cloud native log collection tool preset in the first edge end collects the container logs and container cluster events of the first edge end in real time based on the edge end log collection policy, and stores the container logs and container cluster events of the first edge end to the data storage platform corresponding to the first edge end.

[0142] By applying the device provided by the embodiment of the present invention, a user can configure the log collection policy in the cloud or a certain edge end through the front-end interface. When the cloud receives the policy configuration request sent by the user, it can determine whether the request is a request for configuration for the cloud or a request for configuration for the edge end. If the request is a request for configuration for the cloud, the request can be parsed, and the log collection policy can be configured according to the configuration information obtained by the parsing, so that the cloud native log collection tool collects container logs and container cluster events based on the policy, thereby completing the policy configuration for collecting container logs and container cluster events on the cloud. If the request is a request for configuration for the edge end, the cloud can send the request to the corresponding edge end through the cloud-edge channel, triggering the edge end controller to parse the request, thereby configuring the log collection policy, so that the cloud native log collection tool collects container logs and container cluster events based on the policy, thereby completing the policy configuration for collecting container logs and container cluster events on the edge end. Users can configure the collection policy for log data in the cloud and edge end respectively through a unified front-end interface, without having to manually configure multiple components embedded in each node separately, which is conducive to simplifying the policy configuration process and improving operation and maintenance efficiency. Secondly, the cloud and edge use cloud-native log collection tools to monitor and collect container logs and container cluster events in real time, without the need for invasive configuration within the container or the integration of multiple components. This helps reduce the complexity of the cloud-edge collaboration platform and reduce container security risks.

[0143] exist Figure 3 Based on the device shown, the device provided by the embodiment of the present invention can be further expanded into multiple units. The functions of each unit can be found in the descriptions of the various embodiments provided in the previous text on the log data management method of the cloud-edge collaborative platform, and no further examples will be given here.

[0144] The embodiment of the present invention also provides a log data management system for a cloud-edge collaborative platform, the architecture of which can be as follows: Figure 4 Shown, including:

[0145] A cloud module 401 and multiple edge modules 402; the cloud module 401 is deployed on the cloud of the cloud-edge collaboration platform, the multiple edge modules 402 correspond one-to-one to each edge of the cloud-edge collaboration platform, and each edge module 402 is deployed on the edge corresponding to the edge module;

[0146] The cloud module 401 is configured to: upon receiving a policy configuration request sent by a user through the front-end interface, determine whether the policy configuration request is a request associated with the cloud; if the policy configuration request is a request associated with the cloud, parse the policy configuration request to obtain first configuration information; configure the cloud log collection policy corresponding to the first configuration information in the form of a custom resource, so that the cloud native log collection tool preset in the cloud collects the container logs and container cluster events of the cloud in real time based on the cloud log collection policy, and stores the container logs and container cluster events of the cloud to the corresponding data storage platform of the cloud; if the policy configuration request is not a request associated with the cloud, determine the edge end corresponding to the policy configuration request in each edge end of the cloud-edge collaborative platform; and send the policy configuration request to its corresponding edge end through the preset cloud-edge channel;

[0147] The edge module 402 is configured to: when the edge corresponding to the edge module receives the policy configuration request, parse the policy configuration request to obtain second configuration information; configure the edge log collection policy corresponding to the second configuration information in the form of custom resources, so that the cloud-native log collection tool preset in the edge collects the container logs and container cluster events of the edge in real time based on the edge log collection policy, and stores the container logs and container cluster events of the edge to the data storage platform corresponding to the edge.

[0148] The system provided by the embodiment of the present invention is configured with a cloud module and an edge module respectively. The cloud module is deployed in the cloud and is used to receive the policy configuration request of the system front-end interface and perform the corresponding policy configuration or request forwarding operation. The specific processing process is combined with the above Figure 1 The corresponding processing procedures in the method shown are the same, and reference can be made to the previous embodiment for details. Figure 1 The description of the method shown will not be repeated here.

[0149] The edge module is deployed on the corresponding edge. When the edge receives a policy configuration request, it can execute the corresponding policy configuration operation. The specific processing process is combined with the previous Figure 1 The corresponding processing procedures in the method shown are the same, and reference can be made to the previous embodiment for details. Figure 1 The description of the method shown will not be repeated here.

[0150] By applying the system provided by the embodiment of the present invention, a user can configure the log collection policy in the cloud or a certain edge end through the front-end interface of the system. When the system receives the policy configuration request sent by the user, it can determine whether the request is a request for configuration for the cloud or a request for configuration for the edge end. If the request is a request for configuration for the cloud, the request can be parsed, and the log collection policy can be configured according to the configuration information obtained by the parsing, so that the cloud native log collection tool collects container logs and container cluster events based on the policy, thereby completing the policy configuration for collecting cloud container logs and container cluster events. If the request is a request for configuration for the edge end, the cloud can send the request to the corresponding edge end through the cloud-edge channel, triggering the edge end controller to parse the request, thereby configuring the log collection policy, so that the cloud native log collection tool collects container logs and container cluster events based on the policy, thereby completing the policy configuration for collecting edge container logs and container cluster events. Users can configure the collection policy for log data in the cloud and edge end respectively through a unified front-end interface, without having to manually configure multiple components embedded in each node separately, which is conducive to simplifying the policy configuration process and improving operation and maintenance efficiency. Secondly, the cloud and edge use cloud-native log collection tools to monitor and collect container logs and container cluster events in real time, without the need for invasive configuration within the container or the integration of multiple components. This helps reduce the complexity of the cloud-edge collaboration platform and reduce container security risks.

[0151] It should be noted that the system provided by the embodiment of the present invention can further perform other configurations on the cloud module and the edge module to realize functions such as data query, data deletion and alarm rule configuration. For specific function implementation, please refer to the description of the various embodiments provided in the previous text for the log data management method of the cloud-edge collaborative platform, and no further examples will be given here.

[0152] In order to better illustrate the system provided by the embodiment of the present invention, based on the methods provided in the above embodiments and combined with actual application scenarios, the embodiment of the present invention provides another log data management system for the cloud-edge collaborative platform, and the architecture for implementing system deployment can be as follows: Figure 5As shown. In the cloud-edge collaborative platform in the embodiment of the present invention, the cloud-native log collection tool Loggie and the full-text search engine ElasticSearch are deployed on the cloud and edge respectively. The CloudHub component is deployed on the cloud, which is a component for messages between the edge and the cloud. The EdgeHub component is deployed on the edge, which is used to receive messages sent from the cloud and perform processing on the edge. A cloud module is deployed on the cloud platform, which can receive various requests sent by users through the front-end interface, such as configuration requests for log collection strategies, data query requests / data deletion requests for log data, etc. The cloud platform can analyze whether the received request belongs to a request for operation on the cloud, and the cloud platform will process it. If the request is a configuration request for the log collection strategy, the log collection strategy of Loggie is configured so that it collects the log data on the cloud according to the strategy and stores it in ElasticSearch. If the request is a request for data query or data deletion, the interface of ElasticSearch is called to perform the corresponding operation.

[0153] If the request received by the cloud platform is a request for an operation on the edge, the cloud platform forwards the request to the cloud-edge channel through the CloudHub component, and the cloud-edge channel sends it to the corresponding edge. After receiving the request, the EdgHub component on the edge performs corresponding processing. If the request is a configuration request for a log collection policy, the log collection policy of Loggie is configured so that it collects the log data of the edge according to the policy and stores it in ElasticSearch. If the request is a request for data query or data deletion, the ElasticSearch interface is called to perform the corresponding operation. After the edge completes the operation, the execution result of the operation can be fed back to the cloud platform.

[0154] Specifically, during the configuration of the log collection strategy, the system will create the log collection configuration of the application in the form of CRD in the K8s cluster (including the application to be collected, whether to collect standard output or log file, log storage backend, etc.). After Loggie detects that the configuration has changed, it will start collecting the container logs of the application based on the configuration, and store the collected logs in the backend ElasticSearch. Based on Loggie's real-time collection capabilities, when it detects changes in container log files, it will collect container logs to achieve real-time collection of container logs. At the same time, the system can configure Loggie by default to collect Kubernetes system events (i.e., container cluster events), realize the collection of K8s system events, and store the collected system events in ElasticSearch.

[0155] The system provided by the embodiments of the present invention stores collected logs and events in ElasticSearch and provides a visual interface to facilitate log and event query and retrieval. The system assembles query conditions specified by the user through the interface and calls the ElasticSearch API to implement query and retrieval functions. Users can query the context associated with a particular log on the front end. The system automatically retrieves logs related to the specified log based on parameters such as container name and start and end time, allowing users to trace application logs.

[0156] Through the front-end, users can select the log index to be cleaned, or more fine-grained log deletion conditions, such as timestamp, application, etc. The system can assemble parameters based on these conditions, call the back-end storage interface, that is, the ElasticSearch interface, to perform log data deletion operations to clean up related logs.

[0157] The system provided by the embodiments of the present invention can configure alarm rules by presetting log alarm rules, and also supports users to configure alarm rules through the front-end interface. When an anomaly or critical event triggers an alarm, the system will send an alarm notification message to the notification object configured according to the alarm rule and store the alarm in the alarm history. The back-end storage can use ElasticSearch.

[0158] The system provided by the embodiment of the present invention supports multi-tenant access. Each tenant can only view the projects under that tenant, thereby achieving tenant isolation for container log collection, viewing, and alarm notification. For K8s system events, only system administrators are supported to perform related operations.

[0159] The system provided by the embodiments of this invention utilizes a cloud-native architecture and provides a front-end visual interface. Through an intuitive graphical user interface, users can uniformly set log collection rules for both the cloud and edge without modifying the internal container configuration. This non-invasive configuration method supports both container standard output and container log files, simplifying the configuration process, lowering technical barriers, and significantly improving the user experience.

[0160] The system provided in this embodiment also supports querying the container log context function, allowing users to query log entries before and after a specific log event to obtain the complete context. Users can customize the time window to determine the scope of the context query, and provide a real-time log stream query function to help users monitor the current log generation situation.

[0161] The system provided by the embodiment of the present invention enables users to uniformly configure and monitor container logs on the cloud and edge through a cloud-based front-end interface, covering the entire life cycle of log data from generation to archiving, and providing a one-stop solution, including management content such as collection, storage, query, analysis, visualization, alarm and cleanup. It simplifies the log management method in multiple locations and environments, improves development and operation efficiency, and improves user experience.

[0162] An embodiment of the present invention also provides a storage medium, which includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the log data management method of the cloud-edge collaborative platform as described above.

[0163] The embodiment of the present invention further provides an electronic device, the structural diagram of which is shown in FIG. Figure 6 As shown, it specifically includes a memory 501 and one or more instructions 502, wherein the one or more instructions 502 are stored in the memory 501 and are configured to be executed by one or more processors 503 to perform the following operations:

[0164] When receiving a policy configuration request sent by a user through the front-end interface, determining whether the policy configuration request is a request associated with the cloud;

[0165] If the policy configuration request is a request associated with the cloud, parsing the policy configuration request to obtain first configuration information;

[0166] Configuring a cloud-based log collection policy corresponding to the first configuration information in the form of a custom resource, so that a cloud-native log collection tool preset in the cloud collects container logs and container cluster events in real time based on the cloud-based log collection policy, and stores the container logs and container cluster events in the cloud to a corresponding data storage platform in the cloud;

[0167] If the policy configuration request is not a request associated with the cloud, determining a first edge end corresponding to the policy configuration request among each edge end of the cloud-edge collaboration platform;

[0168] The policy configuration request is sent to the first edge end through a preset cloud-edge channel, so that the first edge end parses the policy configuration request to obtain second configuration information, and configures the edge end log collection policy corresponding to the second configuration information in the form of a custom resource, so that the cloud native log collection tool preset in the first edge end collects the container logs and container cluster events of the first edge end in real time based on the edge end log collection policy, and stores the container logs and container cluster events of the first edge end to the data storage platform corresponding to the first edge end.

[0169] Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple. For relevant parts, refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without expending creative work.

[0170] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0171] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A log data management method for a cloud-edge collaborative platform, characterized in that: The method is applied to the cloud side of the cloud-edge collaboration platform, and the method includes: When receiving a policy configuration request sent by a user through the front-end interface, determining whether the policy configuration request is a request associated with the cloud; If the policy configuration request is a request associated with the cloud, parsing the policy configuration request to obtain first configuration information; Configuring a cloud-based log collection policy corresponding to the first configuration information in the form of a custom resource, so that a cloud-native log collection tool preset in the cloud collects container logs and container cluster events in real time based on the cloud-based log collection policy, and stores the container logs and container cluster events in the cloud to a corresponding data storage platform in the cloud; If the policy configuration request is not a request associated with the cloud, determining a first edge end corresponding to the policy configuration request among each edge end of the cloud-edge collaboration platform; The policy configuration request is sent to the first edge end through a preset cloud-edge channel, so that the first edge end parses the policy configuration request to obtain second configuration information, and configures the edge end log collection policy corresponding to the second configuration information in the form of a custom resource, so that the cloud native log collection tool preset in the first edge end collects the container logs and container cluster events of the first edge end in real time based on the edge end log collection policy, and stores the container logs and container cluster events of the first edge end to the data storage platform corresponding to the first edge end.

2. The log data management method of the cloud-edge collaborative platform according to claim 1 is characterized in that: Also includes: When receiving a data query request sent by a user through the front-end interface, determining whether the data query request is a cloud-related request; If the data query request is a request associated with the cloud, determining first query information corresponding to the data query request; Based on the first query information, calling a data query interface of the data storage platform corresponding to the cloud, and querying the log data corresponding to the first query information in the data storage platform corresponding to the cloud; In the front-end interface, the log data corresponding to the first query information is visually displayed.

3. The log data management method of the cloud-edge collaborative platform according to claim 2 is characterized in that: The determining the first query information corresponding to the data query request includes: Parsing the data query request to obtain parsed data corresponding to the data query request; Determining a query object corresponding to the data query request based on the parsed data; the query object is the container log on the cloud or the container cluster event on the cloud; If the query object is a container cluster event on the cloud, obtaining event parameters corresponding to the query object from the parsed data, and using the event parameters as the first query information; If the query object is the container log of the cloud, determining whether the query mode corresponding to the data query request is a context query; If the query mode corresponding to the data query request is context query, the container name, log identifier and query start and end time are obtained from the parsed data, and the container name, log identifier and query start and end time are used as the first query information.

4. The log data management method of the cloud-edge collaborative platform according to claim 2 is characterized in that: Also includes: If the data query request is not a request associated with the cloud, determining a second edge end corresponding to the data query request among each edge end of the cloud-edge collaboration platform; Sending the data query request to the second edge end through the cloud-edge channel, so that the second edge end determines second query information corresponding to the data query request, and based on the second query information, calling a data query interface of a data storage platform corresponding to the second edge end to query log data corresponding to the second query information in the data storage platform corresponding to the second edge end, and sending the log data corresponding to the second query information to the cloud; When the log data corresponding to the second query information is received, the log data corresponding to the second query information is visually displayed in the front-end interface.

5. The log data management method of the cloud-edge collaborative platform according to claim 1 is characterized in that: Also includes: When receiving a data deletion request sent by the user through the front-end interface, determining whether the data deletion request is a cloud-related request; If the data deletion request is a request associated with the cloud, parsing the data deletion request to obtain first data information; Based on the first data information, calling a data operation interface of a data storage platform corresponding to the cloud, and deleting log data corresponding to the first data information in the data storage platform corresponding to the cloud; If the data deletion request is not a request associated with the cloud, determining a third edge end corresponding to the data deletion request among the edge ends of the cloud-edge collaboration platform; The data deletion request is sent to the third edge end through the cloud-edge channel, so that the third edge end parses the data deletion request to obtain second data information, and based on the second data information, calls the data operation interface of the data storage platform corresponding to the third edge end to delete the log data corresponding to the second data information in the data storage platform corresponding to the third edge end.

6. The log data management method of the cloud-edge collaborative platform according to claim 1 is characterized in that: Also includes: When receiving an alarm configuration request sent by a user through the front-end interface, determining whether the alarm configuration request is a request associated with the cloud; If the alarm configuration request is a request associated with the cloud, parsing the alarm configuration request to obtain first alarm rule information; Setting a cloud alarm rule based on the first alarm rule information, and issuing an abnormal alarm for the cloud based on the cloud alarm rule; If the alarm configuration request is not a request associated with the cloud, determining a fourth edge end corresponding to the alarm configuration request among the edge ends of the cloud-edge collaboration platform; The alarm configuration request is sent to the fourth edge end through the cloud edge channel, so that the fourth edge end parses the alarm configuration request, obtains the second alarm rule information, sets the edge end alarm rule based on the second alarm rule information, and issues an abnormal alarm to the fourth edge end based on the edge end alarm rule.

7. A log data management device for a cloud-edge collaborative platform, characterized in that: The device is applied to the cloud side of the cloud-edge collaboration platform, and the device includes: a judgment unit, configured to, upon receiving a policy configuration request sent by a user through a front-end interface, judge whether the policy configuration request is a cloud-related request; a parsing unit, configured to parse the policy configuration request to obtain first configuration information if the policy configuration request is a request associated with the cloud; a configuration unit configured to configure, in the form of custom resources, a cloud log collection policy corresponding to the first configuration information, so that a cloud-native log collection tool preset in the cloud collects container logs and container cluster events in real time on the cloud based on the cloud log collection policy, and stores the container logs and container cluster events on the cloud to a corresponding data storage platform on the cloud; a determining unit, configured to determine, if the policy configuration request is not a request associated with the cloud, a first edge end corresponding to the policy configuration request among the edge ends of the cloud-edge collaboration platform; A sending unit is used to send the policy configuration request to the first edge end through a preset cloud-edge channel, so that the first edge end parses the policy configuration request, obtains second configuration information, and configures the edge end log collection policy corresponding to the second configuration information in the form of a custom resource, so that the cloud native log collection tool preset in the first edge end collects the container logs and container cluster events of the first edge end in real time based on the edge end log collection policy, and stores the container logs and container cluster events of the first edge end to the data storage platform corresponding to the first edge end.

8. A log data management system for a cloud-edge collaborative platform, characterized in that: include: A cloud module and multiple edge modules; the cloud module is deployed on the cloud of the cloud-edge collaboration platform, the multiple edge modules correspond one-to-one to each edge of the cloud-edge collaboration platform, and each edge module is deployed on the edge corresponding to the edge module; The cloud module is configured to: upon receiving a policy configuration request sent by a user through a front-end interface, determine whether the policy configuration request is a request associated with the cloud; if the policy configuration request is a request associated with the cloud, parse the policy configuration request to obtain first configuration information; In the form of custom resources, a cloud-based log collection strategy corresponding to the first configuration information is configured, so that a cloud-native log collection tool preset in the cloud collects container logs and container cluster events in real time on the cloud based on the cloud-based log collection strategy, and stores the container logs and container cluster events on the cloud to a data storage platform corresponding to the cloud; if the policy configuration request is not a request associated with the cloud, then the edge end corresponding to the policy configuration request is determined in each edge end of the cloud-edge collaborative platform; and the policy configuration request is sent to its corresponding edge end through a preset cloud-edge channel; The edge module is configured to: when the edge end corresponding to the edge module receives the policy configuration request, parse the policy configuration request to obtain second configuration information; In the form of custom resources, configure the edge log collection strategy corresponding to the second configuration information, so that the cloud native log collection tool preset in the edge end collects the container logs and container cluster events of the edge end in real time based on the edge log collection strategy, and stores the container logs and container cluster events of the edge end to the data storage platform corresponding to the edge end.

9. A storage medium, characterized in that: The storage medium includes stored instructions, wherein, when the instructions are executed, the device where the storage medium is located is controlled to execute the log data management method of the cloud-edge collaboration platform as described in any one of claims 1 to 6.

10. An electronic device, characterized in that: It includes a memory and one or more instructions, wherein one or more instructions are stored in the memory and are configured to be executed by one or more processors to perform the log data management method of the cloud-edge collaborative platform as described in any one of claims 1 to 6.

Images