Log data management method of cloud edge collaboration platform and related equipment

Abstract

The invention provides a log data management method of a cloud-side collaboration platform and related equipment, and relates to the field of cloud-side collaboration.The method comprises the steps that when a cloud end receives a strategy configuration request sent by a user through a front end, whether the request is a request associated with the cloud end or not is judged, if yes, the request is analyzed, configuration information is obtained, and the configuration information is sent to the cloud end; a cloud log collection strategy is configured, and log data of the cloud are collected and stored in real time based on the strategy; if the request is not the request associated with the cloud, determining an edge end corresponding to the request; and sending the request to a corresponding edge end through a cloud side channel to enable the edge end to carry out request analysis and obtain configuration information, thereby configuring an edge end log collection strategy, and carrying out real-time collection and storage on log data of the edge end based on the strategy. By applying the method provided by the invention, the user can uniformly configure the log collection strategies of the cloud end and the edge end through the front-end interface, so that the operation and maintenance efficiency can be improved.

Description

Technical Field

[0001] The present invention relates to the field of cloud-edge collaboration, and in particular to a log data management method and related equipment for a cloud-edge collaboration platform. Background Art

[0002] With the development of cloud computing technology, cloud-edge collaboration has been widely used to deploy various business platforms in industrial Internet scenarios. Business platforms based on cloud-edge collaboration can be called cloud-edge collaboration platforms. One of the common application deployment methods in cloud-edge collaboration platforms is to deploy applications through containers. In the container orchestration scenario, the Kubernetes (K8s) system is one of the common container orchestration platforms. In the container application scenario based on the K8s system, log data such as container logs and container cluster events (i.e., K8s system events) need to be collected to facilitate the operation and maintenance of the application environment.

[0003] At present, the collection of log data is usually completed by using component intrusive configuration. For example, the collection of container logs usually configures container log collection components such as log-Pilot in the container. The components collect container logs and store the logs on the data storage platform.

[0004] The cloud-edge collaboration platform usually includes the cloud and multiple edge terminals, that is, there are multiple nodes, and multiple containers are deployed on each node. Based on the existing log data collection method, when it is necessary to configure or update the log data collection rules, relevant personnel need to perform intrusive configuration inside the container of each node to set the data collection rules of the corresponding components. The configuration and update process of log data collection rules is relatively cumbersome, resulting in low operation and maintenance efficiency. Summary of the invention

[0005] In view of this, an embodiment of the present invention provides a log data management method for a cloud-edge collaborative platform to solve the problem that the existing log collection method of the cloud-edge collaborative platform requires manual configuration of each component in each node, the process is relatively cumbersome, and the operation and maintenance efficiency is low.

[0006] An embodiment of the present invention also provides a log data management device for a cloud-edge collaborative platform to ensure the actual implementation and application of the above method.

[0007] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0008] A log data management method for a cloud-edge collaborative platform, the method being applied to a cloud of the cloud-edge collaborative platform, the method comprising:

[0009] When receiving a policy configuration request sent by a user through a front-end interface, determining whether the policy configuration request is a request associated with the cloud;

[0010] If the policy configuration request is a request associated with the cloud, parsing the policy configuration request to obtain first configuration information;

[0011] In the form of custom resources, a cloud log collection strategy corresponding to the first configuration information is configured, so that a cloud native log collection tool preset in the cloud collects container logs and container cluster events in real time on the cloud based on the cloud log collection strategy, and stores the container logs and container cluster events on the cloud to a data storage platform corresponding to the cloud;

[0012] If the policy configuration request is not a request associated with the cloud, determining a first edge end corresponding to the policy configuration request among the edge ends of the cloud-edge collaboration platform;

[0013] The policy configuration request is sent to the first edge end through a preset cloud-edge channel, so that the first edge end parses the policy configuration request to obtain second configuration information, and configures an edge end log collection policy corresponding to the second configuration information in the form of a custom resource, so that the cloud native log collection tool preset in the first edge end collects the container logs and container cluster events of the first edge end in real time based on the edge end log collection policy, and stores the container logs and container cluster events of the first edge end to the data storage platform corresponding to the first edge end.

[0014] The above method may optionally further include:

[0015] When receiving a data query request sent by a user through the front-end interface, determining whether the data query request is a request associated with the cloud;

[0016] If the data query request is a request associated with the cloud, determining first query information corresponding to the data query request;

[0017] Based on the first query information, calling a data query interface of a data storage platform corresponding to the cloud, and querying the log data corresponding to the first query information in the data storage platform corresponding to the cloud;

[0018] In the front-end interface, the log data corresponding to the first query information is visually displayed.

[0019] In the above method, optionally, the determining the first query information corresponding to the data query request includes:

[0020] Performing data parsing on the data query request to obtain parsed data corresponding to the data query request;

[0021] Based on the parsed data, determining a query object corresponding to the data query request; the query object is a container log on the cloud or a container cluster event on the cloud;

[0022] If the query object is a container cluster event on the cloud, obtaining event parameters corresponding to the query object from the parsed data, and using the event parameters as the first query information;

[0023] If the query object is the container log of the cloud, determining whether the query mode corresponding to the data query request is a context query;

[0024] If the query mode corresponding to the data query request is context query, the container name, log identifier and query start and end time are obtained from the parsed data, and the container name, log identifier and query start and end time are used as the first query information.

[0025] The above method may optionally further include:

[0026] If the data query request is not a request associated with the cloud, determining a second edge end corresponding to the data query request among the edge ends of the cloud-edge collaboration platform;

[0027] The data query request is sent to the second edge end through the cloud-edge channel, so that the second edge end determines the second query information corresponding to the data query request, and based on the second query information, calls the data query interface of the data storage platform corresponding to the second edge end to query the log data corresponding to the second query information in the data storage platform corresponding to the second edge end, and sends the log data corresponding to the second query information to the cloud end;

[0028] When the log data corresponding to the second query information is received, the log data corresponding to the second query information is visually displayed in the front-end interface.

[0029] The above method may optionally further include:

[0030] When receiving a data deletion request sent by the user through the front-end interface, determining whether the data deletion request is a request associated with the cloud;

[0031] If the data deletion request is a request associated with the cloud, parsing the data deletion request to obtain first data information;

[0032] Based on the first data information, calling a data operation interface of a data storage platform corresponding to the cloud, and deleting log data corresponding to the first data information in the data storage platform corresponding to the cloud;

[0033] If the data deletion request is not a request associated with the cloud, determining a third edge end corresponding to the data deletion request among the edge ends of the cloud-edge collaboration platform;

[0034] The data deletion request is sent to the third edge terminal through the cloud-edge channel, so that the third edge terminal parses the data deletion request to obtain the second data information, and based on the second data information, calls the data operation interface of the data storage platform corresponding to the third edge terminal to delete the log data corresponding to the second data information in the data storage platform corresponding to the third edge terminal.

[0035] The above method may optionally further include:

[0036] When receiving an alarm configuration request sent by a user through a front-end interface, determining whether the alarm configuration request is a request associated with the cloud;

[0037] If the alarm configuration request is a request associated with the cloud, parsing the alarm configuration request to obtain first alarm rule information;

[0038] Setting a cloud alarm rule based on the first alarm rule information, and issuing an abnormal alarm to the cloud based on the cloud alarm rule;

[0039] If the alarm configuration request is not a request associated with the cloud, determining a fourth edge end corresponding to the alarm configuration request among the edge ends of the cloud-edge collaboration platform;

[0040] The alarm configuration request is sent to the fourth edge end through the cloud edge channel, so that the fourth edge end parses the alarm configuration request, obtains the second alarm rule information, sets the edge end alarm rule based on the second alarm rule information, and issues an abnormal alarm to the fourth edge end based on the edge end alarm rule.

[0041] A log data management device for a cloud-edge collaborative platform, the device is applied to the cloud of the cloud-edge collaborative platform, and the device includes:

[0042] A determination unit, configured to determine whether a policy configuration request is a request associated with the cloud when receiving a policy configuration request sent by a user through a front-end interface;

[0043] a parsing unit, configured to parse the policy configuration request to obtain first configuration information if the policy configuration request is a request associated with the cloud;

[0044] A configuration unit, configured to configure a cloud log collection strategy corresponding to the first configuration information in the form of a custom resource, so that a cloud native log collection tool preset in the cloud collects container logs and container cluster events in real time on the cloud based on the cloud log collection strategy, and stores the container logs and container cluster events on the cloud to a data storage platform corresponding to the cloud;

[0045] A determination unit, configured to determine, if the policy configuration request is not a request associated with the cloud, a first edge end corresponding to the policy configuration request among the edge ends of the cloud-edge collaboration platform;

[0046] A sending unit is used to send the policy configuration request to the first edge end through a preset cloud-edge channel, so that the first edge end parses the policy configuration request to obtain second configuration information, and configures an edge log collection policy corresponding to the second configuration information in the form of a custom resource, so that the cloud-native log collection tool preset in the first edge end collects the container logs and container cluster events of the first edge end in real time based on the edge log collection policy, and stores the container logs and container cluster events of the first edge end to a data storage platform corresponding to the first edge end.

[0047] A log data management system for a cloud-edge collaborative platform, comprising:

[0048] A cloud module and multiple edge modules; the cloud module is deployed on the cloud of the cloud-edge collaborative platform, the multiple edge modules correspond one-to-one to each edge of the cloud-edge collaborative platform, and each edge module is deployed on the edge corresponding to the edge module;

[0049] The cloud module is configured to: upon receiving a policy configuration request sent by a user through a front-end interface, determine whether the policy configuration request is a request associated with the cloud; if the policy configuration request is a request associated with the cloud, parse the policy configuration request to obtain first configuration information; configure a cloud log collection policy corresponding to the first configuration information in the form of a custom resource, so that the cloud native log collection tool preset in the cloud collects the container logs and container cluster events of the cloud in real time based on the cloud log collection policy, and stores the container logs and container cluster events of the cloud to the corresponding data storage platform of the cloud; if the policy configuration request is not a request associated with the cloud, determine the edge end corresponding to the policy configuration request in each edge end of the cloud-edge collaborative platform; send the policy configuration request to its corresponding edge end through a preset cloud-edge channel;

[0050] The edge module is configured as follows: when the edge corresponding to the edge module receives the policy configuration request, the policy configuration request is parsed to obtain second configuration information; in the form of a custom resource, an edge log collection policy corresponding to the second configuration information is configured, so that the cloud native log collection tool preset in the edge collects the container logs and container cluster events of the edge in real time based on the edge log collection policy, and stores the container logs and container cluster events of the edge to the data storage platform corresponding to the edge.

[0051] A storage medium includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the log data management method of the cloud-edge collaborative platform as described above.

[0052] An electronic device includes a memory and one or more instructions, wherein the one or more instructions are stored in the memory and are configured to be executed by one or more processors to implement the log data management method of the cloud-edge collaborative platform as described above.

[0053] A log data management method for a cloud-edge collaborative platform provided based on the above-mentioned embodiment of the present invention includes: when the cloud receives a policy configuration request sent by a user through a front-end interface, determining whether the policy configuration request is a cloud-related request; if the policy configuration request is a cloud-related request, parsing the policy configuration request to obtain first configuration information; configuring a cloud log collection policy corresponding to the first configuration information in the form of a custom resource, so that a cloud-native log collection tool preset in the cloud collects container logs and container cluster events in real time on the cloud based on the cloud log collection policy, and stores the container logs and container cluster events on the cloud to a corresponding data storage platform on the cloud; if If the policy configuration request is not a cloud-related request, then the first edge end corresponding to the policy configuration request is determined among the edge ends of the cloud-edge collaborative platform; the policy configuration request is sent to the first edge end through the preset cloud-edge channel, so that the first edge end parses the policy configuration request to obtain the second configuration information, and configures the edge end log collection policy corresponding to the second configuration information in the form of a custom resource, so that the cloud-native log collection tool preset in the first edge end can collect the container logs and container cluster events of the first edge end in real time based on the edge end log collection policy, and store the container logs and container cluster events of the first edge end to the data storage platform corresponding to the first edge end. By applying the method provided by the embodiment of the present invention, the user can configure the log collection strategy in the cloud or a certain edge end through the front-end interface. When the cloud receives the policy configuration request sent by the user, it can determine whether the request is a request for configuration for the cloud or a request for configuration for the edge end. If the request is a request for configuration for the cloud, the request can be parsed, and the log collection strategy can be configured according to the configuration information obtained by the parsing, so that the cloud native log collection tool collects container logs and container cluster events based on the strategy, thereby completing the policy configuration of cloud container log and container cluster event collection. If the request is a request for configuration for the edge end, the cloud can send the request to the corresponding edge end through the cloud edge channel, triggering the edge end controller to parse the request, thereby configuring the log collection strategy, so that the cloud native log collection tool collects container logs and container cluster events based on the strategy, thereby completing the policy configuration of edge container log and container cluster event collection. Users can configure the collection strategy of log data in the cloud and edge end respectively through a unified front-end interface, without the need to manually configure multiple components embedded in each node, which is conducive to simplifying the policy configuration process and improving operation and maintenance efficiency. Secondly, the cloud and edge use cloud-native log collection tools to monitor and collect container logs and container cluster events in real time, without the need for invasive configuration inside the container or the integration of multiple components. This helps reduce the complexity of the cloud-edge collaborative platform and reduce container security risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0055] Figure 1 A method flow chart of a log data management method for a cloud-edge collaborative platform provided by an embodiment of the present invention;

[0056] Figure 2 A signaling diagram of a log data management process of a cloud-edge collaborative platform provided in an embodiment of the present invention;

[0057] Figure 3 A structural schematic diagram of a log data management device for a cloud-edge collaborative platform provided by an embodiment of the present invention;

[0058] Figure 4 A schematic diagram of the architecture of a log data management system for a cloud-edge collaborative platform provided by an embodiment of the present invention;

[0059] Figure 5 A schematic diagram of the component architecture of a log data management system for a cloud-edge collaborative platform provided by an embodiment of the present invention;

[0060] Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] In this application, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element.

[0063] The embodiment of the present invention provides a log data management method for a cloud-edge collaborative platform. The method can be applied to the cloud of the cloud-edge collaborative platform. The execution subject can be a server or controller in the cloud. The method flow chart of the method is as follows: Figure 1 As shown, the method includes:

[0064] S101: When receiving a policy configuration request sent by a user through a front-end interface, determining whether the policy configuration request is a request associated with the cloud;

[0065] The method provided in the embodiment of the present invention can be applied to the cloud of the cloud-edge collaborative platform, and can be specifically deployed as a log data management system. In the embodiment of the present invention, a visual system front-end interface is designed in advance according to the configuration requirements of the log collection strategy. The front-end interface is provided with a node selection control for configuring the strategy, including nodes that can be selected, specifically including cloud nodes and each edge node. The front-end interface is also provided with input controls for various configuration contents of the log collection strategy, which can include the configuration content of container log collection and the configuration content of container cluster events (K8s system events) collection, such as the container log type that needs to be collected, the container name that needs to be collected, the type of container cluster events that need to be collected, etc. The user can select the node that currently needs to configure the log collection strategy through the front-end interface, such as the cloud or a specified edge end, or select multiple nodes at the same time for batch configuration. The user can enter the relevant configuration data through the front-end interface, and then send a policy configuration request to the system through the trigger control.

[0066] When the cloud-based log data management system receives a policy configuration request from a user, it can obtain the node ID of the node that currently needs to configure the policy from the request information, and identify whether the node that currently needs to configure the policy is the cloud or the edge based on the node ID. If the node that currently needs to configure the policy is the cloud, the policy configuration request is considered to be a cloud-related request. If the node that currently needs to configure the policy is the edge, the policy configuration request is considered not to be a cloud-related request.

[0067] S102: Parse the policy configuration request to obtain first configuration information;

[0068] In the method provided by the embodiment of the present invention, if it is determined that the policy configuration request is a cloud-related request, the cloud performs data analysis on the policy configuration request, obtains the configuration data for the log configuration policy input by the user, and uses the above configuration as the first configuration information.

[0069] S103: configuring a cloud log collection strategy corresponding to the first configuration information in the form of a custom resource, so that a cloud native log collection tool preset in the cloud collects container logs and container cluster events in real time on the cloud based on the cloud log collection strategy, and stores the container logs and container cluster events on the cloud to a data storage platform corresponding to the cloud;

[0070] In the method provided by the embodiment of the present invention, the cloud defines the log collection configuration strategy in the form of a custom resource (Custom Resource Definition, CRD) according to the configuration data in the first configuration information, and uses the obtained log collection configuration strategy as the cloud log collection strategy. CRD is an extension capability of the container orchestration platform K8s, which allows the customization of new resource types to implement business function logic, which will not be introduced in detail here.

[0071] In the method provided by the embodiment of the present invention, cloud native log collection tools can be deployed in advance in the cloud and each edge terminal, such as applying the cloud native log collection tool Loggie, and data storage platforms can be deployed in the cloud and each edge terminal, for example, the data storage platform can be deployed based on the distributed full-text search engine Elasticsearch. The cloud native log collection tool can detect the log collection strategy configured in the system in real time. When a new log collection strategy is configured, the cloud native log collection tool detects that the configuration has changed, and then performs log data collection based on the new log collection strategy, including collecting container logs and container cluster events in the cloud, and storing the collected log data in the data storage platform corresponding to the cloud.

[0072] S104: Determine, among the edge terminals of the cloud-edge collaboration platform, a first edge terminal corresponding to the policy configuration request;

[0073] In the method provided by an embodiment of the present invention, if it is determined that the policy configuration request is not a cloud-related request, the system will determine the edge end associated with the policy configuration request among the various edge ends of the cloud-edge collaborative platform based on the node identifier, that is, the edge end that currently needs to perform policy configuration, and use the edge end as the first edge end.

[0074] It should be noted that if the user selects multiple nodes for batch configuration, there can be multiple first edge terminals corresponding to the policy configuration request, and subsequent operations are performed for each first edge terminal. In the batch configuration scenario, if the node identifier corresponding to the policy configuration request includes the cloud identifier, that is, the policy configuration request is a cloud-related request, in addition to executing the previous steps S102 and S103, step S104 and the subsequent step S105 must also be executed, that is, in addition to configuring the log collection policy in the cloud, the policy configuration request must also be sent to the corresponding edge terminal for configuration operations.

[0075] S105: Send the policy configuration request to the first edge end through a preset cloud-edge channel, so that the first edge end parses the policy configuration request to obtain second configuration information, and configures an edge end log collection policy corresponding to the second configuration information in the form of a custom resource, so that the cloud-native log collection tool preset in the first edge end collects the container logs and container cluster events of the first edge end in real time based on the edge end log collection policy, and stores the container logs and container cluster events of the first edge end to the data storage platform corresponding to the first edge end.

[0076] In the method provided by the embodiment of the present invention, the system can send a policy configuration request to the first edge end through a pre-established cloud-edge channel, triggering the edge end to parse the policy configuration request and perform corresponding configuration operations. That is, when the edge end receives the policy configuration request, it can perform data parsing on the policy configuration request to obtain corresponding configuration data, use the above configuration data as the second configuration information, and define the log collection configuration strategy in the form of a custom resource CRD according to the configuration data in the second configuration information, obtain the edge end log collection strategy of the current edge end, and the cloud native log collection tool in the edge end can collect the container logs and container cluster events in the current edge end in real time according to the edge end log collection strategy, and store them in the data storage platform corresponding to the edge end.

[0077] like Figure 2 As shown, the log data management process provided by the embodiment of the present invention mainly includes:

[0078] When the cloud platform receives a configuration request associated with the cloud, it configures a log data collection strategy based on the configuration data carried in the configuration request, so that the cloud-native log collection tool on the cloud collects log data such as container logs and container cluster events on the cloud in real time according to the collection strategy, and stores the log data on the cloud storage platform.

[0079] When the cloud platform receives a configuration request associated with the edge, it sends the configuration request to the cloud-edge channel, which forwards the configuration request to the specified edge. The edge controller can configure the log data collection strategy based on the configuration data carried in the configuration request, so that the cloud-native log collection tool at the edge can collect the edge container logs and container cluster events in real time according to the collection strategy, and store the log data in the edge storage platform.

[0080] Based on the method provided by the embodiment of the present invention, the user can configure the log collection strategy in the cloud or a certain edge end through the front-end interface. When the cloud receives the policy configuration request sent by the user, it can determine whether the request is a request for configuration for the cloud or a request for configuration for the edge end. If the request is a request for configuration for the cloud, the request can be parsed, and the log collection strategy can be configured according to the configuration information obtained by the parsing, so that the cloud native log collection tool collects container logs and container cluster events based on the strategy, thereby completing the policy configuration of cloud container log and container cluster event collection. If the request is a request for configuration for the edge end, the cloud can send the request to the corresponding edge end through the cloud edge channel, triggering the edge end controller to parse the request, thereby configuring the log collection strategy, so that the cloud native log collection tool collects container logs and container cluster events based on the strategy, thereby completing the policy configuration of edge container log and container cluster event collection. Users can configure the collection strategy of log data in the cloud and edge end respectively through a unified front-end interface, without the need to manually configure multiple components embedded in each node, which is conducive to simplifying the policy configuration process and improving operation and maintenance efficiency. Secondly, the cloud and edge use cloud-native log collection tools to monitor and collect container logs and container cluster events in real time, without the need for invasive configuration inside the container or the integration of multiple components. This helps reduce the complexity of the cloud-edge collaborative platform and reduce container security risks.

[0081] exist Figure 1 On the basis of the method shown, the method provided in the embodiment of the present invention further includes:

[0082] When receiving a data query request sent by a user through the front-end interface, determining whether the data query request is a request associated with the cloud;

[0083] In the method provided by the embodiment of the present invention, the user can query the log data in the cloud or edge through the front-end interface of the log data management system. Specifically, the user can select the node where the log data to be queried is located through the front-end interface, that is, query the log data in the cloud or the log data of a certain edge, and set the corresponding query conditions. For example, if querying the container log, the container name, log identifier or log type of the query can be set. If querying the container cluster event, the event type of the query, the event occurrence time, etc. can be set. Based on the set query conditions, the corresponding data query request is sent to the system.

[0084] When the cloud receives a data query request sent by a user, the node identifier of the node where the currently queried log data is located can be obtained from the request information, and the node identifier can be used to determine whether the currently queried log data is located at the cloud or the edge. If the currently queried log data is the log data of the cloud, then the data query request is considered to be a cloud-related request. If the currently queried log data is the log data of the edge, then the data query request is considered not to be a cloud-related request.

[0085] If the data query request is a request associated with the cloud, determining first query information corresponding to the data query request;

[0086] In the method provided by the embodiment of the present invention, if it is determined that the current data query request is a cloud-related request, the data query request is parsed, the current query condition is determined based on the parsed data, and the corresponding query condition is used as the first query information.

[0087] Based on the first query information, calling a data query interface of a data storage platform corresponding to the cloud, and querying the log data corresponding to the first query information in the data storage platform corresponding to the cloud;

[0088] In the front-end interface, the log data corresponding to the first query information is visually displayed.

[0089] In the method provided by the embodiment of the present invention, the cloud can generate a corresponding interface call request based on the first query information, thereby calling the data query interface of the data storage platform, so that the data storage platform performs data query in the stored log data according to the corresponding query conditions to obtain log data matching the query conditions, that is, the log data corresponding to the first query information. After the cloud obtains the log data corresponding to the first query information, the corresponding log data can be visualized in the front-end interface according to a predetermined display method to complete the query process corresponding to the data query request.

[0090] Based on the method provided in the embodiment of the present invention, the user can query the container log through the log data management system to obtain visual data, which is helpful for the user to carry out operation and maintenance work.

[0091] On the basis of the method provided in the above embodiment, in the method provided in the embodiment of the present invention, the determining the first query information corresponding to the data query request includes:

[0092] Performing data parsing on the data query request to obtain parsed data corresponding to the data query request;

[0093] Based on the parsed data, determining a query object corresponding to the data query request; the query object is a container log on the cloud or a container cluster event on the cloud;

[0094] In the method provided in the embodiment of the present invention, data parsing can be performed on the data query request to obtain the parsed data of the request, and the query object type specified by the user, that is, the type of log data currently required to be queried, is obtained from the parsed data, thereby determining whether the query object is a container log in the cloud or a container cluster event.

[0095] If the query object is a container cluster event on the cloud, obtaining event parameters corresponding to the query object from the parsed data, and using the event parameters as the first query information;

[0096] In the method provided by the embodiment of the present invention, if the current query object is a container cluster event in the cloud, the parsed data will include event parameters of the container cluster event set by the user and to be queried, such as event type, event occurrence time, etc., and the corresponding event parameters in the parsed data will be used as the first query information.

[0097] If the query object is the container log of the cloud, determining whether the query mode corresponding to the data query request is a context query;

[0098] In the method provided by the embodiment of the present invention, when a user queries a container log through the front-end interface, he or she can choose whether to perform a context query, and the query method selected by the user will be carried in the data query request as a request parameter. If the current query object is a container log in the cloud, the query method selected by the user is determined based on the parsed data to be a context query or a separate query.

[0099] If the query mode corresponding to the data query request is context query, the container name, log identifier and query start and end time are obtained from the parsed data, and the container name, log identifier and query start and end time are used as the first query information.

[0100] In the method provided by the embodiment of the present invention, if the query method selected by the user is a context query, the parsed data will include parameters such as the container name of the query selected by the user, the log identifier, and the start and end time of the context query (i.e., the query start and end time). The above parameters can be obtained from the parsed data as the first query information, so as to query the specified container log of the specified container based on the above parameters, as well as the container log generated within a certain time before and after the container log.

[0101] Based on the method provided by the embodiment of the present invention, a context query of a container log can be provided to the user, that is, when the user queries a specified container log, the log entries before and after the container log can be obtained on demand, which is helpful for the user to better understand the log generation situation.

[0102] exist Figure 1 On the basis of the method shown, the method provided in the embodiment of the present invention further includes:

[0103] If the data query request is not a request associated with the cloud, determining a second edge end corresponding to the data query request among the edge ends of the cloud-edge collaboration platform;

[0104] In the method provided by an embodiment of the present invention, if it is determined that the current data query request is not a cloud-related request, the edge end corresponding to the node identifier carried in the data query request can be determined in each edge end, and the edge end can be used as the second edge end.

[0105] The data query request is sent to the second edge end through the cloud-edge channel, so that the second edge end determines the second query information corresponding to the data query request, and based on the second query information, calls the data query interface of the data storage platform corresponding to the second edge end to query the log data corresponding to the second query information in the data storage platform corresponding to the second edge end, and sends the log data corresponding to the second query information to the cloud end;

[0106] In the method provided by the embodiment of the present invention, the cloud can send a data query request to a pre-built cloud-edge channel, and the cloud-edge channel forwards the data query request to the second edge terminal, triggering the edge terminal to parse the data query request and perform corresponding data query operations. When the edge terminal receives the data query request and parses it, it can determine the corresponding query conditions based on the parsed data, use the obtained query conditions as the second query information, and generate a corresponding interface call request based on the second query information, thereby calling the data query interface of the data storage platform of the edge terminal, so that the data query platform of the edge terminal performs data query in the log data stored in it according to the corresponding query conditions, and obtains the log data matched by the second query information. After obtaining the log data corresponding to the second query information, the second edge terminal can feed back the log data to the cloud through the cloud-edge channel.

[0107] When the log data corresponding to the second query information is received, the log data corresponding to the second query information is visually displayed in the front-end interface.

[0108] In the method provided by the embodiment of the present invention, when the cloud receives the log data corresponding to the second query information sent by the second edge terminal, the above log data can be visualized in a predetermined display manner.

[0109] Based on the method provided in the embodiment of the present invention, a unified query can be performed on the log data in the cloud and the log data on the edge, and the queried log data can be displayed to the user in a visual manner. The user can query the log data in the cloud or the edge through a unified front-end interface to obtain visualized data, which is beneficial for the user to carry out operation and maintenance work.

[0110] exist Figure 1 On the basis of the method shown, the method provided in the embodiment of the present invention further includes:

[0111] When receiving a data deletion request sent by the user through the front-end interface, determining whether the data deletion request is a request associated with the cloud;

[0112] In the method provided by the embodiment of the present invention, the user can clean up the log data in the cloud or edge through the front-end interface of the log data management system. Specifically, the user can select the node where the log data to be deleted is located through the front-end interface, that is, whether to delete the log data in the cloud or the log data of a certain edge, and enter the relevant information of the log data to be deleted, such as the type or event identifier of the container cluster event to be deleted, the time when the event was generated, the name of the container where the container log to be deleted is located, the container log identifier, etc.

[0113] When the cloud receives a data deletion request sent by a user, the node identifier of the node where the log data that needs to be deleted is located can be obtained from the request information, and based on the node identifier, it can be determined whether the current data deletion request is a request to delete the log data in the cloud. If the data deletion request is a request to delete the log data in the cloud, then the data deletion request is considered to be a cloud-related request; if the data deletion request is a request to delete the log data in the edge, then the data deletion request is not considered to be a cloud-related request.

[0114] If the data deletion request is a request associated with the cloud, parsing the data deletion request to obtain first data information;

[0115] In the method provided by the embodiment of the present invention, if it is determined that the current data deletion request is a cloud-related request, the cloud will parse the data deletion request, determine the relevant information of the data that currently needs to be deleted based on the parsed data, and use this information as the first data information.

[0116] Based on the first data information, calling a data operation interface of a data storage platform corresponding to the cloud, and deleting log data corresponding to the first data information in the data storage platform corresponding to the cloud;

[0117] In the method provided by the embodiment of the present invention, the cloud can generate a corresponding interface call request based on the first data information, thereby calling the data operation interface of the data storage platform of the cloud, so that the data storage platform can filter out matching log data according to the data information and perform a deletion operation on the above log data.

[0118] If the data deletion request is not a request associated with the cloud, determining a third edge end corresponding to the data deletion request among the edge ends of the cloud-edge collaboration platform;

[0119] In the method provided by an embodiment of the present invention, if it is determined that the current data deletion request is not a cloud-related request, then in each edge terminal, the edge terminal corresponding to the node identifier carried in the data deletion request is determined, and the edge terminal is used as the third edge terminal.

[0120] The data deletion request is sent to the third edge terminal through the cloud-edge channel, so that the third edge terminal parses the data deletion request to obtain the second data information, and based on the second data information, calls the data operation interface of the data storage platform corresponding to the third edge terminal to delete the log data corresponding to the second data information in the data storage platform corresponding to the third edge terminal.

[0121] In the method provided by the embodiment of the present invention, the cloud can send a data deletion request to the cloud-edge channel, and the cloud-edge channel forwards the data deletion request to the third edge end, triggering the edge end to parse the data deletion request, determine the relevant information of the data that currently needs to be deleted based on the parsed data, use this information as the second data information, and generate a corresponding interface call request based on the second data information, thereby calling the data operation interface of the data storage platform of the edge end, so that the data storage platform filters out matching log data according to the data information, and performs corresponding deletion operations.

[0122] Based on the method provided in the embodiment of the present invention, the user can uniformly clean up the log data on the cloud and the edge through the front-end interface, which is conducive to improving the convenience of the user in managing the log data.

[0123] exist Figure 1 On the basis of the method shown, the method provided in the embodiment of the present invention further includes:

[0124] When receiving an alarm configuration request sent by a user through a front-end interface, determining whether the alarm configuration request is a request associated with the cloud;

[0125] In the method provided by the embodiment of the present invention, the user can configure the alarm rules in the cloud or edge through the front-end interface of the log data management system, that is, the user can select the node for which the alarm rules need to be configured, that is, whether the alarm rules of the cloud or the edge need to be configured, and input the alarm rules to be configured, and then send a responsive alarm configuration request to the system. It is understandable that in the cloud-edge collaborative platform, the cloud and edge will perform anomaly detection based on the log data they collect. When anomalies or certain events are detected, alarm processing must be performed according to certain alarm rules.

[0126] When the cloud receives an alarm configuration request sent by a user, the node identifier of the node that currently needs to configure the alarm rules can be obtained from the request information, and whether the current alarm configuration request is a request to configure the alarm rules of the cloud can be determined based on the node identifier. If the alarm configuration request is a request to configure the alarm rules of the cloud, then the alarm configuration request is considered to be a cloud-related request; if the alarm configuration request is a request to configure the alarm rules of the edge, then the alarm configuration request is not considered to be a cloud-related request.

[0127] If the alarm configuration request is a request associated with the cloud, parsing the alarm configuration request to obtain first alarm rule information;

[0128] In the method provided by the embodiment of the present invention, if it is determined that the current alarm configuration request is a cloud-associated request, the cloud parses the alarm configuration request, determines the rule content of the alarm rule configured by the user based on the parsed data, and uses the above rule content as the first alarm rule information.

[0129] Setting a cloud alarm rule based on the first alarm rule information, and issuing an abnormal alarm to the cloud based on the cloud alarm rule;

[0130] In the method provided in the embodiment of the present invention, the cloud can configure the cloud alarm rules based on the first alarm rule information to obtain the cloud alarm rules, and alarm the monitored anomalies according to the cloud alarm rules.

[0131] If the alarm configuration request is not a request associated with the cloud, determining a fourth edge end corresponding to the alarm configuration request among the edge ends of the cloud-edge collaboration platform;

[0132] In the method provided by an embodiment of the present invention, if it is determined that the current alarm configuration request is not a cloud-associated request, an edge end matching the node identifier carried in the alarm configuration request is determined among the edge ends, and the edge end is used as the fourth edge end.

[0133] The alarm configuration request is sent to the fourth edge end through the cloud edge channel, so that the fourth edge end parses the alarm configuration request, obtains the second alarm rule information, sets the edge end alarm rule based on the second alarm rule information, and issues an abnormal alarm to the fourth edge end based on the edge end alarm rule.

[0134] In the method provided by the embodiment of the present invention, the cloud can send the current alarm configuration request to the cloud edge channel, and the cloud edge channel forwards the request to the fourth edge end, triggering the edge end to parse the alarm configuration request, obtain the rule content of the alarm rules configured by the user, thereby setting the alarm rules of the edge end, and issuing abnormal alarms according to the alarm rules.

[0135] Based on the method provided in the embodiment of the present invention, users can customize the alarm rules of the cloud and edge through the front-end interface of the log data management system, which is conducive to users to uniformly configure the alarm rules of the cloud-edge collaborative platform and improve the convenience of operation and maintenance.

[0136] and Figure 1 Corresponding to the log data management method of the cloud-edge collaborative platform shown in the embodiment of the present invention, a log data management device of the cloud-edge collaborative platform is provided for Figure 1 The specific implementation of the method shown in , the device is applied to the cloud side of the cloud-edge collaborative platform, and its structural diagram is as follows Figure 3 As shown, including:

[0137] The judging unit 301 is configured to judge whether the policy configuration request is a request associated with the cloud when receiving the policy configuration request sent by the user through the front-end interface;

[0138] A parsing unit 302, configured to parse the policy configuration request to obtain first configuration information if the policy configuration request is a request associated with the cloud;

[0139] The configuration unit 303 is used to configure the cloud log collection strategy corresponding to the first configuration information in the form of a custom resource, so that the cloud native log collection tool preset in the cloud collects the container logs and container cluster events of the cloud in real time based on the cloud log collection strategy, and stores the container logs and container cluster events of the cloud to the data storage platform corresponding to the cloud;

[0140] A determination unit 304 is configured to determine, if the policy configuration request is not a request associated with the cloud, a first edge end corresponding to the policy configuration request among the edge ends of the cloud-edge collaboration platform;

[0141] The sending unit 305 is used to send the policy configuration request to the first edge end through a preset cloud-edge channel, so that the first edge end parses the policy configuration request to obtain second configuration information, and configures the edge end log collection policy corresponding to the second configuration information in the form of a custom resource, so that the cloud native log collection tool preset in the first edge end, based on the edge end log collection policy, collects the container logs and container cluster events of the first edge end in real time, and stores the container logs and container cluster events of the first edge end to the data storage platform corresponding to the first edge end.

[0142] By using the device provided by the embodiment of the present invention, the user can configure the log collection strategy in the cloud or a certain edge end through the front-end interface. When the cloud receives the policy configuration request sent by the user, it can determine whether the request is a request for configuration for the cloud or a request for configuration for the edge end. If the request is a request for configuration for the cloud, the request can be parsed, and the log collection strategy can be configured according to the configuration information obtained by the parsing, so that the cloud native log collection tool collects container logs and container cluster events based on the strategy, thereby completing the policy configuration of cloud container log and container cluster event collection. If the request is a request for configuration for the edge end, the cloud can send the request to the corresponding edge end through the cloud edge channel, triggering the edge end controller to parse the request, thereby configuring the log collection strategy, so that the cloud native log collection tool collects container logs and container cluster events based on the strategy, thereby completing the policy configuration of edge container log and container cluster event collection. Users can configure the collection strategy of log data in the cloud and edge end respectively through a unified front-end interface, without the need to manually configure multiple components embedded in each node, which is conducive to simplifying the policy configuration process and improving operation and maintenance efficiency. Secondly, the cloud and edge use cloud-native log collection tools to monitor and collect container logs and container cluster events in real time, without the need for invasive configuration inside the container or the integration of multiple components. This helps reduce the complexity of the cloud-edge collaborative platform and reduce container security risks.

[0143] exist Figure 3 Based on the device shown, the device provided by the embodiment of the present invention can be further expanded into multiple units. The functions of each unit can be found in the descriptions of the various embodiments provided in the previous text for the log data management method of the cloud-edge collaborative platform, and no further examples will be given here.

[0144] The embodiment of the present invention also provides a log data management system for a cloud-edge collaborative platform. The architecture of the system can be as follows: Figure 4 As shown, including:

[0145] A cloud module 401 and multiple edge modules 402; the cloud module 401 is deployed on the cloud of the cloud-edge collaborative platform, the multiple edge modules 402 correspond one-to-one to each edge of the cloud-edge collaborative platform, and each edge module 402 is deployed on the edge corresponding to the edge module;

[0146] The cloud module 401 is configured to: upon receiving a policy configuration request sent by a user through a front-end interface, determine whether the policy configuration request is a request associated with the cloud; if the policy configuration request is a request associated with the cloud, parse the policy configuration request to obtain first configuration information; configure a cloud log collection strategy corresponding to the first configuration information in the form of a custom resource, so that the cloud native log collection tool preset in the cloud collects the container logs and container cluster events of the cloud in real time based on the cloud log collection strategy, and stores the container logs and container cluster events of the cloud to the corresponding data storage platform of the cloud; if the policy configuration request is not a request associated with the cloud, determine the edge end corresponding to the policy configuration request in each edge end of the cloud-edge collaborative platform; send the policy configuration request to its corresponding edge end through a preset cloud-edge channel;

[0147] The edge module 402 is configured as follows: when the edge corresponding to the edge module receives the policy configuration request, the policy configuration request is parsed to obtain second configuration information; in the form of custom resources, the edge log collection policy corresponding to the second configuration information is configured, so that the cloud native log collection tool preset in the edge collects the container logs and container cluster events of the edge in real time based on the edge log collection policy, and stores the container logs and container cluster events of the edge to the data storage platform corresponding to the edge.

[0148] The system provided by the embodiment of the present invention is respectively configured with a cloud module and an edge module. The cloud module is deployed in the cloud and is used to receive the policy configuration request of the system front-end interface and perform the corresponding policy configuration or request forwarding operation. The specific processing process is combined with the above Figure 1 The corresponding processing process in the method shown is the same, and reference can be made to the previous embodiment for Figure 1 The description of the method shown will not be repeated here.

[0149] The edge module is deployed on the corresponding edge. When the edge receives a policy configuration request, it can execute the corresponding policy configuration operation. The specific processing process is combined with the previous Figure 1 The corresponding processing process in the method shown is the same, and reference can be made to the previous embodiment for Figure 1 The description of the method shown will not be repeated here.

[0150] By using the system provided by the embodiment of the present invention, the user can configure the log collection strategy in the cloud or a certain edge end through the front-end interface of the system. When the system receives the policy configuration request sent by the user, it can determine whether the request is a request for configuration for the cloud or a request for configuration for the edge end. If the request is a request for configuration for the cloud, the request can be parsed, and the log collection strategy can be configured according to the configuration information obtained by the parsing, so that the cloud native log collection tool collects container logs and container cluster events based on the strategy, thereby completing the policy configuration of cloud container logs and container cluster event collection. If the request is a request for configuration for the edge end, the cloud can send the request to the corresponding edge end through the cloud edge channel, triggering the edge end controller to parse the request, thereby configuring the log collection strategy, so that the cloud native log collection tool collects container logs and container cluster events based on the strategy, thereby completing the policy configuration of edge container logs and container cluster event collection. Users can configure the collection strategy of log data in the cloud and edge ends respectively through a unified front-end interface, without the need to manually configure multiple components embedded in each node, which is conducive to simplifying the policy configuration process and improving operation and maintenance efficiency. Secondly, the cloud and edge use cloud-native log collection tools to monitor and collect container logs and container cluster events in real time, without the need for invasive configuration inside the container or the integration of multiple components. This helps reduce the complexity of the cloud-edge collaborative platform and reduce container security risks.

[0151] It should be noted that the system provided by the embodiment of the present invention can further perform other configurations on the cloud module and the edge module to realize functions such as data query, data deletion and alarm rule configuration. For specific function implementations, please refer to the descriptions in the various embodiments provided for the log data management method of the cloud-edge collaborative platform in the previous text, and no further examples will be given here.

[0152] In order to better illustrate the system provided by the embodiment of the present invention, based on the methods provided in the above embodiments and in combination with actual application scenarios, the embodiment of the present invention provides another log data management system for the cloud-edge collaborative platform, and the architecture for implementing the system deployment can be as follows: Figure 5As shown. In the cloud-edge collaborative platform in the embodiment of the present invention, the cloud-native log collection tool Loggie and the full-text search engine ElasticSearch are deployed on the cloud and edge respectively. The CloudHub component is deployed on the cloud, which is a component for messages between the edge and the cloud. The EdgeHub component is deployed on the edge, which is used to receive messages sent from the cloud and perform processing on the edge. A cloud module is deployed on the cloud platform, which can receive various requests sent by users through the front-end interface, such as configuration requests for log collection strategies, data query requests / data deletion requests for log data, etc. The cloud platform can analyze whether the received request belongs to a request for operation on the cloud, which is processed by the cloud platform. If the request is a configuration request for the log collection strategy, the log collection strategy of Loggie is configured so that it collects the log data on the cloud according to the strategy and stores it in ElasticSearch. If the request is a request for data query or data deletion, the interface of ElasticSearch is called to perform the corresponding operation.

[0153] If the request received by the cloud platform is a request for an operation on the edge, the cloud platform forwards the request to the cloud-edge channel through the CloudHub component, and the cloud-edge channel sends it to the corresponding edge. After receiving the request, the EdgHub component on the edge performs corresponding processing. If the request is a configuration request for the log collection strategy, the log collection strategy of Loggie is configured to collect the log data of the edge according to the strategy and store it in ElasticSearch. If the request is a request for data query or data deletion, the ElasticSearch interface is called to perform the corresponding operation. After the edge completes the operation, the execution result of the operation can be fed back to the cloud platform.

[0154] Specifically, during the configuration of the log collection strategy, the system will create the log collection configuration of the application in the form of CRD in the K8s cluster (including the application to be collected, whether to collect standard output or log file, log storage backend, etc.). After Loggie detects that the configuration has changed, it will start collecting the container logs of the application based on the configuration, and store the collected logs in the backend ElasticSearch. Based on Loggie's real-time collection capabilities, when it detects changes in container log files, it will collect container logs to achieve real-time collection of container logs. At the same time, the system can configure the collection of Kubernetes system events (i.e. container cluster events) based on Loggie by default, realize the collection of K8s system events, and store the collected system events in ElasticSearch.

[0155] The system provided by the embodiment of the present invention stores the collected logs and events in ElasticSearch, provides a visual interface, and facilitates users to query and retrieve logs and events. The system assembles the query conditions specified by the user through the interface and calls the API interface of ElasticSearch to implement the query and retrieval function. Users can query the associated context of a log at the front end, and the system will automatically query the logs associated with the specified log before and after according to parameters such as container name, start and end time, so that users can trace the application logs.

[0156] Through the front-end, users can select the log index to be cleaned, or more fine-grained log deletion conditions, such as timestamp, application, etc. The system can assemble parameters based on these conditions, call the back-end storage interface, that is, the ElasticSearch interface, and perform log data deletion operations to clean up related logs.

[0157] The system provided by the embodiment of the present invention can configure the alarm rules by presetting the log alarm rules, and also supports the user to configure the alarm rules through the front-end page. When an abnormality or a key event triggers an alarm, the system will send the alarm notification message to the notification object according to the notification object configured by the alarm rule, and store the alarm in the alarm history. The back-end storage can use ElasticSearch.

[0158] The system provided by the embodiment of the present invention supports multi-tenant access, and each tenant can only view the projects under the tenant, thereby realizing tenant isolation of container log collection, viewing, and alarm notification. For K8s system events, only system administrators are supported to perform related operations.

[0159] The system provided by the embodiment of the present invention adopts a cloud-native architecture and provides a front-end visual interface. Users can use an intuitive graphical user interface to support unified setting of log collection rules for the cloud and edge without modifying the internal configuration of the container. It supports two methods: collecting container standard output and collecting container log files. This non-intrusive configuration method simplifies the configuration process, lowers the technical threshold, and significantly improves the user experience.

[0160] The system provided in this embodiment also supports the function of querying the container log context, allowing users to query the log entries before and after a specific log event to obtain the complete context. Users can customize the time window to determine the scope of the context query, and provide a real-time log stream query function to help users monitor the current log generation situation.

[0161] The system provided by the embodiment of the present invention enables users to uniformly configure and monitor container logs on the cloud and edge through a front-end interface on the cloud, covering the entire life cycle of log data from generation to archiving, and providing a one-stop solution, including management content such as collection, storage, query, analysis, visualization, alarm and cleanup, which simplifies the log management method in multiple locations and environments, improves development and operation efficiency, and improves user experience.

[0162] An embodiment of the present invention also provides a storage medium, which includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the log data management method of the cloud-edge collaborative platform as described above.

[0163] The embodiment of the present invention further provides an electronic device, the structural diagram of which is shown in FIG. Figure 6 As shown, it specifically includes a memory 501 and one or more instructions 502, wherein the one or more instructions 502 are stored in the memory 501 and are configured to be executed by one or more processors 503 to perform the following operations:

[0164] When receiving a policy configuration request sent by a user through a front-end interface, determining whether the policy configuration request is a request associated with the cloud;

[0165] If the policy configuration request is a request associated with the cloud, parsing the policy configuration request to obtain first configuration information;

[0166] In the form of custom resources, a cloud log collection strategy corresponding to the first configuration information is configured, so that a cloud native log collection tool preset in the cloud collects container logs and container cluster events in real time on the cloud based on the cloud log collection strategy, and stores the container logs and container cluster events on the cloud to a data storage platform corresponding to the cloud;

[0167] If the policy configuration request is not a request associated with the cloud, determining a first edge end corresponding to the policy configuration request among the edge ends of the cloud-edge collaboration platform;

[0168] The policy configuration request is sent to the first edge end through a preset cloud-edge channel, so that the first edge end parses the policy configuration request to obtain second configuration information, and configures an edge end log collection policy corresponding to the second configuration information in the form of a custom resource, so that the cloud native log collection tool preset in the first edge end collects the container logs and container cluster events of the first edge end in real time based on the edge end log collection policy, and stores the container logs and container cluster events of the first edge end to the data storage platform corresponding to the first edge end.

[0169] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.

[0170] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0171] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A log data management method for a cloud-edge collaborative platform, characterized in that: The method is applied to the cloud side of the cloud-edge collaboration platform, and the method includes: When receiving a policy configuration request sent by a user through a front-end interface, determining whether the policy configuration request is a request associated with the cloud; If the policy configuration request is a request associated with the cloud, parsing the policy configuration request to obtain first configuration information; In the form of custom resources, a cloud log collection strategy corresponding to the first configuration information is configured, so that a cloud native log collection tool preset in the cloud collects container logs and container cluster events in real time on the cloud based on the cloud log collection strategy, and stores the container logs and container cluster events on the cloud to a data storage platform corresponding to the cloud; If the policy configuration request is not a request associated with the cloud, determining a first edge end corresponding to the policy configuration request among the edge ends of the cloud-edge collaboration platform; The policy configuration request is sent to the first edge end through a preset cloud-edge channel, so that the first edge end parses the policy configuration request to obtain second configuration information, and configures an edge end log collection policy corresponding to the second configuration information in the form of a custom resource, so that the cloud native log collection tool preset in the first edge end collects the container logs and container cluster events of the first edge end in real time based on the edge end log collection policy, and stores the container logs and container cluster events of the first edge end to the data storage platform corresponding to the first edge end.

2. The log data management method of the cloud-edge collaborative platform according to claim 1 is characterized in that: Also includes: When receiving a data query request sent by the user through the front-end interface, determining whether the data query request is a request associated with the cloud; If the data query request is a request associated with the cloud, determining first query information corresponding to the data query request; Based on the first query information, calling a data query interface of a data storage platform corresponding to the cloud, and querying the log data corresponding to the first query information in the data storage platform corresponding to the cloud; In the front-end interface, the log data corresponding to the first query information is visually displayed.

3. The log data management method of the cloud-edge collaborative platform according to claim 2 is characterized in that: The determining the first query information corresponding to the data query request includes: Performing data parsing on the data query request to obtain parsed data corresponding to the data query request; Based on the parsed data, determining a query object corresponding to the data query request; the query object is a container log on the cloud or a container cluster event on the cloud; If the query object is a container cluster event on the cloud, obtaining event parameters corresponding to the query object from the parsed data, and using the event parameters as the first query information; If the query object is the container log of the cloud, determining whether the query mode corresponding to the data query request is a context query; If the query mode corresponding to the data query request is context query, the container name, log identifier and query start and end time are obtained from the parsed data, and the container name, log identifier and query start and end time are used as the first query information.

4. The log data management method of the cloud-edge collaborative platform according to claim 2 is characterized in that: Also includes: If the data query request is not a request associated with the cloud, determining a second edge end corresponding to the data query request among the edge ends of the cloud-edge collaboration platform; The data query request is sent to the second edge end through the cloud-edge channel, so that the second edge end determines the second query information corresponding to the data query request, and based on the second query information, calls the data query interface of the data storage platform corresponding to the second edge end to query the log data corresponding to the second query information in the data storage platform corresponding to the second edge end, and sends the log data corresponding to the second query information to the cloud end; When the log data corresponding to the second query information is received, the log data corresponding to the second query information is visually displayed in the front-end interface.

5. The log data management method of the cloud-edge collaborative platform according to claim 1 is characterized in that: Also includes: When receiving a data deletion request sent by the user through the front-end interface, determining whether the data deletion request is a request associated with the cloud; If the data deletion request is a request associated with the cloud, parsing the data deletion request to obtain first data information; Based on the first data information, calling a data operation interface of a data storage platform corresponding to the cloud, and deleting log data corresponding to the first data information in the data storage platform corresponding to the cloud; If the data deletion request is not a request associated with the cloud, determining a third edge end corresponding to the data deletion request among the edge ends of the cloud-edge collaboration platform; The data deletion request is sent to the third edge terminal through the cloud-edge channel, so that the third edge terminal parses the data deletion request to obtain the second data information, and based on the second data information, calls the data operation interface of the data storage platform corresponding to the third edge terminal to delete the log data corresponding to the second data information in the data storage platform corresponding to the third edge terminal.

6. The log data management method of the cloud-edge collaborative platform according to claim 1 is characterized in that: Also includes: When receiving an alarm configuration request sent by a user through a front-end interface, determining whether the alarm configuration request is a request associated with the cloud; If the alarm configuration request is a request associated with the cloud, parsing the alarm configuration request to obtain first alarm rule information; Setting a cloud alarm rule based on the first alarm rule information, and issuing an abnormal alarm to the cloud based on the cloud alarm rule; If the alarm configuration request is not a request associated with the cloud, determining a fourth edge end corresponding to the alarm configuration request among the edge ends of the cloud-edge collaboration platform; The alarm configuration request is sent to the fourth edge end through the cloud edge channel, so that the fourth edge end parses the alarm configuration request, obtains the second alarm rule information, sets the edge end alarm rule based on the second alarm rule information, and issues an abnormal alarm to the fourth edge end based on the edge end alarm rule.

7. A log data management device for a cloud-edge collaborative platform, characterized in that: The device is applied to the cloud side of the cloud-edge collaboration platform, and the device includes: A determination unit, configured to determine whether a policy configuration request is a request associated with the cloud when receiving a policy configuration request sent by a user through a front-end interface; a parsing unit, configured to parse the policy configuration request to obtain first configuration information if the policy configuration request is a request associated with the cloud; A configuration unit, configured to configure a cloud log collection strategy corresponding to the first configuration information in the form of a custom resource, so that a cloud native log collection tool preset in the cloud collects container logs and container cluster events in real time on the cloud based on the cloud log collection strategy, and stores the container logs and container cluster events on the cloud to a data storage platform corresponding to the cloud; A determination unit, configured to determine, if the policy configuration request is not a request associated with the cloud, a first edge end corresponding to the policy configuration request among the edge ends of the cloud-edge collaboration platform; A sending unit is used to send the policy configuration request to the first edge end through a preset cloud-edge channel, so that the first edge end parses the policy configuration request to obtain second configuration information, and configures an edge log collection policy corresponding to the second configuration information in the form of a custom resource, so that the cloud-native log collection tool preset in the first edge end collects the container logs and container cluster events of the first edge end in real time based on the edge log collection policy, and stores the container logs and container cluster events of the first edge end to a data storage platform corresponding to the first edge end.

8. A log data management system for a cloud-edge collaborative platform, characterized in that: include: A cloud module and multiple edge modules; the cloud module is deployed on the cloud of the cloud-edge collaborative platform, the multiple edge modules correspond one-to-one to each edge of the cloud-edge collaborative platform, and each edge module is deployed on the edge corresponding to the edge module; The cloud module is configured to: upon receiving a policy configuration request sent by a user through a front-end interface, determine whether the policy configuration request is a request associated with the cloud; if the policy configuration request is a request associated with the cloud, parse the policy configuration request to obtain first configuration information; In the form of custom resources, configure the cloud log collection strategy corresponding to the first configuration information, so that the cloud native log collection tool preset in the cloud collects the container logs and container cluster events of the cloud in real time based on the cloud log collection strategy, and stores the container logs and container cluster events of the cloud to the data storage platform corresponding to the cloud; if the policy configuration request is not a request associated with the cloud, then determine the edge end corresponding to the policy configuration request in each edge end of the cloud-edge collaborative platform; send the policy configuration request to its corresponding edge end through the preset cloud-edge channel; The edge module is configured to: when the edge corresponding to the edge module receives the policy configuration request, parse the policy configuration request to obtain second configuration information; An edge log collection strategy corresponding to the second configuration information is configured in the form of a custom resource, so that the cloud native log collection tool preset in the edge collects the container logs and container cluster events of the edge in real time based on the edge log collection strategy, and stores the container logs and container cluster events of the edge to the data storage platform corresponding to the edge.

9. A storage medium, characterized in that: The storage medium includes stored instructions, wherein, when the instructions are executed, the device where the storage medium is located is controlled to execute the log data management method of the cloud-edge collaborative platform as described in any one of claims 1 to 6.

10. An electronic device, characterized in that: It includes a memory and one or more instructions, wherein one or more instructions are stored in the memory and are configured to be executed by one or more processors to perform the log data management method of the cloud-edge collaborative platform as described in any one of claims 1 to 6.

Images