Cloud native environment monitoring method and system based on fingerprints, terminal and storage medium

Abstract

The invention relates to the technical field of data processing, and particularly provides a fingerprint-based cloud native environment monitoring method and system, a terminal and a storage medium, and the method comprises the steps: employing a container arrangement tool to regularly collect cloud native environment information, the cloud native environment information comprising cluster information, container information and micro-service log information; carrying out filtering and statistical processing on the cloud native environment information to obtain multiple pieces of feature data; generating an identity code for the feature data based on the type of the feature data and related clusters, containers and micro-services; calculating a hash value of the feature data, and storing the identity code and the hash value of the feature data as fingerprints in a fingerprint database; and analyzing the matching property of the fingerprint data collected for two adjacent times in the fingerprint database, and if the two fingerprint data are not matched, judging that the cloud native environment is updated. According to the method, the cloud native environment characterization fingerprints are generated, and accurate monitoring of the cloud native environment can be realized by matching the fingerprints.

Description

Technical Field

[0001] The present invention belongs to the field of data processing technology, and specifically relates to a fingerprint-based cloud native environment monitoring method, system, terminal and storage medium. Background Art

[0002] In the vast field of cloud computing and network security technology, cloud native architecture, as an emerging and highly anticipated architectural paradigm, is leading a wide range of application trends. Based on advanced technologies such as microservice architecture, containerization technology, automated deployment process, and continuous integration / continuous deployment (CI / CD), this architecture significantly improves the flexibility, scalability, and maintenance convenience of applications, injecting strong impetus into enterprise development.

[0003] However, with the increasing popularity of cloud-native applications, there are a large number of microservices, complex call relationships, and frequent changes in container and environment configuration data. Traditional service monitoring methods usually monitor some indicator data and cannot effectively analyze and monitor the frequently changing cloud-native environment. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the present invention provides a fingerprint-based cloud native environment monitoring method, system, terminal and storage medium to solve the above-mentioned technical problems.

[0005] In a first aspect, the present invention provides a fingerprint-based cloud native environment monitoring method, comprising: Use container orchestration tools to regularly collect cloud native environment information, including cluster information, container information, and microservice log information; Filter and statistically process cloud native environment information to obtain multiple feature data; Generate an identity code for the feature data based on the type of the feature data and the clusters, containers, and microservices involved; Calculate the hash value of the feature data, and save the identity code and the hash value of the feature data as a fingerprint in a fingerprint database; Analyze the matching of two consecutive fingerprint data collected in the fingerprint database. If the two do not match, it is determined that the cloud native environment has been updated.

[0006] In an optional implementation, the cloud native environment information is filtered and statistically processed to obtain multiple feature data, including: Pre-setting key parameter items, wherein the key parameter items include error information and resource characterization parameters; Filter out valid information belonging to key parameter items from cloud native environment information; Counting intermediate data based on the valid information, the intermediate data includes the number of nodes, the number of idle nodes, the number of containers, and the number of microservice error messages; Generate early warning information based on valid information using pre-trained neural network models; Integrate resource characterization parameters, intermediate data and warning information in effective information into feature data.

[0007] In an optional implementation, calculating a hash value of the feature data, and saving the identity code and the hash value of the feature data as a fingerprint in a fingerprint database includes: Generating dynamic representation data for the feature data corresponding to each identity code respectively, and calculating a hash value of the dynamic representation data, wherein the dynamic representation data changes as the feature data is updated; Integrate the identity code and hash value of the feature data into structured data, and save the structured data as a fingerprint in a fingerprint database; Traverse all feature data to obtain all fingerprint data of the cloud native environment information collected this time.

[0008] In an optional implementation, comparing the matching of two consecutive fingerprint data collected in the fingerprint database, and if the two do not match, determining that there is an update in the cloud native environment, includes: Compare the fingerprint data collected this time with the fingerprint data collected last time to obtain multiple difference fingerprints; Parse identity codes of multiple differential fingerprints; For multiple differential fingerprints with exactly the same identity code, the data type in the identity code is compared with the data type in the preset alarm rule. If the comparison is consistent, an alarm message of the environment update is generated. If the comparison is inconsistent, only the differential fingerprints with exactly the same identity code are recorded in the monitoring log; For multiple differential fingerprints whose identity codes are not completely identical, the data update factor is located by parsing the identity codes that are not completely identical.

[0009] In a second aspect, the present invention provides a fingerprint-based cloud native environment monitoring system, comprising: An information collection module is used to regularly collect cloud native environment information using a container orchestration tool, where the cloud native environment information includes cluster information, container information, and microservice log information; The information processing module is used to filter and statistically process the cloud native environment information to obtain multiple feature data; An identity identification module, used to generate an identity code for the feature data based on the type of the feature data and the cluster, container, and microservice involved; A fingerprint generation module, used to calculate the hash value of the feature data, and save the identity code and the hash value of the feature data as a fingerprint in a fingerprint database; The fingerprint matching module is used to analyze the matching of two consecutive fingerprint data collected in the fingerprint database. If the two do not match, it is determined that there is an update in the cloud native environment.

[0010] In an optional implementation, the information processing module includes: A filter setting unit, used to pre-set key parameter items, wherein the key parameter items include error information and resource representation parameters; An effective screening unit, used to screen out effective information belonging to key parameter items from the cloud native environment information; An information statistics unit, used to count intermediate data based on the valid information, wherein the intermediate data includes the number of nodes, the number of idle nodes, the number of containers, and the number of microservice error messages; An early warning generation unit, used to generate early warning information based on valid information using a pre-trained neural network model; The information integration unit is used to integrate resource characterization parameters, intermediate data and warning information in the effective information into characteristic data.

[0011] In an optional implementation, the fingerprint generation module includes: A hash calculation unit, used to generate dynamic representation data for the feature data corresponding to each identity code, and calculate a hash value of the dynamic representation data, wherein the dynamic representation data changes as the feature data is updated; A format conversion unit, used to integrate the identity code and hash value of the feature data into structured data, and save the structured data as a fingerprint in a fingerprint database; The data variable unit is used to traverse all feature data and obtain all fingerprint data of the cloud native environment information collected this time.

[0012] In an optional implementation, the fingerprint matching module includes: A fingerprint comparison unit is used to compare the fingerprint data collected this time with the fingerprint data collected last time to obtain multiple difference fingerprints; An identity resolution unit, used to resolve identity codes of multiple differential fingerprints; A first analysis unit is used to compare the data type in the identity code with the data type in the preset alarm rule for multiple difference fingerprints with exactly the same identity code, and generate an alarm message of environment update if the comparison is consistent, and only record the difference fingerprints with exactly the same identity code in the monitoring log if the comparison is inconsistent; The second analyzing unit is used for locating data update factors by analyzing the multiple difference fingerprints whose identity codes are not completely the same.

[0013] In a third aspect, a terminal is provided, including: A storage device for storing fingerprint-based cloud native environment monitoring programs; A processor is used to implement the steps of the fingerprint-based cloud-native environment monitoring method provided in the first aspect when executing the fingerprint-based cloud-native environment monitoring program.

[0014] In a fourth aspect, a computer-readable storage medium is provided, on which a fingerprint-based cloud-native environment monitoring program is stored. When the fingerprint-based cloud-native environment monitoring program is executed by a processor, the steps of the fingerprint-based cloud-native environment monitoring method provided in the first aspect are implemented.

[0015] The beneficial effect of the present invention is that the fingerprint-based cloud native environment monitoring method, system, terminal and storage medium provided by the present invention can reduce the dimension and effectively analyze complex cloud native environment information, generate cloud native environment characterization fingerprints, and accurately monitor the cloud native environment by matching the fingerprints.

[0016] In addition, the invention has a reliable design principle, a simple structure and a very broad application prospect. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 is a schematic flow chart of a method according to an embodiment of the present invention.

[0019] Figure 2 is a schematic block diagram of a system according to an embodiment of the present invention.

[0020] Figure 3 A schematic diagram of the structure of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0023] The fingerprint-based cloud-native environment monitoring method provided in the embodiment of the present invention is executed by a computer device, and accordingly, the fingerprint-based cloud-native environment monitoring system runs in the computer device.

[0024] Figure 1 is a schematic flow chart of a method according to an embodiment of the present invention. Figure 1 The execution subject can be a fingerprint-based cloud native environment monitoring system. According to different requirements, the order of the steps in the flowchart can be changed, and some can be omitted.

[0025] like Figure 1 As shown, the method includes: Step 110: regularly collect cloud native environment information using a container orchestration tool, where the cloud native environment information includes cluster information, container information, and microservice log information; Step 120, filtering and statistically processing the cloud native environment information to obtain multiple feature data; Step 130, generating an identity code for the feature data based on the type of the feature data and the cluster, container, and microservice involved; Step 140, calculating a hash value of the feature data, and saving the identity code and the hash value of the feature data as a fingerprint in a fingerprint database; Step 150: Analyze the matching of two consecutive fingerprint data collected in the fingerprint database. If the two do not match, it is determined that there is an update in the cloud native environment.

[0026] To facilitate understanding of the present invention, the fingerprint-based cloud native environment monitoring method provided by the present invention is further described below based on the principle of the fingerprint-based cloud native environment monitoring method of the present invention and the process of monitoring the cloud native environment based on fingerprints in the embodiment.

[0027] Specifically, fingerprint-based cloud native environment monitoring methods include: S1. Use container orchestration tools to regularly collect cloud native environment information, where the cloud native environment information includes cluster information, container information, and microservice log information.

[0028] Utilize the rich API interfaces provided by container orchestration tools such as Kubernetes, and collect various types of information in the cloud native environment in real time or at preset time intervals through efficient communication methods such as HTTP requests or WebSocket connections.

[0029] Specifically, call the Kubernetes cluster interface to obtain detailed cluster resource information, which covers several key aspects, including: Version information: Through the cluster interface, we can obtain the current version of the Kubernetes cluster, including the major version number, minor version number, and possible patch version number. This helps us understand the update status of the cluster and whether a version upgrade or compatibility check is required.

[0030] Node information: Node information details the status of each node in the cluster, resource allocation (such as CPU, memory, and storage), operating system version, kernel version, and node labels and taints. This information is critical for resource management and troubleshooting.

[0031] Pod information: Pod is the smallest deployable computing unit running in Kubernetes. Through the interface, we can obtain detailed information about all Pods in the cluster, including the Pod's name, status (running, suspended, failed, etc.), namespace, container information (such as image name, port configuration, etc.), and Pod's resource requests and limits.

[0032] Cluster log information: Log information is crucial for monitoring cluster status and troubleshooting. By calling the cluster log API, we can collect logs of various components in the cluster (such as kubelet, etcd, API server, etc.) and logs of applications in Pods.

[0033] User authentication and authorization information: This includes user information in the cluster (such as user name, role), permission configuration (RBAC policy), access control lists (ACLs), etc. This information is critical to ensuring the security of the cluster.

[0034] In addition, based on the microservice governance features and / or API governance features, we can also collect relevant information about the cloud native environment, which helps us understand the interactions between microservices, the availability, performance, and security of APIs, etc.

[0035] We took two main approaches to collecting container information: Through local operation instructions in the container: This includes executing commands such as df -h (to view disk usage) and free -m (to view memory usage) inside the container to obtain the local resource usage and system status of the container.

[0036] Through network interaction within the container: We can use network monitoring tools (such as tcpdump, netstat, etc.) to collect network interaction data between the container and other components, including request frequency, bandwidth usage, latency, etc., to evaluate the network performance and reliability of microservices.

[0037] Finally, through log monitoring components (such as ELK Stack, Prometheus+Grafana, etc.), we can collect and analyze microservice logs in real time, which is very important for monitoring service status, troubleshooting, and optimizing performance. These log information can include service startup logs, error logs, request logs, etc., providing us with the ability to gain in-depth insights into the running status of microservices.

[0038] In a specific example, the main part of the fingerprint can be obtained by collecting the Node, NameSpace, and Service information of the cluster. The following are the methods for collecting Node, NameSpace, and Service respectively: Node fingerprint configuration collection: # Prerequisite: The kubectl command needs to be configured on a machine that can access the k8s cluster.

[0039] bash-5.1# kubectl get nodes -o yaml|grep uid; uid: f988075e-7d3e-41f1-a14d-9a1972a3f3ad; uid: ab2a9911-97db-4a70-a639-ce14b620e272; uid: 8963883e-4a6b-4074-8a22-5fd491e04e7b; uid: b77ce9cf-0c8f-448a-9a7e-9232df2eeeff; uid: 865ed7c5-570b-4806-9a68-1958e4f0dacd; uid: d39258d9-587e-4ccd-af58-55086ae905a5; uid: 4ac95905-3619-4758-bd63-94dc3c20764b; uid: db82daed-3303-4f6e-89f4-0e7c89125705; uid: 9fe94d83-637e-4a0d-97c8-2ffeef66222f; uid: de31fd9a-5dbb-499e-b6a4-5a040658a8ab; uid: ec44849d-d5e8-4610-8979-97f7d29f7bbe; uid: 43a0ac25-09b1-4743-8e7e-84e7221922a6; uid: a8e45ce2-778c-4918-a26f-fb2b9288ca74; uid: 3b9526f8-c3c1-4d92-873e-29bea453d8ae; #By executing kubectl get nodes -o yaml|grep uid, we can get the UID of each server in the K8S cluster. We can see that the cluster has a total of 14 servers.

[0040] NameSpace configuration information collection: bash-5.1# kubectl get ns hone-dev -o yaml; apiVersion: v1; kind: Namespace; metadata: annotations: kubesphere.io / creator:zhangtao22; kubesphere.io / description: Hone basic development platform; creationTimestamp: "2023-03-15T08:07:33Z"; finalizers: - finalizers.kubesphere.io / namespaces; labels: kubernetes.io / metadata.name: hone-dev; kubesphere.io / namespace: hone-dev; kubesphere.io / workspace: gczl-dev; name: hone-dev; ownerReferences: - apiVersion: tenant.kubesphere.io / v1alpha1; blockOwnerDeletion: true; controller: true; kind: Workspace; name: gczl-dev; uid: ae29352e-dba4-4494-9cfb-fcb420fb0ca7; resourceVersion: "22933433"; uid: d27d0dd5-d54e-46a9-b52b-6e5c7729d1cb; spec: finalizers: - kubernetes status: phase: Active。

[0041] Service configuration information collection: bash-5.1# kubectl get svc hone-system -n hone-dev -o yaml; apiVersion: v1; kind: Service; metadata: annotations: kubectl.kubernetes.io / last-applied-configuration: | {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"app":"hone-system","component":"hone-system"},"name":"hone-system","namespace":"hone-dev"},"spec":{"ports":[{"name":"http","port":30022,"protocol":"TCP","targetPort":30022}],"selector":{"app":"hone-system","component":"hone-system","tier":"backend"},"sessionAffinity":"None","type":"NodePort"}}; creationTimestamp: "2023-03-22T08:34:04Z"; labels: app: hone-system; component: hone-system; name: hone-system; namespace: hone-dev; resourceVersion: "24947959"; uid: f487f8dd-0a85-4542-b1f4-4d0ee188b340; spec: clusterIP: 10.233.25.18; clusterIPs: - 10.233.25.18; externalTrafficPolicy: Cluster; internalTrafficPolicy: Cluster; ipFamilies: - IPv4; ipFamilyPolicy: SingleStack; ports: - name: http; nodePort: 32720; port: 30022; protocol: TCP; targetPort: 30022; selector: app: hone-system; component: hone-system; tier: backend; sessionAffinity: None; type: NodePort; status: loadBalancer: {}.

[0042] S2. Filter and statistically process the cloud native environment information to obtain multiple feature data.

[0043] Pre-set key parameter items, which include error information and resource characterization parameters; filter out valid information belonging to key parameter items from cloud native environment information; count intermediate data based on the valid information, which includes the number of nodes, the number of idle nodes, the number of containers, and the number of microservice error information; generate warning information based on valid information using a pre-trained neural network model; integrate resource characterization parameters, intermediate data, and warning information in the valid information into feature data.

[0044] In a specific example, the information processing includes the following steps: Step 1: Pre-set key parameters Error information: including exception information in microservice logs, API request failure information, database connection errors, etc.

[0045] Resource characterization parameters: such as CPU usage, memory usage, disk I / O, network bandwidth, etc., are used to measure the resource status of the node.

[0046] Step 2: Filter out valid information from cloud native environment information Collect environmental information: Collect status information of each node, Pod, and microservice in the cluster through Kubernetes cluster interface, log monitoring components, microservice governance tools, etc.

[0047] Filter key information: Filter out valid information that matches the preset key parameter items from the collected environment information. For example, filter out log entries containing error information and resource representation parameter values ​​of each node.

[0048] Step 3: Count intermediate data Number of nodes: counts the total number of all nodes in the current cluster.

[0049] Number of idle nodes: Based on the resource characterization parameters of the nodes (such as CPU usage and memory occupancy), determine which nodes are idle and count their number.

[0050] Number of containers: counts the total number of containers in all Pods (container groups) in the current cluster.

[0051] Number of microservice error messages: counts the number of error messages generated by each microservice within a specific time window.

[0052] Step 4: Generate warning information using neural network model Train the neural network model: Train a neural network model in advance. The model can receive inputs such as resource characterization parameters and error information, and output warning information (such as whether a failure is likely to occur, the type of failure, the severity of the failure, etc.).

[0053] Generate early warning information: Input the filtered valid information (including resource representation parameters, error information, etc.) into the trained neural network model, and the model generates corresponding early warning information based on the input information.

[0054] Step 5: Integrate into feature data Integrate information: Integrate the resource characterization parameters in the filtered valid information, the statistical intermediate data (number of nodes, number of idle nodes, number of containers, number of microservice error messages), and the generated warning information to form feature data.

[0055] S3. Generate an identity code for the feature data based on the type of the feature data and the clusters, containers, and microservices involved.

[0056] In a specific example, the identity code is generated as follows: Identity code generation rules: Basic code: First, we assign a basic code to different types of feature data. For example, the basic code of resource characterization parameters can be "R", the basic code of intermediate data can be "M", and the basic code of warning information can be "W".

[0057] Cluster Identifier: Next, we assign a unique cluster identifier to each cluster. This identifier can be the cluster name, ID, or any other string that uniquely identifies the cluster. For example, if the cluster name is "Kubernetes-Cluster-A", then its cluster identifier can be "KCA".

[0058] Container / Pod Identifier: For feature data related to a specific container or Pod, we also need to assign them a container / Pod identifier. This identifier can be the name, ID, or any other string that can uniquely identify the container / Pod. For example, if the name of the container / Pod is "my-app-pod-123", then its container / Pod identifier can be "MAP123".

[0059] Microservice ID: For feature data related to a specific microservice, we also need to assign a microservice ID to them. This ID can be the name, version, or any other string that can uniquely identify the microservice. For example, if the name of the microservice is "my-microservice-v1", then its microservice ID can be "MMV1".

[0060] Timestamp: In order to distinguish the feature data generated at different time points of the same type, the same cluster, the same container / Pod or the same microservice, we also need to add a timestamp to the identity code. This timestamp can be the date and time when the feature data is generated, and the format can be "YYYYMMDDHHMMSS".

[0061] Sequence number: If multiple feature data of the same type, the same cluster, the same container / Pod, and the same microservice are generated at the same time, we also need to assign a sequence number to distinguish them. This sequence number can be an increasing integer.

[0062] Identity code example: Assume that we have a feature data of resource characterization parameter type, which comes from the cluster named "Kubernetes-Cluster-A", is related to the container / Pod named "my-app-pod-123", and this data is generated by the microservice named "my-microservice-v1". The feature data was generated at 3:45:30 pm on October 5, 2023, and this is the first such data generated at that time.

[0063] Then, according to our identity code generation rules, the identity code of this feature data can be: R-KCA-MAP123-MMV1-20231005154530-001 This identity code includes the type of feature data (R represents resource characterization parameter), cluster identifier (KCA), container / Pod identifier (MAP123), microservice identifier (MMV1), timestamp (20231005154530) and serial number (001), which can uniquely identify this feature data.

[0064] By generating such identity codes for feature data, we can manage and utilize this data more conveniently and improve the efficiency and accuracy of data processing.

[0065] S4. Calculate the hash value of the feature data, and save the identity code and the hash value of the feature data as a fingerprint in a fingerprint database.

[0066] S401, generating dynamic representation data for feature data corresponding to each identity code respectively, and calculating a hash value of the dynamic representation data, wherein the dynamic representation data changes as the feature data is updated; In an example, the summary part of the collected configuration information is selected according to certain rules for SHA-256 operation. For example, Node selects the UID f988075e-7d3e-41f1-a14d-9a1972a3f3ad of the first node, NameSpace selects the UID d27d0dd5-d54e-46a9-b52b-6e5c7729d1cb of hone-dev to assemble into a new summary string, and the UID of Service is selected in the same way. Finally, the configuration information can be expressed in json format as follows: "node-uid": "f988075e-7d3e-41f1-a14d-9a1972a3f3ad", "namespace-uid": "d27d0dd5-d54e-46a9-b52b-6e5c7729d1cb", "service-uid": "xxxxxxxxx".

[0067] Use Java programming language to perform SHA-256 operation on Json: / ** * SHA256, Secure Hash Algorithm 256-bit * Input: string to be encrypted * Output: 256 bits (16 bytes) or 64 hexadecimal characters (commonly used) * Applications: password management, digital signature, file integrity verification * @param plainString plain text * @return cipherString ciphertext * / public static String sha256(String plainString) { String cipherString = null; try { / / Get an instance (same as SHA-512) MessageDigest messageDigest = MessageDigest.getInstance("SHA-256"); / / Calculate summary byte[] cipherBytes = messageDigest.digest(plainString.getBytes(StandardCharsets.UTF_8)); / / Output as a hexadecimal string StringBuilder sb = new StringBuilder(); for (byte b : cipherBytes) { sb.append(String.format("%02x", b));} cipherString = sb.toString();} catch (Exception e) { e.printStackTrace();} return cipherString;}.

[0068] S402: Integrate the identity code and hash value of the feature data into structured data, and save the structured data as a fingerprint in a fingerprint database; traverse all feature data to obtain all fingerprint data of the cloud native environment information collected this time.

[0069] Combine the identity code and hash value of the feature data to form a structured data. This structured data can be a simple key-value pair, where the key is the "identity code" and the value is the "hash value". Save the above structured data as a fingerprint in the fingerprint database. Repeat the above steps for all feature data of the cloud native environment information collected this time, generate a fingerprint for each feature data, and save it to the fingerprint database.

[0070] The first version of the fingerprint database recommends the use of distributed database systems (such as Cassandra, HBase, etc.). These systems support high-concurrency reading and writing, automatic sharding, horizontal expansion and other features, and can meet the storage needs of large-scale data in a cloud-native environment. The data storage structure is recommended to use a columnar storage structure to optimize the reading and writing performance of the data. By designing reasonable indexing and query optimization strategies, the efficiency of data retrieval and query can be improved to ensure that the system can quickly respond to monitoring and early warning needs. Finally, consider data security, implement strict data access control and encryption measures, and ensure the security and privacy protection of fingerprint data.

[0071] S5. Analyze the matching of two consecutive fingerprint data collected in the fingerprint database. If the two do not match, it is determined that the cloud native environment has been updated.

[0072] Obtain the latest environmental fingerprints from the fingerprint database periodically or in real time through scheduled tasks (such as Cron jobs) or event triggering mechanisms (such as Kubernetes event listeners). Fingerprint comparison uses an efficient fingerprint comparison algorithm to compare and analyze new fingerprints with old fingerprints. By calculating the difference or similarity between fingerprints, determine whether the environment has changed or there is an anomaly. Once a change or anomaly is detected, the system immediately triggers the early warning mechanism and generates a detailed early warning report. The report includes detailed information on the abnormal fingerprint, the content of the change, possible causes, and recommended countermeasures, etc., providing administrators with comprehensive reference information.

[0073] In one implementation of the present invention, the fingerprint data collected this time is compared with the fingerprint data collected last time to obtain multiple differential fingerprints; the identity codes of the multiple differential fingerprints are parsed; for multiple differential fingerprints with exactly the same identity codes, the data type in the identity code is compared with the data type in the preset alarm rule, and if the comparison is consistent, an alarm message of the environment update is generated; if the comparison is inconsistent, only the differential fingerprints with exactly the same identity code are recorded in the monitoring log; for multiple differential fingerprints with not exactly the same identity codes, the data update factor is located by parsing the not exactly the same identity codes.

[0074] Step 1: Compare fingerprint data: Obtaining fingerprint data: extracting the fingerprint data set collected this time (referred to as "this fingerprint data") and the fingerprint data set collected last time (referred to as "last fingerprint data") from the fingerprint database.

[0075] Comparison operation: Compare the fingerprint data in the two sets to find all the different fingerprints, that is, differential fingerprints. These differential fingerprints represent the data that has changed since the last collection.

[0076] Step 2: Analyze the identity code of the difference fingerprint: Extract identity code: Extract the corresponding identity code from each difference fingerprint.

[0077] Classification processing: According to the similarity of identity codes, the difference fingerprints are divided into two categories: difference fingerprints with exactly the same identity codes and difference fingerprints with not exactly the same identity codes.

[0078] Step 3: Process the difference fingerprints with exactly the same identity code: Compare data types: For differential fingerprints with exactly the same identity codes, compare the data type in their identity codes (the basic coding part, such as "R" for resource characterization parameters) with the data type in the preset alarm rules.

[0079] Generate alarm information or record log: If the comparison is consistent, it means that the data of this type has changed, which may mean that the environment has been updated or abnormal. At this time, according to the preset alarm rules, the corresponding environment update alarm information is generated.

[0080] If the comparison is inconsistent, it may mean that the rest of the identity code (such as cluster ID, container / Pod ID, microservice ID, timestamp or serial number) has changed, but the data type itself has not changed. In this case, only the difference fingerprints of these identity codes that are exactly the same are recorded in the monitoring log for subsequent analysis.

[0081] Step 4: Processing different fingerprints with different identity codes: Locating data update factors: For differential fingerprints with different identity codes, the specific factors that cause data updates are located by parsing the various parts of the identity code (cluster ID, container / Pod ID, microservice ID, timestamp, serial number, etc.).

[0082] Record analysis results: Record the analysis results in the monitoring log, including information such as which factors have changed and when the changes occurred.

[0083] Example: Assume that a differential fingerprint is collected this time, and its identity code is "R-KCA-MAP123-MMV1-20231006154530-001", while the identity code of the corresponding fingerprint collected last time is "R-KCA-MAP123-MMV1-20231005154530-001".

[0084] Compare fingerprints: The hash values ​​of the two fingerprints are found to be different, so they are differential fingerprints.

[0085] Parsing the identity code: The first five parts of the identity code (basic code, cluster ID, container / Pod ID, microservice ID, and the first eight digits of the timestamp) are exactly the same, but the last four digits of the timestamp (i.e., seconds) and the serial number are different.

[0086] deal with: Since the basic code "R" represents a resource characterization parameter, which is consistent with the data type in the preset alarm rule, it is judged to be an environment update.

[0087] Generates an alarm message: "The data of the resource representation parameter type was updated on the my-microservice-v1 microservice in the my-app-pod-123 container / Pod in the Kubernetes-Cluster-A cluster at 15:45:30 on October 6, 2023." For differential fingerprints whose identity codes are not exactly the same, if the cluster ID or container / Pod ID has changed, it may indicate data migration or deployment changes; if the microservice ID has changed, it may indicate a microservice version update or replacement; if the timestamp has changed, it obviously means that the data was collected at different time points; if the serial number has changed, it may mean that multiple feature data of the same type, the same cluster, the same container / Pod, and the same microservice were generated at the same time point.

[0088] In some embodiments, the fingerprint-based cloud native environment monitoring system may include multiple functional modules composed of computer program segments. The computer programs of each program segment in the fingerprint-based cloud native environment monitoring system may be stored in a memory of a computer device and executed by at least one processor to perform (see Figure 1 Description) Fingerprint-based cloud native environment monitoring capabilities.

[0089] In this embodiment, the fingerprint-based cloud native environment monitoring system can be divided into multiple functional modules according to the functions it performs, such as Figure 2 As shown. The functional modules of the system 200 may include: an information collection module 210, an information processing module 220, an identity identification module 230, a fingerprint generation module 240 and a fingerprint matching module 250. The module referred to in the present invention refers to a series of computer program segments that can be executed by at least one processor and can complete fixed functions, which are stored in a memory. In this embodiment, the functions of each module will be described in detail in subsequent embodiments.

[0090] An information collection module is used to regularly collect cloud native environment information using a container orchestration tool, where the cloud native environment information includes cluster information, container information, and microservice log information; The information processing module is used to filter and statistically process the cloud native environment information to obtain multiple feature data; An identity identification module, used to generate an identity code for the feature data based on the type of the feature data and the cluster, container, and microservice involved; A fingerprint generation module, used to calculate the hash value of the feature data, and save the identity code and the hash value of the feature data as a fingerprint in a fingerprint database; The fingerprint matching module is used to analyze the matching of two consecutive fingerprint data collected in the fingerprint database. If the two do not match, it is determined that there is an update in the cloud native environment.

[0091] Optionally, as an embodiment of the present invention, the information processing module includes: A filter setting unit, used to pre-set key parameter items, wherein the key parameter items include error information and resource representation parameters; An effective screening unit, used to screen out effective information belonging to key parameter items from the cloud native environment information; An information statistics unit, used to count intermediate data based on the valid information, wherein the intermediate data includes the number of nodes, the number of idle nodes, the number of containers, and the number of microservice error messages; An early warning generation unit, used to generate early warning information based on valid information using a pre-trained neural network model; The information integration unit is used to integrate resource characterization parameters, intermediate data and warning information in the effective information into characteristic data.

[0092] Optionally, as an embodiment of the present invention, the fingerprint generation module includes: A hash calculation unit, used to generate dynamic representation data for the feature data corresponding to each identity code, and calculate a hash value of the dynamic representation data, wherein the dynamic representation data changes as the feature data is updated; A format conversion unit, used to integrate the identity code and hash value of the feature data into structured data, and save the structured data as a fingerprint in a fingerprint database; The data variable unit is used to traverse all feature data and obtain all fingerprint data of the cloud native environment information collected this time.

[0093] Optionally, as an embodiment of the present invention, the fingerprint matching module includes: A fingerprint comparison unit is used to compare the fingerprint data collected this time with the fingerprint data collected last time to obtain multiple difference fingerprints; An identity resolution unit, used to resolve identity codes of multiple differential fingerprints; A first analysis unit is used to compare the data type in the identity code with the data type in the preset alarm rule for multiple difference fingerprints with exactly the same identity code, and generate an alarm message of environment update if the comparison is consistent, and only record the difference fingerprints with exactly the same identity code in the monitoring log if the comparison is inconsistent; The second analyzing unit is used for locating data update factors by analyzing the multiple difference fingerprints whose identity codes are not completely the same.

[0094] Figure 3 A schematic diagram of the structure of a terminal 300 provided in an embodiment of the present invention. The terminal 300 can be used to execute the fingerprint-based cloud native environment monitoring method provided in an embodiment of the present invention.

[0095] The terminal 300 may include: a processor 310, a memory 320 and a communication unit 330. These components communicate via one or more buses. Those skilled in the art will appreciate that the server structure shown in the figure does not limit the present invention, and it may be a bus structure or a star structure, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0096] The memory 320 can be used to store the execution instructions of the processor 310, and the memory 320 can be implemented by any type of volatile or non-volatile storage terminal or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. When the execution instructions in the memory 320 are executed by the processor 310, the terminal 300 can perform some or all of the steps in the following method embodiments.

[0097] The processor 310 is the control center of the storage terminal, and uses various interfaces and lines to connect various parts of the entire electronic terminal. It runs or executes software programs and / or modules stored in the memory 320, and calls data stored in the memory to perform various functions of the electronic terminal and / or process data. The processor can be composed of an integrated circuit (IC), for example, it can be composed of a single packaged IC, or it can be composed of multiple packaged ICs with the same or different functions. For example, the processor 310 can only include a central processing unit (CPU). In the embodiment of the present invention, the CPU can be a single computing core or multiple computing cores.

[0098] The communication unit 330 is used to establish a communication channel so that the storage terminal can communicate with other terminals, receive user data sent by other terminals or send user data to other terminals.

[0099] The present invention also provides a computer storage medium, wherein the computer storage medium may store a program, and when the program is executed, the program may include some or all of the steps in each embodiment provided by the present invention. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0100] Therefore, the present invention can reduce the dimension and effectively analyze complex cloud native environment information, generate cloud native environment characterization fingerprints, and accurately monitor the cloud native environment by matching the fingerprints. The technical effects that can be achieved by this embodiment can be found in the description above and will not be repeated here.

[0101] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention, in essence or in other words, the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, and other media that can store program codes, including several instructions for enabling a computer terminal (which can be a personal computer, a server, or a second terminal, a network terminal, etc.) to execute all or part of the steps of the methods described in each embodiment of the present invention.

[0102] In this specification, the same or similar parts between the various embodiments can be referred to each other. In particular, for the terminal embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiment.

[0103] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or modules, which can be electrical, mechanical or other forms.

[0104] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0105] In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0106] Although the present invention has been described in detail with reference to the accompanying drawings and in combination with the preferred embodiments, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, a person of ordinary skill in the art may make various equivalent modifications or substitutions to the embodiments of the present invention, and these modifications or substitutions shall be within the scope of the present invention. Any person of ordinary skill in the art may easily think of changes or substitutions within the technical scope disclosed by the present invention, and these shall be within the scope of protection of the present invention.

Claims

1. A fingerprint-based cloud native environment monitoring method, characterized in that: include: Use container orchestration tools to regularly collect cloud native environment information, including cluster information, container information, and microservice log information; Filter and statistically process cloud native environment information to obtain multiple feature data; Generate an identity code for the feature data based on the type of the feature data and the clusters, containers, and microservices involved; Calculate the hash value of the feature data, and save the identity code and the hash value of the feature data as a fingerprint in a fingerprint database; Analyze the matching of two consecutive fingerprint data collected in the fingerprint database. If the two do not match, it is determined that the cloud native environment has been updated.

2. The method according to claim 1, characterized in that Filter and statistically process the cloud native environment information to obtain multiple feature data, including: Pre-setting key parameter items, wherein the key parameter items include error information and resource characterization parameters; Filter out valid information belonging to key parameter items from cloud native environment information; Counting intermediate data based on the valid information, the intermediate data includes the number of nodes, the number of idle nodes, the number of containers, and the number of microservice error messages; Generate early warning information based on valid information using pre-trained neural network models; Integrate resource characterization parameters, intermediate data and warning information in effective information into feature data.

3. The method according to claim 2, characterized in that Calculating a hash value of the feature data, and saving the identity code and the hash value of the feature data as a fingerprint in a fingerprint database, including: Generating dynamic representation data for the feature data corresponding to each identity code respectively, and calculating a hash value of the dynamic representation data, wherein the dynamic representation data changes as the feature data is updated; Integrate the identity code and hash value of the feature data into structured data, and save the structured data as a fingerprint in a fingerprint database; Traverse all feature data to obtain all fingerprint data of the cloud native environment information collected this time.

4. The method according to claim 3, characterized in that Compare the matching of two consecutive fingerprint data collected in the fingerprint database. If the two do not match, it is determined that the cloud native environment has been updated, including: Compare the fingerprint data collected this time with the fingerprint data collected last time to obtain multiple difference fingerprints; Parse identity codes of multiple differential fingerprints; For multiple differential fingerprints with exactly the same identity code, the data type in the identity code is compared with the data type in the preset alarm rule. If the comparison is consistent, an alarm message of the environment update is generated. If the comparison is inconsistent, only the differential fingerprints with exactly the same identity code are recorded in the monitoring log; For multiple differential fingerprints whose identity codes are not completely identical, the data update factor is located by parsing the identity codes that are not completely identical.

5. A fingerprint-based cloud native environment monitoring system, characterized in that: include: An information collection module is used to regularly collect cloud native environment information using a container orchestration tool, where the cloud native environment information includes cluster information, container information, and microservice log information; The information processing module is used to filter and statistically process the cloud native environment information to obtain multiple feature data; An identity identification module, used to generate an identity code for the feature data based on the type of the feature data and the cluster, container, and microservice involved; A fingerprint generation module, used to calculate the hash value of the feature data, and save the identity code and the hash value of the feature data as a fingerprint in a fingerprint database; The fingerprint matching module is used to analyze the matching of two consecutive fingerprint data collected in the fingerprint database. If the two do not match, it is determined that there is an update in the cloud native environment.

6. The system according to claim 5, characterized in that The information processing module includes: A filter setting unit, used to pre-set key parameter items, wherein the key parameter items include error information and resource representation parameters; An effective screening unit, used to screen out effective information belonging to key parameter items from the cloud native environment information; An information statistics unit, used to count intermediate data based on the valid information, wherein the intermediate data includes the number of nodes, the number of idle nodes, the number of containers, and the number of microservice error messages; An early warning generation unit, used to generate early warning information based on valid information using a pre-trained neural network model; The information integration unit is used to integrate resource characterization parameters, intermediate data and warning information in the effective information into characteristic data.

7. The system according to claim 6, characterized in that The fingerprint generation module comprises: A hash calculation unit, used to generate dynamic representation data for the feature data corresponding to each identity code, and calculate a hash value of the dynamic representation data, wherein the dynamic representation data changes as the feature data is updated; A format conversion unit, used to integrate the identity code and hash value of the feature data into structured data, and save the structured data as a fingerprint in a fingerprint database; The data variable unit is used to traverse all feature data and obtain all fingerprint data of the cloud native environment information collected this time.

8. The system according to claim 7, characterized in that The fingerprint matching module comprises: A fingerprint comparison unit is used to compare the fingerprint data collected this time with the fingerprint data collected last time to obtain multiple difference fingerprints; An identity resolution unit, used to resolve identity codes of multiple differential fingerprints; A first analysis unit is used to compare the data type in the identity code with the data type in the preset alarm rule for multiple difference fingerprints with exactly the same identity code, and generate an alarm message of environment update if the comparison is consistent, and only record the difference fingerprints with exactly the same identity code in the monitoring log if the comparison is inconsistent; The second analyzing unit is used for locating data update factors by analyzing the multiple difference fingerprints whose identity codes are not completely the same.

9. A terminal, characterized in that: include: A storage device for storing fingerprint-based cloud native environment monitoring programs; A processor, used to implement the steps of the fingerprint-based cloud native environment monitoring method as described in any one of claims 1-4 when executing the fingerprint-based cloud native environment monitoring program.

10. A computer-readable storage medium storing a computer program, characterized in that: The readable storage medium stores a fingerprint-based cloud-native environment monitoring program, and when the fingerprint-based cloud-native environment monitoring program is executed by the processor, the steps of the fingerprint-based cloud-native environment monitoring method according to any one of claims 1 to 4 are implemented.

Images