Global link sensing method and device based on universal time sequence and storage medium
By splitting the log into indicator tags and data content, and using structured storage and timing analysis, the problems of unknown fault identification and log system in large service systems are solved, real-time perception and risk warning of global links are realized, cost reduction and fault location efficiency are improved.
Patent Information
- Application Number
- CN202510388100.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-31
- Publication Date
- 2025-07-04
AI Technical Summary
The prior art is difficult to identify unknown faults in large service systems, traditional buried point solutions are costly and insufficient coverage, semi-structured storage solutions are expensive and have poor reusability, log system retrieval and analysis efficiency is low, and real-time perception and risk warning of global links cannot be achieved.
The original log is split into indicator tags and data content, stored in a time series or relational database using structured log format, combined with timing operations and memory aggregation technology for real-time aggregation analysis, and display multi-level link relationships and timing charts through a visual interface to generate a global link summary and risk warning.
It realizes seamless connection with the existing distributed data acquisition architecture in multiple scenarios, reduces storage costs, improves retrieval and analysis efficiency, supports second-level problem positioning, reduces operation and maintenance costs, improves fault perception and decision-making efficiency, and ensures log content compliance.
Smart Images
Figure CN120256403A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of global link perception, and particularly to a global link perception method, an electronic device, and a storage medium based on general time series. Background Art
[0002] With the increasing complexity of large-scale service systems, failures caused by change operations (such as releases, BugFixes, configuration changes, etc.) have become a common problem in the industry. In the prior art, SkyWalking locates known problems through call chain tracing, but lacks the ability to identify unknown problems and cannot generate a global summary. Traditional instrumentation solutions rely on manual instrumentation, resulting in high R & D costs, insufficient instrumentation coverage, and problems such as passive patching, making it difficult to achieve auditing and insight from a global perspective. Another semi-structured storage solution can aggregate data for specific scenarios, but it relies on a high-performance engine, is costly, and has poor reusability, and cannot effectively support the discovery of unknown problems. In addition, existing logging systems mostly store in an unstructured form, with low retrieval and analysis efficiency, and it is difficult to meet the real-time monitoring requirements of multiple scenarios. Therefore, there is an urgent need for an efficient, general-purpose, and low-cost logging processing technology to achieve real-time perception and risk warning of the global link through structured transformation and time series analysis. Summary of the Invention
[0003] In view of the above technical problems, the technical solution adopted by the present invention is as follows: According to the first aspect of the present application, there is provided a global link perception method based on general time series, the method comprising the following steps: S100, splitting the original log into metric tags (tags) and data content (datas); wherein, the metric tags represent the hierarchical relationship of the business scenario in a tree structure, and the data content is bound to the corresponding metrics; S200, classifying and storing the metric tags and the data content through a structured log format; wherein, the metric tags are stored in a time series database or a relational database, and the data content is stored in a conventional log storage medium; S300, performing real-time aggregation analysis on the metrics using time series operations or in-memory aggregation techniques to generate a global link summary, a fluctuation summary, and a risk warning; S400, dynamically displaying the multi-level linkage relationship of the link metrics, a real-time magnitude time series graph, and a historical comparison analysis graph through a visualization interface.
[0004] Further, step S100 includes the following steps: S110, splitting the original log content into several hierarchical metric tags (tags) and the data content (datas) associated with each level according to the business scenario; wherein, the metric tags are defined in a tree structure, the root node is the main business scenario, and the child nodes are the sub-dimensions of the scenario; S120. Bind the data content to the corresponding metric labels, store the specific business parameters through the "datas" field, and dynamically filter sensitive fields through the "excludeKey" or "includeKey" parameters.
[0005] Further, step S200 includes the following steps: S210. Store the metric labels and their corresponding hierarchical relationships in a time series database or a relational database, where the time series database or the relational database supports aggregation queries based on label combinations; S220. After binding the data content to the corresponding associated metric labels, store it in a conventional log storage medium to retain the complete business context.
[0006] Further, step S300 includes the following steps: S310. Perform real-time aggregation operations on the numerical data of the metric labels, and the operation types include summation, counting, and response time statistics; S320. Adopt in-memory pre-aggregation technology to reduce storage pressure, and periodically write the intermediate results to persistent storage.
[0007] Further, step S400 includes the following steps: S410. Display the global link metrics through an interactive large screen, where the global link metrics support multi-level label linkage; S420. Generate real-time time series graphs, historical comparison graphs, and three-primary-color Diff graphs, display the metric hierarchy in a tree structure, and distinguish the data magnitude differences of today, yesterday, and the same period last week through color superposition.
[0008] Further, the metric labels construct multi-level business scenarios through custom fields, supporting combined retrieval and statistics.
[0009] Further, the custom fields include: channel number, customer number, and elapsed time.
[0010] According to another aspect of the present application, there is also provided a non-transitory computer-readable storage medium, in which at least one instruction or at least one program segment is stored, and at least one instruction or at least one program segment is loaded and executed by a processor to implement the above-mentioned global link perception method based on general time series.
[0011] According to another aspect of the present application, there is also provided an electronic device, including a processor and the above-mentioned non-transitory computer-readable storage medium.
[0012] The present invention has at least the following beneficial effects: The global link perception method based on general timing of the present invention supports multiple scenarios such as HTTP and RPC, adapts to complex business systems, and can be seamlessly docked with the existing distributed data collection architecture without code intrusion; by splitting and classifying structured logs for storage, it reduces storage costs, improves the retrieval and analysis efficiency in multiple scenarios, and supports second-level problem positioning; the timing operation and memory aggregation technology achieve millisecond-level latency, meeting the requirements of real-time monitoring and early warning; it reduces the dependence on high-performance engines, has high resource utilization rate and low hardware requirements, and significantly reduces the operation and maintenance costs; through multi-level linked large screens, timing trees and Diff comparison charts, it intuitively presents the changes in link status, improving the fault perception and decision-making efficiency; it has built-in sensitive information desensitization rules, supports screening key data as needed, and ensures the compliance of log content; it unifies the log format, solves the problem of traditional log chaos, and reduces the costs of team collaboration and cognitive alignment; it adopts efficient, general and low-cost log processing technology, and through structured transformation and timing analysis, realizes real-time perception and risk early warning of the global link. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.
[0015] Figure 1 It is a flowchart of the global link perception method based on general timing provided by the embodiment of the present invention; Figure 2 It is a schematic diagram of the local printing interface provided by the embodiment of the present invention; Figure 3 It is a schematic diagram of the log page provided by the embodiment of the present invention; Figure 4 It is a schematic diagram of the large screen of the flat relationship of the full-link perception indicators provided by the embodiment of the present invention; Figure 5 It is a schematic diagram of the full-link perception timing tree provided by the embodiment of the present invention; Figure 6 It is the historical yesterday's chart provided by the embodiment of the present invention; Figure 7 It is the chart of the same period last week provided by the embodiment of the present invention; Figure 8 It is the perception Diff chart provided by the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0016] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present invention.
[0017] It should be noted that based on this disclosure, those skilled in the art should understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects described herein can be used to implement the device and / or practice the method. Additionally, this device and / or this method can be implemented using other structures and / or functions in addition to one or more of the aspects described herein.
[0018] Next, a global link awareness method based on a general time sequence will be introduced with reference to Figure 1 the flowchart of the global link awareness method based on the general time sequence shown.
[0019] The global link awareness method based on the general time sequence may include the following steps: S100, splitting the original log into metric tags (tags) and data content (datas); wherein, the metric tags represent the business scenario hierarchy in a tree structure, and the data content is bound to the corresponding metrics.
[0020] Further, step S100 includes the following steps: S110, splitting the original log content into several levels of metric tags (tags) and the data content (datas) associated with each level according to the business scenario; wherein, the metric tags are defined in a tree structure, the root node is the main business scenario, and the child nodes are the sub-dimensions of the scenario; S120, binding the data content to the corresponding metric tags, storing the specific business parameters through the datas field, and dynamically filtering sensitive fields through the excludeKey or includeKey parameters.
[0021] In this embodiment, the general form of printing logs is as follows: log.info("Legality verification, order placing user ID: {}", userId); log.info("Risk control behavior verification, order placing user data: {}", o1); log.info("Risk control scalper verification, order placing user data: {}, intercepted data: {}", o1, o2); After analyzing the traditional logs, the following valid data can be obtained: Placing order user Legality verification; Risk control verification; Behavior verification; Scalper verification; According to the above data structure analysis, it is: ROOT root + K-V structure.
[0022] Summary: The root node identifies what things, which is called an indicator in professional terms. The K-V structure represents the data corresponding to the indicator.
[0023] The original log is a whole content. When retrieving, only the whole sentence or part of the content can be used as the search condition, and at most combined with time to query a certain scenario. This method cannot meet the retrieval and monitoring requirements of multiple scenarios.
[0024] And this log system optimizes the original log, splits its whole content into multiple paragraphs, each paragraph can be regarded as a scenario or an indicator, and customizes some common indicators such as channel number, customer number, and elapsed time. After such splitting, we can retrieve, count or monitor multiple scenarios, greatly improving the flexibility and functionality of log use.
[0025] S200, through the structured log format, classify and store the metric tags and data content; among them, the metric tags are stored in the time series database or relational database, and the data content is stored in the conventional log storage medium.
[0026] Further, step S200 includes the following steps: S210, store the metric tags and their corresponding hierarchical relationships in the time series database or relational database, and the time series database or relational database supports aggregation queries based on tag combinations.
[0027] S220, bind the data content with the corresponding associated metric tags and then store it in the conventional log storage medium, retaining the complete business context.
[0028] In this embodiment, except for the standard field query in log retrieval, the rest are retrieved on the K-V. How to reduce the storage cost and query efficiency without affecting log printing? It can be achieved through the following methods: That is: flatten K, and bind V to ROOT root + K.
[0029] So after these analyses and transformation adaptations, the following data is finally obtained: Placing order user, legality verification, userId; Placing order user, risk control verification, behavior verification, o1; Placing order user, risk control verification, scalper verification, o1, o2; The key information identifying a line of log will be converted into metrics: The key data identifying a line of log will be converted into content: step, param, stepStaus.
[0030] Then format it into structured data: Loger().logger().tags("Place an order", "User legitimacy verification").datas(userId).log(); Loger().logger().tags("Place an order", "Risk control verification", "Risk control behavior verification").datas(o1).log(); Loger().logger().tags("Place an order", "Risk control verification", "Scalper verification").datas(o1, o2).log().
[0031] S300. Use time series operation or in-memory aggregation technology to perform real-time aggregation analysis on the metrics, and generate a global link summary, a fluctuation summary, and a risk warning.
[0032] Furthermore, step S300 includes the following steps: S310. Perform real-time aggregation operations on the numerical data of the metric tags, and the operation types include summation, counting, and response time statistics.
[0033] S320. Adopt in-memory pre-aggregation technology to reduce the storage pressure, and periodically write the intermediate results into persistent storage.
[0034] In this embodiment, assuming it is a shopping scenario and it is necessary to observe in real time the number of users intercepted due to legitimacy verification or behavior verification or scalper verification, or how much money is lost, the following structure can be used: Loger().logger().tags("Place an order", "User legitimacy verification").datas(userId).interval(${actual value}, ServiceLog.IntervalTypeEnum.N_SUM).log(); Loger().logger().tags("Place an order", "Risk control verification", "Risk control behavior verification").datas(o1).interval(${actual value}, ServiceLog.IntervalTypeEnum.N_SUM).log(); Loger().logger().tags("Order Placement", "Risk Control Verification", "Scalper Verification").datas(o1, o2).interval(${actual value}, ServiceLog.IntervalTypeEnum.N_SUM).log(); Among them, the first parameter in the interval method is the actual value (numeric type), and the second parameter is the calculation method of the value, including: summation, counting, etc.
[0035] For example: When this technology is implemented for HTTP or RPC types, it will become as follows: Loger().logger().tags("${interface or controller layer name}").datas(${request input parameters}, ${request return value}).interval(${elapsed time}, ServiceLog.IntervalTypeEnum.N_RT).log().
[0036] Finally, take HTTP or RPC as a full - function example, which is actually applicable to all scenarios and can internally customize the rules for desensitizing user - sensitive information.
[0037] S400, Dynamically display the multi - level linkage relationship, real - time magnitude time - series chart, and historical comparison analysis chart of link metrics through a visual interface.
[0038] Furthermore, step S400 includes the following steps: S410, Display global link metrics through an interactive large - screen, and the global link metrics support multi - level label linkage.
[0039] S420, Generate a real - time time - series chart, a historical comparison chart, and a three - primary - color Diff chart, display the metric hierarchy in a tree structure, and distinguish the data magnitude differences of today, yesterday, and the same period last week through color superposition.
[0040] In this embodiment, Example 1: Print the input parameters, return values, and elapsed time of the RPC interface. When printing, exclude the base64Code in the ignored entity attributes to avoid printing overly long text in the log.
[0041] Loger().logger().tags("${name}").userId(111).datas(${request input parameters},${request return value}).desc("input parameters","return").interval(${time taken},ServiceLog.IntervalTypeEnum.N_RT).excludeKey(Collections.singletonList("base64Code")).log();
[0042] When writing the log, the following key parameters are set: tags: Used to identify a specific scenario, which is one of the scenario elements split from the original log and can be used alone or in combination with other scenario elements for retrieval and statistics.
[0043] userId: The unique identifier of the user, which is also a split scenario element and supports retrieval and statistical operations for single or combined scenarios.
[0044] datas: Contains specific data content, which is a split scenario element and can be used for various retrieval and statistical analyses.
[0045] desc: A detailed description of the data in datas, which helps to understand the meaning and usage of the data.
[0046] excludeKey: Used to exclude the content of the specified key in datas to achieve precise data screening.
[0047] interval: The first field records the time, and the second field is similar to the statistical type, providing information related to the time dimension and statistical method for data statistics.
[0048] Through these parameters, the log data can be retrieved and statistically analyzed based on multiple scenarios flexibly, greatly improving the analysis and utilization efficiency of the log data.
[0049] Example 2: The RPC interface prints the input parameters, return value, and time taken, and only prints the userId and price in the serialized entity, focusing only on the key data printing method: Loger().logger().tags("${name}").userId(222).datas(${request input parameter},${request return value}).desc("input parameter","return").interval(${time consumption},ServiceLog.IntervalTypeEnum.N_RT).includeKey(Collections.singletonList("userId","price")).log();
[0050] The finally generated log is formatted into the following structure. Just for example, the delimiter and others should be replaced according to the actual scenario.
[0051] External format: %d{yyyy-MM-dd HH:mm:ss.SSS}|%X{tid}|%level - [%ex#@^@#%msg] - [1:1] - [%thread]|%property{serverEnv}|%property{applicationName}|%nThe msg format
placeholder: #@^@#
[0052] Collect using the existing distributed data collection architecture and extract the data located in the placeholder in msg. The extraction is still divided into two major segments: The first segment is as follows. This segment needs to be collected into a time - series database or a relational database: tag1,tag2,tag3,tag4,tag5...
[0053] The second segment is as follows. This segment needs to be collected into a normal log storage medium: userId,tag1,tag2,tag3,tag4,tag5...,data1,data2,data3,data4...,desc1,desc2,desc3,desc4...,interval.
[0054] The difference between the method in this embodiment and SkyWalking is that SkyWalking focuses on the statistics of method call chains, while the current example focuses on the statistics of log - printed tags.
[0055] Essential difference: The call chain statistics solution targets the code level, while the tags variable arguments represent business attributes and have a subordinate relationship. It emphasizes more on: which business and which sub-scenario, and being able to locate the affected business based on the description of the tags.
[0056] As Figure 2 shown, local printing; as Figure 3 shown, the d2 long text is self-adaptively scrolled to the keyword and the json structure can be expanded for viewing.
[0057] As Figure 4 shown, the full-link perception index tiled relationship large screen; as an example: the red selection is for multi-level linkage. When k= is selected, k will be automatically aggregated and other ks will be grouped; when k and sk are selected, k and sk will be automatically aggregated and tk and fk will be grouped.
[0058] As Figure 5 shown, the full-link perception time series tree, implemented by Echart, with the color being red; real-time graph: the thickness represents the magnitude.
[0059] As Figure 6 shown, the historical yesterday graph (the thickness represents the magnitude) can have a green color.
[0060] As Figure 7 shown, the graph of the same period last week can have a blue color.
[0061] As Figure 8 shown, the perception Diff graph can be implemented by overlaying the three primary colors with 50% transparency; from the Diff graph, different colored areas can be seen. Among them, the cyan-green color is the fully overlapping part, that is: the lowest magnitude; the pure blue color is the magnitude lost yesterday and last week. Taking blue as an example, the same applies to red and yellow; a yellow boundary is presented, that is: the magnitude reduced today compared to yesterday and the same period last week; the analysis of other overlapping colors will not be elaborated here.
[0062] In this embodiment, it supports multiple scenarios such as HTTP and RPC, adapts to complex business systems, and can seamlessly connect to the existing distributed data collection architecture without code intrusion; by splitting and classifying structured logs for storage, it reduces storage costs, improves the retrieval and analysis efficiency in multiple scenarios, and supports second-level problem location; time series operation and in-memory aggregation technology achieve millisecond-level latency, meeting the requirements of real-time monitoring and early warning; it reduces the dependence on high-performance engines, has high resource utilization rate, low hardware requirements, and significantly reduces operation and maintenance costs; through multi-level linked large screens, time series trees, and Diff comparison charts, it intuitively presents the changes in link status, improving the efficiency of fault perception and decision-making; it has built-in sensitive information desensitization rules, supports screening key data as needed, and ensures the compliance of log content; it unifies the log format, solves the problem of traditional log chaos, and reduces the costs of team collaboration and cognitive alignment; it adopts efficient, general, and low-cost log processing technology, and through structured transformation and time series analysis, it realizes real-time perception and risk early warning of the global link.
[0063] It is universal for complex applications and important applications and applicable to all scenarios; index aggregation and operation are universal, applicable to scenarios including but not limited to magnitude, RT, etc., and can also be used for real-time analysis of business; it has high real-time performance with millisecond-level latency; it has high resource utilization rate, low physical hardware requirements, and low resource costs; it can seamlessly connect to all distributed data collection architectures on the market without architecture intrusion; the time complexity of data parsing is low, there is no hierarchical parsing, and the index can also be collected later to reduce the collection pressure; it has a high degree of visualization, can intuitively reflect and feedback application problems, and has the ability to locate problems at the second level; in general scenarios, msg is defined as unstructured data, and it is semi-formatted as a whole, so it normally does not affect the limitations of the original top-level log and is not restricted by the top-level log format; the format is unified, and the hungry singleton pattern has strong specifications for the format, abandoning the problem of format chaos, which is conducive to all users to align their cognitions.
[0064] In addition, although the steps of the methods in the present disclosure are described in a specific order in the drawings, this does not require or imply that these steps must be executed in that specific order, or that all the steps shown must be executed to achieve the desired result. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step for execution, and / or one step may be decomposed into multiple steps for execution, etc.
[0065] An embodiment of the present invention also provides a non-transitory computer-readable storage medium, which can be set in an electronic device to store at least one instruction or at least one segment of a program related to a method in the method embodiment. The at least one instruction or the at least one segment of the program is loaded and executed by the processor to implement the method provided in the above embodiment.
[0066] The program product may employ any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0067] The computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which the readable program code is carried. Such a propagated data signal may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The readable signal medium may also be any readable medium other than the readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device.
[0068] The program code contained on the readable medium may be transmitted with any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
[0069] The program code for performing the operations of this application may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and also including conventional procedural programming languages such as the "C" language or similar programming languages. The program code may execute entirely on the user's computing device, partially on the user's device, as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).
[0070] An embodiment of the present invention also provides an electronic device, including a processor and the foregoing non-transitory computer-readable storage medium.
[0071] The electronic device is merely an example and should not impose any limitation on the functions and scope of use of the embodiments of this application.
[0072] The electronic device is presented in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: the at least one processor described above, the at least one memory described above, and a bus connecting different system components (including the memory and the processor).
[0073] Wherein, the memory stores program code, and the program code can be executed by the processor, so that the processor executes the steps in various embodiments described in this specification.
[0074] The memory may include a readable medium in the form of volatile memory, such as random access memory (RAM) and / or cache memory, and may further include read-only memory (ROM).
[0075] The memory may also include a program / utility having a set (at least one) of program modules. Such program modules include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include the implementation of a network environment.
[0076] The bus may represent one or more of several types of bus structures, including a memory bus or a memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any bus structure in a variety of bus structures.
[0077] The electronic device can also communicate with one or more external devices (such as a keyboard, a pointing device, a Bluetooth device, etc.), and can also communicate with one or more devices that enable a user to interact with the electronic device, and / or communicate with any device that enables the electronic device to communicate with one or more other computing devices (such as a router, a modem, etc.). Such communication can be carried out through an input / output (I / O) interface. Moreover, the electronic device can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter. The network adapter communicates with other modules of the electronic device through the bus. It should be understood that although not shown in the figure, other hardware and / or software modules can be used in combination with the electronic device, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.
[0078] From the description of the above embodiments, those skilled in the art can easily understand that the exemplary embodiments described herein can be implemented by software or by a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (such as a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.
[0079] The embodiments of the present invention further provide a computer program product, which includes program codes. When the program product runs on an electronic device, the program codes are used to cause the electronic device to execute the steps in the methods according to various exemplary embodiments of the present invention described above in this specification.
[0080] Although some specific embodiments of the present invention have been described in detail by way of examples, those skilled in the art should understand that the above examples are only for the purpose of illustration and not for the purpose of limiting the scope of the present invention. Those skilled in the art should also understand that various modifications can be made to the embodiments without departing from the scope and spirit of the present invention.
Claims
1. A global link awareness method based on general timing, characterized in that The method includes the following steps: S100. Split the original log into metric tags (tags) and data content (datas); among them, the metric tags represent the hierarchical relationship of the business scenario in a tree structure, and the data content is bound to the corresponding metric; S200. Classify and store the metric tags and the data content in a structured log format; among them, the metric tags are stored in a time series database or a relational database, and the data content is stored in a conventional log storage medium; S300. Use time series operations or in-memory aggregation technology to perform real-time aggregation analysis on the metrics, and generate a global link summary, a fluctuation summary, and a risk warning; S400. Dynamically display the multi-level linkage relationship, the real-time magnitude time series graph, and the historical comparison analysis graph of the link metrics through a visualization interface.
2. The global link awareness method based on general timing according to claim 1, wherein Step S100 includes the following steps: S110. Split the original log content into several hierarchical metric tags (tags) and the associated data content (datas) at each level according to the business scenario; among them, the metric tags are defined in a tree structure, the root node is the main business scenario, and the child nodes are the sub-dimensions of the scenario; S120. Bind the data content to the corresponding metric tags, store the specific business parameters through the datas field, and dynamically filter sensitive fields through the excludeKey or includeKey parameter.
3. The global link awareness method based on general timing according to claim 1, wherein Step S200 includes the following steps: S210. Store the metric tags and their corresponding hierarchical relationships in a time series database or a relational database, and the time series database or the relational database supports aggregation queries based on tag combinations; S220. After binding the data content to the corresponding associated metric tags, store it in a conventional log storage medium, retaining the complete business context.
4. The global link awareness method based on general timing according to claim 1, characterized in that Step S300 includes the following steps: S310. Perform real-time aggregation operations on the numerical data of the metric tags, and the operation types include summation, counting, and response time statistics; S320. Adopt in-memory pre-aggregation technology to reduce the storage pressure, and periodically write the intermediate results into persistent storage.
5. The global link awareness method based on general timing according to claim 1, characterized in that Step S400 includes the following steps: S410. Display the global link metrics through an interactive large screen, and the global link metrics support multi-level tag linkage; S420. Generate a real-time time series graph, a historical comparison graph, and a three-primary-color Diff graph, display the metric hierarchy in a tree structure, and distinguish the data magnitude differences of today, yesterday, and the same period last week through color overlay.
6. The global link awareness method based on general timing according to claim 5, characterized in that, The metric tags construct a multi-level business scenario through custom fields, and support combined retrieval and statistics.
7. The global link awareness method based on general timing according to claim 6, characterized in that The custom fields include: channel number, customer number, and elapsed time.
8. A non-transitory computer-readable storage medium storing at least one instruction or at least one program segment, characterized in that, The at least one instruction or the at least one program is loaded and executed by a processor to implement the global link perception method based on a general time series as described in any one of claims 1-7.
9. An electronic device, characterized in that, It includes a processor and the non-transitory computer-readable storage medium described in claim 8.