Traffic observation method and device, computer device and storage medium

Abstract

The application relates to a flow observation method and device, computer equipment, a storage medium and a computer program product, and relates to the technical field of cloud computing. The method comprises the following steps: obtaining stored unitized monitoring information based on a preset data acquisition rule; the unitized monitoring information comprises unitized monitoring data and a type identifier of a unitized index type; performing data processing on the unitized monitoring information according to a preset flow observation view category and a data processing rule corresponding to the flow observation view category, to obtain view display information; and constructing a unitized flow observation view according to the view display information, a unitized dimension determined in advance and a preset unitized flow observation view construction rule; the unitized flow observation view is used for flow observation on data of a unitized system. The method enables a user to perform flow observation on data of a unitized system according to multiple dimensions.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cloud computing, and in particular to a traffic observation method and device, computer equipment, a storage medium and a computer program product. BACKGROUND

[0002] A unit is a self-contained set that can complete all business operations, and all services required by the business and data allocated to the unit are included in the set. The unit architecture is to take the unit as the basic unit of system deployment, deploy several units in all machine rooms of the station, and deploy all applications required by the system in any unit. The data is a part of the full data divided horizontally according to a certain dimension (usually customer dimension). Unitization is the process of designing and transforming services and data to meet the unit characteristics.

[0003] In the running process of the unitization system, data such as business data and log data will be generated. In order to ensure the normal operation of the unitization system, the data of the unitization system needs to be observed in traffic to discover abnormal operation of the unitization system in time.

[0004] However, there is currently a lack of traffic observation methods suitable for unitization systems. SUMMARY

[0005] Therefore, it is necessary to provide a traffic observation method, device, computer equipment, computer readable storage medium and computer program product suitable for unitization systems to solve the above technical problems.

[0006] In a first aspect, the present application provides a traffic observation method. The method comprises:

[0007] Based on a preset data acquisition rule, stored unitization monitoring information is acquired; the unitization monitoring information includes unitization monitoring data and type identifiers of the unitization index types; the unitization monitoring data includes unitization metadata and monitoring data corresponding to the unitization metadata;

[0008] According to a preset traffic observation view category and a data processing rule corresponding to the traffic observation view category, the unitization monitoring information is processed to obtain view display information;

[0009] According to the view display information, a predetermined unitization dimension and a preset unitization traffic observation view construction rule, a unitization traffic observation view is constructed; the unitization traffic observation view is used for traffic observation of data of a unitization system.

[0010] In one embodiment, when the traffic observation view category is a cross-unit traffic monitoring view, the step of processing the unitized monitoring information according to the preset traffic observation view category and the data processing rules corresponding to the traffic observation view category to obtain view display information includes:

[0011] For each transaction in the unitized monitoring information, the correspondence between the time corresponding to each node in the transaction information and the unit type corresponding to each node in the transaction information is stored in a temporary cache.

[0012] The correspondences are sorted according to the chronological order, and the node sequence of each node included in the transaction information is determined based on the sorted correspondences.

[0013] Match each node sequence with the preset cell access rules;

[0014] For each unit mutual access rule, the number of transactions corresponding to the unit mutual access rule is updated according to the number of times the unit mutual access rule is successfully matched with the sequence of each node;

[0015] The view display information is determined based on the unitized monitoring information and the number of transactions corresponding to the mutual access rules of each unit.

[0016] In one embodiment, when the traffic observation view category is a unit link view, the step of processing the unitized monitoring information according to the preset traffic observation view category and the data processing rules corresponding to the traffic observation view category to obtain view display information includes:

[0017] For each link in the unitized monitoring information, determine the nodes that the link passes through;

[0018] For each node, the unit type of the node is bound to the node address and the application name of the application included in the node, and the mapping relationship between the unit type, node address and application name is constructed and stored;

[0019] Based on the unitized monitoring information and the mapping relationship, the view display information is determined.

[0020] In one embodiment, constructing the unitized traffic observation view based on the view display information, a pre-determined unitized dimension, and preset unitized traffic observation view construction rules includes:

[0021] For each predetermined unitized dimension, the view display information is aggregated according to the unitized dimension to obtain the view display information corresponding to each unitized object under the unitized dimension;

[0022] Based on the view display information corresponding to each unitized object under each of the aforementioned unitized dimensions, a unitized traffic observation view is constructed.

[0023] In one embodiment, acquiring the stored unitized monitoring information based on preset data acquisition rules includes:

[0024] For each preset traffic observation view category, determine the table name of the data table corresponding to the traffic observation view category;

[0025] Based on the table name of the data table, determine the view variables in the data table, and obtain the variable values ​​corresponding to the view variables in the data table in the data source to obtain unitized monitoring information.

[0026] In one embodiment, the method further includes:

[0027] In response to the data source configuration command, configure the data source corresponding to each of the unitized indicator types;

[0028] In response to the variable definition instruction, each view variable is defined according to the preset view variable and the definition rules corresponding to the view variable;

[0029] In response to the data table setting command, determine the table name of the data table corresponding to each of the traffic observation view categories, as well as the correspondence between the table name and the view variables.

[0030] Secondly, this application also provides a flow rate observation device. The device includes:

[0031] The acquisition module is used to acquire stored unitized monitoring information based on preset data acquisition rules; the unitized monitoring information includes unitized monitoring data and type identifiers of the unitized indicator types; the unitized monitoring data includes unitized metadata and monitoring data corresponding to the unitized metadata;

[0032] The data processing module is used to process the unitized monitoring information according to the preset traffic observation view category and the data processing rules corresponding to the traffic observation view category to obtain view display information.

[0033] The construction module is used to construct a unitized traffic observation view based on the view display information, the pre-determined unitized dimensions, and the preset unitized traffic observation view construction rules; the unitized traffic observation view is used to perform traffic observation on the data of the unitized system.

[0034] In one embodiment, when the traffic observation view category is a cross-cell traffic monitoring view, the data processing module is specifically used for:

[0035] For each transaction in the unitized monitoring information, the correspondence between the time corresponding to each node in the transaction information and the unit type corresponding to each node in the transaction information is stored in a temporary cache.

[0036] The correspondences are sorted according to the chronological order, and the node sequence of each node included in the transaction information is determined based on the sorted correspondences.

[0037] Match each node sequence with the preset cell access rules;

[0038] For each unit mutual access rule, the number of transactions corresponding to the unit mutual access rule is updated according to the number of times the unit mutual access rule is successfully matched with the sequence of each node;

[0039] The view display information is determined based on the unitized monitoring information and the number of transactions corresponding to the mutual access rules of each unit.

[0040] In one embodiment, when the traffic observation view category is a unit link view, the data processing module is specifically used for:

[0041] For each link in the unitized monitoring information, determine the nodes that the link passes through;

[0042] For each node, the unit type of the node is bound to the node address and the application name of the application included in the node, and the mapping relationship between the unit type, node address and application name is constructed and stored;

[0043] Based on the unitized monitoring information and the mapping relationship, the view display information is determined.

[0044] In one embodiment, the building module is specifically used for:

[0045] For each predetermined unitized dimension, the view display information is aggregated according to the unitized dimension to obtain the view display information corresponding to each unitized object under the unitized dimension;

[0046] Based on the view display information corresponding to each unitized object under each of the aforementioned unitized dimensions, a unitized traffic observation view is constructed.

[0047] In one embodiment, the acquisition module is specifically used for:

[0048] For each preset traffic observation view category, determine the table name of the data table corresponding to the traffic observation view category;

[0049] Based on the table name of the data table, determine the view variables in the data table, and obtain the variable values ​​corresponding to the view variables in the data table in the data source to obtain unitized monitoring information.

[0050] In one embodiment, the device further includes:

[0051] The configuration module is used to configure the data source corresponding to each of the unitized indicator types in response to the data source configuration command;

[0052] The definition module is used to define each view variable in response to the variable definition instruction, according to the preset view variables and the definition rules corresponding to the view variables;

[0053] The configuration module is used to respond to the data table configuration command to determine the table name of the data table corresponding to each of the traffic observation view categories, as well as the correspondence between the table name and the view variables.

[0054] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps described in the first aspect.

[0055] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the steps described in the first aspect.

[0056] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps described in the first aspect.

[0057] The aforementioned traffic observation method, device, computer equipment, storage medium, and computer program product acquire stored unitized monitoring information based on preset data acquisition rules. The unitized monitoring information includes unitized monitoring data and type identifiers for the unitized indicator types. The unitized monitoring data includes unitized metadata and monitoring data corresponding to the unitized metadata. According to preset traffic observation view categories and the data processing rules corresponding to those categories, the unitized monitoring information is processed to obtain view display information. Based on the view display information, predetermined unitized dimensions, and preset unitized traffic observation view construction rules, a unitized traffic observation view is constructed. This unitized traffic observation view is used to observe traffic on the data of the unitized system. Thus, by acquiring stored unitized monitoring data including unitized metadata, the unitized monitoring data acquires unit attributes. Based on unitized dimensions and unitized monitoring information, a unitized traffic observation view is constructed, enabling users to observe traffic on the data of the unitized system from multiple dimensions. Attached Figure Description

[0058] Figure 1 This is a diagram illustrating the application environment of the flow observation method in one embodiment;

[0059] Figure 2 This is a schematic diagram of the structure of a unitized system in one embodiment;

[0060] Figure 3 This is a flowchart illustrating a flow observation method in one embodiment;

[0061] Figure 4 This is a flowchart illustrating the data processing steps for unitized monitoring information in one embodiment.

[0062] Figure 5 This is a flowchart illustrating the data processing steps for unitized monitoring information in another embodiment;

[0063] Figure 6 This is a flowchart illustrating the steps involved in constructing a unitized flow observation view in one embodiment;

[0064] Figure 7 This is a flowchart illustrating the steps for obtaining stored unitized monitoring information in one embodiment;

[0065] Figure 8 This is a flowchart illustrating the flow observation method in another embodiment;

[0066] Figure 9 This is a structural block diagram of a flow observation device in one embodiment;

[0067] Figure 10This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0069] The flow observation method provided in this application embodiment can be applied to, for example... Figure 1 The application environment shown is illustrated. The unitized monitoring system 100 includes a data acquisition terminal 102, a data processing node 104, and a server node 106. The unitized monitoring system is used to monitor unitized systems. The unitized monitoring system can be implemented through a cloud computing platform. For example, it can be implemented using Platform as a Service (PaaS). It is understood that the data acquisition terminal 102, data processing node 104, and server node 106 included in the unitized monitoring system 100 can be implemented using individual hardware devices or software modules. A unitized system uses units as the basic unit of system deployment. Each unit deploys all the applications required by the system, and the data in each unit is a portion of the total data after being horizontally divided according to a certain dimension (usually the customer dimension). A unit is a self-contained set capable of completing all business operations, containing all services required by all businesses, as well as the data allocated to that unit. A unit can be a physical unit or a logical unit. It is understood that a unit can be a terminal or a server. A server can be implemented using a standalone server or a server cluster composed of multiple servers. In one embodiment, the unitized system includes: an access unit, a fragmentation unit, and a common unit, such as... Figure 2 As shown. The access unit includes application node A1. The sharding unit includes application node B1, application node C1, application node B2, and application node C2. The common unit includes the application B proxy layer and application node D1. The access unit is responsible for data access. The sharding unit is responsible for data processing. The common unit is responsible for common operations.

[0070] In one embodiment, such as Figure 3 As shown, a flow observation method is provided, which can be applied to... Figure 1 Taking a unitized monitoring system as an example, the following steps are included:

[0071] Step 301: Based on the preset data acquisition rules, acquire the stored unitized monitoring information.

[0072] The unitized monitoring information includes unitized monitoring data and type identifiers for unitized indicator types. The unitized monitoring data includes unitized metadata and the corresponding monitoring data.

[0073] In this embodiment, the monitoring data can be data generated during the operation of the unitized system. For example, the monitoring data can be the business data and log data of the unitized system. Unitized metadata represents the attributes of a unit. Unitized metadata can include physical metadata and logical metadata. Physical metadata represents the attributes of a physical unit. Logical metadata represents the attributes of a logical unit. A physical unit is the unit in which the physical location of a machine environment, such as a server, is located. For example, A-001 represents a server deployed in unit 001 at address A. A logical unit is a unit with a logical meaning and certain business implications. For example, a personal account can be divided into 10 units based on the last four digits of its card number; person-01 represents the unit where the last four digits of the personal account's card number are 1. The type identifier is used to indicate the type of unitized indicator to which the unitized monitoring data belongs.

[0074] In one embodiment, the unitized metric type may include metric class, system class, and link class. The unitized monitoring data for the metric class is related to the transaction action but not to the transaction link, and is used to measure the success or failure of a transaction. The unitized monitoring data for the metric class includes, but is not limited to, transaction volume, transaction time, transaction success identifier, and transaction anomaly type. The unitized monitoring data for the system class is data related to the operating system. The unitized monitoring data for the system class includes, but is not limited to, monitoring data for the Java Virtual Machine (JVM) and the operating system (OS). The unitized monitoring data for the link class is related to the transaction action and involves the transaction link, and is used to represent the transaction link. A transaction link is a series of services involved in a transaction. For example, a payment transaction involves multiple services; before payment, a series of services such as protocol query service, blacklist verification service, and customer authentication service are called. The data from this entire link, strung together, constitutes a complete payment transaction. Link-based unitized monitoring data includes, but is not limited to, link links and call relationships.

[0075] The server node retrieves the unitized monitoring information from the storage space that stores the unitized monitoring information. This unitized monitoring information can be monitoring information directly stored after being collected by the unitized monitoring system, or monitoring information obtained after the unitized monitoring system processes the collected data before storing it. Specifically, for each type of unitized indicator, the server node retrieves the unitized monitoring information of that type from the storage space that stores unitized monitoring information of that type. For example, the server node may retrieve the unitized monitoring information from the data source that stores the unitized monitoring information.

[0076] Step 302: Based on the preset traffic observation view category and the data processing rules corresponding to the traffic observation view category, perform data processing on the unitized monitoring information to obtain the view display information.

[0077] In this embodiment, the traffic observation view category is a category of unitized traffic observation views. The traffic observation view category may include intra-unit traffic monitoring views, cross-unit traffic monitoring views, and unit link views. Intra-unit traffic monitoring views are used to display and monitor traffic within each unit of the unitized system. Cross-unit traffic monitoring views are used to display and monitor traffic across units in the unitized system. Unit link views are used to display and monitor links in the unitized system.

[0078] The server nodes process the unitized monitoring information according to the preset traffic observation view categories and the corresponding data processing rules to obtain the view display information.

[0079] In one example, when the traffic observation view category is intra-cell traffic monitoring view, the server node displays the cell-level monitoring information as the view information.

[0080] Step 303: Construct a unitized traffic observation view based on the view display information, the pre-determined unitized dimensions, and the preset unitized traffic observation view construction rules.

[0081] Among them, the unitized flow observation view is used to observe the flow of data from the unitized system.

[0082] In this embodiment, the unitized monitoring view includes a unitized traffic observation view. The unitized monitoring view is used to monitor the unitized system. The unitization dimension is the dimension used to divide the unitized system. The unitization dimension includes at least a unit dimension. The unit dimension is the dimension that divides the unitized system according to units.

[0083] Server nodes can predetermine the unitization dimensions.

[0084] In one example, server nodes can predetermine the unitization dimension as the unit dimension. This provides aggregation computing capabilities at the unit dimension, helping the development department to drive analysis and problem localization from a unit perspective, making it more accurate in determining the scope of production issues and improving the accuracy of traffic observation.

[0085] In another example, server nodes can pre-determine the unitization dimension as a single unit and other unitization dimensions. This allows for the creation of different levels of monitoring views based on multiple unitization dimensions, satisfying the analysis of production issues with varying impact ranges. This makes it more accurate in determining the scope of production problems and further improves the accuracy of traffic observation.

[0086] Then, the server node constructs the unitized traffic observation view based on the view display information, the pre-determined unitized dimensions, and the preset unitized traffic observation view construction rules.

[0087] In one example, the server node aggregates view display information based on a defined unitization dimension, obtaining the view display information corresponding to each unitized object under that unitization dimension. Then, the server node constructs a unitized traffic observation view based on the view display information corresponding to each unitized object under each unitization dimension and predefined unitized traffic observation view construction rules. Here, a unitized object is the target object under the unitization dimension. For example, the unitized objects under the unitization dimension are various units, including various physical units and various logical units.

[0088] In another example, the server node calculates the results for various preset traffic observation metrics based on the information displayed in the view. These traffic observation metrics are those displayed in the unitized traffic observation view. Then, the server node aggregates the calculation results for each traffic observation metric according to the determined unitized dimension, obtaining the calculation results for each traffic observation metric corresponding to each unitized object within that unitized dimension. Finally, the server node constructs the unitized traffic observation view based on the calculation results for each traffic observation metric corresponding to each unitized object within each unitized dimension and the preset unitized traffic observation view construction rules.

[0089] In one embodiment, the unitization dimension is a unit-form dimension. Unitized objects under the unit-form dimension are physical units and logical units. Traffic observation metrics are anomaly types. The server node aggregates the calculation results corresponding to the traffic observation metrics based on the determined unitization dimension, obtaining the calculation results for each traffic observation metric corresponding to each unitized object under the unitization dimension. Then, the server node constructs a unitized traffic observation view based on the calculation results of the traffic observation metrics corresponding to each unitized object under the unitization dimension and the preset unitized traffic observation view construction rules. Thus, the unitized traffic observation view supports viewing unitized traffic observation views according to different dimensions of physical units and logical units.

[0090] In the aforementioned traffic observation method, stored unitized monitoring information is acquired based on preset data acquisition rules. The unitized monitoring information is then processed according to preset traffic observation view categories and corresponding data processing rules to obtain view display information. Finally, a unitized traffic observation view is constructed based on the view display information, pre-determined unitized dimensions, and preset unitized traffic observation view construction rules. In this way, by acquiring stored unitized monitoring data including unitized metadata, the unitized monitoring data acquires unitized attributes. Furthermore, by constructing a unitized traffic observation view based on unitized dimensions and unitized monitoring information, users can observe traffic from multiple dimensions within the unitized system.

[0091] In one embodiment, such as Figure 4 As shown, when the traffic observation view category is a cross-unit traffic monitoring view, the specific process of processing the unitized monitoring information to obtain the view display information according to the preset traffic observation view category and the corresponding data processing rules includes the following steps:

[0092] Step 401: For each transaction information in the unitized monitoring information, store the correspondence between the time corresponding to each node included in the transaction information and the unit type corresponding to each node included in the transaction information in a temporary cache.

[0093] In this embodiment, the unit type refers to the type of unit in a unitized system. For example, a unitized system may include: an access unit, a fragmentation unit, and a common unit. The fragmentation unit includes: fragmentation unit 1, fragmentation unit 2, fragmentation unit 3, and fragmentation unit 4. The unit type of fragmentation unit 1 is fragmentation unit 1.

[0094] For each transaction record in the unitized monitoring information, the server node establishes a correspondence between the time corresponding to each node in the transaction record and the unit type corresponding to each node. Then, the server node stores the obtained correspondence in a temporary cache. A unit in the unitized system includes multiple nodes. Nodes can be server nodes. The time corresponding to a node can be at the millisecond level. This improves the accuracy of the time corresponding to each node, more accurately distinguishes the order in which a transaction passes through each node, improves the accuracy of the displayed information, and thus improves the accuracy of traffic observation.

[0095] Step 402: Sort the correspondence according to the chronological order, and determine the node sequence of each node included in the transaction information based on the sorted correspondence.

[0096] In this embodiment, the server node sorts the correspondence according to the time of each node in the correspondence, based on the chronological order. Then, the server node sorts the nodes in the correspondence according to the chronological order of the sorted correspondence, thus obtaining the node sequence of the nodes included in the transaction information.

[0097] Step 403: Match each node sequence with the preset unit access rules.

[0098] In this embodiment, the server node presets unit access rules and stores these rules in a unit access rule table. These unit access rules define the mutual access between units in the unitized system. The unit access rules include normal access rules and prohibited access rules. Normal access rules define normal mutual access between units in the unitized system. Prohibited access rules define prohibited mutual access between units in the unitized system. For example, a normal access rule could be access unit 1 accessing fragment unit 1, while a prohibited access rule could be access unit 1 prohibiting access unit 2. The unit access rules are dynamically adjustable. The unit access rules are directional. For example, access unit 1 accessing fragment unit 1 and fragment unit 1 accessing access unit 1 are two different access rules; in reality, fragment unit 1 cannot access access unit 1.

[0099] In one example, the server node can update the cell access rules in real time or periodically and store the updated rules in the cell access rule table. In another example, the server node can respond to an access rule update command, update the cell access rules, and store the updated rules in the cell access rule table. This allows for timely and dynamic adjustments to the cell access rules, improving the accuracy of the displayed information and the accuracy of traffic observation.

[0100] Then, the server node reads all the inter-node access rules from the cell inter-node access rule table. Next, the server node matches each node sequence with the preset cell inter-node access rules.

[0101] Step 404: For each unit mutual access rule, update the number of transactions corresponding to that unit mutual access rule based on the number of times the mutual access rule of that unit is successfully matched with the sequence of each node.

[0102] In this embodiment, for each unit mutual access rule, the server node adds the number of successful matches between the unit mutual access rule and each node sequence to obtain the updated number of transactions corresponding to the unit mutual access rule. Once all node sequences have been matched with the unit mutual access rule, the server node writes the number of transactions corresponding to the unit mutual access rule into the unit mutual access rule table.

[0103] In one embodiment, if the inter-unit access rule of access unit 1 to shard unit 1 is successfully matched with the sequence of each node 10 times, that is, 10 transactions occur, the server node adds 10 to the number of transactions corresponding to the inter-unit access rule to obtain the updated number of transactions corresponding to the inter-unit access rule.

[0104] Step 405: Determine the view display information based on the unitized monitoring information and the number of transactions corresponding to the mutual access rules of each unit.

[0105] In this embodiment, the server node displays the unitized monitoring information and the number of transactions corresponding to the mutual access rules of each unit as view information.

[0106] In the aforementioned traffic observation method, for each transaction in the unitized monitoring information, the correspondence between the time corresponding to each node in the transaction and the unit type corresponding to each node is stored in a temporary cache; the correspondence is sorted according to the chronological order, and the node sequence of each node in the transaction is determined based on the sorted correspondence; each node sequence is matched with a preset unit mutual access rule; for each unit mutual access rule, the number of transactions corresponding to the unit mutual access rule is updated based on the number of times the unit mutual access rule matches each node sequence; and the view display information is determined based on the unitized monitoring information and the number of transactions corresponding to each unit mutual access rule. In this way, when the traffic observation view category is a cross-unit traffic monitoring view, the number of transactions corresponding to each unit's mutual access rule is determined by matching the preset unit mutual access rules with the sequence of each node. Based on the unitized monitoring information and the number of transactions corresponding to each unit's mutual access rule, the view display information is determined. This makes the view display information used to construct the cross-unit traffic monitoring view also include the number of transactions corresponding to each unit's mutual access rule. The operating indicators of the cross-unit traffic monitoring view are richer, improving the accuracy of cross-unit traffic observation of data from the unitized system.

[0107] In one embodiment, such as Figure 5 As shown, when the traffic observation view category is a unit link view, the specific process of processing the unitized monitoring information to obtain the view display information according to the preset traffic observation view category and the corresponding data processing rules includes the following steps:

[0108] Step 501: For each link in the unitized monitoring information, determine the nodes that the link passes through.

[0109] In this embodiment of the application, for each link in the unitized monitoring information, the server node determines each node that the link passes through.

[0110] Step 502: For each node, bind the node's unit type with the node address and the application name of the applications included in the node, and build and store the mapping relationship between the unit type, node address, and application name.

[0111] In this embodiment, for each node, the server node binds the node's unit type to its node address, establishing a mapping relationship between unit type and node address (referred to as the first mapping relationship for ease of distinction). Simultaneously, for each node, the server node binds the node's unit type to the application names of all applications included in that node, establishing a mapping relationship between unit type and application name (referred to as the second mapping relationship for ease of distinction). The server node then stores the first and second mapping relationships. For example, the server node stores the first and second mapping relationships in a database.

[0112] Step 503: Determine the view display information based on the unitized monitoring information and the mapping relationship.

[0113] In this embodiment, the server node displays the unitized monitoring information and the mapping relationship as view information.

[0114] In the aforementioned traffic observation method, for each link in the unitized monitoring information, the nodes traversed by that link are identified. For each node, the unit type of that node is bound to its address and the application name of the applications included in that node, thus constructing and storing the mapping relationship between unit type, node address, and application name. Based on the unitized monitoring information and this mapping relationship, the view display information is determined. Thus, when the traffic observation view category is a unit link view, by establishing the mapping relationship between the unit type and node address and the mapping relationship between the unit type and application name for each node, and determining the view display information based on the unitized monitoring information and these mapping relationships, the view display information used to construct the unit link view also includes the mapping relationship between unit type and node address and the mapping relationship between unit type and application name. This enriches the operational metrics of the unit link view and improves the accuracy of unit link traffic observation of data from the unitized system.

[0115] In one embodiment, such as Figure 6 As shown, the specific process of constructing a unitized traffic observation view, based on the view display information, the pre-determined unitized dimensions, and the preset unitized traffic observation view construction rules, includes the following steps:

[0116] Step 601: For each predetermined unitized dimension, aggregate the view display information according to that unitized dimension to obtain the view display information corresponding to each unitized object under that unitized dimension.

[0117] In this embodiment, the server node can predetermine the unitization dimension. Then, for each predetermined unitization dimension, the server node aggregates the view display information according to that unitization dimension to obtain the view display information corresponding to each unitized object under that unitization dimension.

[0118] In one embodiment, when the traffic observation view category is a cross-cell traffic monitoring view, the cell dimension also includes a cell inter-access dimension. The cell objects under the cell inter-access dimension are normal access scenarios and prohibited access scenarios.

[0119] Step 602: Construct a unitized traffic observation view based on the view display information corresponding to each unitized object under each unitized dimension.

[0120] In this embodiment, the server node predetermines traffic observation metrics. These metrics are those displayed in the unitized traffic observation view. For example, traffic observation metrics include transaction volume, transaction duration, and anomaly type. Then, for each traffic observation metric, the server node calculates the result of that metric for each unitized object under that unitized dimension, based on the view display information corresponding to each unitized object within that unitized dimension and the calculation rules corresponding to that metric. Next, the server node constructs a sub-unitized traffic observation view corresponding to that unitized dimension based on the view display information corresponding to each unitized object within that unitized dimension and the calculation results of each traffic observation metric for each unitized object within that unitized dimension. Finally, the server node combines the sub-unitized traffic observation views corresponding to each unitized dimension to form the unitized traffic observation view.

[0121] In the aforementioned traffic observation method, for each pre-determined unitized dimension, view display information is aggregated according to that unitized dimension to obtain the view display information corresponding to each unitized object under that unitized dimension; based on the view display information corresponding to each unitized object under each unitized dimension, a unitized traffic observation view is constructed. Thus, by aggregating view display information according to preset unitized dimensions and constructing a unitized traffic observation view, it is possible to perform intra-unit traffic observation, cross-unit traffic observation that can display cross-unit traffic distribution, and unit link traffic observation that can display details of traffic scheduling between units in a unitized system.

[0122] In one embodiment, such as Figure 7 As shown, the specific process of obtaining stored unitized monitoring information based on preset data acquisition rules includes the following steps:

[0123] Step 701: For each preset traffic observation view category, determine the table name of the data table corresponding to that traffic observation view category.

[0124] In this embodiment, the server node pre-sets a correspondence between traffic observation view categories and table names. Then, for each pre-set traffic observation view category, the server node determines the table name of the corresponding data table based on the correspondence between the traffic observation view category and the table name.

[0125] Step 702: Based on the table name, determine the view variables in the data table, and retrieve the corresponding variable values ​​from the data table in the data source to obtain the unitized monitoring information.

[0126] In this embodiment, the server node pre-sets the correspondence between table names and view variables in the data table. Then, the server node determines the view variables in the data table based on the correspondence and the table name. Next, the server node retrieves the corresponding variable values ​​from the data table in the data source to obtain unitized monitoring information. Here, the view variables are the variables required for the unitized traffic observation view. The variable values ​​of the view variables are used to characterize the unitized monitoring information. View variables include, but are not limited to, unit type, access type, and link identifier. Access type includes whether access is normal and access method. Whether access is normal includes normal access and abnormal access. For example, the access method could be access unit accessing fragment unit 1.

[0127] In the aforementioned traffic observation method, for each preset traffic observation view category, the table name corresponding to that view category is determined. Based on the table name, the view variables in that table are determined, and the corresponding variable values ​​are retrieved from the data table in the data source to obtain unitized monitoring information. Thus, by determining the table name and view variables corresponding to the traffic observation view category, and retrieving the corresponding variable values, unitized monitoring information is obtained, enabling the acquisition of stored unitized monitoring information and achieving traffic observation of the unitized system's data. Furthermore, this method allows for the dynamic selection of different values ​​in subsequent unitized traffic observation view construction, improving the flexibility and accuracy of traffic observation of the unitized system's data.

[0128] In one embodiment, such as Figure 8 As shown, the flow observation method also includes the following steps:

[0129] Step 801: In response to the data source configuration command, configure the data source corresponding to each unitized indicator type.

[0130] In this embodiment, the data source configuration instruction is used to instruct the server node to configure the data source. The data source configuration instruction may include a unitized metric type and a data source. The server node pre-defines the unitized metric type. Then, the server node parses the data source configuration instruction to obtain the unitized metric type and data source. Next, the server node establishes a correspondence between the unitized metric type and the data source. Then, based on the correspondence between the unitized metric type and the data source, the server node configures the data source corresponding to each unitized metric type. Specifically, in response to the data source configuration instruction, the server node configures the data source corresponding to each unitized metric type through a configuration interface. For example, the configuration interface is the interface provided by the Grafana open-source software. In this way, the configuration interface used supports configuration connections to various common data sources, ensuring the data source configuration for traffic observation.

[0131] In one example, the server node responds to the data source configuration command by configuring the storage space corresponding to each pre-defined unitized indicator type as the data source corresponding to each unitized indicator type.

[0132] In one embodiment, the server node responds to the data source configuration command by configuring the open-source data source ClickHouse as the data source corresponding to the metric class. Simultaneously, the server node configures the open-source data source Elasticsearch as the data source corresponding to the link class.

[0133] Step 802: In response to the variable definition instruction, define each view variable according to the preset view variables and the definition rules corresponding to the view variables.

[0134] In this embodiment, a variable definition instruction is used to instruct a server node to define view variables. The variable definition instruction may include view variables and definition rules. The definition rules corresponding to view variables include, but are not limited to, retrieving the value of the view variable from the table corresponding to the view variable, using a specified fixed value as the value of the view variable, and using a manually entered value as the value of the view variable. The server node parses the variable definition instruction to obtain the view variables and definition rules. Then, the server node establishes a correspondence between view variables and definition rules. Then, based on the correspondence between view variables and definition rules, the server node determines the definition rules corresponding to each view variable. Finally, the server node defines each view variable according to the preset view variables and their corresponding definition rules.

[0135] In one example, the server node can obtain the values ​​of each view variable in real time or periodically to complete the definition of the view variables.

[0136] In one embodiment, the view variable is a physical unit phy_zone. In response to the variable definition instruction, the server node retrieves the value of the view variable from the indicator table corresponding to the view variable, thus completing the definition of the view variable.

[0137] In one embodiment, the view variable is a physical unit `phy_zone`. In response to a variable definition instruction, the server node specifies a portion of the view variable's values ​​as fixed values. Then, the server node uses these specified fixed values ​​as part of the view variable's values ​​and retrieves the other portion of the view variable's values ​​from the corresponding metric table, thus completing the view variable's definition. The division between the two portions of values ​​depends on the specific view variable being defined.

[0138] In one embodiment, the view variable is the trace identifier (traceId). In response to the variable definition instruction, the server node retrieves the value of the view variable from the trace table corresponding to the view variable, thus completing the definition of the view variable.

[0139] In one embodiment, the view variable is the traceId, and the server node responds to the variable definition instruction by using the manually entered value as the value of the view variable, thus completing the definition of the view variable.

[0140] Step 803: In response to the data table setting instruction, determine the table name of the data table corresponding to each flow observation view category, and the correspondence between the table name and the view variable.

[0141] In this embodiment, the data table setting instruction is used to instruct the server node to determine the table name of the data table and the correspondence between the table name and view variables. The data table setting instruction may include the traffic observation view category, the table name, and the view variables. The server node parses the data table setting instruction to obtain the traffic observation view category, the table name, and the view variables. Then, the server node establishes the correspondence between the traffic observation view category and the table name, and the correspondence between the table name and the view variables. Finally, based on the correspondence between the traffic observation view category and the table name, the server node determines the table name of the data table corresponding to each traffic observation view category.

[0142] In the aforementioned traffic observation method, in response to data source configuration commands, the data source corresponding to each unitized indicator type is configured; in response to variable definition commands, each view variable is defined according to preset view variables and their corresponding definition rules; and in response to data table setting commands, the table name of the data table corresponding to each traffic observation view category, as well as the correspondence between the table name and the view variable, are determined. Thus, by configuring the data source, defining view variables, and setting the data table, the stored unitized monitoring information is retrieved, enabling traffic observation of the unitized system's data. Furthermore, this method allows for the dynamic selection of different values ​​in subsequent unitized traffic observation view construction, improving the flexibility and accuracy of traffic observation of the unitized system's data.

[0143] In one embodiment, the unitization dimensions include: unit dimension, application dimension, group dimension, service dimension, fault domain dimension, host IP dimension, and container ID dimension.

[0144] In this embodiment, the unitization dimensions, ordered from largest to smallest after division, include, but are not limited to: unit dimension, application dimension, group dimension, service / resource dimension, fault domain dimension, host IP dimension, and container ID dimension. The unit dimension is the dimension that divides the unitized system according to units. The application dimension, group dimension, service / resource dimension, fault domain dimension, host IP dimension, and container ID dimension are similar to the unit dimension.

[0145] In the aforementioned traffic observation method, the unitized dimensions include: unit dimension, application dimension, group dimension, service dimension, fault domain dimension, host IP dimension, and container ID dimension. This layered drill-down approach to unitized dimensions further narrows the scope of traffic observation, improves the accuracy of determining the impact range of faults, and ultimately enhances the accuracy of traffic observation for data from unitized systems.

[0146] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0147] Based on the same inventive concept, this application also provides a flow observation device for implementing the flow observation method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more flow observation device embodiments provided below can be found in the limitations of the flow observation method described above, and will not be repeated here.

[0148] In one embodiment, such as Figure 9 As shown, a flow observation device 900 is provided, including: an acquisition module 910, a data processing module 920, and a construction module 930, wherein:

[0149] The acquisition module 910 is used to acquire stored unitized monitoring information based on preset data acquisition rules; the unitized monitoring information includes unitized monitoring data and type identifiers of the unitized indicator types; the unitized monitoring data includes unitized metadata and monitoring data corresponding to the unitized metadata;

[0150] Data processing module 920 is used to process the unitized monitoring information according to the preset traffic observation view category and the data processing rules corresponding to the traffic observation view category to obtain view display information;

[0151] The construction module 930 is used to construct a unitized traffic observation view based on the view display information, the pre-determined unitized dimensions, and the preset unitized traffic observation view construction rules; the unitized traffic observation view is used to perform traffic observation on the data of the unitized system.

[0152] Optionally, when the traffic observation view category is a cross-cell traffic monitoring view, the data processing module 920 is specifically used for:

[0153] For each transaction in the unitized monitoring information, the correspondence between the time corresponding to each node in the transaction information and the unit type corresponding to each node in the transaction information is stored in a temporary cache.

[0154] The correspondences are sorted according to the chronological order, and the node sequence of each node included in the transaction information is determined based on the sorted correspondences.

[0155] Match each node sequence with the preset cell access rules;

[0156] For each unit mutual access rule, the number of transactions corresponding to the unit mutual access rule is updated according to the number of times the unit mutual access rule is successfully matched with the sequence of each node;

[0157] The view display information is determined based on the unitized monitoring information and the number of transactions corresponding to the mutual access rules of each unit.

[0158] Optionally, when the traffic observation view category is a unit link view, the data processing module 920 is specifically used for:

[0159] For each link in the unitized monitoring information, determine the nodes that the link passes through;

[0160] For each node, the unit type of the node is bound to the node address and the application name of the application included in the node, and the mapping relationship between the unit type, node address and application name is constructed and stored;

[0161] Based on the unitized monitoring information and the mapping relationship, the view display information is determined.

[0162] Optionally, the building module 930 is specifically used for:

[0163] For each predetermined unitized dimension, the view display information is aggregated according to the unitized dimension to obtain the view display information corresponding to each unitized object under the unitized dimension;

[0164] Based on the view display information corresponding to each unitized object under each of the aforementioned unitized dimensions, a unitized traffic observation view is constructed.

[0165] Optionally, the acquisition module 910 is specifically used for:

[0166] For each preset traffic observation view category, determine the table name of the data table corresponding to the traffic observation view category;

[0167] Based on the table name of the data table, determine the view variables in the data table, and obtain the variable values ​​corresponding to the view variables in the data table in the data source to obtain unitized monitoring information.

[0168] Optionally, the device 900 further includes:

[0169] The configuration module is used to configure the data source corresponding to each of the unitized indicator types in response to the data source configuration command;

[0170] The definition module is used to define each view variable in response to the variable definition instruction, according to the preset view variables and the definition rules corresponding to the view variables;

[0171] The configuration module is used to respond to the data table configuration command to determine the table name of the data table corresponding to each of the traffic observation view categories, as well as the correspondence between the table name and the view variables.

[0172] Each module in the aforementioned flow monitoring device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.

[0173] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 10As shown. The computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a flow observation method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0174] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0175] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0176] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0177] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0178] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0179] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0180] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0181] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A flow rate observation method, characterized in that, The method includes: Based on preset data acquisition rules, the stored unitized monitoring information is acquired; the unitized monitoring information includes unitized monitoring data and type identifiers of the unitized indicator types; the unitized monitoring data includes unitized metadata and monitoring data corresponding to the unitized metadata; Based on the preset traffic observation view categories and the data processing rules corresponding to the traffic observation view categories, the unitized monitoring information is processed to obtain view display information; Based on the view display information, the pre-determined unitized dimensions, and the preset unitized traffic observation view construction rules, a unitized traffic observation view is constructed; the unitized traffic observation view is used to perform traffic observation on the data of the unitized system. Specifically, when the traffic observation view category is a cross-unit traffic monitoring view, the step of processing the unitized monitoring information according to the preset traffic observation view category and the data processing rules corresponding to the traffic observation view category to obtain the view display information includes: For each transaction in the unitized monitoring information, the correspondence between the time corresponding to each node in the transaction information and the unit type corresponding to each node in the transaction information is stored in a temporary cache; the correspondence is sorted according to the chronological order, and the node sequence of each node in the transaction information is determined based on the sorted correspondence; each node sequence is matched with a preset unit mutual access rule; for each unit mutual access rule, the number of transactions corresponding to the unit mutual access rule is updated according to the number of times the unit mutual access rule matches the node sequence; the view display information is determined based on the unitized monitoring information and the number of transactions corresponding to each unit mutual access rule. In the case where the traffic observation view category is a unit link view, the step of processing the unitized monitoring information according to the preset traffic observation view category and the data processing rules corresponding to the traffic observation view category to obtain the view display information includes: For each link in the unitized monitoring information, determine the nodes that the link passes through; for each node, bind the node's unit type with the node address and the application name of the application included in the node, and construct and store the mapping relationship between the unit type, node address, and application name; determine the view display information based on the unitized monitoring information and the mapping relationship.

2. The method according to claim 1, characterized in that, The step of constructing a unitized traffic observation view based on the view display information, the pre-determined unitized dimensions, and the preset unitized traffic observation view construction rules includes: For each predetermined unitized dimension, the view display information is aggregated according to the unitized dimension to obtain the view display information corresponding to each unitized object under the unitized dimension; Based on the view display information corresponding to each unitized object under each of the aforementioned unitized dimensions, a unitized traffic observation view is constructed.

3. The method according to claim 1, characterized in that, The acquisition of stored unitized monitoring information based on preset data acquisition rules includes: For each preset traffic observation view category, determine the table name of the data table corresponding to the traffic observation view category; Based on the table name of the data table, determine the view variables in the data table, and obtain the variable values ​​corresponding to the view variables in the data table in the data source to obtain unitized monitoring information.

4. The method according to claim 3, characterized in that, The method further includes: In response to the data source configuration command, configure the data source corresponding to each of the unitized indicator types; In response to the variable definition instruction, each view variable is defined according to the preset view variable and the definition rules corresponding to the view variable; In response to the data table setting command, determine the table name of the data table corresponding to each of the traffic observation view categories, as well as the correspondence between the table name and the view variables.

5. A flow rate monitoring device, characterized in that, The device includes: The acquisition module is used to acquire stored unitized monitoring information based on preset data acquisition rules; the unitized monitoring information includes unitized monitoring data and type identifiers of the unitized indicator types; the unitized monitoring data includes unitized metadata and monitoring data corresponding to the unitized metadata; The data processing module is used to process the unitized monitoring information according to the preset traffic observation view category and the data processing rules corresponding to the traffic observation view category to obtain view display information. The construction module is used to construct a unitized traffic observation view based on the view display information, the pre-determined unitized dimensions, and the preset unitized traffic observation view construction rules; the unitized traffic observation view is used to perform traffic observation on the data of the unitized system. Specifically, when the traffic observation view category is a cross-cell traffic monitoring view, the data processing module is used for: For each transaction in the unitized monitoring information, the correspondence between the time corresponding to each node in the transaction information and the unit type corresponding to each node in the transaction information is stored in a temporary cache; the correspondence is sorted according to the chronological order, and the node sequence of each node in the transaction information is determined based on the sorted correspondence; each node sequence is matched with a preset unit mutual access rule; for each unit mutual access rule, the number of transactions corresponding to the unit mutual access rule is updated according to the number of times the unit mutual access rule matches the node sequence; the view display information is determined based on the unitized monitoring information and the number of transactions corresponding to each unit mutual access rule. Specifically, when the traffic observation view category is a unit link view, the data processing module is used for: For each link in the unitized monitoring information, determine the nodes that the link passes through; for each node, bind the node's unit type with the node address and the application name of the application included in the node, and construct and store the mapping relationship between the unit type, node address, and application name; determine the view display information based on the unitized monitoring information and the mapping relationship.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

Images