Query method and system for fault location in industrial big data systems

Abstract

The present invention discloses a query method and system for fault location in industrial big data systems. Users can interactively query and explore the large-scale multi-attribute hierarchical data formed by fault propagation in industrial big data systems based on topological regular expressions, and then locate the faults. Based on regular expression queries, the system introduces data flow models and a variety of different tree visualization forms to construct a complete exploratory analysis system, allowing users to effectively explore large-scale multi-attribute hierarchical data according to their needs.

Description

Technical Field

[0001] The present invention relates to the technical field of visualization and human-computer interaction, and in particular to a query method and system for fault location in an industrial big data system. Background Art

[0002] In large-scale industrial big data systems, failures are unexpected, unplanned events that can be caused by a variety of factors, such as power failures, hardware failures, configuration issues, and code errors. Each failure typically involves multiple services in the system, and the failure is typically reported to the service where the failure occurred. The team responsible for that service analyzes the failure. If that service is not the root cause, the failure is redirected to the team responsible for another service until the root cause is found. Research results show that assigning failures to the wrong service team during root cause tracing is extremely time-consuming.

[0003] Quickly and accurately locating the root cause of a fault is a very challenging task. Large industrial big data systems contain a vast number of services, and the dependencies between them are complex. Many dynamic dependencies are implicit to engineers. Services deployed on the same node may also affect each other. The direct cause of a fault and the root cause of the fault are often different. During the root cause tracing process, the engineer responsible for a service often has only a partial understanding of the fault. They are only familiar with their own service and its closely related services. As a result, the fault may be passed from one service team to another, resulting in a long root cause tracing chain, which greatly prolongs the time required for fault analysis and resolution.

[0004] Since faults propagate along the dependencies between services, a hierarchical tree is formed. Analysis of existing fault propagation data reveals that faults caused by different events on the same service often have similar vulnerability propagation paths between dependent services. Therefore, in the process of fault location in industrial big data systems, users can combine the relevant topological structure information and node attribute information of the current fault propagation to query in previously recorded fault propagation data, thereby helping engineers find the root cause of the vulnerability. Therefore, supporting users to efficiently query multi-attribute hierarchical data is an important aspect of fault location in industrial big data systems. Currently, methods for querying multi-attribute hierarchical data include interactive filtering and code writing. However, the process of interactively filtering and visualizing results is cumbersome and time-consuming. Users need to understand the entire hierarchical data and its corresponding visual results, making it impossible to achieve accurate multi-attribute hierarchical data queries. Furthermore, the interactive query process is difficult to reuse for other multi-attribute hierarchical data, which reduces the efficiency of fault location. Although the code-based query method is sufficiently expressive and accurate, it is difficult for ordinary users to learn and has a heavy programming burden, making it unsuitable for application scenarios in industrial big data system fault location. Currently, no method exists to support users in accurately and efficiently querying multi-attribute hierarchical data.

[0005] Based on the above-mentioned technical problems existing in the prior art, the present invention provides a visual query method and system for fault location in industrial big data systems. Summary of the Invention

[0006] The present invention proposes a query method and system for fault location in an industrial big data system.

[0007] The present invention adopts the following technical solutions:

[0008] In one aspect, the present invention provides a visual query method for fault location in an industrial big data system, comprising:

[0009] Step 1: The user uploads multi-attribute hierarchical structure data in JSON format to the system;

[0010] Step 2: Users select different tree visualization formats to display the uploaded multi-attribute hierarchical data;

[0011] Step 3: Users interactively construct a multi-attribute hierarchical data query expression based on topological regular expressions;

[0012] Step 4: The system parses the query expression and uses a multi-attribute automaton algorithm to determine matching results for the user to locate the root cause of the fault;

[0013] Step 5: The system displays the query results, and the user interactively explores the multi-attribute distribution of nodes;

[0014] Step 6: The user iteratively creates multi-attribute hierarchical data query expressions to build a data streaming visual analysis process.

[0015] Furthermore, in step 1, the system parses the multi-attribute hierarchical structure data, integrates all attribute values ​​in the data, and filters to obtain all numerical, categorical, and Boolean node attribute information.

[0016] Furthermore, in step 3, the user interactively constructs a multi-attribute hierarchical structure data query expression based on topological regular expressions, including: constructing a node expression, constructing a path expression, and constructing a subtree expression.

[0017] Furthermore, in step 4, the multi-attribute automaton algorithm inputs the multi-attribute hierarchical data to be queried and the list obtained by parsing the query expression, and the multi-attribute automaton algorithm outputs partial hierarchical data that matches the expression. The output result of the multi-attribute automaton algorithm includes multiple multi-attribute hierarchical data.

[0018] Furthermore, the multi-attribute automaton algorithm traverses each node of the input hierarchical data, matches the query expression based on each node in the hierarchical data using the multi-attribute automaton algorithm, and adopts a lazy matching strategy to judge the state transition condition. The matching process ends when there are no more nodes to match if and only if the last element of the expression contains a greedy quantifier (*). The query process is performed in units of multi-attribute nodes, and each state transition process is a process of matching the data according to the attribute restrictions within the node;

[0019] The node expression will match each data point in the hierarchical data. The matching of path expression and subtree expression both starts from a node and matches the subsequent path. The difference is that there can be multiple paths under the same node that meet the path expression, while there can be at most one subtree that meets the requirements of the subtree expression. The state of the automaton is determined by the sequence number of the expression list element and the number of times the element needs to be matched repeatedly.

[0020] The time complexity of the query process is O(n), O(n×m×l), and O(n×m×l), corresponding to the query of the three target nodes, paths, and subtrees respectively. n represents the number of nodes in the entire hierarchical data tree, m represents the longest path in the tree, and l represents the number of leaf nodes.

[0021] Furthermore, in step 5, the system distinguishes the nodes of the overall data, input data, and output data.

[0022] Furthermore, in step 5, the system displays the query results along with the context information of the query results.

[0023] Furthermore, in step 6, in large-scale industrial big data systems, the number of services is huge and the dependencies between services are complex, resulting in the hierarchical data formed by fault propagation being usually also huge. Therefore, fault location is difficult to achieve through only one query, but is an ongoing process. The system supports users to build data analysis flows to achieve the progressive analysis process.

[0024] In another aspect, the present invention provides a system for implementing a visual query method, comprising:

[0025] The display module is used to display the uploaded multi-attribute hierarchical data in a tree visualization format;

[0026] Matching module, used to parse the query expression and determine the matching result using a multi-attribute automaton algorithm;

[0027] Expression building module, used to interactively build query expressions for multi-attribute hierarchical data;

[0028] The exploration flow construction module is used to interactively build multi-attribute hierarchical data exploration and analysis flows based on data flow models.

[0029] Furthermore, the system also includes a parsing and screening module for parsing multi-attribute hierarchical structure data, integrating all attribute values ​​in the data, and screening to obtain all numerical, categorical, and Boolean node attribute information.

[0030] Compared with the prior art, the present invention has the following advantages:

[0031] The visual query method for fault location in industrial big data systems described in the present invention allows users to interactively query and explore large-scale multi-attribute hierarchical data formed by fault propagation in industrial big data systems based on topological regular expressions, and then locate the faults; based on regular expression queries, the system introduces data flow models and a variety of different tree visualization forms to construct a complete exploratory analysis system, which supports users to effectively explore large-scale multi-attribute hierarchical data according to their needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A schematic diagram of a user's operation process in an embodiment of the present invention;

[0033] Figure 2 A relationship diagram of different query targets in hierarchical data in an embodiment of the present invention;

[0034] Figure 3 This is a deconstruction diagram of the branch expression query result in an embodiment of the present invention;

[0035] Figure 4 Schematic diagram of query results of path and subtree expressions in an embodiment of the present invention;

[0036] Figure 5 A graphical interface constructed for query expressions in an embodiment of the present invention;

[0037] Figure 6 A schematic diagram of exploring data flow in an embodiment of the present invention;

[0038] Figure 7 Schematic diagram of seven different tree visualization forms in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below with reference to the accompanying drawings and specific implementation methods. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other unless there is a conflict.

[0040] Example

[0041] The visual query method for fault location in industrial big data systems includes:

[0042] S1, users upload multi-attribute hierarchical data in JSON format;

[0043] S2, the system uses seven different tree visualization forms to display the uploaded hierarchical data;

[0044] S3, users interactively construct multi-attribute hierarchical data query expressions based on topological regular expressions;

[0045] S4, the system parses the query expression and uses a multi-attribute automaton algorithm to determine the matching results;

[0046] S5, the system displays the query results and supports users to interactively explore the multi-attribute distribution of nodes;

[0047] S6, users iteratively create multi-attribute hierarchical data query expressions to build a data streaming visual analysis process.

[0048] The user's operating process of using the system is as follows Figure 1 As shown, first upload hierarchical data in JSON format and interactively build query expressions or explore the analysis process. The system will query the input data and visualize the query results to the user. The user can interactively analyze the visualized results and draw corresponding conclusions.

[0049] In S1 of the above embodiment, the user uploads multi-attribute hierarchical data in JSON format, and the system automatically parses the data, integrates all attribute values ​​in the data, and filters to obtain all numeric, categorical, and Boolean node attribute information;

[0050] In S2 of the above embodiment, as Figure 7 As shown, the system supports seven different tree visualization forms to display multi-attribute hierarchical data uploaded by users, including two node-link tree visualizations with different layouts, two radial tree visualization forms, icicle diagram visualization form, indented tree visualization form, and indented pixel tree diagram visualization form. Different tree visualization forms focus on displaying different hierarchical data features. Users can choose different tree visualization forms to explore data according to the analysis tasks and the characteristics of hierarchical data.

[0051] The system visualization module displays the hierarchical data obtained by the calculation module query according to the tree visualization form selected by the user. The visualization part of the system is completed based on the visualization toolkit D3. The tree visualization form drawing results can be automatically adjusted according to the window size. After the tree visualization form is displayed on the computer screen, the user can interact with the tree visualization form to view the attribute value of each node separately or interactively select a part of the data to view its attribute value distribution, thereby exploring the hierarchical data. Based on the results of the tree visualization form and analysis requirements, the user can choose to adjust the tree visualization form to analyze the data from different angles;

[0052] In S3 of the above embodiment, if Figure 2 As shown in the figure, users interactively construct hierarchical data query expressions based on topological regular expressions. The construction targets are divided into three categories: nodes, paths, and subtrees:

[0053] A node is the basic unit in hierarchical data, and a node expression represents a node pattern in hierarchical data. Given the multi-attribute nature of hierarchical data, users can use operators to define constraints on attribute values ​​within node expressions, including ">,"≥,"<,"≤," and "=". Attribute values ​​can be inherent attributes related to the topology of hierarchical data, such as height, width, and depth, or unique attributes within different data types, such as the gender and age of nodes in a company's organizational structure. Attribute values ​​in node expressions should be numbers, strings, or Boolean values, and different attribute value constraints are in an AND relationship. Node expressions can not only independently describe nodes that meet user requirements, but can also appear as part of a path or subtree expression. For user convenience, three common node expression wildcards are predefined: root node (^), and leaf node ($). Node expressions are defined as follows: Node represents a node expression, and CustomNode represents a user-defined node expression containing multiple attribute constraints:

[0054] Node:=CustomNode|.|^|$,

[0055] Path expressions consist of node expressions and operators, and are used to describe sequences in hierarchical data. Because paths have a linear structure, path expressions are constructed using operators from traditional regular expressions, including or (|), not (!), and the repetition operator (0..*). The repetition operator allows users to specify an exact number or range of repetitions. For example, 2..5 indicates a repetition count between 2 and 5, and 2..* indicates a repetition count greater than 2. Path queries are performed using a lazy matching strategy. Once a path matches the query expression, it becomes the query result, and the matching process ends. However, if no node expressions appear after the greedy quantifier (*), the matching process ends when no nodes can be matched. Furthermore, because users query paths from hierarchical data, the same node can appear in multiple path query results. For example, the expression Node[name="A"](·)3..* is used to query the evolutionary path of a system fault. This fault propagates from service A through three or more generations, with the following characteristics: Figure 4 The three matching paths are shown in the query path.

[0056] The definition of path expression is as follows, where Path represents the path expression:

[0057] Path:=(Node|!Node) 0..* Path,

[0058] The most significant difference between paths and subtrees in terms of topological structure is that a node in a subtree may have multiple successor nodes, while each node in a path has at most one successor node. Since there is no specific order between the successor nodes of each node, the subtree expression needs to describe the composition of the successor nodes and the order of the nodes along the path. In order to meet the above requirements, a branch operator (Branch) is defined to represent the composition of the subsequent path of a node. Users can define the set of subsequent paths of a node as a branch expression, such as Figure 3 As shown, the tree obtained by visualizing hierarchical data can be decomposed into a node and a branch. A branch consists of several paths under the same node. Each path in the branch expression has a repetition operator (0..*) to limit the number of times the path appears. The definition of the branch expression is as follows, where Branch represents the branch expression:

[0059] Branch:=[ <path1> 0..* ,..., <Path n > 0..* ],

[0060] Each branch follows a root node. Path expressions in branch expressions are matched independently, so the matching results of different path expressions in the same branch expression may overlap. Unlike traditional regular expressions, there is no precedence order between path expressions in branch expressions. This is because the order between sibling nodes in hierarchical data is arbitrary, and the number of times a path appears can be specified in a branch expression. A subtree expression is represented by a sequence consisting of a root node expression and branch expressions, where Branches represents a branch expression sequence, EPT represents an empty string, and Subtree represents a subtree.

[0061] Branches:=(Branch Branches)|EPT,

[0062] Subtree: = Node Branches,

[0063] Among them, Branches represents the sequence of branch expressions, and EPT represents empty. When two branch expressions are connected, the leaf node of the query result of the first branch expression will be used as the root node of the second branch expression. If the subsequent path set of a leaf node in the matching result of the first branch expression matches the second branch expression, the path where the leaf is located will be added to the candidate result of the first branch expression. Since the number of repetitions of the path is limited in the branch expression, the matching result of the entire branch expression can only be determined after the matching of the second branch expression is completed. For example, Node[name='B']{<(·)>2..*}{<(·)2..*)>2..*} means querying the subtree formed by the propagation of system faults starting from service B, and service B has more than two directly related services, and each service also has more than two directly related services. The query result is as follows: Figure 4 As shown in the query subtree, only after completing the query of the second branch expression can we determine whether service B has two directly related services that meet the requirements.

[0064] It is worth noting that some query tasks only restrict the target composition and ignore the relative positions of the elements within the target. Therefore, the query syntax defines the element composition expression to describe the composition of the target, including all and existence Two operators, all of which Only node expressions can appear after the operator, indicating that all targets must meet the requirements of the specific node expression; The node or path operator can appear after the operator, and the number of repetitions can be specified, indicating that a specific node or path must appear in the target, and the number of times must also meet the requirements. The definition of the element composition expression is as follows, where EC stands for element composition expression, and the subscripts "a" and "e" indicate that all the nodes or paths appear in the expression. and existence operator, EPT stands for empty:

[0065]

[0066]

[0067] EC:=(EC a |EC e )EC,

[0068] Summarizing the above syntax definitions, we can get the query target of this multi-attribute hierarchical data query syntax, where Target represents the query target:

[0069] Target:=(Subtree|Path|Node)~EC.

[0070] User in Figure 5 Query expressions for multi-attribute hierarchical data are interactively constructed in the visual editor shown. The symbol panel (a) provides all operators included in the query syntax specification described in s3; the expression panel (b) supports the interactive construction and visual representation of query expressions; the node list panel (c) displays all user-defined node expressions; the node definition panel (d) allows users to set the name, color and properties of the node expression.

[0071] The symbol panel contains all the symbols needed to build query expressions. Each symbol is a button, which represents the following meanings: add an empty rectangle after the currently selected element; set the number of repetitions of the selected element; add a new rectangle and have an OR relationship with the selected element; add a NOT operator to the selected element; add a path in the branch expression; add an exist expression Add an "all" expression Users can click the corresponding symbol button to add it to the expression panel to build a query expression.

[0072] In the Expression panel, query expressions are drawn on the panel in SVG format. Node expressions, as basic elements in query syntax, are represented by a rectangular box. The string inside the rectangular box represents the name of the node expression. Path expressions are composed of several node expressions arranged horizontally on the panel, consistent with the logic of traditional regular expressions. Branch expressions are represented by a square bracket and the elements within it. To more intuitively represent the parallel relationship between the elements within the branch expression and distinguish it from the logic of traditional regular expressions, the elements within the brackets of the branch expression are arranged vertically. The expression structure and the expression components are separated by two vertical lines, and the elements within them are also arranged vertically on the panel. Users can click the element they want to select (rectangular box or bracket) in the Expression panel. After the element is selected, its border turns red. At this time, users can operate on the selected panel by clicking the operator in the Symbol panel. Users can also click the repetition count in the upper right corner of the element to modify it.

[0073] The expression panel contains three buttons in the upper right corner. The Delete button deletes the selected element; the Reset button clears the constructed expression in the expression panel; and the Submit button submits the constructed expression to the calculation module for query. Users can also double-click the title at the top of the editor to name the constructed expression, which will help them recall its meaning. In the Node Definition panel, users can customize the attribute constraints of the selected node expression. The system automatically processes the node attributes of the input hierarchical data, filtering out all numeric, character, and Boolean attributes and importing them into the "Attribute" drop-down box. Users can directly click the desired attribute or enter it manually. The system will automatically complete the manually entered attributes. The "Operator" selection box allows users to select the predicate symbols they want to use: ">", ">=", "<", "<=", and "=". In the "Value" input box, users can enter their own restrictions on the attribute. The system will automatically convert the entered text to the corresponding attribute type. In addition, in "Name" and "Color", users can set the name and color of the current node expression. After the user clicks the Submit button, the system will automatically update the name and color to the rectangular box corresponding to the current node expression in the Expression Panel, making it easier for users to distinguish different node expressions and understand the meaning of each node expression. At the same time, the submitted node expression is also updated in the Node List Panel for user reuse.

[0074] The Node List panel contains all user-defined node expressions. After selecting a rectangle in the Expression panel, you can quickly define a node expression by clicking on it. Three common node expressions are pre-set in the system: wildcard (*), root node (^), and leaf node ($).

[0075] In S4 of the above embodiment, the system parses the multi-attribute hierarchical data query expression constructed by the user into a list form, and each element in the list is defined as a query unit, and each query unit represents a part of the expression. The query unit can be recursively defined, that is, the elements in the query unit may still be in a list form, such as a branch expression and an element composition expression. The system uses a multi-attribute automaton algorithm to query the input hierarchical data according to the expression. The input of the algorithm is the multi-attribute hierarchical data to be queried and the list obtained by parsing the query expression, and outputs the partial hierarchical data that matches the expression. The output result of the matching algorithm includes multiple multi-attribute hierarchical data;

[0076] The matching algorithm will traverse each node of the input hierarchical data and use a multi-attribute automaton algorithm to match the query expression based on each node in the hierarchical data. The state of the automaton is determined by the sequence number of the expression list element and the number of times the element needs to be repeatedly matched. A lazy matching strategy is used to judge the state transition condition. If and only if the last element of the expression contains a greedy quantifier (*), the matching process will end when there are no more nodes to match. The entire query process is performed in units of multi-attribute nodes, and each state transition process is a process of matching data based on the attribute restrictions within the node.

[0077] In S5 of the above embodiment, after the system completes the query of multi-attribute hierarchical data according to the query expression, the user can select the seven different tree visualization forms described in S2 to display the data. The system supports the user to view the topological structure of the hierarchical data for analysis of the query results separately, and also allows it to be displayed as part of the input data or part of the overall data. In order to distinguish the three types of data: overall data, input data, and output data, the system will use color mapping to map the nodes of different data. Specifically, the node color of the output data is orange, the node color of the input data is blue, and the node color of the overall data is gray. The user can display the query results at different granularities and retain the context information of the query results, so that the user can better understand the data.

[0078] It should be noted that in order to assist users in analyzing node attributes in multi-attribute hierarchical data, the system also supports users to click on nodes in the visualization results to view the attributes within each node. The system implements an attribute distribution view, and users can interactively select the node attributes to be analyzed. The system automatically displays the attribute distribution of the output data and input data obtained from the query in the form of a bar chart; in addition, users can interactively select part of the subtree to analyze the attribute distribution of any part.

[0079] In S6 of the above embodiment, the system supports the user to Figure 6 The interface shown interactively constructs an exploration process for hierarchical data, consisting of three panels: the visualization panel (a), where users can select from a variety of tree visualizations to visualize hierarchical data; the analysis flow panel (b), which allows users to construct multi-attribute hierarchical data exploration and analysis processes based on query expressions; and the attribute distribution panel (c), which displays the attribute distribution of multi-attribute hierarchical data. The system draws on the classic data flow model. Users define query expressions in each module of the data flow according to S3. The system queries based on the module input data, and the query results serve as the input for the next module. Users can connect other data flow modules as needed. When the data or expression in a module is updated, the inputs and outputs of all subsequent modules connected to it are automatically updated. Furthermore, to support expression reuse, the system provides a module copying function, allowing users to copy parts of the data flow. The inputs of the copied sub-data flow are determined by the new connection relationships. After the exploration process is completed, the computation module matches the multi-attribute hierarchical data. As described in S3, the query matching process for each expression is designed as a finite state automaton. After the previous query module in the exploration process is completed, the query of the next module is automatically jumped to, until the entire exploration process is complete.

[0080] The present invention is not limited to the above-described embodiments. The above-described embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims.

Claims

1. A visual query method for fault location in industrial big data hierarchical architecture, characterized by: include: Step 1: The user uploads multi-attribute hierarchical structure data in JSON format to the system; Step 2: The system uses different tree visualization forms to display the uploaded multi-attribute hierarchical structure data; Step 3: User interactively constructs a multi-attribute hierarchical structure data query expression based on topological regular expression: User interactively constructs a multi-attribute hierarchical structure data query expression based on topological regular expression, including: constructing a node expression, constructing a path expression, and constructing a subtree expression; Step 4: The system parses the query expression and uses a multi-attribute automaton algorithm to determine the matching result: the multi-attribute automaton algorithm is used to determine the matching result. The input of the multi-attribute automaton algorithm is the multi-attribute hierarchical data to be queried and the list obtained by parsing the query expression. The multi-attribute automaton algorithm outputs partial hierarchical data that matches the expression. The output result of the multi-attribute automaton algorithm contains multiple multi-attribute hierarchical data. The multi-attribute automaton algorithm traverses each node of the input hierarchical data, uses the multi-attribute automaton algorithm to match the query expression based on each node in the hierarchical data, and uses a lazy matching strategy to judge the state transition condition. The matching process ends when there are no more nodes to match if and only if the last element of the expression contains a greedy quantifier. The query process is performed in units of multi-attribute nodes, and each state transition process is a process of matching the data according to the attribute restrictions within the node. Step 5: The system displays the query results, and the user interactively explores the multi-attribute distribution of nodes; Step 6: The user iteratively creates multi-attribute hierarchical data query expressions to build a data streaming visual analysis process.

2. The visual query method for fault location in industrial big data hierarchical architecture according to claim 1 is characterized in that: In step 1, the system parses the multi-attribute hierarchical data, integrates all attribute values ​​in the data, and filters out all numerical, categorical, and Boolean node attribute information.

3. The visual query method for fault location in industrial big data hierarchical architecture according to claim 1 is characterized in that: In step 5, the system distinguishes the nodes of overall data, input data, and output data.

4. The visual query method for fault location in the industrial big data hierarchical architecture according to claim 1 or 3, characterized in that: In step 5, the system displays the query results along with the context information of the query results.

5. A system for implementing the visual query method according to claim 1, characterized in that: include: The display module is used to display the uploaded multi-attribute hierarchical structure data in a tree visualization form; Matching module, used to parse the query expression and determine the matching result using a multi-attribute automaton algorithm; Expression building module, used to interactively build query expressions for multi-attribute hierarchical data; The exploration flow construction module is used to interactively build multi-attribute hierarchical data exploration and analysis flows based on data flow models.

6. The system according to claim 5, characterized in that The system also includes a parsing and screening module for parsing multi-attribute hierarchical structure data, integrating all attribute values ​​in the data and screening to obtain all numerical, categorical and Boolean node attribute information.

Images