Data processing method and device, equipment, medium and product

By building an error graph and using graph theory algorithm to calculate the shortest path, the problems of low efficiency and poor accuracy of processing process interrupt error analysis in the prior art are solved, and the error source and propagation path are quickly and accurately positioned, improving the user experience.

CN119003221APending Publication Date: 2024-11-22CHINA CONSTRUCTION BANK +1
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411086475.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-08
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

In the prior art, when processing process interrupt errors, the analysis efficiency is low and the accuracy is poor, making it difficult to quickly and accurately locate the error source and error propagation path.

Method used

By constructing an error graph corresponding to the pending log data, and using a pre-trained attribute evaluation model to determine the node attributes and edge attributes in the error graph, the graph theory algorithm is used to calculate the shortest path to locate the target error path that causes the interruption of the processing process.

Benefits of technology

It improves the efficiency and accuracy of interrupt error analysis, simplifies the error repair process, and improves the user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119003221A_ABST
    Figure CN119003221A_ABST
Patent Text Reader

Abstract

The invention discloses a data processing method and device, equipment, a medium and a product. The invention relates to the technical field of computer application. The method comprises the steps that when it is detected that a processing process corresponding to a data processing request is interrupted, to-be-processed log data associated with the data processing request is acquired; according to the to-be-processed log data, an error graph is determined, and the error graph is a directed graph comprising at least two nodes; determining a node attribute of each node in the error graph and an edge attribute of a connecting edge through an attribute evaluation model; based on node attributes and edge attributes in the error graph, determining a target error path causing interruption of the processing process; wherein the target error path is the shortest path in all the judged error paths, and the target error path is a branch of the error graph. According to the technical scheme, the effect of improving the efficiency and accuracy of analyzing the interruption error when the interruption of the processing process corresponding to the data processing request is detected can be achieved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computer application technology, and in particular, to a data processing method, device, equipment, medium and product. Background Art

[0002] With the development of social economy, various industries have become more and more complex in order to meet social needs. For example, a transaction task needs to call a large number of interfaces in multiple backends to implement it. Therefore, it is common that a process interruption error will cause multiple error points to burst out, making the error analysis process very complicated.

[0003] In the related art, after a processing interruption error occurs, the error is usually analyzed by manually checking the error points in the error log file one by one to determine the error source and error propagation path, etc. However, this takes a lot of time and the accuracy of the analysis results is difficult to guarantee. Summary of the invention

[0004] Embodiments of the present invention provide a data processing method, apparatus, device, medium and product to solve the technical problem of low efficiency and poor accuracy in analyzing interruption errors when detecting an interruption of a processing process corresponding to a data processing request.

[0005] In a first aspect, an embodiment of the present invention provides a data processing method, wherein the method includes:

[0006] When detecting that a processing process corresponding to a data processing request is interrupted, obtaining to-be-processed log data associated with the data processing request;

[0007] Determine an error graph according to the log data to be processed, wherein the error graph is a directed graph including at least two nodes, the nodes correspond to a data processing interface in the log data to be processed, an interface return type of the data processing interface, interface feedback data returned by the data processing interface, or a data return type of the interface feedback data, and the connection edge of the at least two nodes corresponds to an interface dependency, a data dependency, or an interface and data dependency;

[0008] Inputting the error graph into a pre-trained attribute evaluation model to obtain node attributes of each node in the error graph and edge attributes of the connecting edge;

[0009] Based on the node attributes and edge attributes in the error graph, a target error path causing the processing process to be interrupted is determined; wherein the target error path is the shortest path among all the error paths determined, and the target error path is a branch of the error graph.

[0010] In a second aspect, an embodiment of the present invention further provides a data processing device, wherein the device includes:

[0011] A log data acquisition module, configured to acquire to-be-processed log data associated with a data processing request when detecting that a processing process corresponding to the data processing request is interrupted;

[0012] An error graph determination module is used to determine an error graph according to the log data to be processed, wherein the error graph is a directed graph including at least two nodes, the nodes correspond to a data processing interface in the log data to be processed, an interface return type of the data processing interface, interface feedback data returned by the data processing interface, or a data return type of the interface feedback data, and the connection edge of the at least two nodes corresponds to an interface dependency, a data dependency, or an interface and data dependency;

[0013] An attribute evaluation module, used for inputting the error graph into a pre-trained attribute evaluation model to obtain node attributes of each node in the error graph and edge attributes of the connecting edge;

[0014] The error path identification module is used to determine the target error path that causes the processing process to be interrupted based on the node attributes and edge attributes in the error graph; wherein the target error path is the shortest path among all the error paths determined, and the target error path is a branch of the error graph.

[0015] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the data processing method as described in any one of the embodiments of the present invention is implemented.

[0016] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a data processing method as described in any one of the embodiments of the present invention.

[0017] In a fifth aspect, an embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements a data processing method as described in any one of the embodiments of the present invention.

[0018] In an embodiment of the present invention, when detecting an interruption of a processing process corresponding to a data processing request, log data to be processed associated with the data processing request is obtained; based on the log data to be processed, an error graph is determined, wherein the error graph is a directed graph including at least two nodes, the nodes correspond to a data processing interface in the log data to be processed, an interface return type of the data processing interface, interface feedback data returned by the data processing interface, or a data return type of the interface feedback data, and the connecting edges of at least two nodes correspond to interface dependencies, data dependencies, or interface and data dependencies; the error graph is input into a pre-trained attribute evaluation model to obtain node attributes of each node in the error graph and edge attributes of the connecting edges; based on the node attributes and edge attributes in the error graph, a target error path causing the interruption of the processing process is determined; wherein the target error path is the shortest path among all error paths determined, and the target error path is a branch of the error graph. The technical solution of the present invention can, when detecting a processing process interruption corresponding to a data processing request, construct an error graph including node attributes and edge attributes corresponding to the log data to be processed, and perform shortest path calculation on the error graph, so as to quickly and accurately locate the error source node and the error propagation path corresponding to the node causing the processing process interruption, thereby improving the analysis efficiency and accuracy of the interruption error, facilitating the user to repair the error, and effectively improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 A flowchart of a data processing method provided by an embodiment of the present invention;

[0021] Figure 2 A flowchart of a data processing method provided by an embodiment of the present invention;

[0022] Figure 3 A flowchart of a data processing method provided by an embodiment of the present invention;

[0023] Figure 4 A flowchart of a data processing method provided by an embodiment of the present invention;

[0024] Figure 5 A schematic diagram of the structure of a data processing device provided by an embodiment of the present invention;

[0025] Figure 6The present invention is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0026] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0027] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance. The acquisition, storage, use, and processing of data in the technical solution of the present invention are in compliance with the relevant provisions of national laws and regulations.

[0028] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0029] Figure 1 A flowchart of a data processing method provided by an embodiment of the present invention. The present invention is applicable to the case where interruption errors are located by analyzing log data. The method can be executed by a data processing device, which can be implemented in the form of hardware and / or software. Optionally, it can be implemented by an electronic device, which can be a mobile terminal, a PC or a server. Figure 1 As shown, the method includes:

[0030] S110 . When detecting that the processing process corresponding to the data processing request is interrupted, obtaining to-be-processed log data associated with the data processing request.

[0031] The data processing request may be understood as a data processing request generated based on a user demand operation. In the embodiment of the present invention, the data processing request is related to the application scenario and is not specifically limited here. Optionally, the data processing request may include a transaction request.

[0032] The log data to be processed can be understood as log data to be analyzed for process interruption. In an embodiment of the present invention, the obtained log data to be processed can be associated with a data processing request. Optionally, the log data to be processed can be log data in a target log corresponding to a request identifier of the data processing request.

[0033] S120. Determine an error graph based on the log data to be processed, wherein the error graph is a directed graph including at least two nodes, the nodes correspond to the data processing interface in the log data to be processed, the interface return type of the data processing interface, the interface feedback data returned by the data processing interface, or the data return type of the interface feedback data, and the connecting edges of at least two nodes correspond to the interface dependency, the data dependency, or the interface and data dependency.

[0034] The error graph can be understood as a directed graph determined based on the log data to be processed. In an embodiment of the present invention, the error graph includes at least one of nodes, connecting edges between nodes, directions corresponding to connecting edges, weights of nodes, and weights of connecting edges.

[0035] In an embodiment of the present invention, based on the log data to be processed, the nodes in the error graph may be constructed first, then the connecting edges between the nodes may be constructed, then the directions corresponding to the connecting edges may be determined, and finally the weights of the nodes or connecting edges may be determined.

[0036] In an embodiment of the present invention, data processing requests can be processed through multiple data processing interfaces, and the multiple data processing interfaces can be located in multiple different backends. The interface dependency can be a dependency between data processing interfaces in the same backend or a dependency between data processing interfaces in different backends.

[0037] The data processing interface can be understood as an interface for performing data processing based on a data processing request.

[0038] The interface return type can be understood as the return type of the data processing interface in the log data to be processed. In an embodiment of the present invention, the interface return type can characterize whether there is an error in the data processing interface. That is, the interface return type can include returning an error and / or returning normal. The interface return type of the data processing interface in the log data to be processed can be presented in the form of code.

[0039] The interface feedback data can be understood as the data fed back by the data processing interface. In the embodiment of the present invention, a data processing request can be processed by multiple data processing interfaces, and the interface feedback data fed back by each data processing interface can be different or the same. The interface feedback data corresponding to each interface return type can be one or more.

[0040] The data return type can be understood as the return type of the interface feedback data in the log data to be processed. In an embodiment of the present invention, the data return type can characterize whether there is an error in the interface feedback data. That is, the data return type may include returning an error and / or returning normal. The data return type of the interface feedback data in the log data to be processed can be presented in the form of code.

[0041] Interface dependencies can be understood as dependencies between data processing interfaces in the log data to be processed. In an embodiment of the present invention, interface dependencies are associated with the settings of multiple backends involved in the data processing request, which is not specifically limited here. Exemplarily, the data processing interface includes interface A, interface B, and interface C. The interface dependency can be that interface B and interface C depend on interface A, or that interface C depends on interface B and interface B depends on interface A. Interface dependencies can be presented in the form of code in the log data to be processed.

[0042] Data dependencies can be understood as dependencies between interface feedback data. In an embodiment of the present invention, data dependencies are associated with the settings of the database where the interface feedback data is located, and are not specifically limited here. Exemplarily, the data dependencies include feedback data A, feedback data B, and feedback data C. The interface dependency may be that feedback data B and feedback data C depend on feedback data A, or that feedback data C depends on feedback data B and feedback data B depends on feedback data A. Data dependencies can be presented in the form of code in the log data to be processed.

[0043] The interface and data dependency can be understood as the dependency between the data processing interface and the interface feedback data in the log data to be processed. In an embodiment of the present invention, the interface and data dependency is associated with the settings of multiple backends involved in the data processing request, which is not specifically limited here. Exemplarily, the interface and data dependency can be that feedback data A and feedback data B rely on interface A for data feedback, and feedback data C relies on interface B for feedback.

[0044] S130 , inputting the error graph into a pre-trained attribute evaluation model to obtain node attributes of each node in the error graph and edge attributes of connecting edges.

[0045] Among them, the node attribute can be an attribute that characterizes the criticality of the node in the error graph. Correspondingly, the edge attribute can be an attribute that characterizes the criticality of the connection edge in the error graph. In an embodiment of the present invention, the node attribute and the edge attribute can be preset according to the scene requirements, and are not specifically limited here. Optionally, the node attribute can be a node weight, and the connection edge attribute includes the connection edge weight.

[0046] The present invention discloses a specific method for determining the node attributes of each node in an error graph and the edge attributes of a connecting edge. On the basis of the above embodiment, optionally, the error graph is input into a pre-trained attribute evaluation model to obtain the node attributes of each node in the error graph and the edge attributes of the connecting edge, including:

[0047] The error graph is input into the attribute evaluation model for weight prediction processing, and the node weight of each node in the error graph and the edge weight of the connecting edge are output.

[0048] The attribute evaluation model can be understood as a model for evaluating the node attributes of each node in the error graph and the edge attributes of the connecting edges. In an embodiment of the present invention, the input data of the attribute evaluation model can be an error graph, and the output data can be an error graph with attached node attributes and edge attributes.

[0049] The technical solution of the present invention adopts a deep learning model to perform weight prediction to determine the node attributes of each node in the error graph and the edge attributes of the connecting edges, thereby ensuring the accuracy of the determined node attributes and edge attributes.

[0050] S140. Determine a target error path that causes the processing process to be interrupted based on node attributes and edge attributes in the error graph; wherein the target error path is the shortest path among all error paths determined, and the target error path is a branch of the error graph.

[0051] The target error path may be understood as the shortest path between the node causing the interruption of the processing process and the error source node.

[0052] In an embodiment of the present invention, the node causing the interruption of the processing process may refer to a node corresponding to the data processing interface and / or a node corresponding to the interface feedback data, and the node may be determined based on the log data to be processed. The error source node may refer to a target node whose dependent parent node has no error, or a target node without a parent node. The node may also be determined based on the log data to be processed.

[0053] In the embodiment of the present invention, the node causing the interruption of the processing process may be one or more, the error source node may be one or more, and correspondingly, the target error path may also be one or more. Optionally, each error source node may correspond to a target error path, or each node causing the interruption of the processing process corresponds to a target error path.

[0054] The present invention discloses a specific method for determining a target error path that causes a processing process interruption based on an error graph including node attributes and edge attributes. Based on the above embodiment, optionally, the target error path that causes a processing process interruption is determined based on the node attributes and edge attributes in the error graph, including:

[0055] Determine at least one error path to be selected that causes the processing process to be interrupted according to node attributes, edge attributes and graph theory algorithms in the error graph;

[0056] The target error path is determined according to the path information of the error path to be selected.

[0057] Among them, graph theory algorithm can be understood as an algorithm for processing directed graphs and can be used to calculate the shortest path.

[0058] The error path to be selected can be understood as an error path to be selected, that is, a path that may cause the processing process to be interrupted. The error path to be selected includes the target error path and other error paths except the target error path.

[0059] The path information can be understood as information representing the length of the error path to be selected. In the embodiment of the present invention, the shortest path represented by the path information in the error path to be selected is used as the target error path.

[0060] The technical solution of the present invention uses a graph theory algorithm to calculate the shortest path of an error graph including node attributes and edge attributes to determine the target error path that causes the processing process to be interrupted, thereby ensuring the accuracy of the error path analysis and the accuracy of the determined target error path.

[0061] The present invention discloses a specific method for determining a target error path that causes a processing process interruption based on an error graph including node attributes and edge attributes. Based on the above embodiment, optionally, the target error path that causes a processing process interruption is determined based on the node attributes and edge attributes in the error graph, including:

[0062] The error graph including node attributes and edge attributes is input into the pre-trained path screening model, and the target error path that causes the processing process to be interrupted is output.

[0063] The path screening model can be understood as a model with the function of screening the shortest path. In an embodiment of the present invention, the input data of the path screening model can be an error graph including node attributes and edge attributes, and the output data can be the shortest path corresponding to the node in the error graph that causes the processing process to be interrupted.

[0064] The technical solution of the present invention adopts a deep learning model to perform path screening to determine the target error path that causes the processing process to be interrupted, thereby ensuring the accuracy of the error path analysis and the accuracy of the determined target error path.

[0065] In an embodiment of the present invention, when detecting an interruption of a processing process corresponding to a data processing request, log data to be processed associated with the data processing request is obtained; based on the log data to be processed, an error graph is determined, wherein the error graph is a directed graph including at least two nodes, the nodes correspond to a data processing interface in the log data to be processed, an interface return type of the data processing interface, interface feedback data returned by the data processing interface, or a data return type of the interface feedback data, and the connecting edges of at least two nodes correspond to interface dependencies, data dependencies, or interface and data dependencies; the error graph is input into a pre-trained attribute evaluation model to obtain node attributes of each node in the error graph and edge attributes of the connecting edges; based on the node attributes and edge attributes in the error graph, a target error path causing the interruption of the processing process is determined; wherein the target error path is the shortest path among all error paths determined, and the target error path is a branch of the error graph. The technical solution of the present invention can, when detecting a processing process interruption corresponding to a data processing request, construct an error graph including node attributes and edge attributes corresponding to the log data to be processed, and perform shortest path calculation on the error graph, so as to quickly and accurately locate the error source node and the error propagation path corresponding to the node causing the processing process interruption, thereby improving the analysis efficiency and accuracy of the interruption error, facilitating the user to repair the error, and effectively improving the user experience.

[0066] On the basis of the above embodiment, before determining the error graph according to the log data to be processed, the log data to be processed may be segmented, POS tagged and named entity recognized to determine the data processing interface, interface return type, interface feedback data, data return type, interface dependency, data dependency, interface and data dependency, interface processing time and interface return data time in the log data to be processed, so as to update the log data to be processed, so as to determine the error graph based on the updated log data to be processed. For the specific implementation method, please refer to the detailed description of this embodiment, in which the technical terms that are the same or corresponding to the above embodiment are not repeated here. Figure 2 A flowchart of a data processing method provided by an embodiment of the present invention.

[0067] like Figure 2 As shown, the data processing method comprises the following steps:

[0068] S210: When detecting that the processing process corresponding to the data processing request is interrupted, obtaining to-be-processed log data associated with the data processing request.

[0069] S220, perform word segmentation, part-of-speech tagging and named entity recognition on the log data to be processed, determine the data processing interface, interface return type, interface feedback data, data return type, interface dependency, data dependency, interface and data dependency, interface processing time and interface return data time in the log data to be processed, so as to update the log data to be processed, and determine the error graph based on the updated log data to be processed.

[0070] Specifically, first, the log data to be processed is segmented to obtain multiple preliminary segmented words. In an embodiment of the present invention, the log data to be processed can be presented in the form of code. The code string can be segmented to obtain multiple independent preliminary segmented words, and the preliminary segmented words can be understood as a group of code strings.

[0071] Secondly, identify the part-of-speech features of each preliminary segmentation, and perform part-of-speech tagging on the preliminary segmentation based on the part-of-speech features. Among them, the part-of-speech features may include at least one of interface features, feedback data features, dependency features, and time features. The interface feature may represent that the preliminary segmentation is associated with the data processing interface, the feedback data feature may represent that the preliminary segmentation is associated with the interface feedback data, the dependency feature may represent that the preliminary segmentation is associated with at least one of the interface dependency, data dependency, and interface and data dependency, and the time feature may represent that the preliminary segmentation is associated with the interface processing time and / or the interface return data time.

[0072] In the embodiment of the present invention, part-of-speech tagging can be understood as an operation of tagging part-of-speech features on the preliminary segmentation. In the embodiment of the present invention, the manner of tagging the preliminary segmentation with part-of-speech features is not specifically limited here, and can be specifically set according to the scene requirements. For example, the preliminary segmentation tagged with the part-of-speech features of the interface class features can refer to the preliminary segmentation as a code string corresponding to the data processing interface.

[0073] Finally, named entity recognition is performed on each marked word segment to obtain multiple entity word segmentations to obtain the log data to be processed. In an embodiment of the present invention, named entity recognition can be understood as an operation to identify named entities set by the user. Among them, the named entity can be a data processing interface and / or interface feedback data. For example, for three preliminary word segmentations with part-of-speech features marked with interface-like features; the entity names set by the user of the preliminary word segmentation are identified, for example, A, B and C respectively; then three entity word segmentations are obtained: interface A, interface B and interface C respectively.

[0074] The updated log data to be processed can be understood as the updated data obtained after the original log data to be processed is segmented, POS-tagged and named entity recognized. According to the technical solution of the present invention, the data in the updated log data to be processed can be directly recognized by the computer and an error graph can be constructed.

[0075] S230: Determine an error graph according to the log data to be processed.

[0076] The present invention discloses a specific method for determining an error graph corresponding to log data to be processed. On the basis of the above embodiment, optionally, the error graph is determined according to the log data to be processed, including:

[0077] Determine connection information of at least one first-level node in the error graph and flow information between first-level nodes according to the data processing interface, interface return type, interface dependency, and interface processing time in the log data to be processed;

[0078] Determine at least one secondary node connected to at least one primary node in the error graph, connection information between the secondary nodes, and flow direction information between the secondary nodes based on interface feedback data, data return type, data dependency, interface and data dependency, and interface return data time in the log data to be processed;

[0079] Based on the connection information and flow direction information of the primary nodes, the connection information between the primary nodes and the secondary nodes, and the flow direction information between the secondary nodes, an error graph corresponding to the data processing request is determined.

[0080] Among them, the first-level node can be understood as a node corresponding to a data processing interface. A first-level node can be associated with a data processing interface. The first-level node can be determined based on the data processing interface and the interface return type corresponding to the data processing interface.

[0081] The interface processing time can be understood as the interface call time point recorded in the target log corresponding to the log data to be processed. In the embodiment of the present invention, the interface processing time corresponding to different data processing interfaces can be different or the same.

[0082] The connection information of the first-level nodes can be used to determine the connection edges between the first-level nodes, and the flow information between the first-level nodes can be used to determine the direction of the connection edges. The connection information and flow information between the first-level nodes can be determined based on the interface dependency and the interface processing time.

[0083] The secondary node can be understood as a node corresponding to the interface feedback data. A secondary node can be associated with an interface feedback data. The secondary node can be determined based on the interface feedback data and the data return type corresponding to the interface feedback data. In the technical solution of the present invention, the secondary node can rely on the primary node, just as the interface feedback data relies on the data processing interface for feedback.

[0084] The interface return data time can be understood as the data return time point recorded in the target log corresponding to the log data to be processed.

[0085] The connection information between the secondary nodes can be used to determine the connection edges between the secondary nodes, and the flow direction information between the secondary nodes can be used to determine the direction of the connection edges. The connection information and flow direction information between the secondary nodes can be determined based on the data dependency relationship.

[0086] Interfaces and data dependencies can be used to determine connection information and flow information between primary nodes and secondary nodes.

[0087] The technical solution of the present invention realizes that the constructed error graph comprehensively includes information related to the interruption error in the log data to be processed.

[0088] S240: Input the error graph into a pre-trained attribute evaluation model to obtain node attributes of each node in the error graph and edge attributes of the connecting edges.

[0089] S250: Determine a target error path that causes the processing process to be interrupted based on the node attributes and edge attributes in the error graph.

[0090] In an embodiment of the present invention, by segmenting, part-of-speech tagging and named entity recognition of log data to be processed, the data processing interface, interface return type, interface feedback data, data return type, interface dependency, data dependency, interface and data dependency, interface processing time and interface return data time in the log data to be processed are determined to update the log data to be processed, so as to determine an error graph based on the updated log data to be processed. The technical solution of the present invention extracts the log data to be processed from the log data to be processed by segmenting, part-of-speech tagging and named entity recognition, so that the extracted data can be identified to construct an error graph, and the effect of constructing a directed error graph corresponding to the log data to be processed associated with the data processing request is achieved.

[0091] On the basis of the above embodiment, before the error graph is input into the pre-trained attribute evaluation model, the historical log data corresponding to the historical processing request interruption can be obtained, and the historical error graph corresponding to the historical log data can be determined; the labeling weight value of each historical node and historical connection edge in the historical error graph can be determined according to the pre-set error judgment rules, so as to determine the training samples participating in the attribute evaluation model training based on the historical error graph and the labeling weight value; the historical error graph in the training sample is input into the attribute evaluation model to be trained, and the predicted node weights and predicted edge weights corresponding to the historical error graph are output; based on the predicted node weights, predicted edge weights, and the labeling weight values ​​corresponding to the training samples, the loss value is determined, and the model parameters in the attribute evaluation model to be trained are adjusted based on the loss value; the convergence of the loss function in the attribute evaluation model to be trained is used as the training goal to obtain the attribute evaluation model. The specific implementation method can be referred to the detailed description of this embodiment, in which the technical terms that are the same as or corresponding to the above embodiment are not repeated here, Figure 3 A flowchart of a data processing method provided by an embodiment of the present invention. Figure 3 As shown, the data processing method comprises the following steps:

[0092] S310, obtaining historical log data corresponding to the interruption of the historical processing request, and determining a historical error graph corresponding to the historical log data;

[0093] Among them, historical processing requests can be understood as historical data processing requests. Historical log data can be understood as historical log data. Historical error graphs can be understood as historical error graphs. In the technical solution of the present invention, historical processing requests, historical log data, and historical error graphs can correspond to data processing requests, log data to be processed, and error graphs, respectively, and are of the same type, which will not be repeated here.

[0094] S320. The preset error judgment rule determines the labeling weight value of each historical node and historical connection edge in the historical error graph, so as to determine the training samples participating in the attribute evaluation model training based on the historical error graph and the labeling weight value.

[0095] The historical error graph includes historical nodes, historical connecting edges, the labeling weight value of each historical node, and the labeling weight value of each historical connecting edge.

[0096] In an embodiment of the present invention, the annotation weight value of each historical node and the annotation weight value of each historical connection edge can be annotated based on a preset error judgment rule. The annotation weight value can represent the criticality of the historical node and / or historical connection edge in causing the interruption of the historical process.

[0097] The training samples can be understood as samples used to train the attribute evaluation model. The training samples can be historical error graphs annotated with annotation weight values.

[0098] The error judgment rule can be understood as a rule for judging the criticality of an error. In the technical solution of the present invention, the error judgment rule can be pre-set according to user needs, and is not specifically limited here. Based on the above embodiment, optionally, the error judgment rule includes at least one of a node severity assessment rule, a node error frequency reference rule, a node impact range determination rule, a node repair record rule, and an edge repair record rule.

[0099] The node severity evaluation rules can be understood as rules for evaluating the severity of node errors. The node error severity can be determined based on multiple dimensions, for example, based on the criticality and attention of the data processing interface corresponding to the node in the backend, or based on the setting attributes of the interface feedback data corresponding to the node in the database.

[0100] The node error frequency reference rule can be understood as a rule for determining the node error frequency. The node error frequency can be a frequency level or a frequency value. Exemplarily, the frequency level can include one or more of a high frequency level, a medium frequency level, and a low frequency level.

[0101] The node influence range determination rule can be understood as a rule for determining the node influence range. The node influence range can be an influence range level. Exemplarily, the influence range level can be one or more of a high range level, a medium range level, and a low range level.

[0102] The node repair record rule can be understood as a rule for node repair. The node repair record rule can be associated with the node repair record. Optionally, the node repair record can include the user's node repair method, such as deleting an interface and / or feedback data, adding an interface and / or feedback data, etc., one or more of the above.

[0103] The edge repair record rule can be understood as a rule for edge repair. The edge repair record rule can be associated with the edge repair record. Optionally, the edge repair record can include the user's edge repair method, such as changing the dependency between the interface and / or feedback data corresponding to the edge, deleting the interface and / or feedback data, adding the interface and / or feedback data, etc., one or more of the above.

[0104] S330: Input the historical error graph in the training sample into the attribute evaluation model to be trained, and output the predicted node weights and predicted edge weights corresponding to the historical error graph.

[0105] The attribute evaluation model to be trained can be understood as the attribute evaluation model to be initially constructed for sample training. The predicted node weight can be understood as the weight of each node in the historical error graph predicted by the attribute evaluation model to be trained. The predicted edge weight can be understood as the weight of each connecting edge in the historical error graph predicted by the attribute evaluation model to be trained.

[0106] S340, determining a loss value based on the predicted node weights, the predicted edge weights, and the labeled weight values ​​corresponding to the training samples, and adjusting model parameters in the attribute evaluation model to be trained based on the loss value.

[0107] The loss value may include a loss value between a predicted node weight and a node weight, and / or a loss value between a predicted edge weight and a connection edge weight.

[0108] S350: Taking the convergence of the loss function in the attribute evaluation model to be trained as a training target, and obtaining the attribute evaluation model.

[0109] Specifically, when the loss function in the attribute evaluation model to be trained converges, the model training is stopped to obtain the attribute evaluation model.

[0110] S360: When detecting that the processing process corresponding to the data processing request is interrupted, obtaining to-be-processed log data associated with the data processing request.

[0111] S370: Determine an error graph according to the log data to be processed.

[0112] S380: Input the error graph into a pre-trained attribute evaluation model to obtain node attributes of each node in the error graph and edge attributes of the connecting edges.

[0113] S390: Determine the target error path that causes the processing process to be interrupted based on the node attributes and edge attributes in the error graph.

[0114] In an embodiment of the present invention, by obtaining the historical log data corresponding to the interruption of the historical processing request, and determining the historical error graph corresponding to the historical log data; determining the label weight value of each historical node and the historical connection edge in the historical error graph according to the preset error judgment rule, so as to determine the training sample participating in the attribute evaluation model training based on the historical error graph and the label weight value; inputting the historical error graph in the training sample into the attribute evaluation model to be trained, outputting the predicted node weight and predicted edge weight corresponding to the historical error graph; determining the loss value based on the predicted node weight, the predicted edge weight, and the label weight value corresponding to the training sample, and adjusting the model parameters in the attribute evaluation model to be trained based on the loss value; taking the convergence of the loss function in the attribute evaluation model to be trained as the training target, and obtaining the attribute evaluation model. The technical scheme of the present invention adopts the method of using the historical error graph corresponding to the historical log data as the training sample to train the attribute evaluation model, and obtains a model with the function of evaluating the weights of nodes and connection edges in the directed graph (i.e., the error graph).

[0115] On the basis of the above embodiment, after obtaining the target error path, at least one target node on the target error path and the connecting edges between the target nodes can be displayed in the error graph in the first form. For specific implementation methods, please refer to the detailed description of this embodiment, in which the technical terms that are the same as or corresponding to the above embodiment are not repeated here. Figure 4 A flowchart of a data processing method provided by an embodiment of the present invention. Figure 4 As shown, the data processing method comprises the following steps:

[0116] S410: When detecting that a processing process corresponding to a data processing request is interrupted, obtaining to-be-processed log data associated with the data processing request.

[0117] S420: Determine an error graph according to the log data to be processed.

[0118] S430: Input the error graph into a pre-trained attribute evaluation model to obtain node attributes of each node in the error graph and edge attributes of the connecting edges.

[0119] S440: Determine a target error path that causes the processing process to be interrupted based on the node attributes and edge attributes in the error graph.

[0120] S450. Display at least one target node on the target error path and the connecting edges between the target nodes in the error graph in a first form.

[0121] Specifically, at least one target node on the target error path and the connection edges between the target nodes are displayed in the first form, wherein the target node can be understood as a node in the target error path.

[0122] The first form can be understood as a preset display form. In an embodiment of the present invention, the first form can be preset according to the scene requirements, and is not specifically limited here. Exemplarily, the first form can be a display form of a target color. Among them, the target color can be a default color different from other paths. The default color and the target color can be preset according to user needs, and are not specifically limited here. Exemplarily, the default color can be black and / or gray, and the target color can include one or more of green, red or blue.

[0123] The first form includes a first sub-form of a target node and / or a second sub-form of a connecting edge between target nodes. The first sub-form and the second sub-form may be different or the same. Exemplarily, the target colors corresponding to the first sub-form and the second sub-form may be different.

[0124] The present invention discloses a method for displaying nodes, associated nodes and node associated information in response to user interaction operations. Based on the above embodiments, the data processing method may further include:

[0125] In response to a trigger operation on at least one node or connecting edge in the error graph, displaying associated nodes of at least one node associated with the trigger operation in a second form; and

[0126] Node association information displayed in a second form is obtained, wherein the node association information includes at least one of a device to which the node belongs, a name of a node on which the node depends, and a node abnormality time.

[0127] The triggering operation may include sliding the mouse to at least one node or connecting edge in the error graph, or clicking at least one node or connecting edge in the error graph.

[0128] The second form can be understood as a preset display form. In the embodiment of the present invention, the second form can be preset according to the scene requirements, and is not specifically limited here. Exemplarily, the second form display can be a high-brightness display form.

[0129] The second form includes a third sub-form of the target node and / or a fourth sub-form of the connecting edge between the target nodes. The third sub-form and the fourth sub-form may be different or the same. Exemplarily, the highlighting degree of the third sub-form and the fourth sub-form may be different.

[0130] The node association information can be understood as information associated with the node on which the trigger operation acts. Optionally, the node association information may include the device to which the node belongs, the name of the node it depends on, and the node abnormality time. Among them, the device to which the node belongs can be understood as the backend device to which the data processing interface corresponding to the node belongs. The name of the node it depends on may include the name of the node it depends on, for example, interface A, interface B, or interface C. The node abnormality time can be understood as the time when the node abnormality is recorded in the target log corresponding to the log data to be processed.

[0131] The technical solution of the present invention realizes the display of node association information including the device to which the node belongs, the name of the node it depends on, and the time of node abnormality, which can enable the user to locate the error object that causes the interruption error so that the error object can be repaired, thereby ensuring the high efficiency of error repair.

[0132] In an embodiment of the present invention, at least one target node on a target error path and the connecting edges between the target nodes are displayed in a first form in an error graph. The technical solution of the present invention realizes the visualization of the target error path. By displaying the target error path in a preset first form, it is convenient for users to view and analyze the shortest error path that causes the interruption of the processing process, thereby improving the user experience.

[0133] The present invention combines natural language processing (NLP), graph theory algorithms and deep learning models to quickly locate the shortest error path in the error log that causes the interruption of the data processing request process. As an optional embodiment of the above embodiment, if the overall process of data processing log data based on the construction of a directed graph is explained in steps, the following steps can be taken to achieve it.

[0134] First, the error log is preprocessed. When the processing interruption corresponding to the transaction request is detected, the target log is obtained, and the log data in the target log is cleaned and standardized, for example, redundant information is removed, and timestamps are formatted.

[0135] Furthermore, the log data is updated. Through NLP technologies, such as word segmentation, part-of-speech tagging, and named entity recognition, key information in the log data is identified, such as interfaces, interface feedback data, error types corresponding to interfaces, error types corresponding to interface feedback data, dependencies between interfaces, dependencies between interface feedback data, and timestamps.

[0136] Furthermore, an error graph is constructed. Nodes are constructed based on interfaces, interface feedback data, and error types, and the connection edges and directions of the connection edges between nodes are constructed based on dependencies and timestamps.

[0137] Furthermore, node weights and edge weights are set. The node weights of each node in the error graph and the edge weights of each edge are predicted or evaluated by a deep learning model, wherein the historical error graph can be used as a training sample to train the deep learning model, and the labels corresponding to the training samples can be determined by judging the severity, frequency, impact range and other factors of the error through preset error judgment rules.

[0138] In an embodiment of the present invention, a deep learning model is introduced, and the deep learning model is trained by a historical error graph corresponding to a large amount of historical log data to predict nodes or connecting edges in the error chain of the error graph that are more likely to cause serious errors. The deep learning model can be a classifier for predicting whether each node or connecting edge is a key target in the error chain. The training sample can be determined based on the historical log data and the corresponding repair records.

[0139] In an embodiment of the present invention, the attribute evaluation model is updated and optimized in real time, and the error graph corresponding to the target log after data processing and the attribute information of the error graph obtained by evaluation are used as new training samples to optimize the attribute evaluation model to improve the accuracy of the attribute evaluation model.

[0140] Further, the shortest path is determined by a graph theory algorithm. The shortest path from the error source node to the node causing the processing interruption is found on the error graph by using the Dijkstra algorithm, Bellman-Ford algorithm or other shortest path algorithms, wherein the error source node may refer to the node where the error first occurs.

[0141] Finally, in response to interactive operations, the error information is visualized. The error graph and the shortest path are displayed through visualization tools, so that users can quickly visualize and locate the root cause of the error and the propagation path; in response to user interactive operations, specific nodes or connecting edges can be highlighted to further analyze the error path.

[0142] In an embodiment of the present invention, the algorithms and tools involved above can be integrated into a monitoring system to automatically construct an error graph and calculate the shortest path when a processing process corresponding to a data processing request is detected to be interrupted. No manual operation is required, thereby realizing fully automatic data processing, improving data processing efficiency, and enhancing user experience.

[0143] In the embodiment of the present invention, the integration method of the algorithm and the tool can be preset according to the scenario requirements and is not specifically limited here.

[0144] The technical solution of the present invention improves positioning efficiency and accuracy by constructing an error graph corresponding to the error log and quickly locating the error source node and error propagation path corresponding to the error causing the processing interruption error in the error graph by the shortest path method.

[0145] Figure 5 A schematic diagram of the structure of a data processing device provided by an embodiment of the present invention. Figure 5 As shown, the apparatus includes: a log data acquisition module 510 , an error graph determination module 520 , a property evaluation module 530 and an error path identification module 540 .

[0146] Among them, the log data acquisition module 510 is used to acquire the pending log data associated with the data processing request when detecting the interruption of the processing process corresponding to the data processing request; the error graph determination module 520 is used to determine the error graph according to the pending log data, wherein the error graph is a directed graph including at least two nodes, and the nodes correspond to the data processing interface in the pending log data, the interface return type of the data processing interface, the interface feedback data returned by the data processing interface, or the data return type of the interface feedback data, and the connecting edge of the at least two nodes corresponds to the interface dependency, the data dependency, or the interface and data dependency; the attribute evaluation module 530 is used to input the error graph into the attribute evaluation model obtained by pre-training to obtain the node attribute of each node in the error graph and the edge attribute of the connecting edge; the error path identification module 540 is used to determine the target error path that causes the interruption of the processing process based on the node attributes and edge attributes in the error graph; wherein the target error path is the shortest path among all the error paths determined, and the target error path is a branch of the error graph.

[0147] In an embodiment of the present invention, when detecting an interruption of a processing process corresponding to a data processing request, log data to be processed associated with the data processing request is obtained; based on the log data to be processed, an error graph is determined, wherein the error graph is a directed graph including at least two nodes, the nodes correspond to a data processing interface in the log data to be processed, an interface return type of the data processing interface, interface feedback data returned by the data processing interface, or a data return type of the interface feedback data, and the connecting edges of at least two nodes correspond to interface dependencies, data dependencies, or interface and data dependencies; the error graph is input into a pre-trained attribute evaluation model to obtain node attributes of each node in the error graph and edge attributes of the connecting edges; based on the node attributes and edge attributes in the error graph, a target error path causing the interruption of the processing process is determined; wherein the target error path is the shortest path among all error paths determined, and the target error path is a branch of the error graph. The technical solution of the present invention can, when detecting a processing process interruption corresponding to a data processing request, construct an error graph including node attributes and edge attributes corresponding to the log data to be processed, and perform shortest path calculation on the error graph, so as to quickly and accurately locate the error source node and the error propagation path corresponding to the node causing the processing process interruption, thereby improving the analysis efficiency and accuracy of the interruption error, facilitating the user to repair the error, and effectively improving the user experience.

[0148] On the basis of the above device, optionally, the data processing device further includes:

[0149] A log data update module is used to perform word segmentation, part-of-speech tagging and named entity recognition on the log data to be processed before determining an error graph based on the log data to be processed, and determine the data processing interface, interface return type, interface feedback data, data return type, interface dependency, data dependency, interface and data dependency, interface processing time and interface return data time in the log data to be processed, so as to update the log data to be processed and determine the error graph based on the updated log data to be processed.

[0150] Based on the above device, optionally, the error graph determination module 520 is specifically used to:

[0151] Determine, according to the data processing interface, interface return type, interface dependency, and interface processing time in the log data to be processed, connection information of at least one primary node in the error graph and flow direction information between the primary nodes;

[0152] Determine at least one secondary node connected to the at least one primary node in the error graph, connection information between the secondary nodes, and flow direction information between the secondary nodes based on interface feedback data, data return type, data dependency, interface and data dependency, and interface return data time in the log data to be processed;

[0153] An error graph corresponding to the data processing request is determined based on the connection information and flow direction information of the primary node, the connection information between the primary node and the secondary node, and the flow direction information between the secondary nodes.

[0154] Based on the above device, optionally, the attribute evaluation model is trained based on the following method:

[0155] Acquire historical log data corresponding to the interruption of the historical processing request, and determine a historical error graph corresponding to the historical log data;

[0156] Determine the labeling weight value of each historical node and historical connection edge in the historical error graph according to a preset error judgment rule, so as to determine the training samples participating in the attribute evaluation model training based on the historical error graph and the labeling weight value;

[0157] Inputting the historical error graph in the training sample into the attribute evaluation model to be trained, and outputting the predicted node weights and predicted edge weights corresponding to the historical error graph;

[0158] Determining a loss value based on the predicted node weight, the predicted edge weight, and the labeled weight value corresponding to the training sample, and adjusting a model parameter in the attribute evaluation model to be trained based on the loss value;

[0159] The convergence of the loss function in the attribute evaluation model to be trained is taken as a training goal to obtain the attribute evaluation model.

[0160] Based on the above device, optionally, the error judgment rule includes at least one of a node severity assessment rule, a node error frequency reference rule, a node impact range determination rule, a node repair record rule, and an edge repair record rule.

[0161] Based on the above device, optionally, the node attribute includes a node weight, the edge attribute includes an edge weight, and the attribute evaluation module 530 is specifically used to:

[0162] The error graph is input into the attribute evaluation model for weight prediction processing, and the node weight of each node in the error graph and the edge weight of the connecting edge are output.

[0163] Based on the above device, optionally, the error path identification module 540 is specifically used to:

[0164] Determine at least one error path to be selected that causes the processing process to be interrupted according to the node attributes, edge attributes and graph theory algorithm in the error graph;

[0165] The target error path is determined according to the path information of the error path to be selected.

[0166] Based on the above device, optionally, the error path identification module 540 is specifically used to:

[0167] The error graph including node attributes and edge attributes is input into the pre-trained path screening model, and the target error path that causes the processing process to be interrupted is output.

[0168] On the basis of the above device, optionally, the data processing device further includes:

[0169] The error path display module is used to display at least one target node on the target error path and the connection edges between the target nodes in the error graph in a first form after obtaining the target error path.

[0170] On the basis of the above device, optionally, the data processing device further includes:

[0171] an associated node display module, configured to display associated nodes of at least one node associated with the triggering operation in a second form in response to a triggering operation on at least one node or a connecting edge in the error graph; and

[0172] The associated information display module is used to obtain the node associated information displayed in the second form, wherein the node associated information includes at least one of the device to which the node belongs, the name of the node it depends on, and the node abnormality time.

[0173] Figure 6 The present invention is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 6 A block diagram of an exemplary electronic device 50 suitable for implementing exemplary embodiments of the present invention is shown. Figure 6 The electronic device 50 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0174] like Figure 6 As shown, the electronic device 50 is in the form of a general computing device. The components of the electronic device 50 may include but are not limited to: one or more processors or processing units 501, a system memory 502, and a bus 503 connecting different system components (including the system memory 502 and the processing unit 501).

[0175] Bus 503 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus structures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.

[0176] The electronic device 50 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 50, including volatile and non-volatile media, removable and non-removable media.

[0177] System memory 502 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 504 and / or cache memory 505. Electronic device 50 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 506 may be used to read and write non-removable, non-volatile magnetic media ( Figure 6 not shown, usually called a "hard drive"). Although Figure 6 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 503 via one or more data medium interfaces. The memory 502 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the embodiments of the present invention.

[0178] A program / utility 508 having a set (at least one) of program modules 507 may be stored, for example, in the memory 502, such program modules 507 including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 507 generally perform the functions and / or methods of the embodiments described herein.

[0179] The electronic device 50 may also communicate with one or more external devices 509 (e.g., keyboards, pointing devices, displays 510, etc.), may communicate with one or more devices that enable a user to interact with the electronic device 50, and / or communicate with any device that enables the electronic device 50 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may be performed through an input / output (I / O) interface 511. Furthermore, the electronic device 50 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 512. As shown, the network adapter 512 communicates with other modules of the electronic device 50 through the bus 503. It should be understood that although Figure 6 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 50, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0180] The processing unit 501 executes various functional applications and data processing by running the programs stored in the system memory 502, such as implementing the data processing method provided in the embodiment of the present invention.

[0181] An embodiment of the present invention further provides a storage medium containing computer executable instructions, and when the computer executable instructions are executed by a computer processor, they are used to perform a data processing method, the method comprising:

[0182] Acquire at least one stream data in the target message system, wherein the stream data includes product basic data of at least one product to be selected under at least one basic field, and floating data under a floating field where data can be changed;

[0183] For the streaming data, determining a target product corresponding to the streaming data, and reading a target configuration rule corresponding to the target product from a configuration center;

[0184] Reading a to-be-processed field in the target configuration rule based on a target function, and acquiring target data content in the streaming data according to the to-be-processed field;

[0185] A target processing result of the streaming data is determined according to the target data content and the target configuration rule.

[0186] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0187] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0188] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0189] Computer program code for performing the operation of embodiments of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0190] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the data processing method provided by any embodiment of the present invention.

[0191] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0192] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A data processing method, It is characterized in that include: When detecting that a processing process corresponding to a data processing request is interrupted, obtaining to-be-processed log data associated with the data processing request; Determine an error graph according to the log data to be processed, wherein the error graph is a directed graph including at least two nodes, the nodes correspond to a data processing interface in the log data to be processed, an interface return type of the data processing interface, interface feedback data returned by the data processing interface, or a data return type of the interface feedback data, and the connection edge of the at least two nodes corresponds to an interface dependency, a data dependency, or an interface and data dependency; Inputting the error graph into a pre-trained attribute evaluation model to obtain node attributes of each node in the error graph and edge attributes of the connecting edge; Based on the node attributes and edge attributes in the error graph, a target error path causing the processing process to be interrupted is determined; wherein the target error path is the shortest path among all the error paths determined, and the target error path is a branch of the error graph.

2. The method according to claim 1, It is characterized in that Before determining an error graph according to the log data to be processed, the method further includes: Perform word segmentation, part-of-speech tagging and named entity recognition on the log data to be processed, determine the data processing interface, interface return type, interface feedback data, data return type, interface dependency, data dependency, interface and data dependency, interface processing time and interface return data time in the log data to be processed, so as to update the log data to be processed, and determine the error graph based on the updated log data to be processed.

3. The method according to claim 1 or 2, It is characterized in that The step of determining an error graph according to the log data to be processed includes: Determine, according to the data processing interface, interface return type, interface dependency, and interface processing time in the log data to be processed, connection information of at least one primary node in the error graph and flow direction information between the primary nodes; Determine at least one secondary node connected to the at least one primary node in the error graph, connection information between the secondary nodes, and flow direction information between the secondary nodes based on interface feedback data, data return type, data dependency, interface and data dependency, and interface return data time in the log data to be processed; An error graph corresponding to the data processing request is determined based on the connection information and flow direction information of the primary node, the connection information between the primary node and the secondary node, and the flow direction information between the secondary nodes.

4. The method according to claim 1, It is characterized in that The attribute evaluation model is trained based on the following method: Acquire historical log data corresponding to the interruption of the historical processing request, and determine a historical error graph corresponding to the historical log data; Determine the labeling weight value of each historical node and historical connection edge in the historical error graph according to a preset error judgment rule, so as to determine the training samples participating in the attribute evaluation model training based on the historical error graph and the labeling weight value; Inputting the historical error graph in the training sample into the attribute evaluation model to be trained, and outputting the predicted node weights and predicted edge weights corresponding to the historical error graph; Determining a loss value based on the predicted node weight, the predicted edge weight, and the labeled weight value corresponding to the training sample, and adjusting a model parameter in the attribute evaluation model to be trained based on the loss value; The convergence of the loss function in the attribute evaluation model to be trained is taken as a training goal to obtain the attribute evaluation model.

5. The method according to claim 4, It is characterized in that The error judgment rule includes at least one of a node severity assessment rule, a node error frequency reference rule, a node impact range determination rule, a node repair record rule, and an edge repair record rule.

6. The method according to claim 1, It is characterized in that The node attributes include node weights, the edge attributes include edge weights, and the error graph is input into a pre-trained attribute evaluation model to obtain the node attributes of each node in the error graph and the edge attributes of the edge, including: The error graph is input into the attribute evaluation model for weight prediction processing, and the node weight of each node in the error graph and the edge weight of the connecting edge are output.

7. The method according to claim 1, It is characterized in that The step of determining the target error path causing the interruption of the processing process based on the node attributes and edge attributes in the error graph includes: Determine at least one error path to be selected that causes the processing process to be interrupted according to the node attributes, edge attributes and graph theory algorithm in the error graph; The target error path is determined according to the path information of the error path to be selected.

8. The method according to claim 1, It is characterized in that The step of determining the target error path causing the interruption of the processing process based on the node attributes and edge attributes in the error graph includes: The error graph including node attributes and edge attributes is input into the pre-trained path screening model, and the target error path that causes the processing process to be interrupted is output.

9. The method according to claim 1, It is characterized in that After obtaining the target error path, the method further includes: At least one target node on the target error path and the connecting edges between the target nodes are displayed in a first form in the error graph.

10. The method according to claim 1, It is characterized in that The method further comprises: In response to a trigger operation on at least one node or connecting edge in the error graph, displaying associated nodes of at least one node associated with the trigger operation in a second form; and Obtain node association information displayed in the second form, wherein the node association information includes at least one of a device to which the node belongs, a name of a node on which the node depends, and a node abnormality time.

11. A data processing device, It is characterized in that include: A log data acquisition module, configured to acquire to-be-processed log data associated with a data processing request when detecting that a processing process corresponding to the data processing request is interrupted; An error graph determination module is used to determine an error graph according to the log data to be processed, wherein the error graph is a directed graph including at least two nodes, the nodes correspond to a data processing interface in the log data to be processed, an interface return type of the data processing interface, interface feedback data returned by the data processing interface, or a data return type of the interface feedback data, and the connection edge of the at least two nodes corresponds to an interface dependency, a data dependency, or an interface and data dependency; An attribute evaluation module, used for inputting the error graph into a pre-trained attribute evaluation model to obtain node attributes of each node in the error graph and edge attributes of the connecting edge; The error path identification module is used to determine the target error path that causes the interruption of the processing process based on the node attributes and edge attributes in the error graph; wherein the target error path is the shortest path among all the error paths determined, and the target error path is a branch of the error graph.

12. The device according to claim 11, It is characterized in that The device also includes: The error path display module is used to display at least one target node on the target error path and the connection edges between the target nodes in the error graph in a first form.

13. A data processing electronic device, It is characterized in that The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the data processing method according to any one of claims 1 to 10 when executing the computer program.

14. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the program is executed by a processor, the data processing method according to any one of claims 1 to 10 is implemented.

15. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the computer program implements the data processing method according to any one of claims 1 to 10.

Citation Information

Cited By

  • Error analysis method and system, electronic equipment and storage medium

    CN119337108A