Data tracking method, device, equipment and storage medium for DAG model

Abstract

The present application relates to a data tracking method, device, equipment and storage medium for a DAG model. The DAG model has a business node, and the method includes: when the business node processes task data to obtain processed task data, and transmits the processed task data and the execution identifier corresponding to the business node, obtaining the execution identifier and task execution data corresponding to the business node; the task execution data is the description data of the business node processing the task data; combining the execution identifier and the task execution data into tracking data; and determining the data link to which the tracking data belongs according to the execution identifier. This method can determine the data link to which the tracking data belongs when the same business node can be triggered by multiple business nodes to which it is forward, so as to facilitate systematic data analysis.

Description

Technical Field

[0001] The present application relates to the field of performance analysis, and in particular to a data tracing method, apparatus, computer device, computer-readable storage medium, and computer program product for a DAG model. Background Art

[0002] With the continuous development of the digitalization process, software and hardware systems are often faced with a variety of businesses, and therefore need to track task data for different businesses in order to restore the original on-site data processing process based on various records in execution.

[0003] The existing solution relies on a tree graph model for data tracking. The main feature of the tree graph model is that each complete transaction processing flow is initiated by a business node, and its subsequent processing nodes can only fork but not merge in the processing flow. When multiple business nodes cannot trigger the same subsequent business node separately, task data in some specific scenarios cannot be tracked. Summary of the invention

[0004] Based on this, it is necessary to provide a data tracking method, device, computer equipment, computer-readable storage medium and computer program product for the DAG model to address the above technical problems, so as to determine the data link to which the tracking data of the task data belongs when the same business node can be triggered by multiple forward business nodes, so as to facilitate systematic data analysis.

[0005] In a first aspect, the present application provides a data tracking method for a DAG model, wherein the DAG model has a business node, and the method includes:

[0006] When the service node processes the task data to obtain processed task data and transmits the processed task data and the execution identifier corresponding to the service node, acquiring the execution identifier and task execution data corresponding to the service node; the task execution data is description data of the service node processing the task data;

[0007] combining the execution identification with the task execution data into tracking data;

[0008] The data link to which the tracking data belongs is determined according to the execution identifier.

[0009] In one embodiment, the obtaining the execution identifier and task execution data corresponding to the service node includes:

[0010] Acquire an execution identifier based on the task data, and record the execution identifier through the slot allocated by the service node;

[0011] The task execution data is generated through the process of the service node processing the task data.

[0012] In one embodiment, the service node includes a start node, and the task data processed by the start node includes a start event;

[0013] The acquiring the execution identifier based on the task data and recording the execution identifier through the slot allocated by the service node includes:

[0014] When the starting node generates an execution identifier corresponding to the starting node based on the start event, the execution identifier generated by the starting node is recorded through the slot allocated by the starting node.

[0015] In one of the embodiments, the service node includes a non-starting node;

[0016] The acquiring the execution identifier based on the task data and recording the execution identifier through the slot allocated by the service node includes:

[0017] When the task data is transmitted to the non-starting node, obtaining the execution identifier carried by the task data, and combining and recording the execution identifier carried by the task data through the slot allocated by the non-starting node;

[0018] The slot allocated to the non-starting node supports storing a plurality of the execution identifiers.

[0019] In one of the embodiments, the task execution data includes system configuration data for processing the task and configuration data of the DAG model, as well as task process description data;

[0020] The combining the execution identifier and the task execution data into tracking data includes:

[0021] In the case where the system configuration data and the configuration data of the DAG model are used as the information header, the execution identifier and the task process description data are combined into the information body to obtain the dot data;

[0022] Wherein, the information header corresponds to at least one of the information bodies.

[0023] In one embodiment, before the service node processes the task data, the method further includes:

[0024] Obtaining configuration data of the DAG model;

[0025] Generate configuration data of the execution identifier and configuration data of the task execution data according to the configuration data of the DAG model;

[0026] The configuration data of the execution identifier is used to generate the execution identifier, and the configuration data of the task execution data is used to generate the task execution data.

[0027] In a second aspect, the present application further provides a data tracking device for a DAG model, wherein the DAG model has a service node, and the device includes:

[0028] an acquisition module, configured to acquire the execution identifier and task execution data corresponding to the business node when the business node processes the task data to obtain the processed task data and transmits the processed task data and the execution identifier corresponding to the business node; the task execution data is the description data of the business node processing the task data;

[0029] A tracking module, used for combining the execution identifier and the task execution data into tracking data;

[0030] The tracking module is used to determine the data link to which the tracking data belongs according to the execution identifier.

[0031] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of data tracking in any of the above embodiments are implemented.

[0032] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of data tracking in any of the above embodiments are implemented.

[0033] In a fifth aspect, the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of data tracking in any of the above embodiments are implemented.

[0034] The above-mentioned data tracking method, device, computer equipment, computer-readable storage medium and computer program product for the DAG model, because the business nodes in the DAG model satisfy a one-way dependency relationship, and the direct dependency relationships of all business nodes will not form a loop, can realize the description of the specific execution process and logic of the business system, so that the same business node can be triggered by multiple business nodes in its forward direction. Furthermore, when the business node processes the task data to obtain the processed task data, and transmits the processed task data and the execution identifier corresponding to the business node, the execution identifier and task execution data corresponding to the business node are obtained; the task execution data is the description data of the business node processing the task data, which can accurately reflect the specific process of the business node executing the task, and the execution identifier can be used to mark the executed nodes in the logical link, laying the foundation for real-time dynamic tracking of the data function processing process. On this basis, according to the execution identifier, the tracking data of the data link to which the task execution data belongs is determined, so as to comprehensively and accurately trace the business execution process under each data link for the task execution data under each data link. Since this data tracking process does not require additional data processing by the business nodes, there is no need to modify the business. Instead, tracking data is generated through the underlying platform or professional analysis software, thus forming non-invasive data tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 An application environment diagram of a data tracing method for a DAG model in one embodiment;

[0037] Figure 2 is a flow chart of a data tracking method for a DAG model in one embodiment;

[0038] Figure 3 A flowchart of the steps of executing identification and acquiring task execution data in one embodiment;

[0039] Figure 4 is a flow chart of the system design phase in one embodiment;

[0040] Figure 5 A schematic diagram of a process of starting the initialization and operation phases in one embodiment;

[0041] Figure 6 A schematic diagram of a data analysis process in one embodiment;

[0042] Figure 7 A schematic diagram of the relationship between execution nodes in an embodiment;

[0043] Figure 8 is a structural block diagram of a data tracking device for a DAG model in one embodiment;

[0044] Fig. 9 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0046] The data tracking method for the DAG model provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the terminal 102 communicates with the server 104 through a network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or placed on the cloud or other network servers.

[0047] The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. Head-mounted devices may be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. The server 104 may be an independent physical server, or a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides cloud computing services.

[0048] In an exemplary embodiment, Figure 2 As shown in Figure 1, a data tracking method for the DAG model is provided. Figure 1 The terminal 102 in FIG. 1 is used as an example to illustrate the application of the DAG model with service nodes, including the following steps 202 to 206. Among them:

[0049] Step 202, when the business node processes the task data to obtain the processed task data and transmits the processed task data and the execution identifier corresponding to the business node, obtain the execution identifier and task execution data corresponding to the business node; the task execution data is the description data of the task data processed by the business node.

[0050] The DAG model (Directed Acyclic Graph) is a special graph structure that consists of multiple business nodes and directed edges used to characterize data links. Each directed edge points from one business node to another business node. The business nodes satisfy a unidirectional dependency relationship, and the direct dependency relationships of all business nodes will not form a loop. Specifically, in the DAG model, starting from any business node, it is impossible to return to the original business node through these directed edges, that is, the DAG model does not contain any loops. Each complete business processing flow is described by a DAG graph. Each business node in the DAG model can trigger multiple subsequent business nodes with which it has a data link, forming a fork structure, and different business nodes can also trigger the same subsequent business node to form a convergence structure. For example: the business node of the DAG model has multiple starting nodes, and each starting node can trigger multiple subsequent business nodes with which it has a data link, forming a fork structure; at the same time, different starting nodes can also trigger the same subsequent business node to form a convergence structure.

[0051] A business node is a functional node in the task data processing process, which processes its own task data to obtain processed data with corresponding functions. The data processing process of each business node can be executed according to the business processing logic corresponding to the business node to complete the corresponding data processing process. Optionally, the business node is a logical node, and the data link between different logical nodes is implemented according to the data dependency relationship in the DAG model. The execution identifier can be represented by "traceId".

[0052] Task data is the data required to execute tasks according to the business logic of the business node. When the business node is the starting node, the task data includes the corresponding start event; when the business node is a non-starting node, the task data is the task data transferred to the non-starting node, that is, the processed task data of the forward business node adjacent to the non-starting node. The processed task data is the result obtained by the business node based on its own task data.

[0053] Optionally, the task execution data includes at least static data and dynamic data, the static data includes the system information header and the system DAG graph, and the dynamic data includes the processing result of the service node itself, and the processing result of the service node itself is the system service tracking information. The task execution data includes data generated by task data processing, such as data used to characterize the system service tracking information.

[0054] Task data is processed by each service node, and the processed task data of each service node is used as the task data corresponding to the subsequent service node. Optionally, a data link has a first service node and a second service node that transmit data in sequence; the data processed according to the service logic corresponding to the first service node is the task data of the first service node; and the processed task data of the first service node is transmitted to the second service node and used as the task data of the second service node; in this case, the first service node is the forward service node adjacent to the second service node, and the second service node is the subsequent service node adjacent to the first service node.

[0055] Optionally, the system information header includes system hardware information, operating system information, and scheduling environment information; the system DAG diagram includes data links, that is, the dependency diagram between service nodes that satisfies the DAG. The processing results of the forward service node or the processing results of the service node itself include the scheduling start information of the service node, the start information of task data transmission, system resource occupancy information, etc.

[0056] The execution identifier is used to indicate the data link where the task data is located, so the system operation status can be tracked through each data link indicated by the execution identifier. Since the execution identifier corresponds to the business node, and each business node is used to implement a function in the business process, the recorded information can be analyzed through the execution identifier to track the behavior of the system operation. Optionally, the business node includes a starting node and a non-starting node. The execution identifier is generated for the starting node and is received and transmitted by the non-starting node. When the non-starting node receives multiple execution identifiers, a converged structure is formed and corresponding data processing is performed.

[0057] Optionally, the task data processed by the business node is transmitted, including the link coloring process. Specifically, the execution identifier of the series connection can be transparently transmitted to clarify which data link is being executed and which business node it has been executed to.

[0058] In an optional embodiment, the process of processing task data by the terminal 102, server 104 or other electronic device business node is to transmit the processed task data of the business node and the execution identifier corresponding to the business node, and the terminal 102 obtains the execution identifier and task execution data corresponding to the business node.

[0059] In an optional embodiment, the business processing thread or business processing process in the terminal 102, the process of the business node processing task data, transmits the processed task data of the business node and the execution identifier corresponding to the business node, and the underlying platform thread in the terminal 102 obtains the execution identifier and task execution data corresponding to the business node.

[0060] In an optional embodiment, obtaining the execution identifier and task execution data corresponding to the business node includes: obtaining the execution identifier corresponding to the business node from the task data of the business node; after processing the task data according to the business logic corresponding to the business node, obtaining the task execution data according to the change data of the task data.

[0061] Step 204: Combine the execution identifier and the task execution data into tracking data.

[0062] The tracking data is the dot data of the data link indicated by the execution identifier. Since the tracking data is converted according to the execution identifier, each data link can be determined by the execution identifier, and when different data links converge to the same node, the data link can be analyzed in detail based on the task execution data.

[0063] Since tracking data is dot data, and dot data is generated by business nodes with data changes, the order of appearance of each dot data is not arranged according to the business nodes. In this case, combining the execution identifier and the task execution data into tracking data can make the dot data at different times determine the data link to which each tracking data belongs, so as to achieve traceability.

[0064] In an optional embodiment, combining the execution identifier and the task execution data into the tracking data includes: using the execution identifier and the task execution data as the information body of the static data information header to obtain a data packet or a data frame containing the tracking data.

[0065] Step 206: Determine the data link to which the tracking data belongs according to the execution identifier.

[0066] The data link is a data dependency relationship indicated by an execution identifier, which can be represented by a DAG model. Optionally, the data link is a combination of multiple business nodes, which is used to represent business rules; business rules are the execution relationships between various logical nodes; execution relationships include but are not limited to serial, parallel, conditional branches, etc., which are used to clarify the data processing path under the premise of following the data dependency relationship in the DAG model.

[0067] Optionally, both the business nodes and data links are determined based on DSL (domain-specific language) to focus on the program field under the corresponding business scenario. DSL is a computer language specially designed to solve a certain type of task. For example, a series of business nodes can be defined through JSON or XML, and the combination relationship of business nodes can be defined to form business rules. Then, by defining logical nodes and business rules, a logical link, i.e., a business link, is obtained, thereby realizing the description of the specific execution process and logic of the business system; and then by obtaining the execution identifier, the executed nodes in the logical link are marked, laying the foundation for real-time dynamic tracking of the business execution process.

[0068] In an optional embodiment, determining the data link to which the tracking data belongs according to the execution identifier includes: determining the data link indicated by the execution identifier among the candidate data links in the DAG model, so as to analyze each data link, or displaying the data for analysis according to the data link.

[0069] In an optional embodiment, displaying the data for analysis according to the data link includes: displaying the execution progress of the data link according to the position of the service node in the data link. For example, a line graph is displayed with time as the horizontal axis and the completion degree of the data link as the vertical axis; through the line graph, the situation and time when the task execution of the data link reaches the corresponding percentage are displayed.

[0070] In an optional embodiment, displaying data for analysis according to data links includes: determining a thread for performing data processing on each data link according to each execution identifier; and displaying the progress of each data link according to the type of visualized data of each thread.

[0071] Optionally, the data tracking method in this embodiment is non-intrusive and does not require business modification. The underlying platform automatically generates tracking information, selects a public ID as an identifier in combination with business logic, and fully implements the logical link concatenation and on-site execution restoration of the audit scenario. Optionally, the specific steps of restoration include five steps: link definition, link coloring, link reporting, link storage, and link display.

[0072] Optionally, the process of obtaining tracking data can be called the system tracking function. After the system tracking function exports the tracking information recorded in the previous step to the PC, the tracking data can be analyzed through tools to achieve comprehensive traceability of the business execution process. Charts and other UI methods can be used to interact with users to present the system operation information required by users from different dimensions.

[0073] In the above-mentioned data tracking method for the DAG model, since the business nodes in the DAG model satisfy a one-way dependency relationship, the direct dependency relationships of all business nodes will not form a loop, which can realize the description of the specific execution process and logic of the business system, so that the same business node can be triggered by multiple business nodes in its forward direction. Furthermore, when the business node processes the task data to obtain the processed task data, and transmits the processed task data and the execution identifier corresponding to the business node, the execution identifier and task execution data corresponding to the business node are obtained; the task execution data is the description data of the business node processing the task data, which can accurately reflect the specific process of the business node executing the task, and the execution identifier can be used to mark the executed nodes in the logical link, laying the foundation for real-time dynamic tracking of the data function processing process. On this basis, according to the execution identifier, the tracking data of the data link to which the task execution data belongs is determined, so as to comprehensively and accurately trace the business execution process under each data link for the task execution data under each data link. And since this data tracking process does not require the business node to generate additional data processing, it does not require the business to be modified, but generates tracking data through the underlying platform or professional analysis software, thus forming non-intrusive data tracking.

[0074] In an exemplary embodiment, Figure 3 As shown, the execution identifier and task execution data corresponding to the business node are obtained, including:

[0075] Step 302: Obtain an execution identifier based on the task data, and record the execution identifier through the slot allocated by the service node.

[0076] A slot is a data structure allocated to a service node, and the data structure is used to store the above-mentioned execution identifier. Optionally, a slot includes but is not limited to a data structure such as an array, a linked list, a queue, a stack, etc. Optionally, the number of slots allocated to a service node can be determined according to the number of candidate data links where the service node is located, so as to ensure that the slot can record a sufficient number of execution identifiers.

[0077] In an optional embodiment, obtaining the execution identifier based on the task data includes: obtaining the execution identifier in the task data processing result of the business node. Thus, the business node dynamically generates the corresponding execution identifier to ensure that the data of the DAG model can be tracked when the task data changes of different business nodes are adaptively adjusted.

[0078] In an optional embodiment, obtaining the execution identifier based on the task data includes: extracting the execution identifier corresponding to the service node from the task data transmitted to the service node. Thus, some service nodes do not generate corresponding execution identifiers themselves, but directly extract corresponding execution identifiers to ensure that the number of data links is relatively small.

[0079] In an optional embodiment, the business node includes a starting node, and the task data processed by the starting node includes a startup event; correspondingly, an execution identifier is obtained based on the task data, and the execution identifier is recorded through a slot allocated by the business node, including: when the starting node generates an execution identifier corresponding to the starting node based on the startup event, the execution identifier generated by the starting node is recorded through the slot allocated by the starting node.

[0080] The start node is a business node that performs corresponding logic processing based on the start event. The processed task data of the start node is generated when the start event is detected, and the processed task data of the start node includes the execution identifier corresponding to the start node. The start event is an event corresponding to the start node. Optionally, the start event can be a time point or a user-triggered instruction.

[0081] The slot allocated by the starting node is used to record the execution identifier generated by the starting node itself. Optionally, the amount of data supported by the slot allocated by the starting node can match its own execution identifier; optionally, the amount of data supported by the slot allocated by the starting node can be preset.

[0082] In an optional embodiment, the starting node generates an execution identifier corresponding to the starting node based on the start event, including: when the thread used to run the starting node detects the start event, the program corresponding to the starting node that matches the start event generates the processed task data of the starting node. Optionally, the program corresponding to the starting node uses the processed task data of the starting node as the task data of the subsequent service node adjacent to the starting node according to the data link therein.

[0083] In an optional embodiment, detecting a start event includes: determining that the start event is triggered according to a comparison result of a duration recorded by a timer and a preset duration.

[0084] In this embodiment, as the starting node, its corresponding business logic can generate its corresponding execution identifier based on the startup event, so that the generation process of the execution identifier is automated, and the execution identifier for analysis can be dynamically constructed according to different events. When the execution identifier is obtained, the execution identifier generated by the starting node is recorded through the slot allocated by the starting node to trace back to the source of the data link, so as to analyze the data more accurately.

[0085] In an optional embodiment, the business node includes a non-starting node. Correspondingly, the execution identifier is obtained based on the task data, and the execution identifier is recorded through the slot allocated by the business node, including: when the task data is transmitted to the non-starting node, the execution identifier carried by the task data is obtained, and the execution identifier carried by the task data is combined and recorded through the slot allocated by the non-starting node; wherein the slot allocated by the non-starting node supports the storage of multiple execution identifiers.

[0086] The non-starting node is a service node that processes data based on the task data delivered to the non-starting node. The processed task data of the non-starting node is generated when the task data is delivered to the non-starting node, and the processed task data of the non-starting node includes the execution identifier corresponding to the non-starting node. The execution identifier contained in the processed task data of the non-starting node includes the execution identifier contained in the task data delivered to the non-starting node.

[0087] The slot allocated to the non-starting node supports storing multiple execution identifiers. Optionally, the slot allocated to the non-starting node stores one or more execution identifiers to indicate that the non-starting node can be applied to data tracing of the data link.

[0088] In an optional embodiment, obtaining the execution identifier carried by the task data includes: detecting data transmitted to a non-starting node; and if the task data is detected, obtaining the execution identifier carried by the task data.

[0089] In an optional embodiment, the execution identifiers carried by the task data are combined and recorded through the slots allocated by the non-starting nodes, including: splicing the execution identifiers carried by the task data in sequence through the slots allocated by the non-starting nodes to obtain the spliced ​​execution identifiers; and recording the spliced ​​execution identifiers.

[0090] In an optional embodiment, the execution identifier carried by the task data is combined and recorded through the slots allocated to the non-starting nodes, including: in the slots allocated to the non-starting nodes, the execution identifier carried by the task data is recorded according to the preset combination position corresponding to each starting node in the slot; wherein the preset combination position corresponds to the node identifier corresponding to each starting node.

[0091] In this embodiment, the non-starting node does not create a new execution identifier, but combines the execution identifiers carried by the task data transmitted to itself, thereby reducing the total number of execution identifiers and avoiding the situation where the non-starting node is started due to abnormality. Since the slots allocated to the non-starting node support the storage of multiple execution identifiers, and the slots allocated to the non-starting node combine and record the execution identifiers carried by the task data, each data link converged to the non-starting node can be accurately determined.

[0092] In an optional embodiment, the business node includes a starting node and a non-starting node. Correspondingly, the execution identifier is obtained based on the task data, and the execution identifier is recorded through the slot allocated by the business node, including: if the execution identifier corresponding to the starting node is generated based on the start event corresponding to the starting node, the execution identifier generated by the starting node is recorded through the slot allocated by the starting node; if the task data is transferred to the non-starting node, the execution identifier carried by the task data is combined and recorded through the slot allocated by the non-starting node; wherein the slot allocated by the non-starting node supports the storage of multiple execution identifiers.

[0093] Step 304, generating task execution data through the process of processing task data by the business node.

[0094] The processing result is obtained by processing the task data received by the service node according to the service processing program of the service node. The processing result includes the identification of the service node and the data for executing the corresponding function of the service node.

[0095] In an optional embodiment, task execution data is generated through the process of processing task data by the service node, including: the identifier of the thread called by the process of processing task data by the service node, the start time point, the end time point, the time period, etc., to determine the task execution data of the service node. Thus, the thread used to execute the task data, the time point when the thread starts or ends, and other related data can be tracked.

[0096] In an optional embodiment, the task execution data is generated through the process of processing the task data by the business node, including: searching for the task execution data of the business node in the processed task data according to the preset type and the user-defined type.

[0097] In this embodiment, the execution identifier is obtained based on the task data, and the task data is processed according to the logic of the business node, thereby laying the foundation for the execution identifier to be transmitted along the data link; on this basis, the execution identifier in the task data processing result is recorded through the slot allocated by the business node, so that the corresponding business node has a data structure specifically used to store the execution identifier, so that when multiple business nodes cannot trigger the same subsequent business node separately, the corresponding execution identifier can be recorded more accurately. At the same time, the process of processing task data and generating task execution data can accurately determine the changes in data, so as to accurately determine the task execution situation for the corresponding task.

[0098] In an optional embodiment, the task execution data includes system configuration data and DAG model configuration data for processing the task, as well as task process description data. Correspondingly, the execution identifier and the task execution data are combined into tracking data, including: in the case where the system configuration data and the DAG model configuration data are used as information headers, the execution identifier and the task process description data are combined into an information body to obtain dot data; wherein the information header corresponds to at least one information body.

[0099] System configuration data is used to describe the system that executes the service node to determine the hardware system for data execution. System configuration data includes but is not limited to system hardware information, operating system information, and scheduling environment information. Exemplarily, information header 1 includes system hardware information, which includes but is not limited to CPU lnfo: CPU processor information, including the number of processors, model, etc.; NPU Info: NPU processor information, including the number of processors, model, etc.

[0100] The configuration data of the DAG model is used to describe the data links of the service nodes in the DAG model to determine the data dependency of each data link. Optionally, the configuration data of the DAG model can exist in the form of a graph or a table. For example, the configuration data of the DAG model can be a DAG graph that satisfies the DAG dependency between service nodes.

[0101] The system configuration data and the configuration data of the DAG model are both static data, which are used as the system header. The header is used to provide static data to describe the system information and the overall data link for data processing by at least one information subject. The header is the header structure of a data packet or a data frame.

[0102] The task process description data is dynamic data that changes as the task data of the business node is processed. The task process description data is used to describe the task processing process of the business node for easy traceability. Optionally, the task process description data is used to indicate the scheduling start information of the business node, the start information of the business data transmission, the system resource occupancy information, the running time, the start or end of the message transmission, and other information.

[0103] The information body contains the dynamic change data of the business process. Since the information body is obtained by combining the execution identifier and the description data, the description data can be determined by the execution identifier in the information body, so as to be analyzed through the data link to which each information body belongs.

[0104] Exemplarily, the system information header includes information header 1 and information header 2. Information header 1 records the hardware and system, and information header 2 is the DAG configuration data. The data in information header 1 and information header 2 are both static data. For the information body inside the data packet, it includes information bodies such as point 1 and point 2, so as to perform system tracking and analysis on each business node, such as running time, the start or end of message transmission, etc.

[0105] Exemplarily, point 1 is the information body, which includes the execution identifier TraceID and the task process description data. The information body contains the following content:

[0106] TraceID: serves as the execution identifier of the system tracing transaction;

[0107] Hardware:

[0108] ProcessorType: processor type, including CPU / MPU / CVE, etc.

[0109] ProcessorID: Processor ID: CPU0 / CPU1;

[0110] n Business information (Runnable is the smallest unit of task scheduling);

[0111] NodeID: corresponds to the node of the DAG configuration file;

[0112] RunnableID: corresponds to the Runnable in the DAG configuration file;

[0113] ReaderTopic: Reader and topic corresponding to Runnable;

[0114] WriterTopic: the writer and topic corresponding to Runnable;

[0115] State: NodeID execution status: Start, End.

[0116] n Performance information:

[0117] TimeStamp: The timestamp of the scheduled task, which is consistent within the chip.

[0118] n Extended information: (optional)

[0119] Userinfo: optional extended information of the middleware specified by the user, used for performance analysis of the middleware data that the user is concerned about.

[0120] In an optional embodiment, the execution identifier and the task process description data are combined into an information body to obtain dot data, including: combining each execution identifier and its corresponding task process description data into each information body to obtain dot data.

[0121] In an optional embodiment, the execution identification and the task process description data are combined into an information body to obtain dot data, including: combining the execution identification and the task process description data into an information body; combining the information bodies in time sequence to obtain dot data.

[0122] In this embodiment, the system configuration data and the configuration data of the DAG model are used as the information header of the dot data, so that one or more dynamically changing information bodies can use the same information header, ensuring relatively high efficiency of data transmission. At the same time, the execution identifier and the description data are combined into the information body, so that each description data can be traced back to the data link to which it belongs under high efficiency.

[0123] In an optional embodiment, before the business node processes the task data, the method also includes: obtaining configuration data of the DAG model; generating configuration data of the execution identifier and configuration data of the task execution data based on the configuration data of the DAG model; wherein the configuration data of the execution identifier is used to generate the execution identifier, and the configuration data of the task execution data is used to generate the task execution data.

[0124] The configuration data of the execution identifier is used to configure the generation process of the execution identifier. Optionally, the configuration data of the execution identifier can be used to determine the number of execution identifiers, the corresponding relationship between the execution identifier and the service node, the number of slots of the execution identifier, and other data.

[0125] The configuration data of the task execution data is used to configure the type of task execution data. Optionally, the configuration data of the task execution data can involve various node attributes in the business processing flow to adaptively regulate the content involved in the tracking data. The configuration data of the task execution data at least involves the data processed by the business node; illustratively, the configuration data of the task execution data may involve but is not limited to the number of aggregation nodes, the maximum aggregation degree of the aggregation node, etc.; wherein the aggregation node corresponds to a non-starting node of multiple execution identifiers.

[0126] In an optional embodiment, obtaining the configuration data of the DAG model includes: obtaining the configuration data of the DAG model in the initialization phase. Exemplarily, during the system startup initialization process, by analyzing the static graph, the system can pre-acquire static information such as the number of starting nodes, the number of aggregation nodes, and the maximum aggregation degree of aggregation nodes in the business processing flow; wherein the aggregation node is a non-starting node that combines multiple execution identifiers. Based on the above static information, the system assigns execution identifiers and rules to each starting node in the initialization phase.

[0127] In this embodiment, the configuration data of the DAG model is obtained; according to the configuration data of the DAG model, the configuration data of the execution identifier and the configuration data of the task execution data are generated, and the corresponding foundation is laid through the configuration data, which helps to more efficiently determine the processing flow of the task data according to the business node.

[0128] In a specific embodiment, Figure 4 As shown in the figure, in the system design phase, the trigger relationship definitions between business nodes and business nodes are obtained to describe the dynamic operation of the business through the static graph of DAG. The description graph of business nodes and relationships only needs to meet the rules of DAG graph and be directed acyclic.

[0129] During the system startup initialization process, by analyzing the static graph, the system can pre-acquire information such as the number of starting nodes, the number of aggregation nodes, and the maximum aggregation degree of the aggregation nodes in the business processing flow. Based on the above static information, the system assigns execution identifiers and rules to each initial node during the initialization phase.

[0130] During operation, the system distributes, transfers, splices and tracks the execution identification of the starting node, aggregation node and other nodes in the logical link, realizes the tracking of the dynamic execution process of the business, and records the tracking information of each business node in a non-intrusive way. Figure 5 As shown, after the system is started and initialized, the business DAG graph information and system information are obtained, and the information required for dynamic tracking is generated according to the above information, including the number of tracking IDs matching the DAG graph, the tracking ID position system information, etc.; then, the system runs; during the system operation, the system tracking function dynamically generates the tracking information in the system DAG graph scheduling process according to the above static information, and records and saves the above tracking information, DAG graph information, and system information.

[0131] Finally, if Figure 6 As shown in the figure, by analyzing the dot data tracked by the system through tools, a comprehensive traceability of the business execution process can be achieved.

[0132] In an exemplary embodiment, Figure 7As shown in (a), there are starting nodes Node1, Node5, Node9, and non-starting node Node11, wherein the execution identifiers of starting nodes Node1, Node5, and Node9 are all TraceID, and the number of slots (ID) can be 6 / 7 / 8.

[0133] Node represents a business node that processes business based on a DAG graph. Business processing functions are represented by nodes, and data dependencies between business processing functions are represented by dependencies between nodes. Nodes are divided into start nodes and non-start nodes. Start nodes represent data that does not depend on any other nodes, and are usually started and run at a scheduled time; non-start nodes are nodes that depend on data provided by other nodes to start running.

[0134] TraceID is an execution identifier used to track the system's operating status. It is dynamically generated by the start Node in the DAG graph when it is running, and is passed to the following Nodes as data is sent and received between Nodes. By recording this series of Nodes, data behaviors, and corresponding TraceIDs, the system's operating behavior can be tracked by analyzing the recorded information.

[0135] TraceID Slot is a slot allocated to all starting nodes in the DAG graph for recording TraceID. When multiple nodes converge to one node, the slot set of TraceID transparently transmitted by multiple converged input data of one node is used as the TraceID corresponding to the node, and is used to pass data to subsequent nodes.

[0136] like Figure 7 As shown in (b), Figure 7 The execution identifiers contained in (a) are:

[0137] Node1 TraceID=0X0006000000000....0000000;

[0138] Node5 TraceID=0X000000070000....0000000;

[0139] Node9 TraceID=0X000000000008....0000000.

[0140] like Figure 7As shown in (c), the non-starting node Node11 is also the successor node of the starting node Node1, the starting node Node5, and the starting node Node9. Therefore, the execution identifier corresponding to the non-starting node Node11 is represented as: Node11 TraceID=0X000600070008....0000000.

[0141] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0142] Based on the same inventive concept, the embodiment of the present application also provides a data tracking device for a DAG model for implementing the data tracking method for a DAG model involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more embodiments of the data tracking device for a DAG model provided below can refer to the limitations of the data tracking method for a DAG model above, and will not be repeated here.

[0143] In an exemplary embodiment, Figure 8 As shown, a data tracking device for a DAG model is provided, wherein the DAG model has a business node, and the device includes:

[0144] The acquisition module 802 is used to acquire the execution identifier and task execution data corresponding to the business node when the business node processes the task data to obtain the processed task data and transmits the processed task data and the execution identifier corresponding to the business node; the task execution data is the description data of the business node processing the task data;

[0145] A tracking module 804, configured to combine the execution identifier and the task execution data into tracking data;

[0146] The tracking module 804 is used to determine the data link to which the tracking data belongs according to the execution identifier.

[0147] In one embodiment, the acquisition module 802 is used to:

[0148] Acquire an execution identifier based on the task data, and record the execution identifier through the slot allocated by the service node;

[0149] The task execution data is generated through the process of the service node processing the task data.

[0150] In one embodiment, the service node includes a start node, and the task data processed by the start node includes a start event;

[0151] The acquisition module 802 is used to:

[0152] When the starting node generates an execution identifier corresponding to the starting node based on the start event, the execution identifier generated by the starting node is recorded through the slot allocated by the starting node.

[0153] In one of the embodiments, the service node includes a non-starting node;

[0154] The acquisition module 802 is used to:

[0155] When the task data is transmitted to the non-starting node, obtaining the execution identifier carried by the task data, and combining and recording the execution identifier carried by the task data through the slot allocated by the non-starting node;

[0156] The slot allocated to the non-starting node supports storing a plurality of the execution identifiers.

[0157] In one of the embodiments, the task execution data includes system configuration data for processing the task and configuration data of the DAG model, as well as task process description data;

[0158] The tracking module 804 is used to:

[0159] In the case where the system configuration data and the configuration data of the DAG model are used as the information header, the execution identifier and the task process description data are combined into the information body to obtain the dot data;

[0160] Wherein, the information header corresponds to at least one of the information bodies.

[0161] In one embodiment, before the service node processes the task data, the acquisition module 802 is used to:

[0162] Obtaining configuration data of the DAG model;

[0163] Generate configuration data of the execution identifier and configuration data of the task execution data according to the configuration data of the DAG model;

[0164] The configuration data of the execution identifier is used to generate the execution identifier, and the configuration data of the task execution data is used to generate the task execution data.

[0165] Each module in the above-mentioned data tracking device for the DAG model can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above modules.

[0166] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Fig. 9 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (Near Field Communication, NFC) or other technologies. When the computer program is executed by the processor, a data tracking method for a DAG model is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0167] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0168] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0169] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0170] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0171] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0172] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0173] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0174] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data tracking method for a DAG model, characterized in that: The DAG model has business nodes, and the business nodes include start nodes and non-start nodes. The method is executed by the underlying platform, and the method includes: When the service node processes the task data to obtain processed task data and transmits the processed task data and the execution identifier corresponding to the service node, acquiring the execution identifier and task execution data corresponding to the service node; the task execution data is description data of the service node processing the task data; combining the execution identification with the task execution data into tracking data; determining, according to the execution identifier, a data link to which the tracking data belongs; The task data processed by the starting node includes a start event; and the step of obtaining the execution identifier and task execution data corresponding to the service node includes: When the starting node generates an execution identifier corresponding to the starting node based on the start event, the execution identifier generated by the starting node is recorded through the slot allocated by the starting node; When the task data is transmitted to the non-starting node, the execution identifier carried by the task data is obtained, and the execution identifier carried by the task data is combined and recorded through the slot allocated by the non-starting node; wherein the slot allocated by the non-starting node supports the storage of multiple execution identifiers; The task execution data is generated through the process of the service node processing the task data.

2. The method according to claim 1, characterized in that The task execution data includes system configuration data for processing the task and configuration data of the DAG model, as well as task process description data; The combining the execution identifier and the task execution data into tracking data includes: In the case where the system configuration data and the configuration data of the DAG model are used as the information header, the execution identifier and the task process description data are combined into the information body to obtain the dot data; Wherein, the information header corresponds to at least one of the information bodies.

3. The method according to claim 1, characterized in that Before the service node processes the task data, the method further includes: Obtaining configuration data of the DAG model; Generate configuration data of the execution identifier and configuration data of the task execution data according to the configuration data of the DAG model; The configuration data of the execution identifier is used to generate the execution identifier, and the configuration data of the task execution data is used to generate the task execution data.

4. The method according to claim 1, characterized in that: The processed task data of the starting node includes an execution identifier corresponding to the starting node; and / or the processed task data of the non-starting node includes an execution identifier corresponding to the non-starting node.

5. The method according to claim 1, characterized in that The method further comprises: According to the number of candidate data links where the service node is located, the number of slots allocated to the service node is determined.

6. The method according to claim 1, characterized in that The underlying platform is a terminal or an underlying platform thread.

7. A data tracking device for a DAG model, characterized in that: The DAG model has business nodes, and the business nodes include starting nodes and non-starting nodes. The device is executed by an underlying platform, and the device includes: an acquisition module, for acquiring the execution identifier and task execution data corresponding to the business node when the business node processes the task data to obtain the processed task data and transmits the processed task data and the execution identifier corresponding to the business node; the task execution data is the description data of the task data processed by the business node; the task data processed by the start node includes a start event; A tracking module, used for combining the execution identifier and the task execution data into tracking data; The tracking module is used to determine the data link to which the tracking data belongs according to the execution identifier; In terms of obtaining the execution identifier and task execution data corresponding to the business node, the acquisition module is used to record the execution identifier generated by the starting node through the slot allocated by the starting node when the starting node generates the execution identifier corresponding to the starting node based on the start event; when the task data is passed to the non-starting node, obtain the execution identifier carried by the task data, and combine and record the execution identifier carried by the task data through the slot allocated by the non-starting node; wherein the slot allocated by the non-starting node supports storing multiple execution identifiers; and generate task execution data through the process of processing the task data by the business node.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images