Intelligent index blood relationship tracing interpretation method and system based on atomic computing unit

Abstract

The present application relates to the technical field of semantic retrieval, in particular to an intelligent index blood relationship tracking interpretation method and system based on an atomic computing unit, comprising: establishing an atomic computing unit and input-output mode, configuration parameters and execution engine binding relationship, establishing a blood relationship semantic template according to data dependency, analyzing a blood relationship request to form a parameterized index calculation instance, executing and recording data sources and input-output quantities according to a topological sequence, forming a directed blood graph and a result blood relationship path; inputting the result data item features and the corresponding path nodes and directed edge features into a large model composed of a graph attention coding network, a feature fusion network and a Transformer decoder, forming a path representation according to the directed edge aggregation, and generating a blood relationship interpretation result containing data sources, execution sequence and result formation process after splicing the result features. The present application makes the index interpretation correspond to the actual calculation link and data source.

Description

Technical Field

[0001] This invention relates to the field of semantic retrieval technology, and in particular to a method and system for interpreting intelligent indicator lineage tracing based on atomic computing units. Background Technology

[0002] The field of semantic retrieval technology mainly revolves around the semantic expression and retrieval process of text and structured information. Typically, the terminal proposes the query content, the server analyzes the meaning of the query by combining the existing data descriptions and index relationships in the database, and then finds relevant information based on semantic correspondences and returns the retrieval results.

[0003] Among them, the traditional intelligent indicator lineage tracing interpretation method refers to the method of tracing and explaining the formation source and processing process of data indicators. It usually reads the corresponding data processing records according to the indicator definition, and then combines the data query statement and task execution process to identify the transmission relationship between the indicator and different data objects. It also traces the data source upward or queries the referenced objects downward along the established dependency path, and finally explains the data objects and calculation process that the indicator passed through according to the processing sequence.

[0004] Traditional indicator lineage tracing relies on indicator definitions to read existing data processing records and combines query statements and task execution processes to identify the transmission relationships between data objects. Lineage relationships are mainly organized based on existing records. When the same indicator contains continuous calculation steps, the input and output changes of each step lack a direct correspondence with the actual execution order. After data objects undergo multiple levels of processing, the sequential relationship needs to be confirmed again. The connection between indicator results and specific processing steps is not clear enough, making it difficult to directly locate the calculation steps the results have gone through and the corresponding original data source during subsequent tracing. Summary of the Invention

[0005] The purpose of this invention is to provide an intelligent indicator lineage tracing and interpretation method and system based on atomic computing units. This method establishes a continuous association between atomic computing units, lineage semantic templates, parameterized indicator calculation instances, execution processes, and directed lineage graphs during the indicator calculation process. This allows input data, output data, and data sources generated during indicator calculation to form lineage records according to actual execution relationships. Furthermore, the final indicator calculation result can be traced back to the corresponding original data source and atomic computing unit along the directed lineage graph. Simultaneously, for each result data item, a graph feature of the result and the corresponding result lineage path is constructed. A large model, including a graph structure encoding network, a feature fusion network, and an autoregressive text generation network based on a Transformer decoder, aggregates the graph feature according to directed edges, splices the result feature and path representation, and generates a lineage interpretation result containing the original data source, the execution order of atomic computing units, and the result formation process. This results in a verifiable lineage tracing interpretation based on the actual indicator calculation process.

[0006] To achieve the above objectives, this invention provides a method for interpreting intelligent indicator lineage tracking based on atomic computing units, comprising the following steps: Atomic computing units are established, and input data patterns, output data patterns, configuration parameters, and execution engine binding relationships are defined for each atomic computing unit. The atomic computing unit is determined from the calculation logic of predefined indicators. For computing stages that can be independently called by the execution engine and have definite input data structures, output data structures, and the position of the query conditions, the corresponding computing stage is identified as the atomic computing unit. The input data pattern is determined based on the data structure entering the computing stage; the output data pattern is determined based on the data structure formed after processing by the computing stage; the configuration parameters are determined based on the position of the query conditions in the computing stage; and the execution engine binding relationship is established based on the execution engine's call correspondence to the computing stage.

[0007] A lineage semantic template is established based on the data dependencies between the atomic computing units. For multiple atomic computing units required to complete the same predefined indicator, when the output data generated by one atomic computing unit is received as input data by another atomic computing unit, the data dependency is established between the two atomic computing units, and the sequential relationship between each atomic computing unit is determined according to the data dependency. The atomic computing units required to complete the same predefined indicator and their corresponding data dependencies are written into the same lineage semantic template.

[0008] The lineage request is parsed, and the target indicator and query conditions are determined from the parsed content of the lineage request. The target indicator is matched with the indicator identifier corresponding to each of the lineage semantic templates. When the target indicator matches one of the indicator identifiers, the corresponding lineage semantic template is selected, and the corresponding atomic computing unit and the configuration parameters to be updated are determined according to the query conditions. The query conditions are then written into the corresponding configuration parameters to form a parameterized indicator calculation instance. When the target indicator does not match any of the indicator identifiers, the generation of the parameterized indicator calculation instance is stopped.

[0009] Based on the data dependencies in the parameterized metric calculation instance, the corresponding atomic computing units are executed in topological order, and the execution engine is invoked according to the execution engine binding relationship. The execution engine processes input data that conforms to the input data pattern according to the configuration parameters, and forms output data according to the output data pattern. When the atomic computing unit has downstream data dependencies, the corresponding output data is used as the input data of the downstream atomic computing unit. When the atomic computing unit does not form the downstream data dependencies, the corresponding output data is determined as the metric calculation result.

[0010] Furthermore, in the lineage semantic template, a set of direct predecessor atomic computing units that have the data dependency with the corresponding atomic computing unit is recorded for each atomic computing unit. When the set of direct predecessor atomic computing units is empty, the corresponding atomic computing unit is started; when the set of direct predecessor atomic computing units is not empty, the corresponding atomic computing unit is started after each atomic computing unit in the set has generated the output data. Multiple atomic computing units that simultaneously meet the start conditions and have no data dependency with each other are executed in parallel, and after new output data is generated, the start conditions of the atomic computing units that have not yet been executed are checked again until all the atomic computing units in the lineage semantic template are completed.

[0011] When each atomic computing unit receives input data, it records the data source and the amount of input data. When the atomic computing unit completes data processing and generates output data, it records the output data amount and execution time, and writes the data source, data time range, and configuration parameters actually accessed by the current atomic computing unit into the same lineage execution record. Using the atomic computing unit as a lineage node, directed edges are established according to the actual transmission relationship where the output data generated by one atomic computing unit is received as input data by a downstream atomic computing unit, and the lineage execution record is written into the corresponding lineage node, thereby forming a directed lineage graph.

[0012] Furthermore, when the input data received by a lineage node comes from two or more data sources, data source nodes are established according to the actual access data sources recorded in the lineage execution record, and each data source node is associated with the lineage node that receives the corresponding input data. When the output data generated by the lineage node is received by a downstream lineage node, the corresponding input association is continuously recorded with the directed edge connecting the lineage node and the downstream lineage node, so that the data flow relationship corresponding to different data sources enters the same directed lineage graph.

[0013] For each result data item in the index calculation results, the corresponding terminal atomic computing unit is determined based on the output data that forms the result data item. The atomic computing units and corresponding data sources that participated in forming the result data item are searched backwards from the terminal atomic computing unit along the data transmission direction in the directed lineage graph, and a result lineage path is formed according to the connection order between the searched nodes. The result data item is then associated with the corresponding result lineage path. For each result data item, the discrete fields in the index identifier, result value, and query conditions are mapped to d-dimensional result features after embedding mapping, and the numerical fields are normalized. For the corresponding result lineage path, the lineage node type, atomic computing unit identifier or data source identifier, input / output volume, execution time, data time range, and configuration parameters are encoded as node features, and the directed edge direction is encoded as edge features. The large model includes a graph structure encoding network consisting of at least two graph attention layers and path pooling layers, a feature fusion network consisting of fully connected mapping layers, and an autoregressive text generation network based on a Transformer decoder. The graph-structured encoding network performs adjacent node attention aggregation according to the directed edge direction and performs path pooling according to the node connection order of the resulting lineage path to form a d-dimensional path representation; the feature fusion network concatenates the d-dimensional result features and the d-dimensional path representation in the order of result features first and path representation last to form a 2d-dimensional vector, which is then passed through the fully connected mapping layer to form a d-dimensional joint feature, where d is the hidden layer dimension of the large model; the autoregressive text generation network uses the joint feature as a prefix context to generate a lineage explanation result on a word-by-word basis, including the original data source, the execution order of atomic computing units, configuration parameters, and the formation process of the result data items.

[0014] Further, when the query conditions include a query time range, for the atomic computing unit executing time filtering, the actual data coverage range is determined based on the time field in the input data. When the actual data coverage range covers the query time range, the output data generated by the atomic computing unit executing time filtering is sent to the downstream atomic computing unit according to the data dependency; when the actual data coverage range does not cover the query time range, the atomic computing unit executing time filtering is marked as having data integrity issues in the directed lineage graph, and the actual data coverage range is determined as the available query time range. After forming the available query time range, an adjustment instruction for the query time range is received, the adjusted query time range is obtained from the adjustment instruction, and the query time range in the original query conditions is replaced with the adjusted query time range to generate the adjusted query conditions. When the adjusted query time range falls within the available query time range, the configuration parameters are updated with the adjusted query conditions, and the atomic computation units that filter the execution time and the downstream atomic computation units connected according to the data dependency are re-executed; when the adjusted query time range does not fall within the available query time range, the data integrity exception flag is maintained and re-execution is stopped.

[0015] To address the aforementioned methods, this invention also provides an intelligent indicator lineage tracking and interpretation system based on atomic computing units. The system includes an atomic computing unit library, a lineage semantic template unit, a dynamic execution engine, a lineage tracking unit, and an interpretable output unit. The atomic computing unit library stores the binding relationships between atomic computing units and execution engines. The lineage semantic template unit establishes lineage semantic templates according to the data dependencies between the atomic computing units, selects the lineage semantic template according to a lineage request, and uses the query conditions in the lineage request to form configuration parameters and parameterized indicator calculation instances. The dynamic execution engine executes the atomic computing units according to the topological order determined by the parameterized indicator calculation instances, and calls the execution engines according to the execution engine binding relationships. It processes input data with the configuration parameters to form output data. When downstream data dependencies exist, the output data is used as downstream input; when no downstream data dependencies exist, the output data is used as the indicator. The calculation results are as follows: the lineage tracing unit records the data source and input / output quantities of the atomic computing unit during execution, and writes them into the directed lineage graph according to the execution order; the interpretable output unit includes a feature construction module and a large model, the large model includes a graph structure encoding network, a feature fusion network and an autoregressive text generation network based on a Transformer decoder, the feature construction module forms the result features and the graph features of the corresponding result lineage paths, the graph structure encoding network aggregates node features according to directed edges to form path representations, the feature fusion network concatenates the result features and the path representations to form joint features, and the autoregressive text generation network generates lineage interpretation results based on the joint features.

[0016] Compared with the prior art, the advantages and positive effects of the present invention are as follows: This invention organizes the index calculation process into atomic computing units with input data patterns, output data patterns, configuration parameters, and execution engine binding relationships. Based on the data dependencies between units, a lineage semantic template is established, allowing query conditions parsed from lineage requests to be entered into corresponding configuration parameters and form parameterized index calculation instances. Atomic computing units are executed according to the topological order of data dependencies, while execution events record the data source and input / output quantities, ensuring the directed lineage graph corresponds to the actual calculation order. After constructing result features and corresponding node and edge features for the result lineage path for each result data item, a graph structure encoding network aggregates along directed edges to form path representations. A feature fusion network then concatenates the result features and path representations in a fixed order to form joint features. Finally, a Transformer decoder generates lineage interpretation results based on these joint features, ensuring that the original data source, atomic computing unit execution order, and result formation process in the interpretation text all have corresponding actual calculation link basis. Attached Figure Description

[0017] Figure 1This is the main flowchart of the intelligent indicator lineage tracing interpretation method of the present invention; Figure 2 This is a diagram showing the relationship between the atomic computing units and the lineage semantic template of this invention; Figure 3 This is a topology execution and input / output transfer diagram of the atomic computing unit of this invention; Figure 4 This is the directed kinship map and result tracing relationship diagram of the present invention; Figure 5 This is a diagram of the intelligent indicator lineage tracing and interpretation system architecture of the present invention; Figure 6 This is a diagram illustrating the query time range and data integrity processing of this invention. Detailed Implementation

[0018] The following implementation methods focus on the intelligent indicator lineage tracing and interpretation method and system based on atomic computing units described in this invention. During implementation, atomic computing units are used to carry independent computational steps in the predefined indicator calculation logic. Input data patterns are used to limit the data structure entering the corresponding atomic computing unit, output data patterns are used to limit the data structure formed after processing by the corresponding atomic computing unit, configuration parameters are used to carry the query conditions obtained from the lineage request parsing and applied to specific computational steps, and execution engine binding relationships are used to determine the execution engine executing the corresponding atomic computing unit. Lineage semantic templates are used to record the atomic computing units required to complete the same predefined indicator and the data dependencies between atomic computing units. Parameterized indicator calculation instances are used to represent lineage semantic template instances that have been written into the current query conditions. Directed lineage graphs are used to record the data source relationships, input-output relationships, and execution sequence relationships formed during the actual execution of atomic computing units. Result features are used to represent the indicator identifier, result value, and query conditions corresponding to a single result data item. Graph features are used to represent the node attributes and directed edge directions in the result lineage path corresponding to the result data item; d represents the hidden layer dimension of the large model. The large model consists of a graph structure encoding network, a feature fusion network, and an autoregressive text generation network based on a Transformer decoder. The graph structure encoding network is responsible for forming path representations, the feature fusion network is responsible for concatenating the resulting features and path representations and mapping them to joint features, and the autoregressive text generation network is responsible for generating lineage explanation results word-by-word based on the joint features.

[0019] Example 1 Please see Figures 1 to 4 and Figure 6This embodiment provides a method for intelligent indicator lineage tracing and interpretation based on atomic computing units. This embodiment uses a lineage request as the initial input for this indicator calculation and lineage tracing, uses the predefined indicator calculation logic as the basis for establishing atomic computing units, uses the input-output relationship generated by the actual execution of the atomic computing units as the basis for lineage formation, and uses the indicator calculation results, directed lineage graphs, and the lineage interpretation results generated from both as associated outputs. The method is executed sequentially according to S1 to S5, where S1 completes the establishment of atomic computing units, S2 completes the establishment of lineage semantic templates and the formation of parameterized indicator calculation instances, S3 completes the topological execution and data transfer of atomic computing units, S4 completes the collection of execution records and the formation of directed lineage graphs, and S5 completes the reverse tracing of indicator calculation results, the construction of result features and graph features, graph structure encoding, feature fusion, and the generation of lineage interpretation results.

[0020] S1 establishes atomic computing units, defining input data patterns, output data patterns, configuration parameters, and execution engine binding relationships. The input to S1 is the computational logic of predefined indicators; the processing objects are the various computational stages that make up the predefined indicator computational logic; and the output is an atomic computing unit with clearly defined input boundaries, output boundaries, the position of query conditions, and execution engine call relationships. Once established, subsequent lineage semantic templates, parameterized indicator computation instances, and directed lineage graphs all use the same atomic computing unit as a stable identifier for computational stages, thus ensuring that the execution objects in the computation process and the processing nodes in lineage tracing maintain a correspondence.

[0021] S111: Identify computational steps that can be independently invoked by the execution engine from the computational logic of predefined indicators. During identification, the criteria for boundary determination are whether a computational step has a definite input data structure, can form a definite output data structure after processing, and whether the query conditions have a clear position of action. Computational steps that meet the aforementioned conditions are defined as atomic computational units. The determination of atomic computational units is not solely based on the name of the computational step, but rather on the actual invocation boundary and input-output boundary. Therefore, two adjacent computational steps only form atomic computational units if they can each accept definite input and form definite output. After identification, each atomic computational unit corresponds to a definite processing position in the predefined indicator computational logic, and the subsequent execution engine can execute the atomic computational units according to the corresponding invocation relationship.

[0022] S112: The input data pattern is determined based on the data structure entering the corresponding computation stage, the output data pattern is determined based on the data structure formed after processing by the corresponding computation stage, and the configuration parameters are determined based on the position of the query condition in the corresponding computation stage. When a query condition acts on a certain computation stage, the position of the parameter carrying the query condition is determined as the configuration parameter position of the corresponding atomic computation unit. Therefore, query conditions generated by subsequent lineage requests can be written into the determined atomic computation unit, without being stored separately from the actual indicator computation chain.

[0023] S113: Establish execution engine binding relationships based on the call correspondence between execution engines and computation stages. When an atomic computation unit enters the execution state, the data processing subject of the atomic computation unit is determined according to the corresponding execution engine binding relationship. Then, the determined execution engine receives input data conforming to the input data pattern and corresponding configuration parameters. After execution, output data conforming to the output data pattern is generated. The verifiable state after S1 is completed is: each atomic computation unit has a defined input data pattern, output data pattern, configuration parameter location, and execution engine binding relationship, and can locate the corresponding computation stage from the computation logic of predefined indicators.

[0024] S2 establishes a lineage semantic template based on the data dependencies between atomic computing units, parses the lineage request and selects the lineage semantic template, writes the parsed query conditions into the configuration parameters, and forms a parameterized index calculation instance. The input of S2 includes the atomic computing unit formed in S1, the relationship between the predefined index and the corresponding calculation logic, and the current lineage request. The output is the parameterized index calculation instance corresponding to the current lineage request.

[0025] S211, for the same predefined index, determine the atomic computing units required to complete the calculation of the predefined index. Check the input-output relationship of the determined atomic computing units. When the output data generated by one atomic computing unit is received as input data by another atomic computing unit, establish a data dependency between the two atomic computing units. The direction of the data dependency is determined according to the direction of transmission of output data to downstream input data, from the atomic computing unit that generates the output data to the atomic computing unit that receives the output data.

[0026] S212, determine the sequential relationships between atomic computing units according to data dependencies, and write the atomic computing units required to complete the same predefined index and their corresponding data dependencies into the same lineage semantic template. The lineage semantic template is jointly composed of atomic computing units and the data dependencies between them. Therefore, the lineage semantic template can represent both which computing stages complete the predefined index and the sequential relationship of output data to input data between computing stages. After the lineage semantic template is formed, it corresponds to a specific predefined index and has an index identifier for matching with the target index.

[0027] S213, parse the lineage request and determine the target metric and query conditions from the parsed content. The target metric is used to determine the template to be invoked this time from the formed lineage semantic template, and the query conditions are used to determine the configuration parameters to be written into the atomic computing unit in this metric calculation. Both the target metric and the query conditions are directly derived from the parsed content of the lineage request.

[0028] S214: Match the target metric with the metric identifiers corresponding to each lineage semantic template. When the target metric matches a metric identifier, select the corresponding lineage semantic template and proceed to the configuration parameter writing process. When the target metric does not match any of the metric identifiers, the metric calculation process triggered by the lineage request stops at the instance generation location. Through the aforementioned judgment, only target metrics with corresponding lineage semantic templates enter the subsequent execution chain.

[0029] S215: After selecting the lineage semantic template, based on the content of the query conditions and the position of the query conditions already determined in S1, determine the atomic computing unit receiving the corresponding query conditions and the configuration parameters to be updated. After writing the query conditions into the corresponding configuration parameters, the original atomic computing unit relationships in the lineage semantic template remain unchanged. The atomic computing unit affected by the query conditions carries the configuration parameter status corresponding to this query, thus forming a parameterized index calculation instance. The verifiable state after S2 is completed is: the parameterized index calculation instance can correspond to a lineage request, a matched lineage semantic template, and a set of query conditions already written into the corresponding atomic computing unit.

[0030] S3 executes atomic computation units in topological order according to the data dependencies in the parameterized index calculation instance, and calls the execution engine according to the execution engine binding relationship to complete the continuous processing from input data to output data. The input of S3 is the parameterized index calculation instance formed by S2. During the execution process, the output data of each atomic computation unit is formed. The output data of the intermediate atomic computation units continues to serve as downstream input. The output data that does not form downstream data dependencies forms the index calculation result.

[0031] S311, based on the data dependencies in the parameterized index calculation instance, determine the set of direct predecessor atomic computation units for each atomic computation unit. When an atomic computation unit has no other atomic computation units providing it with input data, the corresponding set of direct predecessor atomic computation units is empty; when an atomic computation unit needs to receive output data from one or more upstream atomic computation units, the corresponding upstream atomic computation units are written into the set of direct predecessor atomic computation units. The set of direct predecessor atomic computation units is used to determine whether the corresponding atomic computation unit has met the execution conditions.

[0032] S312: When the set of direct predecessor atomic computing units is empty, the corresponding atomic computing unit is started; when the set of direct predecessor atomic computing units is not empty, it is checked whether each atomic computing unit in the set has formed output data. Only after each atomic computing unit in the set has formed output data is the atomic computing unit that receives the output data started. When there are still atomic computing units in the set that have not yet formed output data, the corresponding downstream atomic computing unit remains in a waiting state and is not executed prematurely.

[0033] S313, after the atomic computing unit starts, it calls the corresponding execution engine according to the execution engine binding relationship, providing the execution engine with input data and configuration parameters that conform to the input data pattern. The execution engine acts on the current calculation stage according to the configuration parameters and forms output data according to the output data pattern. After forming the output data, it checks whether the current atomic computing unit has downstream data dependencies. When downstream data dependencies exist, the output data is passed to the corresponding downstream atomic computing unit and used as the input data of the downstream atomic computing unit; when no downstream data dependencies exist, the current output data is determined as the index calculation result. Thus, the index calculation result comes from the output data in the actual execution chain that no longer provides downstream input to other atomic computing units.

[0034] S314: When multiple atomic computation units simultaneously meet the startup conditions and there are no data dependencies between them, the multiple atomic computation units are executed in parallel. The parallel-executed atomic computation units call the execution engine according to their corresponding execution engine binding relationships, obtain their respective input data, and form their respective output data. Whenever new output data is formed, it is checked again whether the atomic computation units that have not yet been executed have met the startup conditions corresponding to the set of direct predecessor atomic computation units. This checking and execution process is repeated until all atomic computation units in the lineage semantic template have completed execution. The verification status after S3 includes that each executed atomic computation unit has corresponding input and output data, adjacent atomic computation units with data dependencies can verify the same data transfer relationship, and the final outputs without downstream data dependencies can be verified as index calculation results.

[0035] S4 is triggered by events executed by atomic computing units to record data sources and input / output quantities, and is written into the directed lineage graph according to the actual execution order. S4 corresponds to the execution of atomic computing units in S3. The lineage record is not re-derived based on independent computation descriptions after all computations are completed, but rather obtains the corresponding execution state when each atomic computing unit receives input data and forms output data, and establishes lineage nodes and directed edges accordingly.

[0036] S411, when each atomic computing unit receives input data, it records the data source and the amount of input data. The data source is the data source actually accessed by the current atomic computing unit during this execution or transmitted via upstream output. The amount of input data is the scale of data actually entering the atomic computing unit during the current execution. The input data source and the amount of input data are established to correspond with the current execution of the atomic computing unit.

[0037] In step S412, when the atomic computing unit completes data processing and generates output data, the output data volume and execution time are recorded. The data source actually accessed by the current atomic computing unit, the data time range, and configuration parameters are written together with the input data volume and output data volume into the same lineage execution record. The execution time corresponds to the execution process of the current atomic computing unit from the start of execution to the generation of output data; the data time range corresponds to the time range covered by the data actually accessed by the current atomic computing unit; and the configuration parameters correspond to the query condition status written in S2 and used in S3. Therefore, the same lineage execution record can correspond to the input status, output status, and execution conditions of the same atomic computing unit execution.

[0038] S413 uses the actually executed atomic computation units as lineage nodes. When the output data of an atomic computation unit is received as input data by a downstream atomic computation unit, a directed edge is established between the corresponding two lineage nodes. The direction of the directed edge is consistent with the actual transmission direction of the output data to the downstream input data. The lineage execution record formed in S412 is written to the corresponding lineage node. After continuously recording the lineage nodes and directed edges formed during the actual execution process, a directed lineage graph is formed. The directed lineage graph reflects the actual computation chain that occurred in this parameterized index calculation instance, rather than only reflecting the data dependencies predefined in the lineage semantic template.

[0039] S414, when a lineage node receives input data from two or more data sources, corresponding data source nodes are established according to the actual data sources accessed as recorded in the lineage execution record. Each data source node establishes an input association with the lineage node that actually receives the corresponding input data, ensuring that multiple data sources retain their source boundaries when entering the same atomic computing unit. When the output data generated by a lineage node continues to be received by downstream lineage nodes, the established input associations and the directed edges connecting the current lineage node and downstream lineage nodes are continuously recorded, ensuring that data from different data sources can continue to be traced downstream along the same directed lineage graph after being processed by the same atomic computing unit. Data source nodes are not established for data sources that do not actually participate in the current execution.

[0040] S415, when the query condition includes a query time range, and the currently executing atomic computation unit is responsible for time filtering, the actual data coverage is determined based on the time field in the input data entering the atomic computation unit. The actual data coverage is compared with the query time range. When the actual data coverage covers the query time range, the atomic computation unit completes the time filtering according to the configuration parameters, and the resulting output data continues to enter downstream atomic computation units based on data dependencies. When the actual data coverage does not cover the query time range, the atomic computation unit corresponding to the time filtering execution is marked as having data integrity issues in the directed lineage graph, and the actual data coverage is determined as the available query time range. The anomaly marker directly corresponds to the lineage node where insufficient data coverage occurs, thus enabling the location of the anomaly in the indicator calculation chain.

[0041] S416: After a data integrity exception is generated in S415, an adjustment instruction for the query time range is received. The adjusted query time range is obtained from the adjustment instruction, and the query time range field in the original query conditions generated in S2 is replaced with the adjusted query time range, while keeping the other query fields unchanged, thus generating the adjusted query conditions. The adjusted query time range is compared with the available query time range. When the adjusted query time range falls within the available query time range, the configuration parameters corresponding to the atomic computing unit for execution time filtering are updated with the adjusted query conditions, and the atomic computing unit for execution time filtering and the downstream atomic computing unit based on data dependency connections are re-executed. The newly formed input and output states continue to be written to the corresponding lineage records according to S411 to S413. When the adjusted query time range does not fall within the available query time range, the data integrity exception flag is maintained, and the current re-execution is stopped. The verification status of S4 is that each lineage node in the directed lineage graph can correspond to the actual atomic computing unit executed, the directed edges between nodes can correspond to the actual output data to input data transmission relationship, when multiple data sources are involved, the data source nodes can be checked back separately, when data integrity is abnormal, the corresponding time filtering node can be located, and the adjusted query conditions can be traced back to be generated by the received adjustment instructions.

[0042] S5 establishes a traceability relationship from the index calculation results to the original data source and atomic computing units along the directed kinship graph. It constructs the graph features of the result features and result kinship paths corresponding to the result data items, and generates kinship interpretation results through a graph structure coding network, a feature fusion network, and an autoregressive text generation network based on a Transformer decoder. The input of S5 is the index calculation results formed in S3 and the directed kinship graph formed in S4. The intermediate results processed include the result kinship paths, d-dimensional result features, d-dimensional path representations, and d-dimensional joint features formed for each result data item. The output is the kinship interpretation results associated with each result data item.

[0043] S511: For each result data item in the index calculation results, determine the output data that forms the result data item, and then determine the corresponding terminal atomic computation unit from the output data. The terminal atomic computation unit is the computation node where the actual result data item is formed and the corresponding output data no longer enters other atomic computation units according to downstream data dependencies. Through the correspondence, the result data item is first located to the terminal computation position where the actual result data item is formed.

[0044] S512: Starting from the lineage node corresponding to the end atomic computation unit, a search is performed in the directed lineage graph in the opposite direction to the actual data transmission direction. The reverse search sequentially obtains the upstream atomic computation units that participate in forming the result data item, as well as the data source nodes associated with the upstream atomic computation units, and forms the result lineage path according to the existing connection order between the found nodes. The result lineage path does not regenerate computation nodes or data source nodes that do not exist in the directed lineage graph; each node in the path originates from the directed lineage graph formed in this actual execution.

[0045] S513 establishes a connection between the result data item and the corresponding result lineage path. When an indicator calculation result contains multiple result data items, S511 and S512 are executed for each result data item respectively, so that each result data item can correspond to the data source and atomic computing unit that actually participated in forming the result data item. For the current result data item, the indicator identifier, result value, and discrete fields in the query conditions are input into the embedding layer. The numerical fields are normalized and linearly mapped to form d-dimensional result features. For the current result lineage path, the atomic computing unit identifier or data source identifier is combined with the input / output quantity, execution time, data time range, and configuration parameters according to the node type to form the initial node features, and the directed edge direction is used as the edge direction feature. The graph structure encoding network of the large model consists of at least two graph attention layers and path pooling layers. Each graph attention layer calculates the attention weights of the upstream neighbor node features of the current node according to the directed edge direction and performs weighted aggregation. After two consecutive aggregation layers, a node representation containing two hops of upstream lineage information is obtained. Then, according to the node connection order of the result lineage path, the path pooling layer forms a d-dimensional path representation. The feature fusion network places the d-dimensional result features first and the d-dimensional path representation last, concatenating them at the beginning and end of the feature dimension to form a 2d-dimensional vector, which is then mapped to d-dimensional joint features by a fully connected mapping layer. The autoregressive text generation network based on the Transformer decoder converts the d-dimensional joint features into decoder prefix context. At each lexical generation step, it reads the prefix context and the generated lexical units, sequentially generating the original data source description, the execution order of atomic computation units, query configuration parameters, and the formation process of the result data item, until an end marker is generated, thus forming the lineage interpretation result. The verification status after S5 is completed is: for any result data item, it is possible to backtrack from the data source, computation unit, and configuration parameters in the interpretation result to the corresponding node or lineage execution record in the result lineage path, and to locate the end atomic computation unit from the result data item and further locate the upstream atomic computation units and corresponding original data sources involved in forming the result data item.

[0046] After execution according to S1 to S5, the calculation logic of the predefined indicators is first converted into atomic computing units with defined input-output boundaries and execution engine binding relationships. These atomic computing units then form lineage semantic templates through data dependencies. After the query conditions formed by the lineage request are entered into the corresponding configuration parameters, the execution process generates output data step by step according to the actual data dependencies, culminating in the indicator calculation result. The data source, input data volume, output data volume, and other execution states generated during the same execution process are written into the lineage execution record, forming a directed lineage graph together with the actual data transmission relationships. The final result data item is then mapped back to the actual computing node and the original data source through the directed lineage graph. The process forms a continuous data chain from lineage request, parameterized indicator calculation instance, actual execution, lineage record to result traceability, with the input of each stage obtainable from the explicit output of the previous stage.

[0047] Example 2 Please see Figure 5 This embodiment provides an intelligent indicator lineage tracing and interpretation system based on atomic computing units. Embodiment 1 discloses the processing chain from lineage request to indicator calculation result, directed lineage graph, and lineage interpretation result in the order of method execution. This embodiment carries the same technical chain through system composition and data transfer relationships between modules. The system includes an atomic computing unit library, a lineage semantic template unit, a dynamic execution engine, a lineage tracing unit, and an interpretable output unit. Each unit transmits corresponding data around the same atomic computing unit, parameterized indicator calculation instance, indicator calculation result, and directed lineage graph. The interpretable output unit further completes the construction of result features and graph features, graph structure encoding, feature fusion, and text generation.

[0048] The atomic computing unit library stores the binding relationships between atomic computing units and execution engines. Each atomic computing unit corresponds to a specific computational step in the predefined index calculation logic and has the same input data pattern, output data pattern, and configuration parameters as its corresponding computational step. When a lineage semantic template unit needs to create a lineage semantic template, it retrieves the atomic computing units constituting the predefined index calculation logic from the atomic computing unit library and creates the lineage semantic template based on the data dependencies passed from output data to input data between atomic computing units.

[0049] After receiving a lineage request, the lineage semantic template unit determines the corresponding lineage semantic template based on the request and retrieves the query conditions from it. The query conditions are then configured as parameters according to their position within the corresponding computational stage and written to the corresponding atomic computation unit. The lineage semantic template unit then outputs a parameterized metric computation instance. After the parameterized metric computation instance is sent to the dynamic execution engine, the dynamic execution engine determines the topology execution relationship based on the data dependencies in the instance and calls the corresponding execution engine according to the execution engine binding relationship.

[0050] The dynamic execution engine provides input data and configuration parameters to the execution engine of the corresponding atomic computing unit. The execution engine processes the data and generates output data. When the current atomic computing unit has downstream data dependencies, the dynamic execution engine passes the corresponding output data to the downstream atomic computing unit; when the current atomic computing unit does not have downstream data dependencies, the corresponding output data is output as the index calculation result. Therefore, the execution events provided by the dynamic execution engine to the lineage tracking unit can maintain a one-to-one correspondence with the atomic computing units that actually generate the input and output data.

[0051] The lineage tracing unit, executing alongside the atomic computation units, acquires the actual data sources accessed and the input / output quantities, and forms a directed lineage graph according to the actual execution order of the atomic computation units and the actual data transfer relationship from output to input. Lineage nodes in the directed lineage graph correspond to the actual executed atomic computation units, and directed edges between nodes correspond to the actual data transfer relationships. When the same lineage node receives input data from multiple data sources, each data source forms an input association with its corresponding lineage node.

[0052] The interpretable output unit receives the index calculation results generated by the dynamic execution engine and the directed lineage graph generated by the lineage tracing unit, and obtains the corresponding lineage path for each result data item. The feature construction module in the interpretable output unit maps the index identifier, result value, and query conditions into d-dimensional result features, and encodes the node attributes and directed edge directions in the result lineage path into graph features; the graph structure encoding network consists of at least two graph attention layers and path pooling layers, which aggregate the upstream adjacent node features according to the directed edge direction and pool them according to the path connection order to form a d-dimensional path representation; the feature fusion network concatenates the d-dimensional result features first and the d-dimensional path representation last into a 2d-dimensional vector, and forms a d-dimensional joint feature through a fully connected mapping layer; the autoregressive text generation network based on the Transformer decoder uses the joint feature as a prefix context to generate a lineage interpretation result containing the original data source, the execution order of atomic computing units, query configuration parameters, and the formation process of result data items word by word. The system execution results can be verified through the following correspondences: the index calculation results generated by the dynamic execution engine can correspond to the output of atomic computing units that do not have downstream data dependencies; the directed lineage graph generated by the lineage tracing unit can correspond to actual execution events and data transmission relationships; and the data sources and computing units in the lineage interpretation results generated by the interpretable output unit can be traced back to the directed lineage graph generated in the same calculation. All units of the system jointly carry out the index calculation, lineage recording, result tracing, and interpretation generation processes described in Example 1 through the aforementioned input-output relationships.

[0053] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for interpreting the lineage of intelligent indicators based on atomic computing units, characterized in that, Includes the following steps: Establish atomic computing units, define input data patterns, output data patterns, configuration parameters, and execution engine binding relationships; Establish a lineage semantic template based on the data dependencies between the atomic computing units, parse the lineage request and select the lineage semantic template, write the parsed query conditions into the configuration parameters, and form a parameterized index calculation instance. Based on the data dependencies in the parameterized index calculation instance, the atomic calculation units are executed in topological order, the execution engine is called according to the execution engine binding relationship, the input data that conforms to the input data pattern is processed with the configuration parameters, and the output data is formed according to the output data pattern. When there is a downstream data dependency, the output data is used as the downstream input. When no downstream data dependency is formed, the output data is used as the index calculation result. The atomic computing unit executes events to trigger the recording of data sources and input / output quantities, and writes them into the directed lineage graph in the execution order; A traceability relationship is established along the directed lineage graph from the index calculation results to the original data source and the atomic computing units, and a result lineage path corresponding to each result data item is formed according to the node connection order. For each result data item in the index calculation results, the index identifier, result value, and query conditions of the result data item are encoded as d-dimensional result features. The lineage node type, atomic computing unit identifier, data source identifier, input / output, execution time, data time range, configuration parameters, and directed edge direction in the corresponding result lineage path are encoded as graph features. The result features and the graph features are input into a graph structure encoding network consisting of at least two graph attention layers and path pooling layers, and a feature fusion network consisting of fully connected mapping layers. The network and a large model of an autoregressive text generation network based on a Transformer decoder are described. The graph structure encoding network performs attention aggregation on node features according to the directed edge direction and performs path pooling according to the node connection order of the resulting lineage path to form a d-dimensional path representation. The feature fusion network concatenates the d-dimensional result features and the d-dimensional path representation in the order of result features first and path representation last to form a 2d-dimensional vector, and forms a d-dimensional joint feature through the fully connected mapping layer, where d is the hidden layer dimension of the large model. The autoregressive text generation network uses the joint feature as a prefix context to generate a lineage explanation result containing the original data source, the execution order of atomic computing units, and the formation process of the result data item on a word-by-word basis.

2. The intelligent indicator lineage tracing and interpretation method based on atomic computing units according to claim 1, characterized in that, Establishing the atomic computing unit includes: Identify computational steps from the computational logic of predefined indicators that can be independently called by the execution engine and have a defined input data structure, output data structure, and query condition application location, and define each computational step as an atomic computational unit. The input data pattern is determined based on the data structure entering the calculation stage, the output data pattern is determined based on the data structure formed after processing by the calculation stage, and the configuration parameters are determined based on the position of the query conditions in the calculation stage. The execution engine binding relationship is established based on the correspondence between the execution engine's calls to the computation stage.

3. The intelligent indicator lineage tracing and interpretation method based on atomic computing units according to claim 2, characterized in that, The bloodline semantic template is generated according to the atomic computing units required to complete the same predefined index. When the output data formed by one atomic computing unit is received as input data by another atomic computing unit, a data dependency is established between the two atomic computing units. The sequential relationship of each atomic computing unit is determined according to the data dependency, and each atomic computing unit required to complete the same predefined index and its corresponding data dependency are written into the same lineage semantic template.

4. The intelligent indicator lineage tracing and interpretation method based on atomic computing units according to claim 1, characterized in that, The bloodline request is parsed, and the target indicators and query conditions are determined from the parsed content; The target indicator is matched with the indicator identifier corresponding to each bloodline semantic template. When the target indicator matches an indicator identifier, the corresponding bloodline semantic template is selected. The corresponding atomic computing unit and the configuration parameters to be updated are determined according to the query conditions. The query conditions are written into the corresponding configuration parameters to form the parameterized indicator calculation instance. When the target indicator is inconsistent with each of the indicator identifiers, the generation of the parameterized indicator calculation instance is stopped.

5. The intelligent indicator lineage tracing and interpretation method based on atomic computing units according to claim 1, characterized in that, When each atomic computing unit receives input data, it records the data source and the amount of input data. When the data processing is completed and output data is generated, it records the output data amount and execution time. The data source, data time range and configuration parameters actually accessed by the current atomic computing unit are written into the same lineage execution record. Using the atomic computing unit as a lineage node, a directed edge is established according to the actual transmission relationship where the output data of one atomic computing unit is received as the input data by the downstream atomic computing unit, and the lineage execution record is written into the corresponding lineage node to form the directed lineage graph.

6. The intelligent indicator lineage tracing and interpretation method based on atomic computing units according to claim 5, characterized in that, When the input data received by a lineage node comes from two or more data sources, data source nodes are established according to the actual access data sources recorded in the lineage execution record, and each data source node is associated with the lineage node that receives the corresponding input data. When the output data generated by the lineage node is received by the downstream lineage node, the corresponding input is associated with the directed edge connecting the lineage node and the downstream lineage node and continuously recorded, so that the data flow relationship corresponding to different data sources enters the same directed lineage graph.

7. The intelligent indicator lineage tracing and interpretation method based on atomic computing units according to claim 1, characterized in that, Establishing a traceability relationship between the index calculation results and the original data source and the atomic computing unit includes: for each result data item in the index calculation results, determining the corresponding end atomic computing unit based on the output data that forms the result data item; The atomic computing units and their corresponding data sources that participate in forming the result data items are searched in reverse along the data transmission direction in the directed lineage graph from the end atomic computing unit, and the result lineage path is formed according to the connection order between the searched nodes; The result data items are associated with the corresponding result lineage paths, so that the indicator identifiers, result values ​​and query conditions of the result data items correspond one-to-one with the lineage node attributes and directed edge directions in the result lineage paths, and are respectively used as the extraction objects of the d-dimensional result features and the graph features.

8. The intelligent indicator lineage tracing and interpretation method based on atomic computing units according to claim 3, characterized in that, In the lineage semantic template, a set of direct predecessor atomic computing units that have the data dependency with the corresponding atomic computing unit is recorded for each atomic computing unit; When the set of direct predecessor atomic computing units is empty, the corresponding atomic computing unit is started. When the set of direct predecessor atomic computing units is not empty, the corresponding atomic computing unit is started after each atomic computing unit in the set has generated the output data. Multiple atomic computation units that simultaneously meet the startup conditions and have no data dependencies on each other are executed in parallel. After new output data is generated, the atomic computation units that have not yet been executed are checked to see if they meet the startup conditions, until all atomic computation units in the lineage semantic template are completed.

9. The intelligent indicator lineage tracing and interpretation method based on atomic computing units according to claim 1, characterized in that, When the query conditions include a query time range, for the atomic computing unit that performs time filtering, the actual data coverage range is determined based on the time field in the input data. When the actual data coverage range covers the query time range, the output data generated by the atomic computing unit that performs the execution time filtering is sent to the downstream atomic computing unit according to the data dependency. When the actual data coverage does not cover the query time range, the atomic computing unit that performs the execution time filtering is marked as having data integrity issues in the directed lineage graph, and the actual data coverage is determined as the available query time range. After the available query time range is formed, an adjustment instruction for the query time range is received, the adjusted query time range is obtained from the adjustment instruction, and the query time range in the query conditions is replaced with the adjusted query time range based on the query conditions to generate the adjusted query conditions. When the adjusted query time range falls within the available query time range, the configuration parameters are updated with the adjusted query conditions, and the atomic computing unit that filters the execution time and the downstream atomic computing unit that is connected according to the data dependency are re-executed. If the adjusted query time range does not fall within the available query time range, maintain a data integrity exception flag and stop re-execution.

10. A smart indicator lineage tracing and interpretation system based on atomic computing units, characterized in that, The system includes an atomic computing unit library, a lineage semantic template unit, a dynamic execution engine, a lineage tracing unit, and an interpretable output unit; The atomic computing unit library stores the binding relationship between atomic computing units and execution engines; The bloodline semantic template unit establishes a bloodline semantic template according to the data dependency between the atomic computing units, selects the bloodline semantic template according to the bloodline request, and forms a configuration parameter and parameterized index calculation instance based on the query conditions in the request; The dynamic execution engine executes the atomic computing units according to the topological order determined by the parameterized index calculation instance, calls the execution engine according to the execution engine binding relationship, processes the input data with the configuration parameters to form output data, and uses the output data as downstream input when there is downstream data dependency, and uses the output data as index calculation result when no downstream data dependency is formed. The lineage tracing unit executes the record data source and input / output quantities along with the atomic computing unit, and writes the directed lineage graph in the execution order; The interpretable output unit includes a feature construction module and a large model. The large model includes a graph structure encoding network consisting of at least two graph attention layers and path pooling layers, a feature fusion network consisting of fully connected mapping layers, and an autoregressive text generation network based on a Transformer decoder. For each result data item, the feature construction module encodes the indicator identifier, result value, and query conditions into d-dimensional result features, and encodes the lineage node type, atomic computing unit identifier, data source identifier, input / output, execution time, data time range, configuration parameters, and directed edge direction in the corresponding result lineage path into graph features. The graph structure encoding network performs attention aggregation and path pooling on the node features according to the directed edge direction to form a d-dimensional path representation. The feature fusion network concatenates the d-dimensional result features and the d-dimensional path representation in the order of result features first and path representation last into a 2d-dimensional vector, and forms a d-dimensional joint feature through the fully connected mapping layer, where d is the hidden layer dimension of the large model. The autoregressive text generation network uses the joint feature as a prefix context to generate a lineage explanation result containing the original data source, the execution order of atomic computing units, and the formation process of the result data item on a word-by-word basis.

Images