Report data tracing method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a report data tracing method and device, computer equipment, a readable storage medium and a computer program product. The method comprises the steps of obtaining original report data and first structured query information corresponding to the report data, and performing grammar analysis on the first structured query information to obtain a first analysis result; metadata obtained after the original report data are accessed to the data warehouse system is obtained, and the metadata at least comprises a plurality of storage data tables corresponding to the original report data and a plurality of pieces of second structured query information in the storage process; grammar analysis is carried out on the multiple pieces of second structured query information, a second analysis result is obtained, and the second analysis result represents the incidence relation among the multiple storage data tables; and generating a data traceability result corresponding to the original report data according to the hierarchical information of the plurality of storage data tables in the data warehouse system, the first analysis result and the second analysis result. By adopting the method, the report data traceability effect can be improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a report data tracing method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] With the development of data processing technology, report data tracing technology has emerged. Report data tracing technology can be used to trace the report data stored in the data warehouse system.

[0003] In traditional technology, the metadata of the original report data is usually obtained first, and then the source table of the original report data is determined based on the metadata, so that the blood relationship between the reports can be analyzed, and traceability can be performed based on this.

[0004] However, this method of tracing report data can only parse out the source table directly related to the original report data, and cannot trace the processing process of the report data stored in the data warehouse system. Therefore, the effect of tracing report data is poor. Summary of the invention

[0005] Based on this, it is necessary to provide a report data tracing method, device, computer equipment, computer-readable storage medium and computer program product that can improve the report data tracing effect in response to the above technical problems.

[0006] In a first aspect, the present application provides a report data tracing method, comprising:

[0007] Obtaining original report data and first structured query information corresponding to the report data, and performing syntax parsing on the first structured query information to obtain a first parsing result, wherein the first parsing result represents an association relationship between each data table in the original report data and a post-access data table, wherein the post-access data table is a data table in a data operation layer in a data warehouse system;

[0008] Obtaining metadata of the original report data after it is connected to the data warehouse system, wherein the metadata at least includes a plurality of storage data tables corresponding to the original report data and a plurality of second structured query information in the storage process;

[0009] Performing syntax parsing on the plurality of second structured query information to obtain a second parsing result, wherein the second parsing result represents an association relationship between the plurality of storage data tables;

[0010] A data tracing result corresponding to the original report data is generated according to the hierarchical information of the multiple storage data tables in the data warehouse system, the first analysis result and the second analysis result.

[0011] In one embodiment, the performing grammatical parsing on the first structured query information to obtain a first parsing result includes:

[0012] Acquire table name information of a data table corresponding to the first structured query information in the original report data, wherein the first structured query information is used to process the data table in the original report data into a post-access data table;

[0013] splicing the first structured query information with the table name information to obtain spliced ​​information;

[0014] The concatenated information is parsed using structured query language to obtain a first parsing result.

[0015] In one of the embodiments, before generating the data tracing result corresponding to the original report data according to the hierarchical information of the plurality of storage data tables in the data warehouse system, the first parsing result and the second parsing result, the method further includes:

[0016] According to the system architecture levels at which the multiple storage data tables are located in the data warehouse system, the multiple storage data tables are respectively identified to obtain a level identification of each storage data table;

[0017] The hierarchical identifiers of the plurality of storage data tables are collectively used as hierarchical information.

[0018] In one embodiment, the performing grammatical parsing on the plurality of second structured query information to obtain a second parsing result includes:

[0019] For each of the second structured query information, remove the condition judgment keyword and the database function of the preset type in the second structured query information to obtain the target structured query information;

[0020] All the target structured query information is parsed to obtain a second parsing result.

[0021] In one embodiment, the data tracing result includes a tracing structure tree; the generating of the data tracing result corresponding to the original report data according to the hierarchical information of the plurality of storage data tables in the data warehouse system, the first parsing result and the second parsing result includes:

[0022] Generate a root node, wherein the root node includes the original report data;

[0023] According to the hierarchical information, the first parsing result and the second parsing result, tree nodes under each tree level are recursively generated from the root node to obtain a traceability structure tree, wherein one tree node corresponds to one storage data table or corresponds to one second structured query information.

[0024] In one embodiment, the data warehouse system includes a data operation layer, a data detail layer, a data service layer, and an application data layer; the tree nodes under each tree level are recursively generated from the root node according to the hierarchical information and the second parsing result, to obtain a traceability structure tree, including:

[0025] Generate at least one first tree node corresponding to the root node according to the hierarchical information and the first parsing result, wherein the first tree node is at a first tree level of the source tracing structure tree, and the first tree node corresponds to a storage data table or second structured query information of the data operation layer;

[0026] recursively generating, in sequence and according to the hierarchical information and the second parsing result, at least one second tree node of a second tree level, at least one third tree node of a third tree level, and at least one fourth tree node of a fourth tree level;

[0027] Among them, the second tree node corresponds to the storage data table or the second structured query information of the data detail layer, the third tree node corresponds to the storage data table or the second structured query information of the data service layer, and the fourth tree node corresponds to the storage data table of the application data layer.

[0028] In a second aspect, the present application also provides a report data tracing device, including:

[0029] A first parsing module is used to obtain original report data and first structured query information corresponding to the report data, and perform syntax parsing on the first structured query information to obtain a first parsing result, wherein the first parsing result represents an association relationship between each data table in the original report data and a post-access data table, wherein the post-access data table is a data table in a data operation layer in a data warehouse system;

[0030] An acquisition module, used to acquire metadata of the original report data after it is connected to the data warehouse system, wherein the metadata at least includes a plurality of storage data tables corresponding to the original report data and a plurality of second structured query information in the storage process;

[0031] A second parsing module, configured to perform syntax parsing on the plurality of second structured query information to obtain a second parsing result, wherein the second parsing result represents an association relationship between the plurality of storage data tables;

[0032] A tracing module is used to generate a data tracing result corresponding to the original report data according to the hierarchical information of the multiple storage data tables in the data warehouse system, the first analysis result and the second analysis result.

[0033] In one embodiment, the first parsing module is further used for:

[0034] Obtain table name information of a data table corresponding to the first structured query information in the original report data, wherein the first structured query information is used to process the data table in the original report data into a post-access data table; splice the first structured query information with the table name information to obtain spliced ​​information; perform syntax parsing of a structured query language on the spliced ​​information to obtain a first parsing result.

[0035] In one embodiment, the device further comprises:

[0036] A hierarchical identification module is used to identify the multiple storage data tables according to the system architecture hierarchies at which the multiple storage data tables are located in the data warehouse system, and obtain the hierarchical identification of each storage data table; the hierarchical identifications of the multiple storage data tables are collectively used as hierarchical information.

[0037] In one embodiment, the second parsing module is further used for:

[0038] For each of the second structured query information, condition judgment keywords and preset type of database functions are removed from the second structured query information to obtain target structured query information; syntax analysis is performed on all the target structured query information to obtain a second analysis result.

[0039] In one embodiment, the data tracing result includes a tracing structure tree; the tracing module is further used to:

[0040] Generate a root node, wherein the root node includes the original report data; recursively generate tree nodes under each tree level from the root node according to the hierarchical information, the first parsing result and the second parsing result to obtain a traceability structure tree, wherein one tree node corresponds to one of the storage data tables or corresponds to one second structured query information.

[0041] In one embodiment, the data warehouse system includes a data operation layer, a data detail layer, a data service layer, and an application data layer; the traceability module is also used to:

[0042] According to the hierarchical information and the first parsing result, at least one first tree node corresponding to the root node is generated, wherein the first tree node is at the first tree level of the traceability structure tree, and the first tree node corresponds to the storage data table or the second structured query information of the data operation layer; according to the hierarchical information and the second parsing result, at least one second tree node of the second tree level, at least one third tree node of the third tree level, and at least one fourth tree node of the fourth tree level are recursively generated in sequence; wherein the second tree node corresponds to the storage data table or the second structured query information of the data detail layer, the third tree node corresponds to the storage data table or the second structured query information of the data service layer, and the fourth tree node corresponds to the storage data table of the application data layer.

[0043] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0044] Obtain original report data and first structured query information corresponding to the report data, and perform syntax analysis on the first structured query information to obtain a first analysis result, wherein the first analysis result represents the association relationship between each data table in the original report data and the data table after access, and the data table after access is a data table in the data operation layer in the data warehouse system; obtain metadata of the original report data after accessing the data warehouse system, wherein the metadata at least includes multiple storage data tables corresponding to the original report data and multiple second structured query information in the storage process; perform syntax analysis on the multiple second structured query information to obtain a second analysis result, wherein the second analysis result represents the association relationship between the multiple storage data tables; generate a data tracing result corresponding to the original report data according to the hierarchical information of the multiple storage data tables in the data warehouse system, the first analysis result and the second analysis result.

[0045] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0046] Obtain original report data and first structured query information corresponding to the report data, and perform syntax analysis on the first structured query information to obtain a first analysis result, wherein the first analysis result represents the association relationship between each data table in the original report data and the data table after access, and the data table after access is a data table in the data operation layer in the data warehouse system; obtain metadata of the original report data after accessing the data warehouse system, wherein the metadata at least includes multiple storage data tables corresponding to the original report data and multiple second structured query information in the storage process; perform syntax analysis on the multiple second structured query information to obtain a second analysis result, wherein the second analysis result represents the association relationship between the multiple storage data tables; generate a data tracing result corresponding to the original report data according to the hierarchical information of the multiple storage data tables in the data warehouse system, the first analysis result and the second analysis result.

[0047] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0048] Obtain original report data and first structured query information corresponding to the report data, and perform syntax analysis on the first structured query information to obtain a first analysis result, wherein the first analysis result represents the association relationship between each data table in the original report data and the data table after access, and the data table after access is a data table in the data operation layer in the data warehouse system; obtain metadata of the original report data after accessing the data warehouse system, wherein the metadata at least includes multiple storage data tables corresponding to the original report data and multiple second structured query information in the storage process; perform syntax analysis on the multiple second structured query information to obtain a second analysis result, wherein the second analysis result represents the association relationship between the multiple storage data tables; generate a data tracing result corresponding to the original report data according to the hierarchical information of the multiple storage data tables in the data warehouse system, the first analysis result and the second analysis result.

[0049] The above-mentioned report data tracing method, computer device, computer-readable storage medium and computer program product first obtain the original report data and the first structured query information corresponding to the report data, and perform syntax analysis on the first structured query information to obtain a first analysis result, wherein the first analysis result represents the association relationship between each data table in the original report data and the accessed data table, and the accessed data table is a data table in the data operation layer of the data warehouse system, that is, the processing process of the report data entering the data operation layer of the data warehouse system from the outside can be analyzed, and then the metadata corresponding to the original report data in the data warehouse system is obtained, wherein the metadata at least includes multiple storage data tables corresponding to the original report data and multiple second structured query information in the storage process. Query information; perform syntax parsing on multiple second structured query information to obtain second parsing results, wherein the second parsing results represent the association relationship between multiple storage data tables, so that the association relationship between each storage data table in the data warehouse system and each second structured query information, as well as the system architecture level of each storage data table in the data warehouse system can be parsed, and then according to the hierarchical information of the multiple storage data tables in the data warehouse system, the first parsing results and the second parsing results, the data tracing results corresponding to the original report data are generated, which can realize the tracing of the processing process of storing the original report data in the data warehouse system, and can trace the system architecture level of each storage data table in the data warehouse system during the processing, thereby improving the effect of report data tracing. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0051] Figure 1 This is a flow chart of a report data tracing method in one embodiment of the present application;

[0052] Figure 2 A schematic diagram of a flow chart of performing syntax parsing on first structured query information according to an embodiment of the present application;

[0053] Figure 3 A schematic diagram of a process for generating data tracing results corresponding to original report data in one embodiment of the present application;

[0054] Figure 4 This is a structural block diagram of a report data tracing device in one embodiment of the present application;

[0055] Figure 5This is a diagram of the internal structure of a computer device in one embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0057] In an exemplary embodiment, Figure 1 As shown, a report data tracing method is provided, including the following steps 202 to 208. Among them:

[0058] Step 202, obtain the original report data and the first structured query information corresponding to the report data, and perform syntax analysis on the first structured query information to obtain a first analysis result, wherein the first analysis result represents the association relationship between each data table in the original report data and the accessed data table, and the accessed data table is a data table in the data operation layer in the data warehouse system.

[0059] Among them, the original report data can be the report data output by the report management system, and the report data needs to be stored in the data warehouse system for processing; the data warehouse system usually includes the data operation layer (ODS layer), the data detail layer (DWD layer), the data service layer (DWS layer) and the application data layer (ADS layer).

[0060] It should be noted that when the original report data is connected to the data operation layer of the data warehouse system, it usually needs to be processed to obtain the post-connection data table of the data operation layer.

[0061] As an example, step 202 includes: obtaining original report data, extracting first structured query information from the metadata of the original report data, and performing syntax parsing on the first structured query information to obtain a first parsing result, wherein the first structured query information is a structured query language code used to process the original report data into a post-access data table, and the first parsing result represents the association relationship between each data table in the original report data and the post-access data table, and the post-access data table is a data table in the data operation layer in the data warehouse system.

[0062] Step 204, obtaining metadata of the original report data after it is connected to the data warehouse system, wherein the metadata at least includes a plurality of storage data tables corresponding to the original report data and a plurality of second structured query information in the storage process.

[0063] Among them, after the original report data is connected to the data warehouse system, it will be processed in turn at each system architecture level of the data warehouse system, and a storage data table will be formed at each system architecture level; the second structured query information is a structured query language code used to process the connected original report data in the data warehouse system.

[0064] As an example, step 204 includes: respectively accessing metadata after the original report data of each system architecture level of the data warehouse system is connected to the data warehouse system; extracting multiple storage data tables and second structured query information corresponding to each storage data table from the metadata.

[0065] Step 206: perform syntax parsing on the plurality of second structured query information to obtain a second parsing result, wherein the second parsing result represents the association relationship between the plurality of storage data tables.

[0066] As an example, step 206 includes: performing syntax parsing on each second structured query information respectively to obtain a second parsing result, wherein the second parsing result represents the association relationship between the multiple storage data tables.

[0067] In one embodiment, the plurality of second structured query information is parsed to obtain a second parsing result, including:

[0068] For each second structured query information, condition judgment keywords and preset type of database functions are removed from the second structured query information to obtain target structured query information; syntax analysis is performed on all target structured query information to obtain second analysis results.

[0069] Among them, there may be conditional judgment keywords or some specific types of database functions in the second structured query information, and these conditional judgment keywords and database functions may affect the accuracy of syntax parsing.

[0070] Specifically, for each second structured query information, condition judgment keywords and preset types of database functions are searched in the second structured query information, and the searched condition judgment keywords and preset types of database functions are removed from the second structured query information to obtain the target structured query information; according to the syntax parser, all target structured query information are parsed to obtain the second parsing result.

[0071] Step 208: Generate data tracing results corresponding to the original report data according to the hierarchical information of the plurality of storage data tables in the data warehouse system, the first analysis result, and the second analysis result.

[0072] As an example, step 208 includes: identifying the system architecture level at which each storage data table is located in the data warehouse system to obtain level information; constructing a traceability structure tree corresponding to the original report data, multiple storage data tables, and the second structured query information corresponding to the multiple storage data tables based on the level information, the first parsing result, and the second parsing result, and using the traceability structure tree as the traceability result.

[0073] In one embodiment, before generating data tracing results corresponding to original report data according to the hierarchical information of the plurality of storage data tables in the data warehouse system, the first parsing result, and the second parsing result, the method further includes:

[0074] According to the system architecture levels at which the multiple storage data tables are located in the data warehouse system, the multiple storage data tables are respectively identified to obtain the level identification of each storage data table; the level identifications of the multiple storage data tables are collectively used as the level information.

[0075] Among them, the hierarchical information includes the hierarchical identification of multiple storage data tables, and the hierarchical identification is used to identify at which system architecture level of the data system the storage data table is located, that is, to identify at which layer of the data operation layer, data detail layer, data service layer and application data layer the storage data table is located.

[0076] In the above embodiment, firstly, the original report data and the first structured query information corresponding to the report data are obtained, and the first structured query information is parsed to obtain a first parsing result, wherein the first parsing result represents the association relationship between each data table in the original report data and the accessed data table, and the accessed data table is a data table in the data operation layer of the data warehouse system, that is, the processing process of the report data entering the data operation layer of the data warehouse system from the outside can be parsed, and then the metadata corresponding to the original report data in the data warehouse system is obtained, wherein the metadata at least includes multiple storage data tables corresponding to the original report data and multiple second structured query information in the storage process; the multiple second structured query information The information is parsed grammatically to obtain a second parsing result, wherein the second parsing result represents the association relationship between the multiple storage data tables, so that the association relationship between each storage data table in the data warehouse system and each second structured query information, as well as the system architecture level of each storage data table in the data warehouse system can be parsed, and then according to the hierarchical information of the multiple storage data tables in the data warehouse system, the first parsing result and the second parsing result, the data tracing result corresponding to the original report data is generated, which can realize the tracing of the processing process of storing the original report data in the data warehouse system, and can trace the system architecture level of each storage data table in the data warehouse system during the processing, thereby improving the effect of report data tracing.

[0077] In one embodiment, referring to Figure 2, performing syntax parsing on the first structured query information to obtain a first parsing result, including:

[0078] Step 302: obtaining the table name information of the data table corresponding to the first structured query information in the original report data, wherein the first structured query information is used to process the data table in the original report data into a connected data table.

[0079] The first structured query information usually does not contain relevant identification information of the data table in the original report data. Therefore, if the first structured query information is directly parsed, the association relationship between the original report data and the accessed data table in the data operation layer cannot be directly parsed.

[0080] Step 304: concatenate the first structured query information with the table name information to obtain concatenated information.

[0081] As an example, step 302 to step 304 include: obtaining table name information of each data table in the original report data; splicing the table name information corresponding to each data table in the original report data and the first structured query information to obtain splicing information corresponding to each data table in the original report data.

[0082] Step 306: perform syntax parsing of the structured query language on the concatenated information to obtain a first parsing result.

[0083] As an example, step 306 includes: using a syntax parser, performing syntax parsing of the structured query language on each concatenated information to obtain a first parsing result.

[0084] In the above embodiment, after obtaining the first structured query information of the original data report, the first structured query information is improved by splicing the table name information of the data table in the original report data with the first structured query information, and then the first parsing result is obtained by performing grammatical parsing on the spliced ​​information. The first parsing result can accurately characterize the association relationship between the data table in the original report data and the data table after access in the data operation layer, which helps to improve the accuracy of the first parsing result.

[0085] In one embodiment, referring to Figure 4 The data tracing result includes a tracing structure tree; according to the hierarchical information of multiple storage data tables in the data warehouse system, the first parsing result and the second parsing result, the data tracing result corresponding to the original report data is generated, including:

[0086] Step 402: Generate a root node, wherein the root node includes original report data.

[0087] As an example, step 402 includes: generating a root node with original report data, wherein the root node stores the original report data.

[0088] Step 404, recursively generate tree nodes under each tree level from the root node according to the level information, the first parsing result and the second parsing result, to obtain a traceability structure tree, wherein one tree node corresponds to a storage data table or to one second structured query information.

[0089] Among them, there are different tree levels after the root node. For example, the tree level where the child node of the root node is located can be set as the first level, and the tree level where the child node of the child node is located can be set as the second level. In this way, the root node can be considered as the zeroth level, followed by the first level corresponding to the child node of the root node, the second level corresponding to the child node of the tree node on the first level, the third level corresponding to the child node of the tree node on the second level, and so on, directly reaching the leaf node.

[0090] As an example, step 404 includes: generating a first tree node of the first tree level corresponding to the data operation layer, a second tree node of the second tree level corresponding to the data detail layer, a third tree node of the third tree level corresponding to the data service layer, and a fourth tree node of the fourth tree level corresponding to the application data layer according to the hierarchical information, each storage data table and each second structured query information; establishing a connection relationship between the root node and the first tree node of the first tree level according to the first parsing result, and establishing a connection relationship between the first tree node of the first tree level, the second tree node of the second tree level, the third tree node of the third tree level and the fourth tree node of the fourth tree level based on the second parsing result to obtain a traceability structure tree.

[0091] In one embodiment, the data warehouse system includes a data operation layer, a data detail layer, a data service layer, and an application data layer; according to the hierarchical information, the first parsing result, and the second parsing result, a tree node under each tree level is recursively generated from a root node to obtain a traceability structure tree, including:

[0092] According to the hierarchical information and the first parsing result, at least one first tree node corresponding to the root node is generated, wherein the first tree node is at the first tree level of the traceability structure tree, and the first tree node corresponds to the storage data table or the second structured query information of the data operation layer; according to the hierarchical information and the second parsing result, at least one second tree node of the second tree level, at least one third tree node of the third tree level, and at least one fourth tree node of the fourth tree level are recursively generated in sequence; wherein the second tree node corresponds to the storage data table or the second structured query information of the data detail layer, the third tree node corresponds to the storage data table or the second structured query information of the data service layer, and the fourth tree node corresponds to the storage data table of the application data layer.

[0093] Specifically, according to the hierarchical information, the storage data table of the data operation layer and the corresponding second structured query information are selected from each storage data table, and each tree node of the first level is generated according to the storage data table of the data operation layer and the second structured query information, wherein the first tree node of the first level stores the storage data table of the data operation layer or the second structured query information, and according to the first parsing result, a connection relationship between the root node and the first tree node of the first level is established; according to the hierarchical information, the storage data table of the data detail layer and the corresponding second structured query information are selected from each storage data table, and according to the storage data table of the data detail layer and the second structured query information, a second tree node of the second level is generated, wherein the second tree node of the second level stores the storage data table of the data detail layer or the second structured query information, and according to the second parsing result, a connection relationship between the first tree node of the first level and the second tree node of the second level is established; according to the hierarchical information, the storage data table of the data service layer and the corresponding second structured query information are selected from each storage data table, and according to the storage data table of the data service layer and The second structured query information generates a third tree node of the third level, wherein the third tree node of the third level stores the storage data table of the data service layer or the second structured query information, and according to the second parsing result, establishes a connection relationship between the second tree node of the second level and the third tree node of the third level, or establishes a connection relationship between the first tree node of the first level and the third tree node of the third level; according to the level information, selects the storage data table of the application data layer and the corresponding second structured query information from each storage data table, and generates a fourth tree node of the fourth level according to the storage data table of the application data layer and the second structured query information, wherein the fourth tree node of the fourth level stores the storage data table of the application data layer or the second structured query information, and according to the second parsing result, establishes a connection relationship between the third tree node of the third level and the fourth tree node of the fourth level, or establishes a connection relationship between the second tree node of the second level and the fourth tree node of the fourth level, or establishes a connection relationship between the first tree node of the first level and the fourth tree node of the fourth level; finally, a traceability structure tree is obtained.

[0094] In the above embodiment, a root node corresponding to the original report data is first generated, and then, according to the hierarchical information, storage data tables and second structured query information at different system architecture levels in the data warehouse system are selected to generate tree nodes at different tree levels; and then, according to the first parsing result and the second parsing result, a connection relationship between the root node and the tree nodes at different tree levels is established to obtain a traceability structure tree. In this way, the traceability structure tree can clearly express the data warehouse level at which each storage data table and the second structured query information are located in the data warehouse system, and can clearly display the association information between all data tables and structured query information. Therefore, the entire processing process of connecting the original report data to the data warehouse system can be indirectly displayed clearly, the traceability of the original report data is completed, and the effect of tracing the report data can be improved.

[0095] In a complete embodiment, original report data and first structured query information corresponding to the report data are obtained; table name information of each data table in the original report data is obtained; the table name information corresponding to each data table in the original report data and the first structured query information are spliced ​​to obtain splicing information corresponding to each data table in the original report data; through a syntax parser, each splicing information is parsed in a structured query language to obtain a first parsing result; the first parsing result represents the association relationship between each data table in the original report data and the data table after access, and the data table after access is a data table in the data operation layer in the data warehouse system.

[0096] Furthermore, the metadata after the original report data of each system architecture level of the data warehouse system is connected to the data warehouse system is obtained; multiple storage data tables and second structured query information corresponding to each storage data table are extracted from the metadata; for each second structured query information, condition judgment keywords and preset type of database functions are searched in the second structured query information, and the found condition judgment keywords and preset type of database functions are removed from the second structured query information to obtain target structured query information; according to the syntax parser, all target structured query information are parsed to obtain second parsing results, wherein the second parsing results represent the association relationship between multiple storage data tables.

[0097] Further, a root node is generated with the original report data, wherein the root node stores the original report data; according to the hierarchical information, a storage data table of the data operation layer and the corresponding second structured query information are selected from each storage data table, and each tree node of the first level is generated according to the storage data table of the data operation layer and the second structured query information, wherein the first tree node of the first level stores the storage data table of the data operation layer or the second structured query information, and according to the first parsing result, a connection relationship between the root node and the first tree node of the first level is established; according to the hierarchical information, a storage data table of the data detail layer and the corresponding second structured query information are selected from each storage data table, and according to the storage data table of the data detail layer and the second structured query information, a second tree node of the second level is generated, wherein the second tree node of the second level stores the storage data table of the data detail layer or the second structured query information, and according to the second parsing result, a connection relationship between the first tree node of the first level and the second tree node of the second level is established; according to the hierarchical information, a storage data table of the data service layer and the corresponding second structured query information are selected from each storage data table, And according to the storage data table of the data service layer and the second structured query information, a third tree node of the third level is generated, wherein the third tree node of the third level stores the storage data table of the data service layer or the second structured query information, and according to the second parsing result, a connection relationship is established between the second tree node of the second level and the third tree node of the third level, or a connection relationship is established between the first tree node of the first level and the third tree node of the third level; according to the level information, the storage data table of the application data layer and the corresponding second structured query information are selected from each storage data table, and according to the storage data table of the application data layer and the second structured query information, a fourth tree node of the fourth level is generated, wherein the fourth tree node of the fourth level stores the storage data table of the application data layer or the second structured query information, and according to the second parsing result, a connection relationship is established between the third tree node of the third level and the fourth tree node of the fourth level, or a connection relationship is established between the second tree node of the second level and the fourth tree node of the fourth level, or a connection relationship is established between the first tree node of the first level and the fourth tree node of the fourth level; finally, a traceability structure tree is obtained.

[0098] The above embodiment realizes the tracing of the processing process of storing the original report data in the data warehouse system, and can trace the system architecture level of each storage data table in the data warehouse system during the processing, thereby improving the effect of report data tracing.

[0099] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0100] Based on the same inventive concept, the embodiment of the present application also provides a report data tracing device for implementing the report data tracing method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more report data tracing device embodiments provided below can refer to the limitations of the report data tracing method above, and will not be repeated here.

[0101] In an exemplary embodiment, Figure 4 As shown, a report data tracing device is provided, including: a first parsing module 502, an acquisition module 504, a second parsing module 506 and a tracing module 508, wherein:

[0102] A first parsing module 502 is used to obtain the original report data and the first structured query information corresponding to the report data, and perform syntax parsing on the first structured query information to obtain a first parsing result, wherein the first parsing result represents the association relationship between each data table in the original report data and the accessed data table, and the accessed data table is a data table in the data operation layer in the data warehouse system;

[0103] An acquisition module 504 is used to acquire metadata of the original report data after it is connected to the data warehouse system, wherein the metadata at least includes a plurality of storage data tables corresponding to the original report data and a plurality of second structured query information in the storage process;

[0104] A second parsing module 506 is used to perform syntax parsing on the plurality of second structured query information to obtain a second parsing result, wherein the second parsing result represents the association relationship between the plurality of storage data tables;

[0105] The tracing module 508 is used to generate a data tracing result corresponding to the original report data according to the hierarchical information of the multiple storage data tables in the data warehouse system, the first analysis result and the second analysis result.

[0106] In one embodiment, the first parsing module is further used for:

[0107] Obtain table name information of a data table corresponding to the first structured query information in the original report data, wherein the first structured query information is used to process the data table in the original report data into a post-access data table; splice the first structured query information with the table name information to obtain spliced ​​information; perform syntax parsing of a structured query language on the spliced ​​information to obtain a first parsing result.

[0108] In one embodiment, the device further comprises:

[0109] A hierarchical identification module is used to identify the multiple storage data tables according to the system architecture hierarchies at which the multiple storage data tables are located in the data warehouse system, and obtain the hierarchical identification of each storage data table; the hierarchical identifications of the multiple storage data tables are collectively used as hierarchical information.

[0110] In one embodiment, the second parsing module is further used for:

[0111] For each of the second structured query information, condition judgment keywords and preset type of database functions are removed from the second structured query information to obtain target structured query information; syntax analysis is performed on all the target structured query information to obtain a second analysis result.

[0112] In one embodiment, the data tracing result includes a tracing structure tree; the tracing module is further used to:

[0113] Generate a root node, wherein the root node includes the original report data; recursively generate tree nodes under each tree level from the root node according to the hierarchical information, the first parsing result and the second parsing result to obtain a traceability structure tree, wherein one tree node corresponds to one of the storage data tables or corresponds to one second structured query information.

[0114] In one embodiment, the data warehouse system includes a data operation layer, a data detail layer, a data service layer, and an application data layer; the traceability module is also used to:

[0115] According to the hierarchical information and the first parsing result, at least one first tree node corresponding to the root node is generated, wherein the first tree node is at the first tree level of the traceability structure tree, and the first tree node corresponds to the storage data table or the second structured query information of the data operation layer; according to the hierarchical information and the second parsing result, at least one second tree node of the second tree level, at least one third tree node of the third tree level, and at least one fourth tree node of the fourth tree level are recursively generated in sequence; wherein the second tree node corresponds to the storage data table or the second structured query information of the data detail layer, the third tree node corresponds to the storage data table or the second structured query information of the data service layer, and the fourth tree node corresponds to the storage data table of the application data layer.

[0116] Each module in the above-mentioned report data tracing device can be implemented in whole or in part by software, hardware and their combination. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the corresponding operations of each of the above modules.

[0117] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 5 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface, the display unit and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a method for tracing the source of report data is implemented. Those skilled in the art can understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0118] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0119] Obtain original report data and first structured query information corresponding to the report data, and perform syntax analysis on the first structured query information to obtain a first analysis result, wherein the first analysis result represents the association relationship between each data table in the original report data and the data table after access, and the data table after access is a data table in the data operation layer in the data warehouse system; obtain metadata of the original report data after accessing the data warehouse system, wherein the metadata at least includes multiple storage data tables corresponding to the original report data and multiple second structured query information in the storage process; perform syntax analysis on the multiple second structured query information to obtain a second analysis result, wherein the second analysis result represents the association relationship between the multiple storage data tables; generate a data tracing result corresponding to the original report data according to the hierarchical information of the multiple storage data tables in the data warehouse system, the first analysis result and the second analysis result.

[0120] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0121] Obtain table name information of a data table corresponding to the first structured query information in the original report data, wherein the first structured query information is used to process the data table in the original report data into a post-access data table; splice the first structured query information with the table name information to obtain spliced ​​information; perform syntax parsing of a structured query language on the spliced ​​information to obtain a first parsing result.

[0122] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0123] According to the system architecture levels at which the multiple storage data tables are located in the data warehouse system, the multiple storage data tables are respectively identified to obtain a level identification of each storage data table; and the level identifications of the multiple storage data tables are collectively used as level information.

[0124] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0125] For each of the second structured query information, condition judgment keywords and preset type of database functions are removed from the second structured query information to obtain target structured query information; syntax analysis is performed on all the target structured query information to obtain a second analysis result.

[0126] In one embodiment, the data tracing result includes a tracing tree; when the processor executes the computer program, the processor further implements the following steps:

[0127] Generate a root node, wherein the root node includes the original report data; recursively generate tree nodes under each tree level from the root node according to the hierarchical information, the first parsing result and the second parsing result to obtain a traceability structure tree, wherein one tree node corresponds to one of the storage data tables or corresponds to one second structured query information.

[0128] In one embodiment, the data warehouse system includes a data operation layer, a data detail layer, a data service layer, and an application data layer; when the processor executes the computer program, the following steps are also implemented:

[0129] According to the hierarchical information and the first parsing result, at least one first tree node corresponding to the root node is generated, wherein the first tree node is at the first tree level of the traceability structure tree, and the first tree node corresponds to the storage data table or the second structured query information of the data operation layer; according to the hierarchical information and the second parsing result, at least one second tree node of the second tree level, at least one third tree node of the third tree level, and at least one fourth tree node of the fourth tree level are recursively generated in sequence; wherein the second tree node corresponds to the storage data table or the second structured query information of the data detail layer, the third tree node corresponds to the storage data table or the second structured query information of the data service layer, and the fourth tree node corresponds to the storage data table of the application data layer.

[0130] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0131] Obtain original report data and first structured query information corresponding to the report data, and perform syntax analysis on the first structured query information to obtain a first analysis result, wherein the first analysis result represents the association relationship between each data table in the original report data and the data table after access, and the data table after access is a data table in the data operation layer in the data warehouse system; obtain metadata of the original report data after accessing the data warehouse system, wherein the metadata at least includes multiple storage data tables corresponding to the original report data and multiple second structured query information in the storage process; perform syntax analysis on the multiple second structured query information to obtain a second analysis result, wherein the second analysis result represents the association relationship between the multiple storage data tables; generate a data tracing result corresponding to the original report data according to the hierarchical information of the multiple storage data tables in the data warehouse system, the first analysis result and the second analysis result.

[0132] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0133] Obtain table name information of a data table corresponding to the first structured query information in the original report data, wherein the first structured query information is used to process the data table in the original report data into a post-access data table; splice the first structured query information with the table name information to obtain spliced ​​information; perform syntax parsing of a structured query language on the spliced ​​information to obtain a first parsing result.

[0134] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0135] According to the system architecture levels at which the multiple storage data tables are located in the data warehouse system, the multiple storage data tables are respectively identified to obtain a level identification of each storage data table; and the level identifications of the multiple storage data tables are collectively used as level information.

[0136] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0137] For each of the second structured query information, condition judgment keywords and preset type of database functions are removed from the second structured query information to obtain target structured query information; syntax analysis is performed on all the target structured query information to obtain a second analysis result.

[0138] In one embodiment, the data tracing result includes a tracing structure tree; when the computer program is executed by the processor, the following steps are also implemented:

[0139] Generate a root node, wherein the root node includes the original report data; recursively generate tree nodes under each tree level from the root node according to the hierarchical information, the first parsing result and the second parsing result to obtain a traceability structure tree, wherein one tree node corresponds to one of the storage data tables or corresponds to one second structured query information.

[0140] In one embodiment, the data warehouse system includes a data operation layer, a data detail layer, a data service layer, and an application data layer; when the computer program is executed by the processor, the following steps are also implemented:

[0141] According to the hierarchical information and the first parsing result, at least one first tree node corresponding to the root node is generated, wherein the first tree node is at the first tree level of the traceability structure tree, and the first tree node corresponds to the storage data table or the second structured query information of the data operation layer; according to the hierarchical information and the second parsing result, at least one second tree node of the second tree level, at least one third tree node of the third tree level, and at least one fourth tree node of the fourth tree level are recursively generated in sequence; wherein the second tree node corresponds to the storage data table or the second structured query information of the data detail layer, the third tree node corresponds to the storage data table or the second structured query information of the data service layer, and the fourth tree node corresponds to the storage data table of the application data layer.

[0142] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0143] Obtain original report data and first structured query information corresponding to the report data, and perform syntax analysis on the first structured query information to obtain a first analysis result, wherein the first analysis result represents the association relationship between each data table in the original report data and the data table after access, and the data table after access is a data table in the data operation layer in the data warehouse system; obtain metadata of the original report data after accessing the data warehouse system, wherein the metadata at least includes multiple storage data tables corresponding to the original report data and multiple second structured query information in the storage process; perform syntax analysis on the multiple second structured query information to obtain a second analysis result, wherein the second analysis result represents the association relationship between the multiple storage data tables; generate a data tracing result corresponding to the original report data according to the hierarchical information of the multiple storage data tables in the data warehouse system, the first analysis result and the second analysis result.

[0144] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0145] Obtain table name information of a data table corresponding to the first structured query information in the original report data, wherein the first structured query information is used to process the data table in the original report data into a post-access data table; splice the first structured query information with the table name information to obtain spliced ​​information; perform syntax parsing of a structured query language on the spliced ​​information to obtain a first parsing result.

[0146] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0147] According to the system architecture levels at which the multiple storage data tables are located in the data warehouse system, the multiple storage data tables are respectively identified to obtain a level identification of each storage data table; and the level identifications of the multiple storage data tables are collectively used as level information.

[0148] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0149] For each of the second structured query information, condition judgment keywords and preset type of database functions are removed from the second structured query information to obtain target structured query information; syntax analysis is performed on all the target structured query information to obtain a second analysis result.

[0150] In one embodiment, the data tracing result includes a tracing structure tree; when the computer program is executed by the processor, the following steps are also implemented:

[0151] Generate a root node, wherein the root node includes the original report data; recursively generate tree nodes under each tree level from the root node according to the hierarchical information, the first parsing result and the second parsing result to obtain a traceability structure tree, wherein one tree node corresponds to one of the storage data tables or corresponds to one second structured query information.

[0152] In one embodiment, the data warehouse system includes a data operation layer, a data detail layer, a data service layer, and an application data layer; when the computer program is executed by the processor, the following steps are also implemented:

[0153] According to the hierarchical information and the first parsing result, at least one first tree node corresponding to the root node is generated, wherein the first tree node is at the first tree level of the traceability structure tree, and the first tree node corresponds to the storage data table or the second structured query information of the data operation layer; according to the hierarchical information and the second parsing result, at least one second tree node of the second tree level, at least one third tree node of the third tree level, and at least one fourth tree node of the fourth tree level are recursively generated in sequence; wherein the second tree node corresponds to the storage data table or the second structured query information of the data detail layer, the third tree node corresponds to the storage data table or the second structured query information of the data service layer, and the fourth tree node corresponds to the storage data table of the application data layer.

[0154] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0155] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0156] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A report data tracing method, characterized in that: The method comprises: Obtaining original report data and first structured query information corresponding to the report data, and performing syntax parsing on the first structured query information to obtain a first parsing result, wherein the first parsing result represents an association relationship between each data table in the original report data and a post-access data table, wherein the post-access data table is a data table in a data operation layer in a data warehouse system; Obtaining metadata of the original report data after it is connected to the data warehouse system, wherein the metadata at least includes a plurality of storage data tables corresponding to the original report data and a plurality of second structured query information in the storage process; Performing syntax parsing on the plurality of second structured query information to obtain a second parsing result, wherein the second parsing result represents an association relationship between the plurality of storage data tables; A data tracing result corresponding to the original report data is generated according to the hierarchical information of the multiple storage data tables in the data warehouse system, the first analysis result and the second analysis result.

2. The method according to claim 1, characterized in that The step of performing grammatical parsing on the first structured query information to obtain a first parsing result includes: Acquire table name information of a data table corresponding to the first structured query information in the original report data, wherein the first structured query information is used to process the data table in the original report data into a post-access data table; splicing the first structured query information with the table name information to obtain spliced ​​information; The concatenated information is parsed using structured query language to obtain a first parsing result.

3. The method according to claim 1, characterized in that Before generating the data tracing result corresponding to the original report data according to the hierarchical information of the plurality of storage data tables in the data warehouse system, the first parsing result and the second parsing result, the method further includes: According to the system architecture levels at which the multiple storage data tables are located in the data warehouse system, the multiple storage data tables are respectively identified to obtain a level identification of each storage data table; The hierarchical identifiers of the plurality of storage data tables are collectively used as hierarchical information.

4. The method according to claim 1, characterized in that: The step of performing grammatical parsing on the plurality of second structured query information to obtain a second parsing result includes: For each of the second structured query information, remove the condition judgment keyword and the database function of the preset type in the second structured query information to obtain the target structured query information; All the target structured query information is parsed to obtain a second parsing result.

5. The method according to claim 1, characterized in that The data tracing result includes a tracing structure tree; the data tracing result corresponding to the original report data is generated according to the hierarchical information of the multiple storage data tables in the data warehouse system, the first parsing result and the second parsing result, including: Generate a root node, wherein the root node includes the original report data; According to the hierarchical information, the first parsing result and the second parsing result, tree nodes under each tree level are recursively generated from the root node to obtain a traceability structure tree, wherein one tree node corresponds to one storage data table or corresponds to one second structured query information.

6. The method according to claim 5, characterized in that The data warehouse system includes a data operation layer, a data detail layer, a data service layer and an application data layer; the tree nodes under each tree level are recursively generated from the root node according to the hierarchical information and the second parsing result, to obtain a traceability structure tree, including: Generate at least one first tree node corresponding to the root node according to the hierarchical information and the first parsing result, wherein the first tree node is at a first tree level of the source tracing structure tree, and the first tree node corresponds to a storage data table or second structured query information of the data operation layer; recursively generating, in sequence and according to the hierarchical information and the second parsing result, at least one second tree node of a second tree level, at least one third tree node of a third tree level, and at least one fourth tree node of a fourth tree level; Among them, the second tree node corresponds to the storage data table or the second structured query information of the data detail layer, the third tree node corresponds to the storage data table or the second structured query information of the data service layer, and the fourth tree node corresponds to the storage data table of the application data layer.

7. A report data tracing device, characterized in that: The device comprises: A first parsing module is used to obtain original report data and first structured query information corresponding to the report data, and perform syntax parsing on the first structured query information to obtain a first parsing result, wherein the first parsing result represents an association relationship between each data table in the original report data and a post-access data table, wherein the post-access data table is a data table in a data operation layer in a data warehouse system; An acquisition module, used to acquire metadata of the original report data after it is connected to the data warehouse system, wherein the metadata at least includes a plurality of storage data tables corresponding to the original report data and a plurality of second structured query information in the storage process; A second parsing module, configured to perform syntax parsing on the plurality of second structured query information to obtain a second parsing result, wherein the second parsing result represents an association relationship between the plurality of storage data tables; A tracing module is used to generate a data tracing result corresponding to the original report data according to the hierarchical information of the multiple storage data tables in the data warehouse system, the first analysis result and the second analysis result.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images