Data processing method and data processing device

By establishing multiple environments in the data center and comparing the data table blood relationship diagram in real time, the problem of the inability to effectively verify the blood relationship differences of the data table when revising pre-stored programs in the existing technology is solved, and the system accuracy and timeliness are improved, and the speed of digital transformation is promoted.

CN119537256APending Publication Date: 2025-02-28INVENTEC PUDONG TECH CORPOARTION +1
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202311086765.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-08-25
Publication Date
2025-02-28

AI Technical Summary

Technical Problem

When the existing technology revises pre-stored programs in the data center, it is impossible to effectively verify the differences in blood relationships in the data table in real time, making it difficult to detect errors, increasing the risk of modification and affecting the speed of digital transformation.

Method used

By establishing a development environment, data quality verification environment and formal environment in the data center, comparing the data table relationship diagrams of different environments in real time, generating comparison results and executing notification functions, ensuring data consistency and system accuracy.

Benefits of technology

Real-time comparison of blood relationships of data tables is realized, the system accuracy and timeliness of revising pre-stored programs is improved, data consistency is ensured, errors and modification risks are reduced, and digital transformation is promoted.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119537256A_ABST
    Figure CN119537256A_ABST
Patent Text Reader

Abstract

The invention provides a data processing method used for a data center. The method comprises the following steps of: obtaining pre-stored programs of a development environment, a data quality verification environment and a formal environment by the data center; respectively generating data table blood relationship diagrams of the development environment, the data quality verification environment and the formal environment according to pre-stored programs of the development environment, the data quality verification environment and the formal environment; wherein the step comprises analyzing each pre-stored program for each environment to judge a directed graph of each pre-stored program and combining the directed graphs of all pre-stored programs in each environment to generate a data table blood relationship graph of each environment; comparing the data table blood relationship diagrams of the development environment, the data quality verification environment and the formal environment to generate a comparison result; and executing a notification function according to the comparison result.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a data processing method and a data processing device, in particular to a data processing method and a data processing device capable of comparing the differences in blood relationship of data tables in different environments in real time. Background Art

[0002] Data centers usually include data tables and stored procedures. Data tables can be used as data carriers to store data. Stored procedures are responsible for the conversion and related logic between data tables and data. When software engineers or developers in data centers modify the logical relationship between data tables and data, they need to modify stored procedures. Since there are a large number of stored procedures in data centers and the logic between them is interrelated, when software engineers in data centers add or modify a stored procedure, it may cause changes to the relationship between the data tables in the data center. The existing verification method is to perform sampling verification on the data before and after the stored procedure is modified. However, this verification method is time-consuming and cannot confirm the complete correctness. Moreover, many errors often rely on data users to discover anomalies and feedback to the software engineers in the data center. However, erroneous data will cause the company to suffer potential and unassessable losses, so modifying stored procedures is a very dangerous job for software engineers in data centers. As the number of stored procedures increases, the risk of modification will also increase proportionally, which in turn affects the speed of the company's digital transformation. Therefore, the existing technology is in need of improvement. Summary of the invention

[0003] In order to solve the above problems, the present invention provides a data processing method and a data processing device that can compare the differences in blood relationship of data tables in different environments in real time to solve the above problems.

[0004] The present invention provides a data processing method for a data center, comprising: obtaining pre-stored programs of a development environment, a data quality verification environment and a formal environment from the data center; generating data table lineage relationship diagrams of the development environment, the data quality verification environment and the formal environment respectively according to the pre-stored programs of the development environment, the data quality verification environment and the formal environment, wherein this step comprises: for each of the development environment, the data quality verification environment and the formal environment, parsing each pre-stored program to determine a directed graph of each pre-stored program; and merging the directed graphs of all pre-stored programs in each environment to generate a data table lineage relationship diagram of each environment, wherein this step comprises merging a first node used to represent the source data table of a first pre-stored program and a second node used to represent the target data table of a second pre-stored program when it is determined that a source data table of a first pre-stored program and a target data table of a second pre-stored program are the same data table; comparing the data table lineage relationship diagrams of the development environment, the data quality verification environment and the formal environment to generate a comparison result; and executing a notification function according to the comparison result.

[0005] The present invention provides a data processing device for a data center, comprising: a storage device for storing instructions; and a processing circuit configured to execute the instructions, wherein the instructions include: obtaining a development environment, a data quality verification environment, and a pre-stored program of a formal environment from the data center; generating a data table lineage relationship diagram of the development environment, the data quality verification environment, and the formal environment according to the pre-stored programs of the development environment, the data quality verification environment, and the formal environment, wherein this step includes: parsing each pre-stored program of the development environment, the data quality verification environment, and the formal environment for each environment; The program is used to determine the directed graph of each stored program; and merge the directed graphs of all stored programs in each environment to generate a data table lineage graph for each environment, wherein this step includes merging a first node used to represent the source data table of a first stored program and a second node used to represent the target data table of a second stored program when it is determined that a source data table of the first stored program and a target data table of the second stored program are the same data table; comparing the data table lineage graphs of the development environment, the data quality verification environment and the formal environment to generate a comparison result; and executing a notification function based on the comparison result. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 The figure is a flowchart of a data processing method according to an embodiment of the present invention.

[0007] Figure 2A schematic diagram of a directed graph of pre-stored programs according to an embodiment of the present invention.

[0008] Figure 3 It is a schematic diagram of a source directed subgraph in a data table blood relationship graph according to an embodiment of the present invention.

[0009] Figure 4 It is a schematic diagram of a target directed subgraph in a data table blood relationship graph according to an embodiment of the present invention.

[0010] Figure 5 It is a schematic diagram for comparing the data table lineage relationship diagrams of the development environment and the data quality verification environment of an embodiment of the present invention.

[0011] Figure 6 It is a schematic diagram for comparing the data table lineage relationship diagrams of the data quality verification environment of an embodiment of the present invention and the formal environment.

[0012] Component number description

[0013] 10 Process

[0014] 202, 206, 208, 212, 214 nodes

[0015] 204, 210 Directed edges

[0016] 302 Source directed subgraph

[0017] 304, 404 Input Menu

[0018] 402 Target directed subgraph

[0019] Steps S100, S102, S104, S106, S108, S110 DETAILED DESCRIPTION

[0020] The embodiment of the present invention can be applied to a data center or a database system. In order to prevent errors caused by adding or modifying stored programs, the embodiment of the present invention can build a development environment, a data quality assurance system environment, and a formal environment in the data center. The development environment can be used to add or modify stored programs, and confirm that the stored programs can be executed normally after the addition or modification, and there are no errors in the program coding. In the development environment, developers or software engineers can load original data and stored programs and add or modify the stored programs according to requirements. After adding or modifying the stored programs, the new version data and stored programs of the development environment that have been added or modified can be copied to the data quality assurance environment. The data quality assurance environment can provide data users with confirmation whether the data changes meet the requirements after the addition or modification of the stored programs. In the data quality assurance environment, data users are provided with various testing methods to perform tests to determine whether the data changes meet the requirements. When data user testing finds any problems, flaws or defects, developers or software engineers can process the problems found in the data quality verification environment in the development environment, and then copy the processed data and stored programs to the data quality verification environment to conduct relevant tests again until the expected results are obtained or no problems occur. The formal environment is the real operating data center environment. Therefore, when the data quality verification environment completes the data user test and confirms that the data changes meet the requirements, the data center developers or software engineers can synchronize the modified new version of the stored program to the formal environment.

[0021] Please refer to Figure 1 , Figure 1 FIG. 1 is a schematic diagram of a process 10 of a data processing method according to an embodiment of the present invention. The process 10 includes the following steps:

[0022] Step S100: Start.

[0023] Step S102: Obtaining pre-stored programs of the development environment, data quality verification environment, and formal environment from the data center.

[0024] Step S104: Generate data table lineage relationship diagrams for the development environment, data quality verification environment, and formal environment respectively according to the pre-stored programs for the development environment, data quality verification environment, and formal environment.

[0025] Step S106: Compare the data table lineage diagrams of the development environment, the data quality verification environment, and the formal environment to generate a comparison result.

[0026] Step S108: Execute a notification function according to the comparison result.

[0027] Step S110: End.

[0028] According to process 10, in step S102, the embodiment of the present invention can obtain the pre-stored programs of the development environment, the pre-stored programs of the data quality verification environment, and the pre-stored programs of the formal environment from the data center. Then, in step S104, the data table lineage relationship diagrams of the development environment, the data quality verification environment, and the formal environment are respectively generated according to the pre-stored programs of the development environment, the pre-stored programs of the data quality verification environment, and the pre-stored programs of the formal environment. In one embodiment, the data table lineage relationship diagram of the development environment is generated according to the pre-stored programs of the development environment. Each pre-stored program of the development environment can be parsed to determine the directed graph of each pre-stored program. For example, a pre-stored program parsing engine can be used to parse the pre-stored program and generate a directed graph of the pre-stored program. For example, the SQL parser parsing engine of the Python program software can be used to parse the pre-stored program and generate a directed graph of the pre-stored program. Each pre-stored program may include at least one source data table and at least one target data table. Each pre-stored program may only include at least one source data table. Each pre-stored program may also only include at least one target data table. The directed graph of the stored procedure may include nodes for representing the source data table and / or the target data table. The directed graph of the stored procedure may include directed edges for representing the stored procedure, and the direction of the directed edges points from the source data table to the target data table. Each stored procedure may correspond to at least one directed graph.

[0029] For example, see Figure 2 ,like Figure 2 As shown in the left half, the first stored procedure is INT_CUSTOMER_SATISFACTION_SP.STOREDPROCEDURE.SQL. A directed graph of the first stored procedure includes node 202, node 206, and directed edge 204. Node 202 is used to represent a target data table of the first stored procedure, wherein the target data table of the first stored procedure is data table TP1_INT_CUSTOMER_SATISFACTION. Node 206 is used to represent a source data table of the first stored procedure, wherein the source data table of the first stored procedure is data table ODS_CUSTOMER_SATISFACTION. Directed edge 204 is used to represent the aforementioned first stored procedure, wherein the direction of directed edge 204 points from node 202 of the source data table to node 206 of the target data table. Please continue to refer to Figure 2 ,like Figure 2As shown in the left half, the second stored procedure is ODS_CUSTOMER_SATISFACTION_SP.STOREDPROCEDURE.SQL, and a directed graph of the second stored procedure includes nodes 208, 212, and directed edges 210. Node 208 is used to represent a target data table of the second stored procedure, wherein the target data table of the second stored procedure is the data table ODS_CUSTOMER_SATISFACTION. Node 212 is used to represent a source data table of the second stored procedure, wherein the source data table of the second stored procedure is the data table TP1_ODS_CUSTOMER_SATISFACTION. Directed edge 208 is used to represent that the direction of directed edge 208 of the second stored procedure is from node 212 of the source data table to node 208 of the target data table.

[0030] In step S104, after parsing the directed graph of all pre-stored programs of the development environment, a data table lineage graph of the development environment can be generated by merging the directed graphs of all pre-stored programs of the development environment. The data table lineage graph can represent the relationship between data tables. For example, the directed graphs of all pre-stored programs of the development environment can be compared. When it is determined that a source data table of a pre-stored program and a target data table of another pre-stored program are the same data table, the node used to represent the source data table of the pre-stored program can be merged with the node used to represent the target data table of the other pre-stored program to form a corresponding data table lineage graph. For example, Figure 2 As shown in the left half, when it is determined that the source data table of the first stored program and the target data table of the second stored program are the same data table (i.e., data table ODS_CUSTOMER_SATISFACTION_CUSTOMER_SATISFACTION), nodes 206 and 208 can be merged to form node 214, and the two directed graphs are combined in this way to form a part of the data table lineage relationship diagram. In this case, node 214 is used to represent the source data table of the first stored program and the target data table of the second stored program, and the directed graph of the first stored program and the directed graph of the second stored program are connected and merged. Through the aforementioned merging method, the directed graphs in all stored programs in the development environment can be merged and converted into a data table lineage relationship diagram of the development environment. The data table lineage relationship diagram may include at least one source directed subgraph and / or at least one target directed subgraph. The source directed subgraph may be a directed subgraph with a data table as the end point. The target directed subgraph may be a directed subgraph with a data table as the starting point. For example, please refer to Figure 3 , Figure 3 It is a schematic diagram of a source directed subgraph in a data table blood relationship diagram according to an embodiment of the present invention. Figure 3The middle part shows a source directed subgraph 302 corresponding to the data table INT_CUSTOMER_SATISFACTION. The node used to represent the data table INT_CUSTOMER_SATISFACTION in the source directed subgraph 302 is the end node. The source directed subgraph 302 includes parent nodes related to the data table INT_CUSTOMER_SATISFACTION. For example, please refer to Figure 4 , Figure 4 It is a schematic diagram of a target directed subgraph in a data table blood relationship graph according to an embodiment of the present invention. Figure 4 The middle part shows a target directed subgraph 402 corresponding to the data table INT_CUSTOMER_SATISFACTION. The node used to represent the data table INT_CUSTOMER_SATISFACTION in the target directed subgraph 402 is a starting node. The target directed subgraph 402 includes child nodes related to the data table INT_CUSTOMER_SATISFACTION.

[0031] Similarly, according to the aforementioned method of generating a data table lineage relationship diagram of a development environment, a data table lineage relationship diagram of a data quality verification environment can be generated based on the pre-stored programs of a data quality verification environment. Each pre-stored program of the data quality verification environment can be parsed to determine the directed graph of each pre-stored program, and after parsing the directed graphs of all pre-stored programs of the data quality verification environment, the directed graphs of all pre-stored programs of the data quality verification environment can be merged to generate a data table lineage relationship diagram for the data quality verification environment. Similarly, a data table lineage relationship diagram of a formal environment can be generated based on the pre-stored programs of the formal environment. Each pre-stored program of the formal environment can be parsed to determine the directed graph of each pre-stored program, and after parsing the directed graphs of all pre-stored programs of the formal environment, the directed graphs of all pre-stored programs of the formal environment can be merged to generate a data table lineage relationship diagram for the formal environment.

[0032] In one embodiment, after the data table lineage diagrams of the development environment, the data quality verification environment, and the formal environment are generated, an input menu may be generated for the user to input and select. Figure 3 and Figure 4 ,like Figure 3 and Figure 4Input menus 304 and 404 are shown. Users can select the desired environment type, data table type (schema), data table name and other fields through the input menu. For example, you can click the item "DEV" in the environment type field to select the development environment, click the item "INT" in the data table type field to select the INT data table, and click the item "INT_CUSTOMER_SATISFACTION" in the data table name field to select the data table INT_CUSTOMER_SATISFACTION. This embodiment can visually display the input menu for the user to view and input the items they want to select, and visually display the relevant information of the selected data table blood relationship diagram. Figure 3 and Figure 4 As shown, this embodiment can visualize the input menu for the user to view and input the desired item, and visualize the content of the source directed subgraph of the selected data table and the corresponding stored program, source data table, target data table and other information. In this way, it can provide users (such as administrators or engineers of the notification data center) with a quick and clear understanding of the stored program data structure relationship in the corresponding environment.

[0033] In step S106, the data table lineage relationship diagrams of the development environment, the data quality verification environment, and the formal environment can be compared to generate a comparison result. For example, the data table lineage relationship diagrams of at least two environments among the development environment, the data quality verification environment, and the formal environment are compared to generate a comparison result. For example, the data table lineage relationship diagrams of any two environments among the development environment, the data quality verification environment, and the formal environment are compared to generate a comparison result. For example, the data table lineage relationship diagrams of the development environment and the data quality verification environment are compared to generate a comparison result. The data table lineage relationship diagrams of the data quality verification environment and the formal environment are compared to generate a comparison result. In addition, a specific data table can also be selected for comparison to generate a comparison result. For example, the source directed subgraph or target directed subgraph corresponding to at least one data table of the data table lineage relationship diagrams of any two environments among the development environment, the data quality verification environment, and the formal environment are compared to generate a comparison result. For example, common types of data tables include DM data tables, INT data tables, ODS data tables, and STAGE data tables. A data table whose name is prefixed with DM is called a DM data table, for example, data table DM_XXXX. A data table whose name is prefixed with INT is called an INT data table, and so on. The DM data table is usually located at the end point of the data table lineage relationship. Since the DM data table is usually the end point data table in the data table lineage relationship, if there are differences and inconsistencies between two environments, they can be quickly and easily compared. In one embodiment, the source directed subgraph corresponding to the DM data table of the data table lineage relationship diagram of any two environments among the development environment, the data quality verification environment, and the formal environment can be compared to generate a comparison result.

[0034] In step S108, a notification function may be executed according to the comparison result of step S106. When the comparison result shows that there is a difference between the data table lineage relationship diagrams of any two environments, the embodiment of the present invention may generate and send a notification signal to remind the administrator or engineer of the data center to execute the notification function. For example, Figure 5 The source directed subgraph corresponding to the data table INT_CUSTOMER_X in the data table lineage relationship diagram of the development environment and the data quality verification environment is respectively displayed. Figure 6 The source directed subgraph corresponding to the data table INT_CUSTOMER_X in the data table lineage relationship diagram in the data quality verification environment and the formal environment is shown respectively. Figure 5 and Figure 6As shown in the figure, in the development environment, data quality verification environment and formal environment, the data table INT_CUSTOMER_X is the end point of the source directed subgraph. After comparing the data quality verification environment with the development environment, the comparison result shows that the data quality verification environment and the development environment have the same source directed subgraph, while the source directed subgraph of the formal environment is different from that of the development environment and the data quality verification environment. Figure 5 As shown, in step S106, the software engineer selects the development environment and the data quality verification environment for comparison, and the comparison result shows that the source directed subgraph corresponding to the data table INT_CUSTOMER_X of the development environment and the source directed subgraph corresponding to the data table INT_CUSTOMER_X of the data quality verification environment are the same. Figure 6 As shown, when the software engineer selects the formal environment and the data quality verification environment for comparison in step S106, the comparison result shows that the source directed subgraph corresponding to the data table INT_CUSTOMER_X in the formal environment is different from the source directed subgraph corresponding to the data table INT_CUSTOMER_X in the data quality verification environment. Figure 6 As shown, the source data table of the stored procedure INT_CUSTOMER_X_SP.STOREDPROCEDURE.SQL corresponding to level 0 is inconsistent. More specifically, in the formal environment, the source data table of the stored procedure INT_CUSTOMER_X_SP.STOREDPROCEDURE.SQL corresponding to level 0 is the data table ODS_CUSTOMER_X. However, in the data quality verification environment, the source data table of the stored procedure INT_CUSTOMER_X_SP.STOREDPROCEDURE.SQL corresponding to level 0 is the data table TP1_INT_CUSTOMER_X. This also means that the update operation when the data and stored procedures of the quality verification environment are synchronized to the formal environment may be abnormal or the update operation has not been performed. In this case, the comparison result shows that there are differences in the source directed subgraphs in the blood relationship diagram of the data tables of the formal environment data and the quality verification environment. Based on the comparison result showing that there are differences between the formal environment data and the quality verification environment, a notification signal can be generated and sent to notify the administrator or engineer of the data center to perform the notification function. In this way, administrators or engineers of the data center can easily and immediately identify whether an error exists based on the notification signal.

[0035] Therefore, during the application service development process, when the software engineer modifies the stored program in the development environment, he can compare the data table lineage diagram of the development environment with the data table lineage diagram of the data quality verification environment to determine whether there is a difference between the two. In this way, the correctness of the overall system after the stored program is modified can be confirmed and the timeliness of application service development can be greatly improved. During the application service development process, when the modified stored program is synchronized to the formal environment, the software engineer can compare the data table lineage diagram of the formal environment with the data quality verification environment to determine whether there is a difference between the two, so as to confirm whether the operation of synchronously updating the formal environment is correctly executed. When an error occurs in the synchronized update of the modified stored program to the formal environment, the embodiment of the present invention will be able to quickly find the affected stored program. In one embodiment, a monitoring cycle can be set, such as every day, every week, or every specific time, so that the data center executes the steps of process 10 every monitoring cycle to compare the data table lineage diagrams in the development environment, the data quality verification environment, and the formal environment, and the comparison results are sent back to the data center, thereby realizing an automated testing and notification mechanism for the software engineers in the data center and avoiding manual input errors to achieve complete test automation. At the same time, automated testing will also greatly improve the efficiency of the production testing process.

[0036] A person skilled in the art may combine, modify or change the above-described embodiments according to the spirit of the present invention, but is not limited thereto. All of the above statements, steps, and / or processes (including recommended steps) may be implemented by hardware, software, firmware (i.e., a combination of hardware devices and computer instructions, where the data in the hardware devices are read-only software data), electronic systems, or a combination of the above devices. The hardware may include analog, digital and hybrid circuits (i.e., microcircuits, microchips or silicon chips). For example, the hardware may be an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a programmable logic component, a coupled hardware component, or a combination of the above hardware. In other embodiments, the hardware may include a general purpose processor, a microprocessor, a controller, a digital signal processor (DSP), or a combination of the above hardware. The software may be a combination of program codes, a combination of instructions and / or a combination of functions (functionality), which is stored in a storage device, such as a computer readable recording medium or a non-transitory computer-readable medium. For example, a computer-readable recording medium may include a read-only memory (ROM), a flash memory (Flash Memory), a random-access memory (RAM), a subscriber identity module (SIM), a hard disk, a floppy disk, or a compact disk read-only memory (CD-ROM / DVD-ROM / BD-ROM), but is not limited thereto. An embodiment of the present invention may include a data processing device applied to a data center, the data processing device including a processing circuit and a storage device. The process steps and embodiments of the present invention may be compiled into a program code or instruction form and stored in the storage device of the data processing device. The processing circuit of the data processing device can be used to read and execute the program code or instructions stored in the storage device to implement all the aforementioned steps and functions.

[0037] In summary, the data processing method of the embodiment of the present invention can obtain the data table lineage diagram of each environment. When software engineers develop application services for the data center, they can compare the differences in data table lineages in different environments in real time, which will effectively improve the system correctness and timeliness when adding and modifying pre-stored programs. At the same time, it can also confirm whether the modified pre-stored programs are indeed synchronized with the formal environment, thereby effectively improving and optimizing the speed of the company's digital transformation.

[0038] The above description is only a preferred embodiment of the present invention. All equivalent changes and modifications made according to the scope of the patent application of the present invention should fall within the scope of the present invention.

Claims

1. A data processing method, characterized in that: For a data center, including: Obtaining pre-stored programs of a development environment, a data quality verification environment, and a formal environment from the data center; Generate data table lineage relationship diagrams of the development environment, the data quality verification environment, and the formal environment respectively according to the pre-stored programs of the development environment, the data quality verification environment, and the formal environment, wherein this step includes: For each of the development environment, the data quality verification environment, and the formal environment, parsing each pre-stored program to determine a directed graph of each pre-stored program; and Merging the directed graphs of all stored programs in each environment to generate a data table lineage graph for each environment, wherein this step includes merging a first node representing the source data table of a first stored program with a second node representing the target data table of the second stored program when it is determined that a source data table of a first stored program and a target data table of a second stored program are the same data table; Comparing the data table lineage diagrams of the development environment, the data quality verification environment, and the formal environment to generate a comparison result; and According to the comparison result, a notification function is executed.

2. The data processing method according to claim 1, characterized in that: The step of comparing the data table lineage diagrams of the development environment, the data quality verification environment, and the formal environment to generate the comparison result comprises: The data table lineage relationship diagrams of at least two environments among the development environment, the data quality verification environment, and the formal environment are compared to generate the comparison result.

3. The data processing method according to claim 1, characterized in that: The step of comparing the data table lineage diagrams of the development environment, the data quality verification environment, and the formal environment to generate the comparison result comprises: The source directed subgraph or the target directed subgraph corresponding to at least one data table of the data table lineage relationship diagram of at least two environments among the development environment, the data quality verification environment and the formal environment is compared to generate the comparison result.

4. The data processing method according to claim 1, characterized in that: The step of executing the notification function according to the comparison result comprises: When the comparison result shows that there are differences in the data table lineage relationship diagrams of any two environments among the development environment, the data quality verification environment and the formal environment, a notification signal is generated and sent to execute the notification function.

5. The data processing method according to claim 1, characterized in that: Also includes: An input menu is generated for input selection and the input menu and the related information of the data table blood relationship diagram selected by the input menu are presented in a visual manner.

6. A data processing device, characterized in that: For a data center, including: a storage device for storing instructions; and a processing circuit configured to execute the instructions, wherein the instructions include: Obtaining pre-stored programs of a development environment, a data quality verification environment, and a formal environment from the data center; Generate data table lineage relationship diagrams of the development environment, the data quality verification environment, and the formal environment respectively according to the pre-stored programs of the development environment, the data quality verification environment, and the formal environment, wherein this step includes: For each of the development environment, the data quality verification environment, and the formal environment, parsing each pre-stored program to determine a directed graph of each pre-stored program; and Merging the directed graphs of all stored programs in each environment to generate a data table lineage graph for each environment, wherein this step includes merging a first node representing the source data table of a first stored program with a second node representing the target data table of the second stored program when it is determined that a source data table of a first stored program and a target data table of a second stored program are the same data table; Comparing the data table lineage diagrams of the development environment, the data quality verification environment, and the formal environment to generate a comparison result; and According to the comparison result, a notification function is executed.

7. The data processing device according to claim 6, characterized in that: The instructions also include: The data table lineage relationship diagrams of at least two environments among the development environment, the data quality verification environment, and the formal environment are compared to generate the comparison result.

8. The data processing device according to claim 6, characterized in that: The instructions also include: The source directed subgraph or the target directed subgraph corresponding to at least one data table of the data table lineage relationship diagram of at least two environments among the development environment, the data quality verification environment and the formal environment is compared to generate the comparison result.

9. The data processing device according to claim 6, characterized in that: The instructions also include: When the comparison result shows that there are differences in the data table lineage relationship diagrams of any two environments among the development environment, the data quality verification environment and the formal environment, a notification signal is generated and sent to execute the notification function.

10. The data processing device according to claim 6, characterized in that: The instructions also include: An input menu is generated for input selection and the input menu and the related information of the data table blood relationship diagram selected by the input menu are presented in a visual manner.

Citation Information

Cited By

  • Production scheduling method based on new energy wharf tractor

    CN121785280A