Data relationship construction method, device, electronic device and storage medium

Abstract

The embodiment of the present application relates to the field of data processing technology, and discloses a method, device, electronic device and storage medium for constructing a data relationship, the method comprising: replacing a target parameter in an initial operation instruction for a target data warehouse with a first table name corresponding to the target parameter to obtain a target operation instruction including a first table name, the first table name being the table name of a first table, the first table being a data table for the operation corresponding to the target operation instruction to be executed; and, obtaining the first table name included in the target operation instruction, and determining a dependent table corresponding to the first table from a second table stored in the target data warehouse according to the first table name; and, establishing a corresponding relationship between the first table and the dependent table. Implementing the embodiment of the present application can improve the accuracy of the corresponding relationship between the data tables established in the data warehouse.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, device, electronic device and storage medium for constructing a data relationship. Background Art

[0002] As a data support system for various decision-making systems, the data warehouse can be used to store various business data used in daily life by the decision-making systems, making it easy to retrieve at any time.

[0003] In practice, it is found that due to the large amount of data in the data warehouse, the decision-making system usually obtains the data tables required for the business through the correspondence between data tables to improve the efficiency of obtaining. However, the accuracy of the correspondence established by the traditional data table correspondence establishment method is low, which is not conducive to the decision-making system to obtain the data tables required for the business through the correspondence between data tables. Summary of the invention

[0004] The embodiments of the present application disclose a method, device, electronic device and storage medium for constructing data relationships, which can improve the accuracy of the corresponding relationship between data tables established in a data warehouse.

[0005] A first aspect of an embodiment of the present application discloses a method for constructing a data relationship, the method comprising:

[0006] Replacing a target parameter in an initial operation instruction for a target data warehouse with a first table name corresponding to the target parameter, so as to obtain a target operation instruction including the first table name, wherein the first table name is a table name of a first table, and the first table is a data table for executing an operation corresponding to the target operation instruction;

[0007] Obtaining a first table name included in the target operation instruction, and determining a dependent table corresponding to the first table from a second table stored in the target data warehouse according to the first table name;

[0008] A correspondence between the first table and the dependent table is established.

[0009] A second aspect of an embodiment of the present application discloses a device for constructing a data relationship, including:

[0010] a replacement unit, configured to replace a target parameter in an initial operation instruction for a target data warehouse with a first table name corresponding to the target parameter, so as to obtain a target operation instruction including the first table name, wherein the first table name is a table name of a first table, and the first table is a data table for executing an operation corresponding to the target operation instruction;

[0011] a determining unit, configured to obtain a first table name included in the target operation instruction, and determine a dependent table corresponding to the first table from a second table stored in the target data warehouse according to the first table name;

[0012] An establishing unit is used to establish a corresponding relationship between the first table and the dependent table.

[0013] A third aspect of the present application discloses an electronic device, including:

[0014] A memory storing executable program code;

[0015] a processor coupled to the memory;

[0016] The processor calls the executable program code stored in the memory to execute the method for constructing data relations disclosed in the first aspect of the embodiment of the present application.

[0017] A fourth aspect of an embodiment of the present application discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the method for constructing a data relationship disclosed in the first aspect of an embodiment of the present application.

[0018] The fifth aspect of the embodiments of the present application discloses a computer program product. When the computer program product runs on a computer, the computer executes part or all of the steps of any one of the methods of the first aspect of the embodiments of the present application.

[0019] A sixth aspect of an embodiment of the present application discloses an application publishing platform, which is used to publish a computer program product. When the computer program product runs on a computer, the computer executes part or all of the steps of any one of the methods of the first aspect of the embodiment of the present application.

[0020] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0021] When receiving an operation instruction for a data warehouse, the related technology usually directly parses the operation instruction to index the dependency table in the data warehouse according to the parsed content, while the embodiment of the present application takes into account that directly parsing the operation instruction is prone to interference due to reasons such as duplicate field names and code annotation information, which may cause parsing errors, thereby causing subsequent indexing to the wrong dependency table, and then establishing an erroneous correspondence. Therefore, when receiving an operation instruction, the embodiment of the present application can replace the target operation in the initial operation instruction with the corresponding first table name by means of parameter replacement, and then use the low probability of table name errors in the operation instruction and the characteristics that the table name cannot be wrapped to match the dependency table of the first table in multiple second tables stored in the target data warehouse, thereby reducing the probability of errors in the matched dependency table and improving the accuracy of the correspondence between the data tables established in the data warehouse. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 It is a schematic diagram of an application scenario of a method for constructing a data relationship disclosed in an embodiment of the present application;

[0024] Figure 2 It is a flowchart of a method for constructing a data relationship disclosed in an embodiment of the present application;

[0025] Figure 3 It is a flowchart of another method for constructing a data relationship disclosed in an embodiment of the present application;

[0026] Figure 4 It is a flowchart of another method for constructing a data relationship disclosed in an embodiment of the present application;

[0027] Figure 5 It is a structural schematic diagram of a data relationship construction device disclosed in an embodiment of the present application;

[0028] Figure 6 It is a structural schematic diagram of another data relationship construction device disclosed in an embodiment of the present application;

[0029] Figure 7 It is a structural schematic diagram of an electronic device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0031] It should be noted that the terms "first", "second", "third" and "fourth" in the specification and claims of the present application are used to distinguish different objects rather than to describe a specific order. The terms "including" and "having" in the embodiments of the present application and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device including a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] The embodiments of the present application disclose a method, device, electronic device and storage medium for constructing data relationships, which can improve the accuracy of the corresponding relationship between data tables established in a data warehouse.

[0033] The technical solution of the present application will be described in detail below in conjunction with specific embodiments.

[0034] In order to more clearly illustrate a method, device, electronic device and storage medium for constructing a data relationship disclosed in an embodiment of the present application. First, an application scenario suitable for the method for constructing a data relationship is introduced. The method for constructing a data relationship can be applied to the construction of table-level blood relationship in various data warehouses. Among them, the data warehouse is a subject-oriented, integrated, stable data set that reflects historical changes and can be used to support various management decision-making systems; and the table-level blood relationship represents the corresponding relationship between the various tables stored in the data warehouse, so that the management decision-making system can quickly obtain various tables required for the business through blood relationship.

[0035] In actual operation, when various management decision systems want to write or read data in the data warehouse, they can usually use operation instructions written in SQL (Structured Query Language) to indicate the data they want to write or read (of course, operation instructions written in SQL also include operation instructions for modifying data, deleting data, etc., which are not limited here), and then send the operation instructions to the executor (for example: spark computing engine, SQL executor, etc., which are not limited here) for execution.

[0036] In one embodiment, if Figure 1As shown, the method for constructing data relationships disclosed in the embodiment of the present application can intercept a target operation instruction that has undergone parameter replacement (for example, replacing a parameter referring to a table name with the table name of the first table to be written into the data warehouse, replacing a parameter referring to a date with a date, etc., which are not limited here) during the process of the management decision system 110 sending the operation instruction to the executor 120 for execution, and obtain the first table name included in the target operation instruction; and then utilize the low probability of error in the table name in the operation instruction and the fact that the table name cannot be wrapped to match the dependent table of the first table in the data warehouse according to the first table name, thereby reducing the probability of error in the indexed dependent table and improving the accuracy of the correspondence between the data tables established in the data warehouse.

[0037] See also Figure 2 , Figure 2 This is a flow chart of a method for constructing a data relationship disclosed in an embodiment of the present application. The method for constructing a data relationship can be applied to various electronic devices such as personal computers, laptops, mobile phones, and various network servers. The present embodiment of the present application is described using electronic devices as an example, and should not be limited to the present embodiment of the present application. The method for constructing a data relationship may include the following steps:

[0038] 202. Replace the target parameter in the initial operation instruction for the target data warehouse with the first table name corresponding to the target parameter to obtain a target operation instruction including the first table name, where the first table name is the table name of the first table, and the first table is the data table corresponding to the operation to be executed by the target operation instruction.

[0039] There can be multiple data warehouses according to the type, source or function of the data stored in them, so that the management decision system can generate initial operation instructions (instructions written in SQL, which can be used to read, write or change the data stored in the data warehouse; optionally, the initial operation instructions can also be shell scripts, not limited here) for the target data warehouse (any one of the multiple data warehouses set up, not limited here) according to business needs, and send the initial operation instructions to the executor (for example: spark computing engine, SQL executor, etc., not limited here) for execution to access the corresponding data resources in the target data warehouse.

[0040] It is found in practice that after obtaining the initial operation instruction for the target data warehouse, the related technology usually directly parses the initial operation instruction and accesses the corresponding data resources in the target data warehouse according to the parsed content. However, since the direct parsing of the initial operation instruction is prone to parsing errors due to interference from reasons such as duplicate field names and code comment information, the related technology is prone to indexing the wrong dependency table and then establishing an erroneous data correspondence. In this regard, the embodiment of the present application may not parse the initial operation instruction first, but replace the target operation in the initial operation instruction with the corresponding first table name by parameter replacement, and then use the low probability of error in the table name in the operation instruction and the characteristics that the table name cannot be wrapped to match the dependency table of the first table in the multiple second tables stored in the target data warehouse, thereby reducing the probability of error in the matched dependency table and improving the accuracy of the correspondence between the data tables established in the data warehouse.

[0041] In this regard, before matching the dependency table, the electronic device needs to first determine the first table name of the data table corresponding to the target operation instruction to be executed based on the parameters included in the initial operation instruction. In the actual implementation process, the initial operation instruction usually includes multiple different parameters to refer to different information, such as: parameters referring to date information, parameters referring to name information, etc., which are not limited here. Therefore, when a certain target information carried by the initial operation instruction is modified, only the parameters corresponding to the target information need to be modified, and all the target information carried by the initial operation instruction will be uniformly modified, thereby facilitating maintenance.

[0042] Therefore, optionally, in step 202, the electronic device may replace the target parameter referring to the first table name in the initial operation instruction with the first table name corresponding to the target parameter before the initial operation instruction is sent to the executor for execution, so as to obtain a target operation instruction including the first table name, thereby facilitating the subsequent extraction of the first table name by the electronic device.

[0043] In one embodiment, the electronic device can call an execution function in a public script (a program pre-stored in the electronic device, which can use functions such as replacement of execution parameters and deletion of annotation information) to replace the target parameter referring to the first table name in the initial operation instruction with the first table name corresponding to the target parameter, thereby improving the efficiency of parameter replacement.

[0044] It should be noted that the initial operation instruction may be a data write instruction for the target data warehouse, or a data read instruction for the target data warehouse, etc., which is not limited here. When the initial operation instruction is a data write instruction for the target data warehouse, the first table name recorded in it may be the table name of the first table, and the first table is the table to be written into the target data warehouse. The embodiment of the present application is described by taking the first table as the table to be written into the target data warehouse as an example, which should not be limited to the embodiment of the present application.

[0045] In another embodiment, the electronic device may also replace other parameters included in the initial operation instruction with the corresponding information, thereby realizing visualization of the operation instruction, so that the user can intuitively understand the various attributes of the initial operation instruction. Optionally, the electronic device may replace the parameter referring to the date information in the initial operation instruction with the date information it refers to, so as to facilitate the user to know the date information such as the generation date and execution date of the initial operation instruction.

[0046] For example, assuming that parameter a refers to the generation date of the initial operation instruction, and parameter b specifies the execution date of the initial operation instruction, when performing parameter replacement, parameter a in the initial operation instruction can be replaced with the generation date of the initial operation instruction, and parameter b in the initial operation instruction can be replaced with the execution date of the initial operation instruction.

[0047] 204. Obtain a first table name included in the target operation instruction, and determine a dependent table corresponding to the first table in a second table stored in the target data warehouse according to the first table name.

[0048] In an embodiment of the present application, before the initial operation instruction is sent to the executor for execution, the electronic device replaces the target parameter referring to the first table name in the initial operation instruction with the first table name corresponding to the target parameter to obtain the target operation instruction including the first table name. After the target operation instruction has been preprocessed such as parameter replacement, it is usually sent to the executor for execution. After the target operation instruction is sent to the executor for execution, the target operation instruction is usually parsed, responded to, and other operations, which may cause the information recorded in the target operation instruction to be disordered. Therefore, in order to obtain an accurate table name, the electronic device can intercept these target operation instructions in the process of sending the target operation instructions to the executor, and obtain the first table name included therein, as a basis for subsequent dependent table matching, thereby improving the accuracy of the dependent table matched subsequently.

[0049] Furthermore, after obtaining the first table name, the electronic device can match the first table name with the second table names of multiple second tables stored in the target data warehouse one by one to determine the target second table whose second table name matches the first table name as the dependent table corresponding to the first table. Since the table name recorded in the target operation instruction has a small error probability and cannot be wrapped, the first table name obtained by the electronic device from the target operation instruction is usually correct, thereby improving the accuracy of the dependent table corresponding to the first table matched according to the first table name; in addition, the electronic device exhaustively matches all table names in the entire target data warehouse, and also ensures that the matched dependent table is definitely an existing table, and the matched table will not be a code snippet, a table alias, or other messy characters, further reducing the probability of error in the indexed dependent table and improving the accuracy of the correspondence between data tables established in the data warehouse.

[0050] 206. Establish a correspondence between the first table and the dependent table.

[0051] In the embodiment of the present application, after the electronic device determines the dependency table corresponding to the first table in the target data warehouse, it can write the first table and its dependency table together into the result file to establish a corresponding relationship between the two. Optionally, after the electronic device writes the first table and its dependency table together into the result file, it can parse the content of the result file, and perform correction judgment and / or filtering operations on the parsed content, and then establish a corresponding relationship between the first table and its dependency table to improve the accuracy of the corresponding relationship between the established data tables.

[0052] In an implementable application scenario, when a user uses a mobile browser to browse news, the browser usually inserts advertisements in the news. In order to deliver these advertisements, the server corresponding to the mobile browser needs to generate a write instruction to write the first table recording the advertisement data into the advertisement delivery data warehouse. If this advertisement is a targeted advertisement directed to the "pregnant women group", then in order to prevent other users who are not "pregnant women group" from seeing the advertisement, the data relationship construction device disclosed in the embodiment of the present application can be used to intercept the write instruction generated by the server corresponding to the mobile browser, and replace the parameter of the first table name referring to the first table in the write instruction with the first table name, and obtain the first table name from the instruction; then match the second table recording the "pregnant women group" data in the advertisement delivery data warehouse according to the first table name, and establish a corresponding relationship between the first table and the dependent table. As a result, the targeted advertisement can be delivered to the "pregnant women group", which improves the efficiency of advertisement delivery.

[0053] In implementing the methods disclosed in the above embodiments, it is considered that directly parsing the operation instruction is prone to parsing errors due to interference from reasons such as duplicate field names and code comment information, which may lead to subsequent indexing of the wrong dependent table and the establishment of an incorrect corresponding relationship. Therefore, the first table name of the first table to be written into the data warehouse included in the operation instruction can be determined by parameter replacement; and then the first table name is used to match the dependent table of the first table in the data warehouse by utilizing the low probability of error in the table name in the operation instruction and the characteristics that the table name cannot be wrapped, thereby reducing the probability of error in the dependent table indexed and improving the accuracy of the corresponding relationship between the data tables established in the data warehouse.

[0054] See also Figure 3 , Figure 3 This is a flow chart of another method for constructing a data relationship disclosed in an embodiment of the present application. The method for constructing a data relationship can be applied to various electronic devices such as personal computers, laptops, mobile phones, and various network servers. The present embodiment of the application is described using electronic devices as an example, which should not be limited to the present embodiment of the application. The method for constructing a data relationship may include the following steps:

[0055] 302. Determine the parameters arranged at fixed positions in the initial operation instruction for the target data warehouse as target parameters.

[0056] As described above, in an embodiment of the present application, the initial operation instruction for the data warehouse may include multiple different parameters to refer to different information, such as: parameters referring to date information, parameters referring to table name information, etc., which are not limited here. In this regard, in order to avoid the electronic device mistakenly replacing all or part of the parameters referring to other information as the first table name when performing parameter replacement, thereby replacing the wrong table name. Optionally, the electronic device may determine the parameters arranged in a fixed position among the multiple parameters included in the initial operation instruction for the data warehouse as the target parameter referring to the first table name. For example, the electronic device may determine the second parameter arranged from the beginning among the multiple parameters included in the initial operation instruction for the data warehouse as the target parameter referring to the first table name.

[0057] By implementing the above method, it is possible to prevent the electronic device from mistakenly replacing all or part of the parameters referring to other information as the first table name when performing parameter replacement, thereby avoiding the situation where an erroneous table name is replaced.

[0058] 304. Replace the target parameter with the corresponding first table name according to the defined replacement relationship to obtain a target operation instruction including the first table name.

[0059] In an embodiment of the present application, the electronic device can first define a replacement relationship between the parameters included in the initial instruction and the information they refer to, and write the replacement relationship into a public script subsequently called by the electronic device. When the electronic device runs the public script, the target parameter can be replaced with the first table name according to the replacement relationship defined in the public script, thereby obtaining a target operation instruction including the first table name.

[0060] In practice, it is found that the target operation instruction not only includes various parameters, but also often includes codes and annotation information for explaining the meaning of the codes or parameters, and the presence of such annotation information easily causes the subsequent electronic device to obtain an incorrect table name when obtaining the first table name included in the target operation instruction. Optionally, after the electronic device replaces the target parameter with the first table name according to the defined replacement relationship, it can delete the annotation information included in the initial operation instruction to obtain the target operation instruction including the first table name.

[0061] By implementing the above method, the electronic device can delete the annotation information in the initial operation instruction to make the content of the target operation instruction cleaner, thereby preventing the subsequent electronic device from obtaining an erroneous table name when obtaining the first table name included in the target operation instruction.

[0062] 306. Obtain a first table name included in the target operation instruction, and determine a dependent table corresponding to the first table in a second table stored in the target data warehouse according to the first table name.

[0063] In the embodiment of the present application, when the electronic device replaces the target parameter in the initial operation instruction with the first table name to obtain the target operation instruction, it can also add a converted identifier in the target operation instruction to indicate that the target parameter in the target operation instruction has been replaced with the first table name, and the first table name can be extracted. Therefore, optionally, the electronic device can intercept the target operation instruction including the above-mentioned converted identifier in the operation instruction (including the initial operation instruction and the target operation instruction) sent to the executor for execution, and obtain the first table name included in the intercepted target operation instruction.

[0064] By implementing the above method, the electronic device can quickly and accurately intercept the target operation instruction that has undergone preprocessing operations such as parameter replacement through the converted identifier, thereby improving the efficiency and accuracy of obtaining the first table name, and further improving the efficiency and accuracy of subsequent dependent table indexing.

[0065] After the electronic device obtains the first table name included in the target operation instruction; optionally, the electronic device can store the first table name in a first file, and add the first table name and other identification information to the name of the first file (for example: the storage path of the file, the first table name, the data recorded in the first table corresponding to the first table name, and the identification information of the target data warehouse, etc., not limited here), so that the electronic device can quickly determine the various information of the first table name stored therein by reading the file name of the first file.

[0066] As described above, multiple data warehouses may be provided according to the type, source or function of the data stored therein, so in order to avoid the electronic device needing to match the first table name with the second table stored in each data warehouse. Optionally, the electronic device may determine the target data warehouse corresponding to the above identification information in multiple data warehouses based on the identification information of the target data warehouse included in the name of the first file. For example, the identification information corresponding to the first table name includes "ad", and "ad" is the identification of the advertising delivery data warehouse. The electronic device may determine the advertising delivery data warehouse in multiple data warehouses and determine it as the target data warehouse. Furthermore, the electronic device may determine the dependent table corresponding to the first table in the second table stored in the target data warehouse based on the first table name included in the name of the first file.

[0067] By implementing the above method, after the electronic device obtains the first table name included in the target operation instruction, it can store it in a file and name the file with the identification information corresponding to the first table name, thereby facilitating the electronic device to quickly determine the various information of the first table name stored therein by reading the file name of the first file, thereby improving the efficiency of storing and reading the first table name.

[0068] 308. Establish a correspondence between the first table and the dependent table.

[0069] By implementing the methods disclosed in the above embodiments, the first table name of the first table to be written into the data warehouse, which is included in the operation instruction, can be determined by parameter replacement; and then, by utilizing the low probability of table name error in the operation instruction and the characteristics that the table name cannot be wrapped, the dependent table of the first table is matched in the data warehouse by the first table name, thereby reducing the probability of error in the indexed dependent table and improving the accuracy of the corresponding relationship between the data tables established in the data warehouse; and, the annotation information in the initial operation instruction can be deleted to make the content of the target operation instruction cleaner, so as to avoid the subsequent electronic device from obtaining the wrong table name when obtaining the first table name included in the target operation instruction; and, the target operation instruction that has been pre-processed by parameter replacement and other operations can be quickly and accurately intercepted by the converted identifier, thereby improving the efficiency and accuracy of obtaining the first table name, and then improving the efficiency and accuracy of subsequent dependent table indexing; after obtaining the first table name included in the target operation instruction, it can be stored in a file, and the file can be named with the identification information corresponding to the first table name, so that the electronic device can quickly determine the various information of the first table name stored by reading the file name of the first file, thereby improving the efficiency of storing and reading the first table name.

[0070] See also Figure 4 , Figure 4 This is a flow chart of another method for constructing a data relationship disclosed in the embodiment of the present application. The method for constructing a data relationship can be applied to various electronic devices such as personal computers, laptops, mobile phones, and various network servers. The embodiment of the present application is described by taking electronic devices as an example, which should not be limited to the embodiment of the present application. The method for constructing a data relationship may include the following steps:

[0071] 402. Replace the target parameter in the initial operation instruction for the target data warehouse with the first table name corresponding to the target parameter to obtain a target operation instruction including the first table name, where the first table name is the table name of the first table, and the first table is the data table corresponding to the operation to be executed by the target operation instruction.

[0072] 404. Obtain a first table name included in the target operation instruction, and match the first table name with second table names of multiple second tables stored in the target data warehouse one by one to determine a target second table whose second table name matches the first table name.

[0073] In the embodiment of the present application, after obtaining the first table name included in the target operation instruction, the electronic device can obtain the second table names of multiple second tables stored in the target data warehouse, and match the first table name with the multiple second table names one by one (optionally, the electronic device can perform full-character cyclic matching of the first table name with the multiple second table names one by one) to determine the target second table whose second table name matches the first table name. This ensures that the matched target second table is definitely an existing table, and there will be no code snippets, table aliases, or other messy characters matched, further reducing the probability of errors in the indexed dependent table and improving the accuracy of the correspondence between the data tables established in the data warehouse.

[0074] As an optional implementation, after the electronic device matches the first table name with the second table names of multiple second tables stored in the target data warehouse one by one to determine the target second table whose second table name matches the first table name, it can obtain the third table and the fourth table from the metadata database used to build the target data warehouse, wherein the third table records the first identifiers corresponding to each database included in the target data warehouse, and the fourth table records the second identifiers corresponding to the second tables included in each database.

[0075] It should be noted that the target data warehouse in the embodiment of the present application can be built based on Hive (Hive is a data warehouse tool, and the electronic device can build the target data warehouse based on the data warehouse tool). In actual operation, the built target data warehouse may include multiple databases to store different types of data, and each database includes multiple second tables of the same type. Among them, the second table name of each second table is usually composed of its corresponding second identifier and the first identifier corresponding to the database stored in it. These identifiers can be recorded in two tables in the metadata database of the target data warehouse, among which the third table can be used to record the first identifier corresponding to each database included in the target database, and the fourth table can be used to record the second identifier corresponding to each second table included in each database.

[0076] In this regard, the electronic device can generate second table names of multiple second tables stored in the target data warehouse based on the third table and the fourth table, wherein each second table name is composed of a second identifier recorded in the fourth table by its corresponding second table, and a first identifier corresponding to the database stored in the second table.

[0077] In one embodiment, the electronic device can store all generated second table names in a second file, and add identification information corresponding to the target data warehouse (for example, the type of data stored in the target data warehouse, the amount of data stored, etc.) to the name of the second file, thereby facilitating the electronic device to find the second file according to the name of the second file and read the second table name recorded therein.

[0078] By implementing the above method, the electronic device can obtain identification information from the metadata database for constructing the target data warehouse to generate the second table name of the second table stored in the target data warehouse, so as to ensure that the second table name of the second table stored in the target data warehouse is not missed and the accuracy of the second table name is guaranteed, thereby reducing the probability of errors in the matched dependent table and improving the accuracy of the correspondence between the data tables established in the data warehouse.

[0079] 406. Determine the target second table as a dependent table corresponding to the first table.

[0080] In the embodiment of the present application, after matching the first table name with the second table names of multiple second tables stored in the target data warehouse one by one, the target second table whose second table name matches the first table name can be determined as the dependent table corresponding to the first table. It should be noted that the dependent table is a table that has a corresponding relationship with the first table, such as a pre-dependent table of the first table, or the data recorded in the dependent table is related to the data recorded in the first table, etc., which is not limited here.

[0081] However, in practice, it is found that the second table name of the target second table matches the first table name when the second table name matches part of the first table name. For example, the first table name is "tbl_hour" and the second table name is "tbl", where "tbl" in the first table name and the second table name matches. However, this only indicates that the first table name contains "tbl", but does not indicate that the target second table is a dependent table of the first table.

[0082] In this regard, optionally, the electronic device can add non-blank characters (for example, integers such as 0 to 9, letters such as a to z, or underscores, etc.) to the second table name of the target second table to obtain a temporary second table name; then the electronic device can determine whether the temporary second table name still matches the first table name based on a regular expression (the regular expression can be set by the developer based on the added non-blank characters, for example: "_0-9a-z", "_0-9", etc., not limited here); if the temporary second table name does not match the first table name, it means that the second table name of the target second table is a complete match with the first table name, rather than a partial match, and the electronic device can determine the target second table as the dependent table corresponding to the first table.

[0083] Implementing the above method can prevent the electronic device from determining a second table whose name only partially matches the first table name as a dependent table corresponding to the first table, thereby reducing the probability of errors in the indexed dependent table and improving the accuracy of the correspondence between data tables established in the data warehouse.

[0084] Optionally, the electronic device can read a first file recording a first table name and a second file recording a second table name of a second table stored in a target data warehouse, and convert all the table names recorded in the first file and the second file into lowercase to unify the characters to facilitate subsequent processing; the electronic device can then delete the annotation information in the first file and the second file to prevent subsequent table name matching from being affected by the annotation information; the electronic device can then match the first table name and multiple second table names one by one, and add non-empty characters to the second table name of the target second table whose second table name matches the first table name to obtain a temporary second table name. If it is determined according to the regular expression that the temporary second table name does not match the first table name, it means that the second table name is a complete match with the first table name, rather than a partial match, and the target second table is the dependent table corresponding to the first table.

[0085] 408. Establish a correspondence between the first table and the dependent table.

[0086] By implementing the methods disclosed in the above-mentioned embodiments, the first table name of the first table to be written into the data warehouse, which is included in the operation instruction, can be determined by means of parameter replacement; and then, by utilizing the low probability of table name errors in the operation instruction and the characteristics that table names cannot be wrapped, the dependent table of the first table can be matched in the data warehouse with the first table name, thereby reducing the probability of errors in the indexed dependent table and improving the accuracy of the correspondence between data tables established in the data warehouse; and, identification information can be obtained from the metadata library for constructing the target data warehouse to generate the second table name of the second table stored in the target data warehouse, so as to ensure that the second table name of the second table stored in the target data warehouse is not omitted and the accuracy of the second table name is guaranteed, thereby reducing the probability of errors in the matched dependent table and improving the accuracy of the correspondence between data tables established in the data warehouse; and, it can be avoided that the second table whose second table name only partially matches the first table name is also determined as the dependent table corresponding to the first table, thereby reducing the probability of errors in the indexed dependent table and improving the accuracy of the correspondence between data tables established in the data warehouse.

[0087] See also Figure 5 , Figure 5 5 is a schematic diagram of a structure of a data relationship construction device disclosed in an embodiment of the present application. The data relationship construction device can apply the electronic device disclosed in the above embodiments. The data relationship construction device may include: a replacement unit 501, a determination unit 502 and an establishment unit 503, wherein:

[0088] A replacement unit 501 is used to replace a target parameter in an initial operation instruction for a target data warehouse with a first table name corresponding to the target parameter, so as to obtain a target operation instruction including a first table name, where the first table name is a table name of a first table, and the first table is a data table corresponding to the operation to be performed by the target operation instruction;

[0089] A determination unit 502 is used to obtain a first table name included in the target operation instruction, and determine a dependent table corresponding to the first table in a second table stored in the target data warehouse according to the first table name;

[0090] The establishing unit 503 is used to establish a corresponding relationship between the first table and the dependent table.

[0091] The device for constructing the data relationship described above is implemented in consideration of the fact that directly parsing the operation instruction is prone to errors in parsing due to interference from reasons such as duplicate field names and code annotation information, which may result in subsequent indexing of the wrong dependent table, and thus establishing an incorrect corresponding relationship. Therefore, the first table name of the first table to be written into the data warehouse, which is included in the operation instruction, can be determined by parameter replacement; and then, by utilizing the low probability of error in the table name in the operation instruction and the fact that the table name cannot be wrapped, the first table name is used to match the dependent table of the first table in the data warehouse, thereby reducing the probability of error in the dependent table indexed, and improving the accuracy of the corresponding relationship between the data tables established in the data warehouse.

[0092] See also Figure 6 , Figure 6 It is a structural diagram of another data relationship construction device disclosed in an embodiment of the present application. The data relationship construction device can also apply the electronic devices disclosed in the above embodiments. Figure 6 The construction device of the data relationship shown can be composed of Figure 5 The data relationship shown is constructed by optimizing the device, and Figure 5 Compared with the construction device of the data relationship shown, Figure 6 The data relationship construction device shown includes a replacement unit 501 for replacing the target parameter in the initial operation instruction for the target data warehouse with the first table name corresponding to the target parameter, so as to obtain the target operation instruction including the first table name, specifically, the method can be:

[0093] The replacement unit 501 is used to determine the parameters arranged in a fixed position in the initial operation instruction for the target data warehouse as target parameters; and replace the target parameters with the corresponding first table name according to the defined replacement relationship to obtain the target operation instruction including the first table name.

[0094] The device for constructing the data relationship described above can determine the first table name of the first table to be written into the data warehouse, which is included in the operation instruction, by means of parameter replacement; and then, utilizing the low probability of an error in the table name in the operation instruction and the fact that the table name cannot be wrapped, the dependent table of the first table can be matched in the data warehouse by the first table name, thereby reducing the probability of an error in the indexed dependent table and improving the accuracy of the correspondence between the data tables established in the data warehouse.

[0095] As an optional implementation manner, the replacing unit 501 is used to replace the target parameter with the first table name according to the defined replacement relationship to obtain the target operation instruction including the first table name in the following manner:

[0096] The replacement unit 501 is used to replace the target parameter with the first table name according to the defined replacement relationship, and delete the comment information included in the initial operation instruction to obtain the target operation instruction including the first table name, where the comment information is information used to explain the meaning of the code or parameters included in the initial operation instruction.

[0097] By implementing the above-mentioned data relationship construction device, the annotation information in the initial operation instruction can be deleted to make the content of the target operation instruction cleaner and avoid obtaining an erroneous table name when subsequently obtaining the first table name included in the target operation instruction.

[0098] As an optional implementation, Figure 6 The data relationship construction device shown may further include: an adding unit 504, wherein:

[0099] An adding unit 504 is used to store the first table name in a first file after the determining unit 502 obtains the first table name included in the target operation instruction, and to add the first table name and identification information corresponding to the target data warehouse to the name of the first file;

[0100] Furthermore, the method in which the determining unit 502 determines the dependent table corresponding to the first table in the second table stored in the target data warehouse according to the first table name may specifically be:

[0101] The determination unit 502 is used to determine the target data warehouse according to the identification information included in the name of the first file; and determine the dependent table corresponding to the first table in the second table stored in the target data warehouse according to the first table name included in the name of the first file.

[0102] The device for constructing the data relationship described above can, after obtaining the first table name included in the target operation instruction, store it in a file and name the file with the identification information corresponding to the first table name, so that by reading the file name of the first file, various information of the first table name stored therein can be quickly determined, thereby improving the efficiency of storing and reading the first table name.

[0103] As an optional implementation manner, the target operation instruction includes a converted identifier, which is used to indicate that the target parameter in the target operation instruction has been replaced with the first table name, and the method for the determination unit 502 to obtain the first table name included in the target operation instruction may specifically be:

[0104] The determination unit 502 is configured to intercept a target operation instruction including a converted identifier in an operation instruction sent to an executor for execution; and obtain a first table name included in the target operation instruction.

[0105] The device for constructing data relations described above can quickly and accurately intercept target operation instructions that have undergone preprocessing operations such as parameter replacement through converted identifiers, thereby improving the efficiency and accuracy of obtaining the first table name, and further improving the efficiency and accuracy of subsequent dependent table indexes.

[0106] As an optional implementation, the method in which the determining unit 502 determines the dependent table corresponding to the first table in the second table stored in the target data warehouse according to the first table name may specifically be:

[0107] The determination unit 502 is used to match the first table name with the second table names of multiple second tables stored in the target data warehouse one by one to determine the target second table whose second table name matches the first table name; and determine the target second table as a dependent table corresponding to the first table.

[0108] The above-mentioned data relationship construction device is implemented to exhaustively match all table names in the entire target data warehouse, ensuring that the matched dependent table is definitely an existing table, and there will be no code snippets, table aliases or other messy characters matched, thereby reducing the probability of errors in the indexed dependent table and improving the accuracy of the corresponding relationship between the data tables established in the data warehouse.

[0109] As an optional implementation manner, the determination unit 502 may determine the target second table as the dependent table corresponding to the first table in the following manner:

[0110] The determination unit 502 is used to add non-blank characters to the second table name of the target second table to obtain a temporary second table name; and if it is determined according to the regular expression that the temporary second table name does not match the first table name, determine that the target second table is a dependent table corresponding to the first table.

[0111] Implementing the above-mentioned data relationship construction device can avoid determining a second table whose name only partially matches the first table name as a dependent table corresponding to the first table, thereby reducing the probability of errors in the indexed dependent table and improving the accuracy of the correspondence between data tables established in the data warehouse.

[0112] As an optional implementation, Figure 6 The data relationship construction device shown may further include: an acquisition unit 505 and a generation unit 506, wherein:

[0113] The acquiring unit 505 is used to acquire a third table and a fourth table from a metadata library used to construct the target data warehouse before the determining unit 502 matches the first table name with the second table names of multiple second tables stored in the target data warehouse one by one to determine the target second table whose second table name matches the first table name, wherein the third table records the first identifiers corresponding to the various databases included in the target data warehouse, and the fourth table records the second identifiers corresponding to the various second tables included in the various databases;

[0114] The generating unit 506 is used to generate second table names of multiple second tables stored in the target data warehouse according to the third table and the fourth table, each second table name is composed of a second identifier corresponding to the corresponding second table and a first identifier corresponding to the database storing the corresponding second table.

[0115] The device for constructing the data relationship described above can obtain identification information from the metadata database for constructing the target data warehouse to generate the second table name of the second table stored in the target data warehouse, so as to ensure that the second table name of the second table stored in the target data warehouse is not missed and the accuracy of the second table name is guaranteed, thereby reducing the probability of errors in the matched dependent tables and improving the accuracy of the corresponding relationships between the data tables established in the data warehouse.

[0116] See also Figure 7 , Figure 7 Schematic diagram of the structure of an electronic device disclosed in the embodiment of the present application. Figure 7 As shown, the electronic device may include:

[0117] A memory 701 storing executable program codes;

[0118] a processor 702 coupled to the memory 701;

[0119] The processor 702 calls the executable program code stored in the memory 701 to execute the method for constructing the data relationship disclosed in the above embodiments.

[0120] An embodiment of the present application discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the method for constructing a data relationship disclosed in the above embodiments.

[0121] The embodiments of the present application also disclose an application publishing platform, wherein the application publishing platform is used to publish a computer program product, wherein when the computer program product runs on a computer, the computer executes part or all of the steps of the method in the above method embodiments.

[0122] It should be understood that the "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. Those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required for the present application.

[0123] In the various embodiments of the present application, it should be understood that the size of the serial numbers of the above-mentioned processes does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0124] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0125] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0126] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-accessible memory. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several requests for a computer device (which can be a personal computer, a server or a network device, etc., specifically a processor in a computer device) to perform some or all of the steps of the above-mentioned methods of various embodiments of the present application.

[0127] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0128] The above is a detailed introduction to a method, device, electronic device and storage medium for constructing a data relationship disclosed in an embodiment of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for a person skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for constructing a data relationship, characterized in that: The method comprises: Without parsing the initial operation instruction for the target data warehouse, replacing the target parameter in the initial operation instruction for the target data warehouse with the first table name corresponding to the target parameter, so as to obtain a target operation instruction including the first table name, wherein the first table name is the table name of a first table, the first table is a data table for the operation corresponding to the target operation instruction to be executed, and the target operation instruction includes a converted identifier, and the converted identifier is used to indicate that the target parameter in the target operation instruction has been replaced with the first table name; Intercepting a target operation instruction including the converted identifier in an operation instruction sent to the executor for execution; Obtaining a first table name included in the target operation instruction; Store the first table name in a first file, and add the first table name and identification information corresponding to the target data warehouse to the name of the first file; Determine a target data warehouse according to the identification information included in the name of the first file; According to the first table name included in the name of the first file, determining, in the second table stored in the target data warehouse, a dependent table corresponding to the first table; A correspondence between the first table and the dependent table is established.

2. The method according to claim 1, characterized in that The step of replacing the target parameter in the initial operation instruction for the target data warehouse with the first table name corresponding to the target parameter to obtain the target operation instruction including the first table name includes: Determine the parameters arranged at fixed positions in the initial operation instruction for the target data warehouse as target parameters; The target parameter is replaced with the corresponding first table name according to the defined replacement relationship to obtain a target operation instruction including the first table name.

3. The method according to claim 2, characterized in that The step of replacing the target parameter with the first table name according to the defined replacement relationship to obtain a target operation instruction including the first table name includes: The target parameter is replaced with the first table name according to the defined replacement relationship, and the comment information included in the initial operation instruction is deleted to obtain a target operation instruction including the first table name, wherein the comment information is information used to explain the meaning of the code or parameters included in the initial operation instruction.

4. The method according to claim 1, characterized in that: The step of determining, according to the first table name, a dependent table corresponding to the first table in a second table stored in the target data warehouse includes: Matching the first table name with the second table names of multiple second tables stored in the target data warehouse one by one to determine a target second table whose second table name matches the first table name; The target second table is determined as a dependent table corresponding to the first table.

5. The method according to claim 4, characterized in that The step of determining the target second table as a dependent table corresponding to the first table includes: Adding a non-blank character to the second table name of the target second table to obtain a temporary second table name; If it is determined according to the regular expression that the temporary second table name does not match the first table name, the target second table is determined to be a dependent table corresponding to the first table.

6. The method according to any one of claims 4 or 5, characterized in that: Before matching the first table name with the second table names of multiple second tables stored in the target data warehouse one by one to determine a target second table whose second table name matches the first table name, the method further includes: Acquire a third table and a fourth table from a metadata library used to construct a target data warehouse, wherein the third table records first identifiers corresponding to each database included in the target data warehouse, and the fourth table records second identifiers corresponding to each second table included in each database; According to the third table and the fourth table, second table names of multiple second tables stored in the target data warehouse are generated, and each second table name consists of a second identifier corresponding to the corresponding second table and a first identifier corresponding to the database stored in the corresponding second table.

7. A data relationship construction device, characterized in that: include: a replacement unit, configured to replace a target parameter in the initial operation instruction for the target data warehouse with a first table name corresponding to the target parameter without parsing the initial operation instruction for the target data warehouse, so as to obtain a target operation instruction including the first table name, wherein the first table name is a table name of a first table, the first table is a data table for an operation corresponding to the target operation instruction to be executed, and the target operation instruction includes a converted identifier, wherein the converted identifier is used to indicate that the target parameter in the target operation instruction has been replaced with the first table name; A determination unit is used to intercept a target operation instruction including the converted identifier in an operation instruction sent to an executor for execution; and obtain a first table name included in the target operation instruction, store the first table name in a first file, and add the first table name and the identification information corresponding to the target data warehouse to the name of the first file; determine the target data warehouse according to the identification information included in the name of the first file; determine a dependent table corresponding to the first table in a second table stored in the target data warehouse according to the first table name included in the name of the first file; An establishing unit is used to establish a corresponding relationship between the first table and the dependent table.

8. An electronic device, comprising a memory storing executable program code, and a processor coupled to the memory; wherein: The processor calls the executable program code stored in the memory to execute the method for constructing a data relationship according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, wherein: The computer program enables a computer to execute the method for constructing a data relationship as described in any one of claims 1 to 6.

Images