Blood relationship generation method and device, computer equipment and storage medium

Abstract

The application belongs to the technical field of data analysis, and provides a blood relationship generation method and device, computer equipment and a storage medium, wherein the method comprises the following steps: adding operation information of each DDL operation to a preset operation data table, screening out target operation information which has not been subjected to hook processing from the operation data table according to a label of each operation information, analyzing the target operation information, determining a source data table to which the target operation information belongs and an associated data table related to the source data table, respectively reading data table information of the source data table and the associated data table, obtaining first data table information of the source data table and second data table information of the associated data table, and generating blood relationship according to the first data table information and the second data table information, so that the generation of blood relationship of a relational database is realized, the operation logic between data tables in the relational database is clearly known, and data can be better used.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data analysis, and in particular, relates to a blood relationship generation method and device, computer equipment and a storage medium. BACKGROUND

[0002] Information systems generate massive amounts of data, and data must have a storage location. Current general data analysis products that provide cross-data source query analysis and data calculation services basically use a relational database as a unified data aggregation center.

[0003] A relational database is a database built on the basis of a relational model. Its advantages include maintaining data consistency, small data update overhead, a relational model that is easier to understand than other models such as a mesh or hierarchy, and a common SQL language that makes it very convenient to operate a relational database and perform complex queries such as join.

[0004] In the face of massive and continuously increasing data of various types, it is necessary to know where the data comes from and how it changes over time, so it is necessary to analyze the blood relationship of the data. The existing upstream and downstream relationships between tables in a relational database cannot generate the blood relationship of the relational database, and it is difficult to clearly understand the operation logic between the tables, which makes it difficult to better use the data in the tables. SUMMARY

[0005] The main purpose of the present application is to provide a blood relationship generation method and device, computer equipment and a storage medium to generate the blood relationship of a relational database, so as to clearly understand the operation logic between the tables and facilitate better use of the data in the tables.

[0006] To achieve the above-mentioned purpose of the application, the present application provides a blood relationship generation method, which comprises:

[0007] Obtaining all DDL operations on the relational database within a preset time period to obtain a plurality of DDL operations;

[0008] Obtaining the operation information of each DDL operation, adding the operation information of each DDL operation to a preset operation data table, and obtaining a label previously assigned to the operation information of each DDL operation; wherein the label is used to mark whether the operation information has been processed by hook;

[0009] According to the label of each operation information, filtering out the operation information that has not been processed by hook from the operation data table to obtain target operation information;

[0010] The target operation information is parsed to determine a source data table to which the target operation information belongs and an associated data table related to the source data table;

[0011] Data table information of the source data table and the associated data table is read respectively to obtain first data table information of the source data table and second data table information of the associated data table;

[0012] A blood relationship is generated according to the first data table information and the second data table information.

[0013] Preferably, the parsing of the target operation information to determine the source data table to which the target operation information belongs and the associated data table related to the source data table comprises:

[0014] The target operation information is parsed to determine a source data table to which the target operation information belongs and an associated data table related to the source data table;

[0015] A syntax tree of the target operation information is generated according to the analysis result;

[0016] The syntax tree is accessed and traversed to determine the source data table to which the target operation information belongs and the associated data table related to the source data table.

[0017] Preferably, the generating of the blood relationship according to the first data table information and the second data table information comprises:

[0018] The generation time of the source data table is determined according to the first data table information to obtain a first generation time;

[0019] The generation time of the associated data table is determined according to the second data table information to obtain a second generation time;

[0020] The source data table and the associated data table are sorted according to the first generation time and the second generation time to obtain a sorting result;

[0021] The blood relationship is generated according to the sorting result.

[0022] In an embodiment, the adding of the operation information of each DDL operation to a preset operation data table comprises:

[0023] The operation information of each DDL operation is added to the preset operation data table after corresponding schema information is added, wherein the schema information is a collection of objects of the relational database, and the objects include data types, indexes, views or stored procedures.

[0024] Preferably, the generating of the blood relationship according to the first data table information and the second data table information comprises:

[0025] obtaining a structured query language code corresponding to the first data table information and the second data table information;

[0026] analyzing the structured query language code corresponding to the first data table information and the second data table information;

[0027] if the operation logic between the source data table and the associated data table is recorded in the structured query language code, it is determined that the source data table and the associated data table have a blood relationship, and a blood relationship between the source data table and the associated data table is generated.

[0028] In an embodiment, the blood relationship is generated according to the first data table information and the second data table information, comprising:

[0029] obtaining a field in the source data table according to the first data table information to obtain a first field;

[0030] obtaining a field in the associated data table according to the second data table information to obtain a second field;

[0031] performing an outer join on the source data table and the associated data table, and calculating a similarity between data of the first field in the source data table and data of the second field in the associated data table;

[0032] if the similarity is greater than a preset similarity threshold, it is determined that the source data table and the associated data table have a blood relationship, and a blood relationship between the source data table and the associated data table is generated.

[0033] Preferably, the similarity between the data of the first field in the source data table and the data of the second field in the associated data table is calculated, comprising:

[0034] converting the data of the first field in the source data table into a vector form to obtain a first vector;

[0035] converting the data of the second field in the associated data table into a vector form to obtain a second vector;

[0036] calculating a cosine distance between the first vector and the second vector to obtain the similarity.

[0037] The application also provides a blood relationship generation device, comprising:

[0038] a first obtaining module for obtaining all DDL operations on a relational database within a preset time period to obtain a plurality of DDL operations;

[0039] An adding module is configured to acquire operation information of each DDL operation, add the operation information of each DDL operation into a preset operation data table, and acquire a label previously assigned to the operation information of each DDL operation, wherein the label is used to mark whether the operation information is subjected to hook processing.

[0040] A screening module is configured to screen out operation information not subjected to hook processing from the operation data table according to the label of each operation information, and obtain target operation information.

[0041] An analyzing module is configured to analyze the target operation information, and determine a source data table to which the target operation information belongs and an associated data table related to the source data table.

[0042] A second acquiring module is configured to read data table information of the source data table and the associated data table respectively, and obtain first data table information of the source data table and second data table information of the associated data table.

[0043] A generating module is configured to generate a blood relationship according to the first data table information and the second data table information.

[0044] The application further provides a computer device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the steps of the method according to any one of the above embodiments when executing the computer program.

[0045] The application further provides a computer readable storage medium, which stores a computer program, and the computer program implements the steps of the method according to any one of the above embodiments when executed by a processor.

[0046] The blood relationship generation method, device, computer equipment and storage medium provided by the application obtain all DDL operations of a relational database in a preset time period to obtain a plurality of DDL operations, obtain operation information of each DDL operation respectively, add the operation information of each DDL operation to a preset operation data table, and obtain a label assigned to the operation information of each DDL operation in advance. According to the label of each operation information, the operation information that has not been processed by hook is filtered out from the operation data table to obtain target operation information. The target operation information is analyzed to determine a source data table to which the target operation information belongs and an associated data table related to the source data table. The data table information of the source data table and the associated data table is read respectively to obtain first data table information of the source data table and second data table information of the associated data table. The blood relationship is generated according to the first data table information and the second data table information. Since the operation information that has not been processed by hook is difficult to track the operation logic between tables, the target operation information that has not been processed by hook is filtered out in the application. The target operation information is analyzed to determine the source data table to which the target operation information belongs and the associated data table related to the source data table. The blood relationship between the source data table and the associated data table is generated through the data table information of the source data table and the associated data table. Therefore, the generation of the blood relationship of the relational database is realized, so that the operation logic between the source data table and the associated data table in the relational database can be clearly known, and the data can be better used. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 A flowchart of a blood relationship generation method according to an embodiment of the application;

[0048] Figure 2 A structural schematic block diagram of a blood relationship generation device according to an embodiment of the application;

[0049] Figure 3 A structural schematic block diagram of a computer equipment according to an embodiment of the application.

[0050] The implementation, functional features and advantages of the application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical scheme and advantages of the application more clear, the application will be further described in detail below with reference to the embodiments and the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the application and not to limit the application.

[0052] The blood relationship generation method provided in the application takes a server as an execution subject, and the server can be an independent server or a cloud server providing basic cloud computing services such as cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, content delivery network (CDN), and big data and artificial intelligence platform.

[0053] The blood relationship generation method is used to solve the technical problem that the existing technology cannot generate the blood relationship of a relational database and cannot clearly know the operation logic between tables in the relational database, so that the data in the tables cannot be better used. Figure 1 In one embodiment, the blood relationship generation method comprises the following steps.

[0054] S11, acquiring all DDL operations on the relational database within a preset time period to obtain a plurality of DDL operations;

[0055] S12, acquiring operation information of each DDL operation respectively, adding the operation information of each DDL operation to a preset operation data table, and acquiring a label pre-assigned to the operation information of each DDL operation; wherein the label is used to mark whether the operation information is processed by hook.

[0056] S13, according to the label of each operation information, screening out operation information not processed by hook from the operation data table to obtain target operation information;

[0057] S14, analyzing the target operation information to determine a source data table to which the target operation information belongs and an associated data table related to the source data table;

[0058] S15, reading data table information of the source data table and the associated data table respectively to obtain first data table information of the source data table and second data table information of the associated data table;

[0059] S16, generating a blood relationship according to the first data table information and the second data table information.

[0060] In the embodiment, a trigger can be defined on the server, the trigger is used to capture all DDL operations of the relational database within a preset time period, and all DDL operations are written into a custom operation data table. The operation data table contains a plurality of DDL operations, operation information of each DDL operation, and a label of the operation information of each DDL operation, which is used to mark whether the operation information is processed by hook.

[0061] DDL (Data Definition Language) is a data definition language used to define database objects: databases, tables, columns, etc., such as creating, deleting, modifying database or table information, etc. The DDL operation of the database includes database viewing, switching, creating and deleting, etc.

[0062] The hook is obtained by reading the interface of the data system, analyzing the flow through the interface to obtain flow data and integrating it, even if different data systems are connected, the corresponding hook can be obtained to realize the hook adaptation to different systems. It can be understood that when the data system executes each SQL statement, the metadata information in the data system will change, at this time, the blood relationship data corresponding to the SQL statement can be obtained through the hook. The blood relationship data corresponding to the SQL statement includes but is not limited to the input data, output data and association information of the SQL statement. The association information includes but is not limited to the execution time start time, execution time length, execution object, username of the user submitting the SQL statement, task name of the SQL statement corresponding task and code of the SQL statement.

[0063] In addition, when processing the hook, a sub-process can be set on the operation information to monitor a certain message of a specified window, and the monitored window can be created by other processes. When the message arrives, it is processed before the target window processing function. We can install a custom hook in the server to monitor the occurrence of a specific event in the server and complete a specific function, such as creating, deleting, modifying database or table information, etc.

[0064] The blood relationship is a data generation relationship based on directly generating one data table from another data table.

[0065] The embodiment filters the operation information that has not been processed by the hook from the operation data table, takes the operation information that has not been processed by the hook as the target operation information, analyzes the target operation information, obtains the source data table to which the target operation information belongs and the dependent associated data table, that is, which source data table of the relational database and its associated data table are operated, then reads the data table information of the source data table and the associated data table respectively, obtains the first data table information of the source data table and the second data table information of the associated data table, and generates the blood relationship according to the first data table information and the second data table information. The operation information includes operation types such as creating, deleting and modifying the relational database. The first data table information includes the generation time and modification time of the source data table, and the second data table information includes the generation time and modification time of the associated data table.

[0066] The data table information is information of defining fields, types, primary keys, foreign keys, indexes and other basic attributes of a table in a database.

[0067] The blood relationship generation method provided in the application obtains all DDL operations on the relational database within a preset time period, obtains a plurality of DDL operations, respectively obtains operation information of each DDL operation, adds the operation information of each DDL operation to a preset operation data table, and obtains a label assigned to the operation information of each DDL operation in advance. According to the label of each operation information, the operation information that has not been processed by hook is filtered out from the operation data table to obtain target operation information. The target operation information is analyzed to determine a source data table to which the target operation information belongs and an associated data table related to the source data table. The data table information of the source data table and the associated data table is respectively read to obtain first data table information of the source data table and second data table information of the associated data table. The blood relationship is generated according to the first data table information and the second data table information. Since the operation information that has not been processed by hook is difficult to track the operation logic between tables, the target operation information that has not been processed by hook is filtered out, the target operation information is analyzed to determine the source data table to which the target operation information belongs and the associated data table related to the source data table, and the blood relationship between the source data table and the associated data table is generated through the data table information of the source data table and the associated data table. Therefore, the generation of the blood relationship of the relational database is realized, so that the operation logic between the source data table and the associated data table in the relational database can be clearly known, and the data can be better used.

[0068] In an embodiment, the analyzing the target operation information to determine the source data table to which the target operation information belongs and the associated data table related to the source data table can specifically include:

[0069] Performing lexical and syntactic analysis on the target operation information to obtain an analysis result;

[0070] Generating a syntax tree of the target operation information according to the analysis result;

[0071] Accessing and traversing the syntax tree to determine the source data table to which the target operation information belongs and the associated data table related to the source data table.

[0072] The embodiment performs lexical and syntax analysis on the target operation information, obtains lexical types, syntax types and syntax change rules used by the target operation information, and forms an analysis result, generates a syntax tree of the target operation information according to the analysis result, and the syntax tree contains lexical types, syntax types and operation data tables used by each stage of the target operation information. The source data table and the associated data table related to the source data table to which the target operation information belongs can be determined by accessing and traversing the syntax tree.

[0073] In an embodiment, the generating the blood relationship according to the first data table information and the second data table information can specifically include:

[0074] determining the generation time of the source data table according to the first data table information to obtain a first generation time;

[0075] determining the generation time of the associated data table according to the second data table information to obtain a second generation time;

[0076] sorting the source data table and the associated data table according to the first generation time and the second generation time to obtain a sorting result;

[0077] generating the blood relationship according to the sorting result.

[0078] The embodiment needs to determine the specific precedence relationship (i.e., the blood relationship) between each data table. Considering that each data table has its generation time, and the generation time difference of different data tables can reflect the generation precedence relationship of each data table. Therefore, the embodiment can use the generation time of each data table to determine the precedence relationship between the data tables.

[0079] Specifically, when the associated data table is multiple, the source data table and all the associated data tables are sorted according to the generation time of the source data table and each associated data table to obtain a sorting result between each data table. The blood relationship between the source data table and each associated data table can be determined according to the sorting result, such as constructing the relationship between the source data table and each associated data table according to the sorting result to obtain the blood relationship. The blood relationship can be in the form of a list or a graph. The source data table and the associated data table represent the input table and the output table of a table, respectively.

[0080] In an embodiment, the adding the operation information of each DDL operation to the preset operation data table can specifically include:

[0081] adding the corresponding schema information to the operation information of each DDL operation and then adding to the preset operation data table; wherein the schema information is a collection of objects of the relational database, and the objects include data types, indexes, views or stored procedures.

[0082] The embodiment needs to externally force the operation information to add corresponding schema information, and add the operation information with the added schema information to the operation data table. The schema information is the organization and structure of the database, and contains schema objects, which can be tables, columns, data types, views, stored procedures, relationships, primary keys, foreign keys, and the like. The schema information can be represented by a visual graph, which shows the database objects and their relationships with each other. In the SQL environment, the schema is a collection of database objects, and the so-called database objects are commonly known as data types, indexes, views, or stored procedures, etc.

[0083] In an embodiment, the generating the blood relationship according to the first data table information and the second data table information can specifically include:

[0084] Obtaining structured query language codes corresponding to the first data table information and the second data table information;

[0085] Analyzing the structured query language codes corresponding to the first data table information and the second data table information;

[0086] If the operation logic between the source data table and the associated data table is recorded in the structured query language code, it is determined that the source data table and the associated data table have a blood relationship, and the blood relationship between the source data table and the associated data table is generated.

[0087] The blood relationship between data tables refers to the data generation relationship based on one data table directly generating another data table. Data tables are usually stored in a relational database. In the process of daily business activities of users, the database needs to be queried, updated and managed, and data needs to be accessed from the database. These data usually exist in the form of data tables. When querying data and managing the database, a structured query language (SQL) needs to be used. The structured query language is a special-purpose programming language that can be used to access data in a database and query, update and manage the database.

[0088] When querying data, a structured query language code corresponding to the query operation is generated. The structured query language code is used to record which data table (i.e. an upstream data table) is used to obtain another data table (i.e. a downstream data table) through what operation logic. The operation logic includes: performing statistics on the data of certain fields in the data table, or involving addition, subtraction, multiplication and division operations of the data.

[0089] The structured query language code records the table names of the upstream data table and the downstream data table and the operation logic therebetween. Therefore, when judging whether the data tables have the blood relationship, the embodiment can analyze the structured query language codes corresponding to the first data table information and the second data table information. In the process of analyzing a large amount of structured query language codes, if it is found that the structured query language code records the operation logic between the data table and the associated data table, it is determined that the data table and the associated data table have the blood relationship, and the blood relationship between the data table and the associated data table can also be generated.

[0090] In an embodiment, the generating the blood relationship according to the first data table information and the second data table information can specifically include:

[0091] obtaining a field in the source data table according to the first data table information to obtain a first field;

[0092] obtaining a field in the associated data table according to the second data table information to obtain a second field;

[0093] performing an outer join on the source data table and the associated data table to calculate the similarity of the data of the first field in the source data table and the data of the second field in the associated data table;

[0094] if the similarity is greater than a preset similarity threshold, determining that the source data table and the associated data table have the blood relationship, and generating the blood relationship between the source data table and the associated data table.

[0095] In the embodiment, the field in the source data table can be obtained to obtain the first field, and the field in the associated data table can be obtained to obtain the second field. The source data table and the associated data table can be subjected to an outer join. The condition of the outer join can be that the data of the fields having the data blood relationship are equal, so as to facilitate mining whether there are column fields having the blood relationship on the basis of the existence of the data blood relationship between the column fields of the two data tables.

[0096] The embodiment can obtain the data of the first field in the source data table and the data of the second field in the associated data table by performing the outer join on the source data table and the associated data table. According to the obtained data, the similarity of the data of the first field in the source data table and the data of the second field in the associated data table can be calculated. For example, the cosine similarity algorithm can be used to calculate the similarity of the data of the first field and the second field. If the calculated similarity is greater than the similarity threshold, it can be determined that the source data table and the associated data table have the blood relationship, and the blood relationship between the source data table and the associated data table is generated. The blood relationship can be mined from multiple aspects, accurate and comprehensive blood relationship can be obtained, and the implementation is convenient and the cost is low. The blood relationship mining can be applied to various databases.

[0097] In an embodiment, the calculating the similarity between the data of the first field in the source data table and the data of the second field in the associated data table can specifically include:

[0098] converting the data of the first field in the source data table into a vector form to obtain a first vector;

[0099] converting the data of the second field in the associated data table into a vector form to obtain a second vector;

[0100] calculating the cosine distance between the first vector and the second vector to obtain the similarity.

[0101] In the embodiment, the data of the first field in the data table can be converted into a vector form by using a word vector tool to obtain a first vector, and the data of the second field in the associated data table can be converted into a vector form by using the word vector tool to obtain a second vector. The word vector tool can include Distributed representation and word2vec. Distributed representation is to map each word into a K-dimensional real number vector (K is generally a hyperparameter in the model) through training, and to determine the semantic similarity between the words through the distance (such as cosine similarity, Euclidean distance, etc.) between the words. Word2vec is an efficient tool for representing words as real-valued vectors. By using the deep learning idea, the processing of text content is simplified to vector operation in a K-dimensional vector space, and the similarity in the vector space can be used to represent the semantic similarity in the text.

[0102] Then, the cosine distance between the first vector and the second vector is calculated respectively, and the cosine distance is taken as the similarity between the data of the first field in the data table and the data of the second field in the associated data table.

[0103] When the data is a word, the word vector output by Word2vec can be used for many natural language processing related work, such as clustering, finding synonyms, part-of-speech analysis, etc. If the word is regarded as a feature, Word2vec can map the feature to a K-dimensional vector space, and can seek a more deep feature representation for text data. Specifically, Word2vec takes the words in the context of a word as input, and takes the word itself as output, that is, seeing a context, it is expected to be able to guess the word and its meaning. Through training in a large corpus, a weight model from the input layer to the hidden layer is obtained, and after the training is completed, the weight of each dimension of each word to the hidden layer is obtained, that is, the vector of each word.

[0104] With reference to Figure 2 In the embodiment, a blood relationship generation device is also provided, which comprises:

[0105] The first obtaining module 11 is configured to obtain all DDL operations on the relational database within a preset time period, and obtain a plurality of DDL operations.

[0106] The adding module 12 is configured to obtain operation information of each DDL operation respectively, add the operation information of each DDL operation to a preset operation data table, and obtain a label pre-assigned to the operation information of each DDL operation, wherein the label is used to mark whether the operation information is subjected to hook processing.

[0107] The screening module 13 is configured to screen out operation information not subjected to hook processing from the operation data table according to the label of each operation information, and obtain target operation information.

[0108] The parsing module 14 is configured to parse the target operation information, and determine a source data table to which the target operation information belongs and an associated data table related to the source data table.

[0109] The second obtaining module 15 is configured to read data table information of the source data table and the associated data table respectively, and obtain first data table information of the source data table and second data table information of the associated data table.

[0110] The generating module 16 is configured to generate a blood relationship according to the first data table information and the second data table information.

[0111] In the embodiment, a trigger can be defined on the server, the trigger is used to capture all DDL operations of the relational database within a preset time period, and all DDL operations are written into a custom operation data table. The operation data table includes a plurality of DDL operations, operation information of each DDL operation, and a label of the operation information of each DDL operation. The label is used to mark whether the operation information is subjected to hook processing.

[0112] The DDL (Data Definition Language) is used to define database objects such as databases, tables, columns, etc., for example, to create, delete, modify database or table information, etc. The DDL operation of the database includes database viewing, switching, creating, deleting, etc.

[0113] The hook is an interface for reading data systems, and the interface analyzes the flow to obtain flow data and integration. Even if different data systems are connected, the corresponding hook can be obtained to realize the hook adaptation to different systems. It can be understood that when the data system executes a SQL statement, the metadata information in the data system will change. At this time, the blood relationship data corresponding to the SQL statement can be obtained through the hook. The blood relationship data corresponding to the SQL statement includes but is not limited to the input data, output data and association information of the SQL statement. The association information includes but is not limited to the execution time start time, execution time length, execution object, username of the user submitting the SQL statement, task name of the SQL statement corresponding task and code of the SQL statement.

[0114] In addition, when the hook is processed, a sub-process can be set on the operation information to monitor a certain message of a specified window, and the monitored window can be created by other processes. When the message arrives, it is processed before the target window processing function. We can install a custom hook in the server to monitor the occurrence of a specific event in the server and complete specific functions, such as creating, deleting, modifying database or table information, etc.

[0115] The blood relationship is a data generation relationship based on direct generation of one data table to another data table.

[0116] The embodiment filters the operation information that has not been processed by the hook from the operation data table, takes the operation information that has not been processed by the hook as the target operation information, analyzes the target operation information, obtains the source data table to which the target operation information belongs and the dependent associated data table, that is, which source data table of the relational database and its associated data table are operated, then reads the data table information of the source data table and the associated data table respectively, obtains the first data table information of the source data table and the second data table information of the associated data table, and generates the blood relationship according to the first data table information and the second data table information. The operation information includes an operation type, such as creating, deleting or modifying a relational database. The first data table information includes the generation time and modification time of the source data table, and the second data table information includes the generation time and modification time of the associated data table.

[0117] The data table information is the definition of the fields, types, primary keys, foreign keys, indexes, etc. of a table in the database, and these basic attributes constitute the data table information of the data table in the database.

[0118] As described above, it can be understood that each component of the blood relationship generation device proposed in the application can realize the function of any one of the blood relationship generation methods described above, and the specific structure will not be described again.

[0119] Reference is made to Figure 3 In an embodiment of the present application, a computer device is also provided, and the internal structure of the computer device can be as shown in Figure 3 The computer device includes a processor, a memory, a network interface and a database connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a storage medium and an internal memory. The storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the storage medium to run. The database of the computer device is configured to store related data of the blood relationship generation method. The network interface of the computer device is configured to communicate with an external terminal through a network connection. The computer program is executed by the processor to implement a blood relationship generation method.

[0120] The processor executes the blood relationship generation method, and the blood relationship generation method includes the following steps.

[0121] All DDL operations on the relational database within a preset time period are obtained to obtain a plurality of DDL operations.

[0122] Operation information of each DDL operation is obtained, and the operation information of each DDL operation is added to a preset operation data table. A label previously assigned to the operation information of each DDL operation is obtained. The label is used to mark whether the operation information has been processed by hook.

[0123] According to the label of each operation information, operation information that has not been processed by hook is filtered from the operation data table to obtain target operation information.

[0124] The target operation information is parsed to determine a source data table to which the target operation information belongs and an associated data table related to the source data table.

[0125] Data table information of the source data table and the associated data table is read respectively to obtain first data table information of the source data table and second data table information of the associated data table.

[0126] The blood relationship is generated according to the first data table information and the second data table information.

[0127] An embodiment of the present application also provides a computer readable storage medium having a computer program stored thereon. The computer program is executed by a processor to implement a blood relationship generation method, and the blood relationship generation method includes the following steps.

[0128] All DDL operations on the relational database within a preset time period are obtained to obtain a plurality of DDL operations.

[0129] Obtain operation information of each DDL operation respectively, add the operation information of each DDL operation to a preset operation data table, and obtain a label previously given to the operation information of each DDL operation; wherein the label is used to mark whether the operation information has been processed by hook;

[0130] According to the label of each operation information, filter out operation information that has not been processed by hook from the operation data table to obtain target operation information;

[0131] Parse the target operation information, determine a source data table to which the target operation information belongs and an associated data table related to the source data table;

[0132] Respectively read data table information of the source data table and the associated data table to obtain first data table information of the source data table and second data table information of the associated data table;

[0133] Generate blood relationship according to the first data table information and the second data table information.

[0134] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The computer program can be stored in a computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium provided by the present application and used in the embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM) and memory bus dynamic RAM (RDRAM) and the like.

[0135] In summary, the greatest benefit of the present application is:

[0136] The blood relationship generation method and device, computer device and storage medium provided by the application obtain all DDL operations of a relational database in a preset time period, obtain a plurality of DDL operations, obtain operation information of each DDL operation respectively, add the operation information of each DDL operation to a preset operation data table, and obtain a label assigned to the operation information of each DDL operation in advance. According to the label of each operation information, the operation information that has not been processed by hook is filtered out from the operation data table to obtain target operation information. The target operation information is analyzed to determine a source data table to which the target operation information belongs and an associated data table related to the source data table. The data table information of the source data table and the associated data table is read respectively to obtain first data table information of the source data table and second data table information of the associated data table. The blood relationship is generated according to the first data table information and the second data table information. Since the operation information that has not been processed by hook is difficult to track the operation logic between tables, the target operation information that has not been processed by hook is filtered out, the target operation information is analyzed to determine the source data table to which the target operation information belongs and the associated data table related to the source data table, and the blood relationship between the source data table and the associated data table is generated through the data table information of the source data table and the associated data table, so that the generation of the blood relationship of the relational database is realized, and the operation logic between the source data table and the associated data table in the relational database is clearly known, so that the data can be better used.

[0137] It should be noted that in this document, the terms "comprising", "including", or any other variant thereof are intended to cover non-exclusive inclusions, such that processes, devices, articles or methods including a series of elements not only include those elements, but also include other elements not explicitly listed, or include elements inherent to such processes, devices, articles or methods. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0138] The above is only the preferred embodiment of the application, and does not limit the patent scope of the application. Any equivalent structure or equivalent process transformation using the content of the specification and drawings, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the application.

Claims

1. A method for generating a blood relationship, characterized by, The method comprises the following steps: acquiring all DDL operations on a relational database within a preset time period to obtain a plurality of DDL operations; acquiring operation information of each DDL operation respectively, adding the operation information of each DDL operation to a preset operation data table, and acquiring a label previously assigned to the operation information of each DDL operation; wherein the label is used to mark whether the operation information has been processed by hook; according to the label of each operation information, screening out operation information that has not been processed by hook from the operation data table to obtain target operation information; parsing the target operation information to determine a source data table to which the target operation information belongs and an associated data table related to the source data table; respectively reading data table information of the source data table and the associated data table to obtain first data table information of the source data table and second data table information of the associated data table; generating a blood relationship according to the first data table information and the second data table information; wherein the generating of the blood relationship according to the first data table information and the second data table information comprises: determining the generation time of the source data table according to the first data table information to obtain a first generation time; determining the generation time of the associated data table according to the second data table information to obtain a second generation time; sorting the source data table and the associated data table according to the first generation time and the second generation time to obtain a sorting result; generating a blood relationship according to the sorting result; wherein when the associated data table is multiple, the source data table and all the associated data tables are sorted according to the generation time of the source data table and each associated data table to obtain a sorting result between each data table, the blood relationship between the source data table and each associated data table can be determined according to the sorting result, and the source data table and the associated data table represent the input table and the output table of a table respectively; wherein the adding of the operation information of each DDL operation to the preset operation data table comprises: adding corresponding schema information to the operation information of each DDL operation and then adding it to the preset operation data table; wherein the schema information is a collection of objects of the relational database, and the objects include data types, indexes, views or stored procedures.

2. The method of claim 1, wherein, The parsing of the target operation information to determine the source data table to which the target operation information belongs and the associated data table related to the source data table comprises: performing lexical and syntactic analysis on the target operation information to obtain an analysis result; generating a syntax tree of the target operation information according to the analysis result; visiting and traversing the syntax tree to determine the source data table to which the target operation information belongs and the associated data table related to the source data table.

3. The method of claim 1, wherein, The generating of the blood relationship according to the first data table information and the second data table information comprises: acquiring structured query language codes corresponding to the first data table information and the second data table information; analyzing the structured query language codes corresponding to the first data table information and the second data table information; If the operation logic between the source data table and the associated data table is recorded in the structured query language code, it is determined that there is a blood relationship between the source data table and the associated data table, and a blood relationship between the source data table and the associated data table is generated.

4. The method of claim 1, wherein, The blood relationship is generated according to the first data table information and the second data table information, including: According to the first data table information, the fields in the source data table are obtained, and the first fields are obtained. According to the second data table information, the fields in the associated data table are obtained, and the second fields are obtained. The source data table and the associated data table are outer joined, and the similarity of the data of the first field in the source data table and the data of the second field in the associated data table is calculated. If the similarity is greater than a preset similarity threshold, it is determined that there is a blood relationship between the source data table and the associated data table, and a blood relationship between the source data table and the associated data table is generated.

5. The method of claim 4, wherein, The similarity of the data of the first field in the source data table and the data of the second field in the associated data table is calculated, including: The data of the first field in the source data table is converted into a vector form to obtain a first vector. The data of the second field in the associated data table is converted into a vector form to obtain a second vector. The cosine distance of the first vector and the second vector is calculated to obtain the similarity.

6. A blood relationship generating apparatus characterized by comprising: Including: The first acquisition module is used for acquiring all DDL operations of the relational database in a preset time period to obtain a plurality of DDL operations. The adding module is used for acquiring operation information of each DDL operation respectively, adding the operation information of each DDL operation to a preset operation data table, and acquiring a label assigned to the operation information of each DDL operation in advance; wherein the label is used to mark whether the operation information is processed by hook. The screening module is used for screening operation information which is not processed by hook from the operation data table according to the label of each operation information to obtain target operation information. The analysis module is used for analyzing the target operation information to determine the source data table to which the target operation information belongs and the associated data table related to the source data table. The second acquisition module is used for reading data table information of the source data table and the associated data table respectively to obtain first data table information of the source data table and second data table information of the associated data table. The generation module is used for generating a blood relationship according to the first data table information and the second data table information. The blood relationship is generated according to the first data table information and the second data table information, including: According to the first data table information, the generation time of the source data table is determined to obtain a first generation time. According to the second data table information, the generation time of the associated data table is determined to obtain a second generation time. According to the first generation time and the second generation time, the source data table and the associated data table are sorted to obtain a sorting result. The blood relationship is generated according to the sorting result. When the association data tables are multiple, the source data table and all the association data tables are sorted according to the generation time of the source data table and each association data table, and the sorting result between each data table is obtained, the blood relationship between the source data table and each association data table can be determined according to the sorting result, and the source data table and the association data table respectively represent the input table and the output table of a table. The operation information of each DDL operation is added to a preset operation data table, including: The operation information of each DDL operation is added to a preset operation data table, including:

7. A computer device, characterized by The operation information of each DDL operation is added to a preset operation data table, including: The operation information of each DDL operation is added to a preset operation data table, including: A processor; A memory; 8. A computer-readable storage medium, characterized in that, The memory stores a computer program, and the processor executes the computer program to realize the blood relationship generation method in any one of claims 1 to 5. The computer readable storage medium stores a computer program, and the processor executes the computer program to realize the blood relationship generation method in any one of claims 1 to 5.

Images