Data table lookup method and device, equipment, storage medium and computer program product

Abstract

The invention discloses a data table lookup method and device, equipment, a storage medium and a computer program product, and relates to the technical field of data lookup, the method comprises the following steps: when a data table query statement is received, converting the data table query statement into a semantic vector; performing similarity matching on the semantic vector and a table vector set and a non-table vector set in a to-be-queried database to obtain a first similarity set and a second similarity set; the initialization vector is normalized based on the first similarity set and the second similarity set, a normalized vector is obtained, and the initialization vector is constructed based on the knowledge graph of the database to be queried; and calculating a correlation score, corresponding to the data table query statement, of each component in the normalized vector, and searching a target data table from the to-be-queried database based on the correlation score. Based on the method, deep mining and relational network construction are performed on the to-be-queried database, so that the target data table can be accurately searched from the database.

Description

Technical Field

[0001] This application relates to the field of data retrieval technology, and in particular to a data table retrieval method, apparatus, device, storage medium, and computer program product. Background Technology

[0002] In the financial industry, data analysts need to quickly locate target data tables that meet business needs in a database containing massive amounts of data tables. This process (referred to as "data finding") is a key prerequisite for data-driven business decisions.

[0003] Current mainstream data retrieval methods rely on matching keywords in the query statement to the database. This approach can fail to find the target table when there's a semantic difference between the business requirements expressed by the keywords and the table's metadata (such as table name, comments, and database). For example, data related to "utility bill payment coupons" might actually be stored in the "mobile banking activities" table, but since the keywords don't directly overlap, the search results could be inaccurate. Therefore, the industry urgently needs a method that can accurately locate the target table even when there's a semantic difference between the keywords in the query statement and the target table. Summary of the Invention

[0004] The main objective of this application is to provide a data table lookup method, apparatus, device, storage medium, and computer program product, which aims to solve the technical problem that existing technologies cannot accurately find the target data table from the database when there is a semantic difference between the keywords in the data table query statement and the target data table.

[0005] To achieve the above objectives, this application provides a data table lookup method, which includes the following steps:

[0006] Upon receiving a data table query statement, the data table query statement is converted into a semantic vector;

[0007] The semantic vectors are matched with the set of table vectors and the set of non-table vectors in the database to be queried to obtain the first similarity set and the second similarity set.

[0008] The initialization vector is normalized based on the first similarity set and the second similarity set to obtain a normalized vector. The initialization vector is constructed based on the knowledge graph of the database to be queried.

[0009] Calculate the relevance score of each component in the normalized vector corresponding to the query statement of the data table, and search for the target data table from the database to be queried based on the relevance score.

[0010] In one embodiment, before the step of performing similarity matching between the semantic vector and the set of table vectors and the set of non-table vectors in the database to be queried, respectively, to obtain a first similarity set and a second similarity set, the method further includes:

[0011] Obtain the knowledge graph of the database to be queried, and classify the nodes contained in the knowledge graph to obtain table type nodes and non-table type nodes;

[0012] The table-type nodes and the non-table-type nodes are vectorized respectively to obtain the table vector set and the non-table vector set of the database to be queried.

[0013] In one embodiment, the knowledge graph is constructed based on the node dependencies of the database to be queried, and the node dependencies are determined based on the following steps:

[0014] Obtain the node attributes of each node in the database to be queried, wherein the node attributes include node ID, node name, node type, and node content;

[0015] The node dependencies of the database to be queried are determined based on the node attributes. The node dependencies include at least the lineage relationships between tables and between tables and the business system.

[0016] In one embodiment, the step of normalizing the initialization vector based on the first similarity set and the second similarity set to obtain a normalized vector includes:

[0017] A first vector set is selected from the set of table vectors based on the similarity ranking of the first similarity set, and a second vector set is selected from the set of non-table vectors based on the similarity ranking of the second similarity set.

[0018] Construct an initialization vector, the length of which is the same as the number of nodes in the knowledge graph of the database to be queried, and the initialization components of the initialization vector correspond one-to-one with the nodes of the knowledge graph;

[0019] The initialization components are assigned values ​​based on the first vector set and the second vector set to obtain the assigned components;

[0020] The initialization vector is normalized based on the assigned components to obtain a normalized vector.

[0021] In one embodiment, the step of assigning values ​​to the initialization components based on the first vector set and the second vector set includes:

[0022] Assign the similarity score of the first vector set in the first similarity set to the initialization component corresponding to the first vector set;

[0023] The similarity score of the second vector set in the second similarity set is assigned to the initialization component corresponding to the second vector set.

[0024] In one embodiment, the step of calculating the relevance score of each component in the normalized vector corresponding to the query statement of the data table, and searching for the target data table from the database to be queried based on the relevance score, includes:

[0025] Based on the normalized vector and the adjacency matrix of the knowledge graph, calculate the relevance score of each component in the normalized vector corresponding to the query statement in the data table.

[0026] After selecting the target component from the normalized vector based on the ranking of the relevance scores, the target data table corresponding to the target component is searched from the database to be queried.

[0027] Furthermore, to achieve the above objectives, this application also proposes a data table lookup device, which includes:

[0028] The data conversion module is used to convert the data table query statement into a semantic vector when it receives the data table query statement;

[0029] The similarity matching module is used to perform similarity matching between the semantic vector and the set of table vectors and the set of non-table vectors in the database to be queried, respectively, to obtain a first similarity set and a second similarity set;

[0030] The vector normalization module is used to normalize the initialization vector based on the first similarity set and the second similarity set to obtain a normalized vector, wherein the initialization vector is constructed based on the knowledge graph of the database to be queried;

[0031] The data table lookup module is used to calculate the relevance score of each component in the normalized vector corresponding to the data table query statement, and to search for the target data table from the database to be queried based on the relevance score.

[0032] In addition, to achieve the above objectives, this application also proposes a data table lookup device, the device comprising: a memory, a processor, and a data table lookup program stored in the memory and executable on the processor, the data table lookup program being configured to implement the steps of the data table lookup method as described above.

[0033] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a data table lookup program is stored, and when the data table lookup program is executed by a processor, it implements the steps of the data table lookup method as described above.

[0034] In addition, to achieve the above objectives, the present invention also provides a computer program product, the computer program product including a data table lookup program, which, when executed by a processor, implements the steps of the data table lookup method as described above.

[0035] Upon receiving a data table query statement, this application converts the query statement into a semantic vector; it then performs similarity matching between the semantic vector and the set of table vectors and the set of non-table vectors in the database to be queried, respectively, to obtain a first similarity set and a second similarity set; based on the first similarity set and the second similarity set, it normalizes the initialization vector to obtain a normalized vector, which is constructed based on the knowledge graph of the database to be queried; it calculates the relevance score of each component in the normalized vector corresponding to the data table query statement, and searches for the target data table in the database to be queried based on the relevance score. The method described in this application performs similarity matching between the semantic vector corresponding to the query statement of the data table and the set of table vectors and non-table vectors in the database to be queried. The obtained first similarity set and second similarity set are used to normalize the initial vector constructed based on the knowledge graph of the database to be queried to obtain a normalized vector. Finally, the target data table is searched from the database to be queried based on the relevance score of each component in the normalized vector corresponding to the query statement of the data table. Thus, while performing in-depth mining of the database to be queried, a relationship network between the data of each node in the database to be queried is also constructed. This enables the accurate retrieval of the target data table from the database even when there are semantic differences between the keywords in the query statement of the data table and the target data table. Attached Figure Description

[0036] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0037] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a flowchart illustrating the first embodiment of the data table lookup method of this application;

[0039] Figure 2 This is a flowchart illustrating the second embodiment of the data table lookup method of this application;

[0040] Figure 3 This is a schematic diagram of the node vectorization of the data table lookup method in this application;

[0041] Figure 4 This is a flowchart illustrating the third embodiment of the data table lookup method of this application;

[0042] Figure 5 This is a diagram illustrating the administrator permission acquisition method for the data table lookup in this application;

[0043] Figure 6 This is a structural block diagram of the first embodiment of the data table lookup device of this application;

[0044] Figure 7 This is a schematic diagram of the structure of the data table lookup device for the hardware operating environment involved in the embodiments of this application.

[0045] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0046] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of this application and are not intended to limit this application.

[0047] It should be noted that the executing entity of the embodiments of this application can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device capable of performing the above functions, such as the aforementioned data table lookup device. The following embodiments will be described using a data table lookup device as an example.

[0048] This application provides a data table lookup method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the data table lookup method of this application.

[0049] In this embodiment, the data table lookup method includes the following steps:

[0050] Step S10: Upon receiving a data table query statement, convert the data table query statement into a semantic vector.

[0051] It should be understood that the above data table query statement can be a natural statement issued by a user intending to find the target data table from the database to be queried, such as "find the data table corresponding to the coupon data of users who paid their utility bills last week". After receiving the above data table query statement, a pre-trained text embedding model (such as Sentence-BERT, SimCSE, etc.) can be used to convert the data table query statement into a high-dimensional, numerical vector (i.e., the above semantic vector). The position of this vector in the vector space represents the semantics of the data table query statement.

[0052] Step S20: Perform similarity matching between the semantic vector and the set of table vectors and the set of non-table vectors in the database to be queried, respectively, to obtain the first similarity set and the second similarity set.

[0053] It should be noted that the aforementioned table vector set can be a set of semantic vectors corresponding to table data (i.e., table metadata, such as table name, comments, and database) in the database to be queried, and the aforementioned non-table vector set can be a set of semantic vectors corresponding to non-table data in the database to be queried. Non-table data can include metadata of fields (such as field name, field comments, and database), metadata of field code values ​​(such as field code value name, field code value comments, and database), and metadata of the business system (such as business system name, business system comments, and database). This embodiment does not impose any limitations on this.

[0054] In a specific implementation, the cosine similarity between the semantic vector corresponding to the query statement in the data table and the semantic vectors contained in the table vector set can be calculated to achieve similarity matching, thereby obtaining the first similarity set mentioned above; the cosine similarity between the semantic vector corresponding to the query statement in the data table and the semantic vectors contained in the non-table vector set can be calculated to achieve similarity matching, thereby obtaining the second similarity set mentioned above.

[0055] Step S30: Normalize the initialization vector based on the first similarity set and the second similarity set to obtain a normalized vector. The initialization vector is constructed based on the knowledge graph of the database to be queried.

[0056] It should be noted that the initialization vector mentioned above is a vector of length N, where N is the total number of nodes in the knowledge graph. Each position in the initialization vector corresponds to a specific node in the knowledge graph (such as a table, a field, or a business system), and each node in the knowledge graph corresponds one-to-one with table data or non-table data in the database to be queried.

[0057] In practical implementation, a first number of table nodes with high relevance to the data table query statement can be selected from the knowledge graph based on the first similarity set mentioned above, and a second number of non-table nodes with high relevance to the data table query statement can be selected from the knowledge graph based on the second similarity set mentioned above. The criteria for determining high relevance to the data table query statement can refer to the first and second similarity sets mentioned above. For example, the table nodes corresponding to the top N similarity values ​​(N can be customized, for example, set to 10) in the first similarity set can be determined as table nodes with high relevance to the data table query statement, and the non-table nodes corresponding to the top M similarity values ​​(M can be customized, for example, set to 5) in the second similarity set can be determined as non-table nodes with high relevance to the data table query statement. Then, the initialization vector is normalized based on these table nodes and non-table nodes with high relevance to the data table query statement, thereby obtaining the normalized vector mentioned above.

[0058] Step S40: Calculate the relevance score of each component in the normalized vector corresponding to the query statement of the data table, and search for the target data table from the database to be queried based on the relevance score.

[0059] It should be understood that, based on the specific implementation process of the above steps, the normalized vector only considers the relevance of table data, non-table data in the database to be queried, and the data table query statement from the perspective of semantic matching. This method may miss some target data tables when there are semantic differences between the business requirements expressed in the data table query statement and the table data and non-table data in the database to be queried. For example, data related to "utility payment coupons" is actually stored in the "mobile banking activities" data table, but the keywords of the two do not directly overlap, leading to deviations in the search results. Therefore, in this specific implementation, the PPR (Personalized PageRank) algorithm can be used to calculate the relevance score of each component in the normalized vector corresponding to the data table query statement. This allows for simultaneous consideration of semantic matching and node structure association from the perspective of the relevance score, ultimately finding the target data table related to the data table query statement from the database to be queried based on the relevance score.

[0060] In this embodiment, upon receiving a data table query statement, the query statement is converted into a semantic vector. The semantic vector is then matched with the set of table vectors and the set of non-table vectors in the database to be queried, respectively, to obtain a first similarity set and a second similarity set. Based on the first and second similarity sets, an initialization vector is normalized to obtain a normalized vector, which is constructed based on the knowledge graph of the database to be queried. The relevance score of each component in the normalized vector corresponding to the data table query statement is calculated, and the target data table is searched from the database to be queried based on the relevance score. In this embodiment, the method described above performs similarity matching between the semantic vector corresponding to the query statement of the data table and the set of table vectors and non-table vectors in the database to be queried. The obtained first similarity set and second similarity set are used to normalize the initial vector constructed based on the knowledge graph of the database to be queried to obtain a normalized vector. Finally, the target data table is searched from the database to be queried based on the relevance score of each component in the normalized vector corresponding to the query statement of the data table. Thus, while performing in-depth mining of the database to be queried, a relationship network between the data of each node in the database to be queried is also constructed. This enables the accurate search of the target data table from the database even when there is a semantic difference between the keywords in the query statement of the data table and the target data table.

[0061] refer to Figure 2 , Figure 2 This is a flowchart illustrating the second embodiment of the data table lookup method of this application.

[0062] In one feasible implementation, prior to step S20, the following may also be included:

[0063] Step S11: Obtain the knowledge graph of the database to be queried, and classify the nodes contained in the knowledge graph to obtain table type nodes and non-table type nodes.

[0064] It should be noted that the above-mentioned table-type nodes can be nodes in the knowledge graph that correspond to table data in the database to be queried, and the above-mentioned non-table-type nodes can be nodes in the knowledge graph that correspond to non-table data in the database to be queried.

[0065] Step S12: Vectorize the table type nodes and the non-table type nodes respectively to obtain the table vector set and non-table vector set of the database to be queried.

[0066] In the specific implementation, you can refer to Figure 3 , Figure 3 This is a schematic diagram of the node vectorization of the data table lookup method in this application. Combined with... Figure 3For table-type nodes, a large model (such as a pre-trained text embedding model) can be used to vectorize the table name and table summary corresponding to the table-type node to obtain the semantic vector for the table node. For non-table-type nodes, a large model can be used to vectorize the field name and field annotation corresponding to the field node to obtain the semantic vector for the field node; the description corresponding to the field code value node can be vectorized to obtain the semantic vector for the field code value node; and the system description and system name corresponding to the business system node can be vectorized to obtain the semantic vector for the business system node.

[0067] In one feasible implementation, the knowledge graph is constructed based on the node dependencies of the database to be queried, and the node dependencies are determined based on the following steps:

[0068] Step S1: Obtain the node attributes of each node in the database to be queried. The node attributes include node ID, node name, node type, and node content.

[0069] Step S2: Determine the node dependency relationship of the database to be queried based on the node attributes. The node dependency relationship includes at least the lineage relationship between tables and the lineage relationship between tables and business systems.

[0070] It should be noted that the database to be queried contains four types of nodes: "tables," "fields," "field code values," and "business systems." Each node has four attributes: node ID, node name, node type, and node content. The node ID is automatically assigned by the knowledge graph; the node types are table, field, field code value, and business system; the node names are the corresponding table name, field name, field code value name, and business system name; and the node content is the corresponding table summary, field comment, field code value, and business system summary. Furthermore, this embodiment also performs data collection and lineage extraction: collecting basic information such as table creation statements, field definitions, and the database system to which the tables belong in the database to be queried; extracting upstream and downstream dependencies between data tables (such as data extraction, transformation, and loading chains); and clarifying the lineage relationships between tables and between tables and business systems. In addition, this embodiment supplements and enhances the metadata of each node: for fields lacking comments, this embodiment uses the table name, other known fields and their comments, and the business background of the table as context input to the large model to generate field comments; for each table, this embodiment combines the meaning of all fields in the table, data flow (upstream and downstream systems / table associations), and other information to generate a detailed table summary using the large model; for business systems, this embodiment inputs the table summary and system business positioning, and the large model generates a business system summary.

[0071] Based on the above data collection, lineage extraction, and metadata supplementation and enhancement, this embodiment can define node dependency relationships as "containing field" (table-field), "field code value" (field-code value), "belonging to system" (table-business system), and "upstream table, downstream table" (table-table lineage relationship), thereby constructing a complete and multi-relationship knowledge graph corresponding to the database to be queried.

[0072] This embodiment acquires a knowledge graph of the database to be queried and classifies the nodes contained in the knowledge graph to obtain table-type nodes and non-table-type nodes. The table-type nodes and non-table-type nodes are vectorized respectively to obtain a set of table vectors and a set of non-table vectors for the database to be queried. The knowledge graph is constructed based on the node dependencies of the database to be queried, and the node dependencies are determined based on the following steps: obtaining the node attributes of each node in the database to be queried, the node attributes including node ID, node name, node type, and node content; determining the node dependencies of the database to be queried based on the node attributes, the node dependencies at least including the lineage relationships between tables and between tables and business systems. This embodiment, by vectorizing the table-type nodes and non-table-type nodes in the knowledge graph respectively to obtain the set of table vectors and a set of non-table vectors for the database to be queried, provides a data foundation for subsequent semantic matching between the database to be queried and the data table query statements. Simultaneously, this embodiment also constructs a knowledge graph based on the node dependencies of the database to be queried, thereby providing a data foundation for subsequent node structure matching between the database to be queried and the data table query statements.

[0073] refer to Figure 4 , Figure 4 This is a flowchart illustrating the third embodiment of the data table lookup method of this application.

[0074] In one feasible implementation, step S30 may include:

[0075] Step S301: Select a first vector set from the table vector set according to the similarity ranking of the first similarity set, and select a second vector set from the non-table vector set according to the similarity ranking of the second similarity set.

[0076] It should be noted that the first vector set mentioned above includes table vectors from several table vector sets that are highly relevant to the data table query statement, and the second vector set mentioned above includes non-table vectors from several non-table vector sets that are highly relevant to the data table query statement.

[0077] Step S302: Construct an initialization vector. The length of the initialization vector is the same as the number of nodes contained in the knowledge graph of the database to be queried, and the initialization components of the initialization vector correspond one-to-one with the nodes of the knowledge graph.

[0078] Step S303: Assign values ​​to the initialization components based on the first vector set and the second vector set to obtain the assigned components.

[0079] Step S304: Normalize the initialization vector based on the assigned components to obtain a normalized vector.

[0080] In the specific implementation, the initialization components can be assigned values ​​based on the first and second vector sets. After assignment, all non-zero score values ​​in the initialization components are scaled so that the sum of all elements in the entire initialization vector is 1. This is a standard prerequisite for the PPR algorithm; this vector represents the probability that the Random Walker starts from each node. After normalization, nodes with scores have a higher probability, nodes with higher scores have an even higher probability, and nodes with zero scores have a probability of 0.

[0081] In one feasible implementation, step S303 may include:

[0082] Step S3031: Assign the similarity score of the first vector set in the first similarity set to the initialization component corresponding to the first vector set.

[0083] Step S3032: Assign the similarity score of the second vector set in the second similarity set to the initialization component corresponding to the second vector set.

[0084] In the specific implementation, assuming that the first vector set is the top N (N value can be customized, for example, it can be set to 10) preferred table vectors in the table vector set ranked by similarity, and the second similarity set is the top M (M value can be customized, for example, it can be set to 5) preferred non-table vectors in the non-table vector set ranked by similarity, then the similarity scores of the above N preferred table vectors and the above M preferred non-table vectors can be assigned to their respective initialization components. For other initialization components (i.e., the initialization components corresponding to other vectors besides the preferred table vectors and preferred non-table vectors), the value can be assigned to 0.

[0085] In one feasible implementation, step S40 may include:

[0086] Step S401: Calculate the relevance score of each component in the normalized vector corresponding to the query statement in the data table based on the normalized vector and the adjacency matrix of the knowledge graph.

[0087] Step S402: After selecting the target component from the normalized vector according to the ranking of the correlation scores, search for the target data table corresponding to the target component in the database to be queried.

[0088] Understandably, in this embodiment, the normalized vector and the adjacency matrix of the knowledge graph can be input into the PPR algorithm for calculation, thereby obtaining the relevance score of each component in the normalized vector corresponding to the query statement in the data table. Finally, based on the ranking of the relevance scores, the target component is selected from the normalized vector (for example, the top 10% of the components in the normalized vector are the target components), and the target data table corresponding to the target component is searched from the database to be queried. The normalized vector can represent the initial preference distribution of the PPR algorithm, used to tell the algorithm user which nodes are of most interest.

[0089] Furthermore, all database-related data involved in this embodiment and the above embodiments were obtained with the administrator's permission; that is, when this application is applied to a specific product or technology, the administrator's permission is required to obtain and process the relevant data, and the processing of the relevant data must comply with the relevant laws, regulations, and regulatory standards of the relevant countries and regions. For example, when it is necessary to obtain data from a database to be queried, a data retrieval prompt can be displayed in the administrator's management interface. After receiving confirmation from the administrator regarding the data retrieval prompt, the terminal can obtain the data from the database to be queried. Figure 5 As shown, Figure 5 This is a diagram illustrating the administrator permission acquisition method for the data table lookup in this application.

[0090] In this embodiment, a first vector set is selected from the table vector set based on the similarity ranking of the first similarity set, and a second vector set is selected from the non-table vector set based on the similarity ranking of the second similarity set. An initialization vector is constructed, the length of which is the same as the number of nodes in the knowledge graph of the database to be queried, and the initialization components of the initialization vector correspond one-to-one with the nodes of the knowledge graph. The similarity score of the first vector set in the first similarity set is assigned to the initialization component corresponding to the first vector set, and the similarity score of the second vector set in the second similarity set is assigned to the initialization component corresponding to the second vector set, resulting in assigned components. The initialization vector is normalized based on the assigned components to obtain a normalized vector. The relevance score of each component in the normalized vector corresponding to the query statement of the data table is calculated based on the normalized vector and the adjacency matrix of the knowledge graph. After selecting the target component from the normalized vector according to the ranking of the relevance scores, the target data table corresponding to the target component is searched in the database to be queried. In this embodiment, the method described above assigns values ​​to the initialization components based on the first set of vectors with the highest similarity ranking in the table vector set and the second set of vectors with the highest similarity ranking in the non-table vector set. Then, based on the assigned components, the initialization vectors are normalized into normalized vectors, thereby clarifying the relevance between the table vectors, non-table vectors, and data table query statements in the database to be queried from the perspective of semantic matching. Furthermore, this embodiment also inputs the normalized vectors and the adjacency matrix of the knowledge graph into the PPR algorithm for calculation to obtain the relevance score of each component in the normalized vector corresponding to the data table query statement. This clarifies the relevance between the table vectors, non-table vectors, and data table query statements in the database to be queried from the perspective of combining semantic matching and node structure association, thus providing the final direction for finding the target data table.

[0091] Reference Figure 6 , Figure 6 This is a structural block diagram of the first embodiment of the data table lookup device of this application.

[0092] like Figure 6 As shown, the data table lookup device proposed in this application includes:

[0093] The data conversion module 601 is used to convert the data table query statement into a semantic vector when it receives the data table query statement;

[0094] The similarity matching module 602 is used to perform similarity matching between the semantic vector and the set of table vectors and the set of non-table vectors in the database to be queried, respectively, to obtain a first similarity set and a second similarity set;

[0095] The vector normalization module 603 is used to normalize the initialization vector based on the first similarity set and the second similarity set to obtain a normalized vector, wherein the initialization vector is constructed based on the knowledge graph of the database to be queried;

[0096] The data table lookup module 604 is used to calculate the relevance score of each component in the normalized vector corresponding to the data table query statement, and to search for the target data table from the database to be queried based on the relevance score.

[0097] In this embodiment, upon receiving a data table query statement, the query statement is converted into a semantic vector. The semantic vector is then matched with the set of table vectors and the set of non-table vectors in the database to be queried, respectively, to obtain a first similarity set and a second similarity set. Based on the first and second similarity sets, an initialization vector is normalized to obtain a normalized vector, which is constructed based on the knowledge graph of the database to be queried. The relevance score of each component in the normalized vector corresponding to the data table query statement is calculated, and the target data table is searched from the database to be queried based on the relevance score. In this embodiment, the method described above performs similarity matching between the semantic vector corresponding to the query statement of the data table and the set of table vectors and non-table vectors in the database to be queried. The obtained first similarity set and second similarity set are used to normalize the initial vector constructed based on the knowledge graph of the database to be queried to obtain a normalized vector. Finally, the target data table is searched from the database to be queried based on the relevance score of each component in the normalized vector corresponding to the query statement of the data table. Thus, while performing in-depth mining of the database to be queried, a relationship network between the data of each node in the database to be queried is also constructed. This enables the accurate search of the target data table from the database even when there is a semantic difference between the keywords in the query statement of the data table and the target data table.

[0098] Based on the first embodiment of the data table lookup device described in this application, a second embodiment of the data table lookup device of this application is proposed.

[0099] In this embodiment, the similarity matching module 602 is further configured to obtain the knowledge graph of the database to be queried, and classify the nodes contained in the knowledge graph to obtain table type nodes and non-table type nodes; and to vectorize the table type nodes and the non-table type nodes respectively to obtain the table vector set and non-table vector set of the database to be queried.

[0100] Furthermore, the knowledge graph is constructed based on the node dependencies of the database to be queried. The node dependencies are determined based on the following steps: obtaining the node attributes of each node in the database to be queried, wherein the node attributes include node ID, node name, node type, and node content; determining the node dependencies of the database to be queried based on the node attributes, wherein the node dependencies include at least the lineage relationships between tables and the lineage relationships between tables and business systems.

[0101] Furthermore, the vector normalization module 603 is also used to select a first vector set from the table vector set according to the similarity ranking of the first similarity set, and select a second vector set from the non-table vector set according to the similarity ranking of the second similarity set; construct an initialization vector, the length of which is the same as the number of nodes contained in the knowledge graph of the database to be queried, and the initialization components of the initialization vector correspond one-to-one with the nodes of the knowledge graph; assign values ​​to the initialization components according to the first vector set and the second vector set to obtain assigned components; and normalize the initialization vector based on the assigned components to obtain a normalized vector.

[0102] Furthermore, the vector normalization module 603 is also used to assign the similarity score of the first vector set in the first similarity set to the initialization component corresponding to the first vector set; and to assign the similarity score of the second vector set in the second similarity set to the initialization component corresponding to the second vector set.

[0103] Furthermore, the data table lookup module 604 is also used to calculate the relevance score of each component in the normalized vector corresponding to the data table query statement based on the normalized vector and the adjacency matrix of the knowledge graph; after selecting the target component from the normalized vector according to the ranking of the relevance scores, it searches for the target data table corresponding to the target component in the database to be queried.

[0104] Other embodiments or specific implementations of the data table lookup device of this application can be found in the above-described method embodiments, and will not be repeated here.

[0105] This application provides a data table lookup device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the data table lookup method in Embodiment 1 above.

[0106] The following is for reference. Figure 7The diagram illustrates a structural schematic of a data table lookup device suitable for implementing embodiments of this application. The data table lookup device in embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 7 The data table lookup device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0107] like Figure 7 As shown, the data table lookup device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in the read-only memory 1002 or a program loaded from the storage device 1003 into the random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the data table lookup device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the data table lookup device to communicate wirelessly or wiredly with other devices to exchange data. Although the figures show data table lookup devices with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0108] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0109] The data table lookup device provided in this application, employing the data table lookup method described in the above embodiments, can solve the technical problem that existing technologies cannot accurately locate the target data table from the database when there is a semantic difference between the keywords in the data table query statement and the target data table. Compared with the prior art, the beneficial effects of the data table lookup device provided in this application are the same as those of the data table lookup method provided in the above embodiments, and other technical features of this data table lookup device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0110] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0111] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0112] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the data table lookup method in the above embodiments.

[0113] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0114] The aforementioned computer-readable storage medium may be included in the data table lookup device; or it may exist independently and not be assembled into the data table lookup device.

[0115] The aforementioned computer-readable storage medium carries one or more programs that, when executed by a data table lookup device, enable the data table lookup device to write computer program code for performing the operations of this application in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++; and also conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, such as a local area network (LAN) or a wide area network (WAN), or connected to an external computer (e.g., via the Internet using an Internet service provider).

[0116] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0117] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0118] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described data table lookup method. This solves the technical problem in the prior art where the target data table cannot be accurately located from the database when there is a semantic difference between the keywords in the data table query statement and the target data table. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the data table lookup method provided in the above embodiments, and will not be repeated here.

[0119] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the data table lookup method described above.

[0120] The computer program product provided in this application can solve the technical problem of data table lookup. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as the beneficial effects of the data table lookup method provided in the above embodiments, and will not be repeated here.

[0121] The above description is only a part of the embodiments of this application and does not limit the scope of protection of this application. All equivalent structural transformations made under the technical concept of this application and using the content of this application specification and drawings, or direct / indirect applications in other related technical fields, are included in the scope of protection of this application.

Claims

1. A data table lookup method, characterized in that, The method includes the following steps: Upon receiving a data table query statement, the data table query statement is converted into a semantic vector; The semantic vectors are matched with the set of table vectors and the set of non-table vectors in the database to be queried to obtain the first similarity set and the second similarity set. The initialization vector is normalized based on the first similarity set and the second similarity set to obtain a normalized vector. The initialization vector is constructed based on the knowledge graph of the database to be queried. Calculate the relevance score of each component in the normalized vector corresponding to the query statement of the data table, and search for the target data table from the database to be queried based on the relevance score.

2. The data table lookup method as described in claim 1, characterized in that, Before the step of performing similarity matching between the semantic vector and the set of table vectors and the set of non-table vectors in the database to be queried, respectively, to obtain the first similarity set and the second similarity set, the method further includes: Obtain the knowledge graph of the database to be queried, and classify the nodes contained in the knowledge graph to obtain table type nodes and non-table type nodes; The table-type nodes and the non-table-type nodes are vectorized respectively to obtain the table vector set and the non-table vector set of the database to be queried.

3. The data table lookup method as described in claim 2, characterized in that, The knowledge graph is constructed based on the node dependencies of the database to be queried, and the node dependencies are determined based on the following steps: Obtain the node attributes of each node in the database to be queried, wherein the node attributes include node ID, node name, node type, and node content; The node dependencies of the database to be queried are determined based on the node attributes. The node dependencies include at least the lineage relationships between tables and between tables and the business system.

4. The data table lookup method as described in claim 1, characterized in that, The step of normalizing the initialization vector based on the first similarity set and the second similarity set to obtain a normalized vector includes: A first vector set is selected from the set of table vectors based on the similarity ranking of the first similarity set, and a second vector set is selected from the set of non-table vectors based on the similarity ranking of the second similarity set. Construct an initialization vector, the length of which is the same as the number of nodes in the knowledge graph of the database to be queried, and the initialization components of the initialization vector correspond one-to-one with the nodes of the knowledge graph; The initialization components are assigned values ​​based on the first vector set and the second vector set to obtain the assigned components; The initialization vector is normalized based on the assigned components to obtain a normalized vector.

5. The data table lookup method as described in claim 4, characterized in that, The step of assigning values ​​to the initialization components based on the first vector set and the second vector set includes: Assign the similarity score of the first vector set in the first similarity set to the initialization component corresponding to the first vector set; The similarity score of the second vector set in the second similarity set is assigned to the initialization component corresponding to the second vector set.

6. The data table lookup method as described in claim 1, characterized in that, The step of calculating the relevance score of each component in the normalized vector corresponding to the query statement of the data table, and searching for the target data table from the database to be queried based on the relevance score, includes: Based on the normalized vector and the adjacency matrix of the knowledge graph, calculate the relevance score of each component in the normalized vector corresponding to the query statement in the data table. After selecting the target component from the normalized vector based on the ranking of the relevance scores, the target data table corresponding to the target component is searched from the database to be queried.

7. A data table lookup device, characterized in that, The data table lookup device includes: The data conversion module is used to convert the data table query statement into a semantic vector when it receives the data table query statement; The similarity matching module is used to perform similarity matching between the semantic vector and the set of table vectors and the set of non-table vectors in the database to be queried, respectively, to obtain a first similarity set and a second similarity set; The vector normalization module is used to normalize the initialization vector based on the first similarity set and the second similarity set to obtain a normalized vector, wherein the initialization vector is constructed based on the knowledge graph of the database to be queried; The data table lookup module is used to calculate the relevance score of each component in the normalized vector corresponding to the data table query statement, and to search for the target data table from the database to be queried based on the relevance score.

8. A data table lookup device, characterized in that, The device includes: a memory, a processor, and a data table lookup program stored in the memory and executable on the processor, the data table lookup program being configured to implement the steps of the data table lookup method as described in any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a data table lookup program is stored on the storage medium. When the data table lookup program is executed by a processor, it implements the steps of the data table lookup method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes a data table lookup program, which, when executed by a processor, implements the steps of the data table lookup method as described in any one of claims 1 to 6.

Images