Data table query method and device

Abstract

The invention discloses a data table query method and device, and relates to the technical field of big data. A specific embodiment of the method comprises the steps of obtaining a query request, wherein the query request indicates one or more query keywords; according to the query keyword and an incidence relation graph corresponding to a plurality of data tables in a data warehouse, determining a heat value corresponding to each data table; determining the similarity between each data table and the query keyword; according to the popularity value and the similarity, determining an arrangement sequence of the plurality of data tables in the data warehouse; and according to the arrangement sequence, generating a query result corresponding to the query request. According to the embodiment, the popularity value and the similarity corresponding to each data table can be determined according to the query keyword and the incidence relation graph corresponding to the multiple data tables, and the data tables are sorted according to the popularity values and the similarity, so that the queried data tables are closer to the query intention of a user, the workload of manual troubleshooting is reduced, and the user experience is improved. And the searching efficiency of the data table is improved.

Description

Technical Field

[0001] The present invention relates to the field of big data technology, and in particular to a method and device for querying a data table. Background Art

[0002] In the existing data storage process, data tables generated by different businesses are usually stored in the database. When the data tables need to be processed or searched, they need to be searched from the database based on partial content or keywords in the table name. However, when there are many types of businesses, a large number of similar data tables will be found only by partial content or keywords in the table name, which requires manual investigation one by one, and cannot be searched quickly and accurately. Summary of the invention

[0003] In view of this, an embodiment of the present invention provides a method and device for querying a data table. According to the query keywords and the association relationship diagrams corresponding to multiple data tables, the popularity value and similarity corresponding to each data table can be determined respectively, and the data tables can be sorted according to the popularity value and similarity, so that the data tables obtained by the query are closer to the user's query intention, reducing the workload of manual investigation and improving the search efficiency of the data table.

[0004] To achieve the above objective, according to one aspect of an embodiment of the present invention, a method for querying a data table is provided.

[0005] A data table query method according to an embodiment of the present invention includes: obtaining a query request indicating multiple query keywords; determining a heat value corresponding to each of the data tables according to an association relationship diagram corresponding to the query keywords and multiple data tables in a data warehouse; determining a similarity between each of the data tables and the query keyword; determining an arrangement order of the multiple data tables in the data warehouse according to the heat value and the similarity; and generating a query result corresponding to the query request according to the arrangement order.

[0006] Optionally, link information of each data table in the data warehouse is obtained by means of hyperlinks; the link information includes upstream data tables and downstream data tables corresponding to each of the data tables; the link information is stored in a graph database to generate the association relationship graph; wherein the association relationship graph includes nodes corresponding to each data table, and upstream data tables and downstream data tables corresponding to each data table, respectively, and directed edges between each node.

[0007] Optionally, the query keyword includes: table name keyword; determining the heat value corresponding to each data table according to the query keyword and the association relationship diagram corresponding to multiple data tables in the data warehouse, including: determining a first node corresponding to a first data table from the association relationship diagram, wherein the table name of the first data table contains the table name keyword; determining a second node connected to the first node according to the first node and the directed edge connected to the first node; and respectively determining the heat value of the second data table corresponding to the first data table and the second node.

[0008] Optionally, respectively determining the heat value of the second data table corresponding to the first data table and the second node includes: taking any one of the first node or the second node as the current node: according to the directionality of each directed edge corresponding to the current node, determining the authority value and hub value corresponding to the current node through a hyperlink-based topic search algorithm; wherein the authority value is determined according to the number of edges pointing to the current node, and the hub value is determined according to the number of edges pointing from the current node to other nodes; adding the authority value and the hub value corresponding to the current node to obtain the heat value of the data table corresponding to the current node.

[0009] Optionally, the query keywords include: attribute keywords; determining the similarity between each of the data tables and the query keywords includes: inputting the attribute keywords into a pre-trained dual-tower model, and outputting the similarity between each of the data tables and the attribute keywords.

[0010] Optionally, the inputting of the attribute keywords into a pre-trained dual-tower model and the outputting of the similarity between each of the data tables and the attribute keywords include: determining the business description information corresponding to each data table; inputting the business description information and the attribute keywords into the input layer of the dual-tower model, and obtaining a first vector corresponding to the business description information and a second vector corresponding to the attribute keywords according to the output of the input layer; and inputting the first vector and the second vector into the matching layer of the dual-tower model to obtain the similarity.

[0011] Optionally, the attribute keywords are input into a pre-trained dual-tower model, and the similarity between each of the data tables and the attribute keywords is output, including: using the matching layer to calculate the distance between the first vector and the second vector; using a normalized exponential function and the distance to determine the similarity.

[0012] Optionally, the method further includes: obtaining a search page according to a search term input by a user; the search page includes a plurality of search statements; manually annotating each of the search statements, and using the annotated statements as training samples for the twin tower model.

[0013] Optionally, determining the arrangement order of multiple data tables in the data warehouse based on the heat value and the similarity includes: determining weight values ​​corresponding to the heat value and the similarity, respectively; determining the arrangement order of multiple data tables in the data warehouse based on the heat value, the similarity, and the weight value.

[0014] To achieve the above objective, according to another aspect of an embodiment of the present invention, a data table query device is provided.

[0015] A data table query device according to an embodiment of the present invention includes: an acquisition module, used to acquire a query request, wherein the query request indicates multiple query keywords; a calculation module, used to determine the heat value corresponding to each of the data tables according to the query keyword and the association relationship diagram corresponding to the multiple data tables in the data warehouse; determine the similarity between each of the data tables and the query keyword; a sorting module, used to determine the arrangement order of the multiple data tables in the data warehouse according to the heat value and the similarity; and a generation module, used to generate a query result corresponding to the query request according to the arrangement order.

[0016] To achieve the above objective, according to another aspect of an embodiment of the present invention, an electronic device for querying a data table is provided.

[0017] An electronic device for querying a data table according to an embodiment of the present invention comprises: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement a method for querying a data table according to an embodiment of the present invention.

[0018] To achieve the above objective, according to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided.

[0019] A computer-readable storage medium according to an embodiment of the present invention stores a computer program, and when the program is executed by a processor, a data table query method according to an embodiment of the present invention is implemented.

[0020] An embodiment of the above invention has the following advantages or beneficial effects: based on the query keywords and the association relationship diagram corresponding to multiple data tables, the popularity value and similarity corresponding to each data table can be determined respectively, and the data tables can be sorted according to the popularity value and similarity, so that the data table obtained by the query is closer to the user's query intention, reducing the workload of manual investigation and improving the search efficiency of the data table.

[0021] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0023] Figure 1 is a schematic flow chart of a method for querying a data table according to an embodiment of the present invention;

[0024] Figure 2 is a schematic diagram of a process of generating an association relationship graph according to an embodiment of the present invention;

[0025] Figure 3 is a schematic diagram of a process for determining a heat value according to an embodiment of the present invention;

[0026] Figure 4 is a schematic diagram of an association relationship between a first node and a second node according to an embodiment of the present invention;

[0027] Figure 5 is a schematic diagram of a process of determining heat values ​​of a first data table and a second data table respectively corresponding to each first node and a second node according to an embodiment of the present invention;

[0028] Figure 6 is a schematic diagram of the overall structure of a double-tower model according to an embodiment of the present invention;

[0029] Figure 7 is a schematic diagram of a process for determining similarity according to an embodiment of the present invention;

[0030] Figure 8 is a schematic diagram of a flow chart of matching layer calculation according to an embodiment of the present invention;

[0031] Fig. 9 is a schematic diagram of a process for determining a double-tower model training sample according to an embodiment of the present invention;

[0032] Fig.10 is a schematic diagram of a process of sorting multiple data tables in a data warehouse according to an embodiment of the present invention;

[0033] Fig.11 is a schematic diagram of main modules of a data table query device according to an embodiment of the present invention;

[0034] Fig.12 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0035] Fig.13 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0037] It should be pointed out that the embodiments of the present invention and the technical features therein may be combined with each other without conflict.

[0038] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, storage and other aspects of user personal information involved in the technical solution of this disclosure are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security and national security.

[0039] Figure 1 It is a schematic diagram of main steps of a method for querying a data table according to an embodiment of the present invention.

[0040] like Figure 1 As shown, the data table query method of the embodiment of the present invention mainly includes the following steps:

[0041] Step S101: obtaining a query request, where the query request indicates one or more query keywords;

[0042] Step S102: determining the heat value corresponding to each data table according to the query keyword and the association relationship diagram corresponding to multiple data tables in the data warehouse;

[0043] Step S103: Determine the similarity between each data table and the query keyword;

[0044] Step S104: determining the arrangement order of multiple data tables in the data warehouse according to the heat value and the similarity;

[0045] Step S105: Generate a query result corresponding to the query request according to the arrangement order.

[0046] In order to search for the data table the user wants, the user needs to input one or more query keywords in the query request of step S101, so as to search according to the query keywords and obtain the data table that is closest to the user's query intention.

[0047] It should be noted that in the prior art, data tables are also queried based on keywords input by users, but keywords are usually part of the data table name. When the number of data tables is large, a large number of data tables containing keywords will be searched, and users are required to check them one by one to determine whether they are the target data tables to be searched. In other words, the prior art only provides simple keyword queries, while in the present invention, the data tables obtained by searching based on the query keywords can be further sorted through the heat value and similarity determined respectively in step S102 and step S103, thereby automatically determining which data tables are closer to the user's search intent, and generating query results based on the sorting order. Reducing the amount of data that users have to view one by one also improves the query efficiency of data tables.

[0048] For the association relationship diagram corresponding to the data table in step S102, in an optional embodiment, the generation process is as follows: Figure 2 As shown, including:

[0049] Step S201: Acquire link information of each data table in the data warehouse by crawling hyperlinks with a crawler; the link information includes an upstream data table and a downstream data table corresponding to each data table;

[0050] Step S202: Store the link information into a graph database to generate an association relationship graph; wherein the association relationship graph includes nodes corresponding to each data table, and the upstream data table and downstream data table corresponding to each data table, node attributes of each of the nodes, and directed edges between each node.

[0051] In the data warehouse storing data tables, each data table corresponds to a hyperlink, and each hyperlink can be crawled by crawler technology to obtain the link information of each data table in step S201, that is, by clicking each hyperlink, the upstream data tables and downstream data tables of each data table can be crawled.

[0052] The graph database in step S202 is a data management system that uses points and edges as basic storage units and is designed with efficient storage and query of graph data as the principle. A graph is a set of nodes and directed edges, where nodes represent entities (which can be understood as individual data tables), and directed edges represent the relationships between entities, that is, the data tables and the relationships between data tables are described through the structure of nodes and directed edges. The direction of the directed edge can explain the upstream and downstream relationship between the data tables corresponding to the two nodes, and the attributes of each node are the fields in the data table corresponding to the node and the description information of the data table.

[0053] In an optional embodiment, the query keyword may include multiple types of keywords, such as table name keywords, that is, keywords for searching for data table names; the process of determining the heat value in step S102, such as Figure 3 As shown, including:

[0054] Step S301: determining a first node corresponding to a first data table from an association relationship graph; the table name of the first data table includes a table name keyword;

[0055] Step S302: determining a second node connected to the first node according to the first node and the directed edge connected to the first node;

[0056] Step S303: respectively determine the heat values ​​of the first data table and the second data table corresponding to the second node.

[0057] Exemplarily, the association relationship between the first node and the second node may be as follows: Figure 4 As shown, first, according to the table name keyword, a part of the first data table whose table name contains the indicated keyword is filtered out from the association relationship diagram, and the first node corresponding to the first data table is formed. Figure 4 The root set in. Among them, the table name keyword is sku, and Table A, Table B, Table C and Table H are all data tables containing sku in the table name. After obtaining the root set, according to the directed edges connected to Table A, Table B, Table C and Table H respectively, multiple second nodes connected to the first node can be determined, namely Table D, Table G, Table I, Table E, Table F, among which, it can be seen from the directed edges that Table D is the upstream data table of Table B, Table G is the downstream data table of Table A and the upstream data table of Table C, Table I is the upstream data table of Table H, Table F is the downstream data table of Table C, Table E is the upstream data table of Table B, and the basic set is composed of the first node and the second node.

[0058] After obtaining the root set and the basic set, the heat values ​​of the first data table and the second data table corresponding to each first node and the second node can be determined. The specific determination process is as follows: Figure 5 As shown, including:

[0059] Taking any first node or second node as the current node: executing step S501 and step S502;

[0060] Step S501: according to the directionality of each directed edge corresponding to the current node, determine the authority value and hub value corresponding to the current node through a hyperlink-based topic search algorithm; wherein the authority value is determined according to the number of edges pointing to the current node, and the hub value is determined according to the number of edges from the current node to other nodes;

[0061] Step S502: Add the authority value and the hub value corresponding to the current node to obtain the heat value of the data table corresponding to the current node.

[0062] Among them, the hypertext-Induced Topic Search (HITS) algorithm based on hyperlinks is an algorithm used to sort web pages in the prior art. However, during the research and development process, the inventor discovered that the algorithm can also be used to sort data tables. The original Hub page in the HITS algorithm can be understood as the upstream data table in the data table, and the original Authority page in the HITS algorithm can be understood as the downstream data table in the data table.

[0063] In the HITS algorithm, after the user enters a keyword, the algorithm calculates two values ​​for the returned matching pages: one is the hub score and the other is the authority score. These two values ​​are interdependent and influence each other. The so-called hub score refers to the sum of the authority scores of multiple pages i pointing to page p, that is, The authority value is the sum of the hub values ​​of page p pointing to multiple pages m, that is, Through multiple iterations of the pivot value and the authority value, until the number of iterations reaches the set upper limit or the change range of the pivot value and the authority value is less than the set threshold. Figure 4 For example, assuming that table C is the current node, the authority value is the number of edges pointing to it, that is, two edges from table B and table G pointing to table C, that is, the authority value is 2, and the hub value is one edge from table C pointing to table F, that is, the hub value is 1. By adding the authority value and the hub value, the heat value of table C is 3 (1+2=3).

[0064] It should be noted that in a data warehouse, data tables are usually divided into three storage levels: fdm layer (i.e., original data layer), cdm layer (middle layer), and app layer. The data tables of the cdm layer, as the tables of the middle layer of the data warehouse, will depend on the data tables of the fdm layer, and will also be depended on by the data tables of the app layer. In the process of use, in order to save computing resources, it is usually necessary to use the data tables of the cdm layer for subsequent calculations, which reduces the process of re-associating the fdm layer data tables to generate the cdm layer data tables. Therefore, it is best to directly search for the cdm layer data tables that meet the query conditions. In view of this, the embodiment of the present invention applies the HITS algorithm to the process of sorting the data table, which perfectly matches the processing logic that the cdm layer table is used first, that is, compared with the fdm layer and the app layer, the cdm layer is between the fdm layer and the app layer, and is cited more times as upstream and downstream, so the heat value must be higher. Therefore, through the embodiment of the present invention, the HITS algorithm is used to calculate the heat value, and the data table is sorted according to the heat value, so that the data table of the cdm layer can be directly obtained, saving subsequent computing resources.

[0065] In another optional embodiment, the query keywords also include attribute keywords, and each node in the association relationship diagram corresponds to an attribute (i.e., a field in a data table), so the attribute keyword can be understood as a search term for the field. The process of determining the similarity in step S102 may further include: inputting the attribute keyword into a pre-trained dual-tower model, and outputting the similarity between each data table and the attribute keyword. Among them, the dual-tower model (Deep Structured Semantic Models, DSSM), also called the deep semantic matching model, is a model developed by Microsoft for calculating semantic similarity. Its overall structure is divided into three layers, namely, the input layer, the representation layer, and the matching layer. Figure 6 As shown. The input layer converts the two comparison texts (Query object and Document object) into low-dimensional vectors and provides them to the representation layer, where the representation layer can use a complex deep learning network (such as a fully connected neural network), and then converts the low-dimensional vectors into 128-dimensional vectors through the representation layer. Finally, the matching layer calculates the cosine similarity of the 128-dimensional vectors corresponding to the two comparison texts.

[0066] Specifically, the process of determining the similarity can be as follows Figure 7 As shown, including:

[0067] Step S701: Determine the business description information corresponding to each data table;

[0068] Step S702: inputting the business description information and the attribute keywords into the input layer of the dual-tower model, and obtaining a first vector corresponding to the business description information and a second vector corresponding to the attribute keywords according to the output of the input layer;

[0069] Step S703: Input the first vector and the second vector into the matching layer in the twin-tower model to obtain similarity.

[0070] The business description information corresponding to the data table can be understood as the words extracted from the data table that need to be semantically compared with the attribute keywords. Figure 6 The Document object in the property key is Figure 6 The Query object in .

[0071] In an optional embodiment, by segmenting the Document object and the Query object, single words corresponding to the Document object and the Query object can be obtained. Since the vector dimension of commonly used double words in Chinese words reaches the million level, the vector space is too large, and the commonly used single words have only about 15,000 dimensions, the embodiment of the present invention selects single words as the input of the input layer.

[0072] For the complex deep learning network of the representation layer, the embodiment of the present invention uses a multi-layer fully connected neural network. The specific establishment process is as follows: the input x is mapped, the first hidden layer of the neural network is l1, w1 is the weight matrix of the first layer, and the i-th hidden layer is l i , w i is the weight matrix of the i-th layer, b i is the bias term of the i-th layer, and the entire neural network can be expressed by the following formula:

[0073] l1=w1x

[0074] l i =f(w i l i-1 +b i ),i=2,…,N-1

[0075] y=f(w n l n-1 +b n )

[0076] Among them, the tanh function is used as the activation function of the hidden layer and the output layer. The first layer of the network converts the input 15,000-dimensional vector into a 300-dimensional vector. The second layer of the network vector dimension remains unchanged. The third layer of the vector converts the 300-dimensional vector into a 128-dimensional vector. That is, after passing through three layers of neural networks, 128-dimensional first and second vectors are obtained.

[0077] For the calculation process of the matching layer, in an optional embodiment, as Figure 8 As shown, including:

[0078] Step S801: Calculate the distance between the first vector and the second vector using a matching layer;

[0079] Step S802: Determine similarity using a normalized exponential function and distance.

[0080] The distance is solved by the general formula for calculating cosine similarity. The closer the distance, the greater the similarity, and the farther the distance, the smaller the similarity. After calculating the similarity, the maximum likelihood estimation is used to minimize the loss function, and finally the stochastic gradient descent (SGD) is used to converge the entire twin tower model, and finally the parameters {w i ,b i}.

[0081] In a further optional embodiment, the input in the dual tower model training process can be as follows Fig. 9 As shown, including:

[0082] Step S901: obtaining a search page according to the search term input by the user; the search page includes multiple search statements;

[0083] Step S902: manually label each search sentence, and use the labeled sentences as training samples for the twin-tower model.

[0084] In other words, the present invention can obtain more realistic training samples by taking the search sentences in the search web pages as input and marking the search sentences as positive samples and negative samples by manual annotation.

[0085] After obtaining the heat value and similarity corresponding to each data table, multiple data tables in the data warehouse can be sorted according to the heat value and similarity. The specific sorting process is as follows: Fig.10 As shown, including:

[0086] Step S1001: Determine the weight values ​​corresponding to the heat value and the similarity respectively;

[0087] Step S1002: Determine the arrangement order of multiple data tables in the data warehouse according to the heat value, similarity, and weight value.

[0088] Among them, the weight value of the heat value is greater than or equal to the weight value of the similarity. By adding similarity, the search results obtained only based on the heat value can be changed to achieve a correction effect, thereby obtaining search results that are more in line with the user's search intent.

[0089] For the process of generating the query result according to the arrangement order in step S105, the top n data tables can be selected as the query result according to the actual application needs, where n is determined by the user's custom setting.

[0090] According to the data table query method of an embodiment of the present invention, based on the query keywords and the association relationship diagrams corresponding to multiple data tables, the popularity value and similarity corresponding to each data table can be determined respectively, and the data tables can be sorted according to the popularity value and similarity, so that the data table obtained by the query is closer to the user's query intention, reducing the workload of manual investigation and improving the search efficiency of the data table.

[0091] Fig.11 4 is a schematic diagram of main modules of a data table query device according to an embodiment of the present invention.

[0092] like Fig.11 As shown, the data table query device 1100 of the embodiment of the present invention includes:

[0093] An acquisition module 1101 is used to acquire a query request, where the query request indicates a plurality of query keywords;

[0094] The calculation module 1102 is used to determine the heat value corresponding to each of the data tables according to the query keyword and the association relationship diagram corresponding to the multiple data tables in the data warehouse; determine the similarity between each of the data tables and the query keyword;

[0095] A sorting module 1103 is used to determine the arrangement order of multiple data tables in the data warehouse according to the heat value and the similarity;

[0096] The generating module 1104 is used to generate the query result corresponding to the query request according to the arrangement order.

[0097] In an optional embodiment of the present invention, the generation module 1104 is also used to obtain link information of each data table in the data warehouse by means of hyperlinks; the link information includes upstream data tables and downstream data tables corresponding to each of the data tables; the link information is stored in a graph database to generate the association relationship graph; wherein the association relationship graph includes nodes corresponding to each data table, and upstream data tables and downstream data tables corresponding to each data table, respectively, and directed edges between each node.

[0098] In an optional embodiment of the present invention, the query keywords include: table name keywords; the calculation module 1102 is also used to determine the first node corresponding to the first data table from the association relationship graph; the table name of the first data table contains the table name keyword; based on the first node and the directed edge connected to the first node, determine the second node connected to the first node; and respectively determine the heat value of the second data table corresponding to the first data table and the second node.

[0099] In an optional embodiment of the present invention, the calculation module 1102 is also used to take any one of the first node or the second node as the current node: according to the directionality of each directed edge corresponding to the current node, determine the authority value and hub value corresponding to the current node through a hyperlink-based topic search algorithm; wherein the authority value is determined according to the number of edges pointing to the current node, and the hub value is determined according to the number of edges pointing to other nodes from the current node; the authority value and the hub value corresponding to the current node are added to obtain the heat value of the data table corresponding to the current node.

[0100] In an optional embodiment of the present invention, the query keywords include: attribute keywords; the calculation module 1102 is also used to input the attribute keywords into a pre-trained double-tower model, and output the similarity between each of the data tables and the attribute keywords.

[0101] In an optional embodiment of the present invention, the calculation module 1102 is also used to determine the business description information corresponding to each data table; input the business description information and the attribute keywords into the input layer of the dual-tower model, and obtain a first vector corresponding to the business description information and a second vector corresponding to the attribute keywords according to the output of the input layer; input the first vector and the second vector into the matching layer in the dual-tower model to obtain the similarity.

[0102] In an optional embodiment of the present invention, the calculation module 1102 is further used to calculate the distance between the first vector and the second vector using the matching layer; and determine the similarity using a normalized exponential function and the distance.

[0103] In an optional embodiment of the present invention, the device also includes a training module, which is used to obtain a search page based on a search term input by a user; the search page includes multiple search statements; each of the search statements is manually labeled, and the labeled statements are used as training samples of the twin tower model.

[0104] In an optional embodiment of the present invention, the sorting module 1103 is also used to determine the weight values ​​corresponding to the heat value and the similarity respectively; and determine the arrangement order of multiple data tables in the data warehouse according to the heat value, the similarity, and the weight value.

[0105] According to the data table query device of the embodiment of the present invention, the popularity value and similarity corresponding to each data table can be determined respectively according to the query keyword and the association relationship diagram corresponding to multiple data tables, and the data tables are sorted according to the popularity value and similarity, so that the data table obtained by the query is closer to the user's query intention, reducing the workload of manual investigation and improving the search efficiency of the data table.

[0106] Fig.12 An exemplary system architecture 1200 is shown to which the data table query method or data table query device according to the embodiment of the present invention can be applied.

[0107] like Fig.12 As shown, the system architecture 1200 may include terminal devices 1201, 1202, 1203, a network 1204, and a server 1205. The network 1204 is used to provide a medium for communication links between the terminal devices 1201, 1202, 1203 and the server 1205. The network 1204 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0108] Users can use terminal devices 1201, 1202, 1203 to interact with server 1205 through network 1204 to receive or send data, etc. Various communication client applications can be installed on terminal devices 1201, 1202, 1203, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0109] The terminal devices 1201 , 1202 , and 1203 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers, etc.

[0110] The server 1205 may be a server that provides various services, such as a background management server that provides support for query requests sent by users using the terminal devices 1201, 1202, and 1203. The background management server may analyze and process the received query requests and other data, and feed back the processing results (such as query results) to the terminal device.

[0111] It should be noted that the data table query method provided in the embodiment of the present invention is generally executed by the server 1205 , and accordingly, the data table query device is generally set in the server 1205 .

[0112] It should be understood that Fig.12 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0113] Reference below Fig.13 , which shows a schematic diagram of the structure of a computer system 1300 of a terminal device suitable for implementing an embodiment of the present invention. Fig.13 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0114] like Fig.13 As shown, the computer system 1300 includes a central processing unit (CPU) 1301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1302 or a program loaded from a storage part 1308 into a random access memory (RAM) 1303. In the RAM 1303, various programs and data required for the operation of the system 1300 are also stored. The CPU 1301, the ROM 1302, and the RAM 1303 are connected to each other via a bus 1304. An input / output (I / O) first interface 1305 is also connected to the bus 1304.

[0115] The following components are connected to the I / O first interface 1305: an input section 1306 including a keyboard, a mouse, etc.; an output section 1307 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1308 including a hard disk, etc.; and a communication section 1309 including a network first interface card such as a LAN card, a modem, etc. The communication section 1309 performs communication processing via a network such as the Internet. A drive 1310 is also connected to the I / O first interface 1305 as needed. A removable medium 1311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1310 as needed, so that a computer program read therefrom is installed into the storage section 1308 as needed.

[0116] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1309, and / or installed from the removable medium 1311. When the computer program is executed by the central processing unit (CPU) 1301, the above-mentioned functions defined in the system of the present invention are executed.

[0117] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0118] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0119] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The modules described may also be set in a processor, for example, it may be described as: a processor includes an acquisition module, a calculation module, a sorting module and a generation module. The names of these modules do not constitute a limitation on the modules themselves in some cases, for example, the acquisition module may also be described as a "module for obtaining query requests".

[0120] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: obtaining a query request, the query request indicating one or more query keywords; determining the heat value corresponding to each of the data tables according to the association relationship diagram corresponding to the query keyword and multiple data tables in the data warehouse; determining the similarity between each of the data tables and the query keyword; determining the arrangement order of the multiple data tables in the data warehouse according to the heat value and the similarity; generating the query result corresponding to the query request according to the arrangement order.

[0121] According to the technical solution of the embodiment of the present invention, based on the query keywords and the association relationship diagram corresponding to multiple data tables, the popularity value and similarity corresponding to each data table can be determined respectively, and the data tables can be sorted according to the popularity value and similarity, so that the data tables obtained by the query are closer to the user's query intention, reducing the workload of manual investigation and improving the search efficiency of the data table.

[0122] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for querying a data table, characterized in that: include: Obtaining a query request, wherein the query request indicates one or more query keywords; Determine the heat value corresponding to each of the data tables according to the query keyword and the association relationship diagram corresponding to the multiple data tables in the data warehouse; Determining the similarity between each of the data tables and the query keyword; Determining an arrangement order of multiple data tables in the data warehouse according to the heat value and the similarity; Generate a query result corresponding to the query request according to the arrangement order.

2. The method according to claim 1, characterized in that Also includes: Acquire link information of each data table in the data warehouse by crawling hyperlinks with a crawler; the link information includes upstream data tables and downstream data tables corresponding to each data table; The link information is stored in a graph database to generate the association relationship graph; wherein the association relationship graph includes nodes corresponding to each data table, and the upstream data tables and downstream data tables corresponding to each data table, node attributes of each node, and directed edges between each node.

3. The method according to claim 2, characterized in that The query keyword includes: a table name keyword; the determining of the heat value corresponding to each of the data tables according to the query keyword and the association relationship diagram corresponding to the multiple data tables in the data warehouse includes: Determine a first node corresponding to a first data table from the association relationship graph; the table name of the first data table includes the table name keyword; Determine, according to the first node and the directed edge connected to the first node, a second node connected to the first node; The heat values ​​of the first data table and the second data table corresponding to the second node are determined respectively.

4. The method according to claim 3, characterized in that The determining the heat values ​​of the first data table and the second data table corresponding to the second node respectively includes: Take any of the first or second nodes as the current node: According to the directionality of each directed edge corresponding to the current node, the authority value and the hub value corresponding to the current node are determined by a hyperlink-based topic search algorithm; wherein the authority value is determined according to the number of edges pointing to the current node, and the hub value is determined according to the number of edges from the current node to other nodes; The authority value and the hub value corresponding to the current node are added together to obtain the heat value of the data table corresponding to the current node.

5. The method according to claim 1, characterized in that: The query keyword includes: an attribute keyword; the determining the similarity between each of the data tables and the query keyword includes: The attribute keywords are input into the pre-trained dual-tower model, and the similarities between each of the data tables and the attribute keywords are output.

6. The method according to claim 5, characterized in that The step of inputting the attribute keywords into the pre-trained dual-tower model and outputting the similarity between each of the data tables and the attribute keywords includes: Determine the business description information corresponding to each data table; Input the business description information and the attribute keywords into the input layer of the dual-tower model, and obtain a first vector corresponding to the business description information and a second vector corresponding to the attribute keywords according to the output of the input layer; The first vector and the second vector are input into a matching layer in the twin-tower model to obtain the similarity.

7. The method according to claim 6, characterized in that The step of inputting the attribute keywords into the pre-trained dual-tower model and outputting the similarity between each of the data tables and the attribute keywords includes: calculating a distance between the first vector and the second vector using the matching layer; The similarity is determined using a normalized exponential function and the distance.

8. The method according to claim 5, characterized in that Also includes: Obtaining a search page according to a search term input by a user; the search page includes a plurality of search statements; Each of the search sentences is manually labeled, and the labeled sentences are used as training samples for the twin-tower model.

9. The method according to claim 1, characterized in that: Determining the arrangement order of the multiple data tables in the data warehouse according to the heat value and the similarity includes: Determine the weight values ​​corresponding to the heat value and the similarity respectively; An arrangement order of multiple data tables in the data warehouse is determined according to the heat value, the similarity, and the weight value.

10. A data table query device, characterized in that: include: An acquisition module, used to acquire a query request, wherein the query request indicates a plurality of query keywords; A calculation module, used to determine the heat value corresponding to each of the data tables according to the query keyword and the association relationship diagram corresponding to the multiple data tables in the data warehouse; determine the similarity between each of the data tables and the query keyword; A sorting module, used to determine the arrangement order of multiple data tables in the data warehouse according to the heat value and the similarity; A generating module is used to generate a query result corresponding to the query request according to the arrangement order.

11. An electronic device for querying a data table, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 9.

12. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images