Method, device and equipment for generating query sql based on hive metadata

Abstract

The application relates to a method, device and equipment for generating a query SQL based on hive metadata, the method comprising the following steps: obtaining metadata information, storing the metadata information to a preset mysql database after marking the metadata information, and obtaining initial metadata; determining field lists of different tables based on the initial metadata, connecting the field lists of different tables, generating a data table association relationship, storing the data table association relationship to a preset subject domain, storing the preset subject domain to the preset mysql database; obtaining a target data relationship in the data model, generating an SQL template for querying the target data relationship by using a recursive algorithm, and developing the SQL template by modification. The application solves the problem of low development efficiency caused by the fact that the data is extracted by means of a big data component due to the writing of a complex SQL statement, the relationship between metadata and data tables needs to be maintained, and an operator needs to write a complex SQL according to the relationship between tables.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of big data query, in particular to a method and device for generating query sql based on hive metadata. BACKGROUND

[0002] With the advancement of big data informatization construction, the business of each department has gathered a large amount of business data in the data warehouse. Subsequently, there are a large number of number extraction requirements. In order to complete these requirements, the operator needs to be quite familiar with the relationship between data tables, and extracts numbers by writing complex sql statements with the help of big data components.

[0003] However, this brings two problems, first, the metadata and the relationship between the data tables need to be maintained; second, the operator needs to write complex sql according to the relationship between each table, and the efficiency decreases with the increase of the complexity of the relationship, which needs to be solved urgently. SUMMARY

[0004] The present application provides a method and device for generating query sql based on hive metadata, to solve the problem of low development efficiency caused by writing complex sql statements with the help of big data components to extract numbers, which requires maintaining the metadata and the relationship between the data tables, and the operator needs to write complex sql according to the relationship between each table.

[0005] The first aspect of the present application provides a method for generating query sql based on hive metadata, comprising the following steps: obtaining metadata information, and storing the labeled metadata information to a preset mysql database to obtain initial metadata; based on the initial metadata, determining the field list of different tables, connecting the field list of different tables, generating the data table association relationship, and storing the data table association relationship as a data model to a preset topic domain, and then storing the preset topic domain to the preset mysql database; obtaining the target data relationship in the data model, and generating a sql template for querying the target data relationship by using a recursive algorithm, so as to extract number development by modifying the sql template.

[0006] According to the above technical means, the problem of low development efficiency caused by writing complex sql statements with the help of big data components to extract numbers, which requires maintaining the metadata and the relationship between the data tables, and the operator needs to write complex sql according to the relationship between each table, is solved, the maintenance cost of administrator is reduced, and the development efficiency is improved.

[0007] Further, the metadata information is acquired and labeled, and then stored in a preset mysql database, including: acquiring existing database and data table information through a hive metastore api (configuration unit storage interface); based on the existing database, acquiring the metadata information in the data table information by using the hive metastore api and an hdfs (Hadoop Distributed File System) api; based on preset labeling data, labeling the metadata information, and storing the labeled metadata information in the preset mysql database.

[0008] According to the above technical means, the information of the existing database and the data table is acquired through the hive metastore api, an operable interface is provided, and the metadata information is labeled conveniently.

[0009] Further, the metadata information includes at least one of a table name, a table note, table field information, a hard disk space size of original data of the table, and a distributed file system HDFS space occupied by data of the table.

[0010] According to the above technical means, the metadata information in the database and the data table is acquired.

[0011] Further, the preset labeling data includes at least one of a table Chinese name, an update period, and a preset loading strategy.

[0012] According to the above technical means, the introduced metadata is labeled through the operable page, so that the metadata is more readable.

[0013] Further, the data model is a tree structure composed of a fact table and a plurality of dimension tables, wherein the fact table is a root node.

[0014] According to the above technical means, the tree structure refers to a data structure in which a “one-to-many” tree relationship exists between data elements, the data structure is clear, and the database query rate is high.

[0015] Further, the data table association relationship includes a first association relationship between the fact table and any one of the plurality of dimension tables, and a second association relationship between any two of the plurality of dimension tables.

[0016] According to the above technical means, the data of two tables is connected, and the consistency and integrity of the data in the two tables are constrained.

[0017] The second aspect embodiment of the application provides a device for generating a query SQL based on hive metadata, comprising: a labeling module configured to obtain metadata information, store the labeled metadata information to a preset mysql database, and obtain initial metadata; a storage module configured to determine field lists of different tables based on the initial metadata, connect the field lists of the different tables, generate a data table association relationship, store the data table association relationship as a data model to a preset topic domain, and store the preset topic domain to the preset mysql database; and a generation module configured to obtain a target data relationship in the data model, and generate a SQL template for querying the target data relationship by using a recursive algorithm, so as to perform data development by modifying the SQL template.

[0018] Further, the labeling module is specifically configured to obtain existing database and data table information by using a hive metastore API, obtain the metadata information in the data table information by using the hive metastore API and a hdfs API based on the existing database, label the metadata information based on preset labeling data, and store the labeled metadata information to the preset mysql database.

[0019] Further, the device for generating a query SQL based on hive metadata has the following characteristics: the metadata information comprises at least one of a table name, a table note, table field information, a hard disk space size of original data of a table, and a distributed file system HDFS space occupied by data of the table.

[0020] Further, the device for generating a query SQL based on hive metadata has the following characteristics: the preset labeling data comprises at least one of a Chinese table name, an update period, and a preset loading strategy.

[0021] Further, the device for generating a query SQL based on hive metadata has the following characteristics: the data model is a tree structure composed of a fact table and a plurality of dimension tables, and the fact table is a root node.

[0022] Further, the device for generating a query SQL based on hive metadata has the following characteristics: the data table association relationship comprises a first association relationship between the fact table and any one of the plurality of dimension tables, and a second association relationship between any two of the plurality of dimension tables.

[0023] The third aspect embodiment of the application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the processor executes the program to implement the method for generating a query SQL based on hive metadata as described in the above embodiments.

[0024] The fourth aspect of the present application provides a computer readable storage medium, which stores a computer program. The program is executed by a processor to implement the method for generating a query sql based on hive metadata according to the above embodiments.

[0025] Thus, by obtaining metadata information, storing the labeled metadata information to a preset mysql database to obtain initial metadata, determining the field list of different tables based on the initial metadata, connecting the field list of different tables, generating a data table association relationship, storing the data table association relationship as a data model to a preset subject domain, storing the preset subject domain to the preset mysql database, obtaining a target data relationship in the data model, generating a sql template for querying the target data relationship by using a recursive algorithm, and modifying the sql template for development, the problem of low development efficiency caused by writing complex sql statements to assist in data extraction by using big data components, the need to maintain the relationship between metadata and data tables, and the need for operators to write complex sql according to the relationship between tables is solved, the maintenance cost of administrators is reduced, and the development efficiency is improved.

[0026] The additional aspects and advantages of the present application will be partially given in the following description, partially will become obvious from the following description, or will be understood by practicing the present application. BRIEF DESCRIPTION OF DRAWINGS

[0027] The above and / or additional aspects and advantages of the present application will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:

[0028] Figure 1 A flowchart of a method for generating a query sql based on hive metadata according to an embodiment of the present application is shown;

[0029] Figure 2 A mysql database ER diagram for storing hive table relationships according to an embodiment of the present application is shown;

[0030] Figure 3 A flowchart of a recursive method for generating a query sql according to an embodiment of the present application is shown;

[0031] Figure 4 A block diagram of a device for generating a query sql based on hive metadata according to an embodiment of the present application is shown;

[0032] Figure 5 A block diagram of an electronic device according to an embodiment of the present application is shown.

[0033] Explanation of reference signs:

[0034] 10 - device for generating query sql based on hive metadata, 100 - labeling module, 200 - storage module, 300 - generating module, 501 - memory, 502 - processor, 503 - communication interface. DETAILED DESCRIPTION

[0035] Embodiments of the present application are described below in detail, examples of which are shown in the accompanying drawings, in which the same or similar reference signs represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by reference to the accompanying drawings are exemplary and are intended to explain the present application, and cannot be understood as limiting the present application.

[0036] The method, device and equipment for generating query sql based on hive metadata of the embodiments of the present application are described below with reference to the accompanying drawings. In view of the problem of low development efficiency caused by the need to maintain the relationship between metadata and data tables and the need for operators to write complex sql according to the relationship between tables due to the use of big data components to extract data by writing complex sql statements, the present application provides a method for generating query sql based on hive metadata. In this method, metadata information is obtained, the metadata information is labeled and stored in a preset mysql database to obtain initial metadata. Based on the initial metadata, the field list of different tables is determined, the field lists of different tables are connected, the data table association relationship is generated, the data table association relationship is stored as a data model in a preset topic domain, the preset topic domain is stored in a preset mysql database, the target data relationship in the data model is obtained, a recursive algorithm is used to generate a sql template for querying the target data relationship, and the sql template is modified to develop the data. Thus, the problem of low development efficiency caused by the need to maintain the relationship between metadata and data tables and the need for operators to write complex sql according to the relationship between tables due to the use of big data components to extract data by writing complex sql statements is solved, the maintenance cost of administrators is reduced, and the development efficiency is improved.

[0037] Specifically, Figure 1 A flowchart of a method for generating query sql based on hive metadata provided by the embodiments of the present application is shown.

[0038] As Figure 1 shown, the method for generating query sql based on hive metadata includes the following steps:

[0039] In step S101, metadata information is obtained, and the metadata information is labeled and stored in a preset mysql database to obtain initial metadata.

[0040] The hive is a data warehouse tool based on a distributed system infrastructure Hadoop, and is used for data extraction, transformation and loading. The hive is a mechanism for storing, querying and analyzing large-scale data stored in the Hadoop.

[0041] Optionally, in some embodiments, the metadata information is acquired and stored in the preset mysql database after being marked, and the method comprises the following steps: acquiring the existing database and data table information through the hive metastore api; acquiring the metadata information in the data table information based on the existing database by using the hive metastore api and the hdfs api; marking the metadata information based on the preset marking data, and storing the marked metadata information in the preset mysql database.

[0042] The metadata information acquired from the hive and the hdfs and the information marked by the operator is stored in the mysql.

[0043] Optionally, in some embodiments, the metadata information can include at least one of a table name, a table remark, table field information, a hard disk space size where original data of the table is located, and a distributed file system HDFS space size occupied by data of the table.

[0044] It can be understood that the existing database and the data table information are acquired through the hive metastore api, an operable interface is provided, an administrator introduces metadata of a table by selecting the table, and the following information is acquired through the hive metastore api and the hdfs api: a table name, a table remark, table field information (field English name, field type, whether a partition field, remark), a hard disk space size where original data of the table is located, and a HDFS space size occupied by data of the table.

[0045] Optionally, in some embodiments, the preset marking data includes at least one of a table Chinese name, an update period and a preset loading strategy.

[0046] It should be understood that the operator can make some markings on the introduced metadata through the operable page, so that the metadata is more readable, and the following information can be supplemented: a table Chinese name, an update period (day, week, month, year, real time), a loading strategy (incremental addition, full coverage, full snapshot, upsert).

[0047] In step S102, based on the initial metadata, the field list of different tables is determined, the field list of different tables is connected, the data table association relationship is generated, and the data table association relationship is stored as a data model to a preset theme domain. After the preset theme domain is stored to the preset mysql database.

[0048] Optionally, in some embodiments, the data model is a tree structure composed of a fact table and a plurality of dimension tables, wherein the fact table is a root node.

[0049] The fact table stores actual data or detailed numerical values, and is generally composed of dimension codes and fact data; the dimension table stores data with independent attributes and hierarchical structure, and is generally composed of dimension codes and corresponding dimension descriptions (labels), and can be associated with the fact table.

[0050] Optionally, in some embodiments, the data table association relationship includes a first association relationship between the fact table and any one of the plurality of dimension tables and a second association relationship between any two of the plurality of dimension tables.

[0051] Specifically, the concept of theme domain is introduced, the data model is located under the theme domain, and a relationship diagram of multiple tables corresponding to one model is as shown in Figure 2 It is specified that one model is a tree structure composed of one fact table and a plurality of dimension tables, and the fact table is a root node. Through an operable interface, multiple tables stored in the mysql are introduced, the table type (fact table or dimension table) is marked, the association relationship between the fields of the fact table and the dimension table and between the fields of the dimension tables is established, and is stored in the mysql database. Among them, the theme domain is a larger concept, the theme is slightly smaller, and the entity is the smallest. The theme here is the data model, and the entity represents the relationship diagram of multiple tables.

[0052] In step S103, the target data relationship in the data model is obtained, and a recursive algorithm is used to generate a sql template for querying the target data relationship, so as to modify the sql template for development.

[0053] Specifically, a data model under a certain theme domain is selected, the data relationship of the model is read, and a recursive function is used to generate a query sql between the relationship tables. The function is described below in combination with a specific example, and a flowchart of the recursive method of generating the query sql is as shown in Figure 3 .

[0054] Step S301: input local variables (sql, modelId, tableId, tabNum) and global variable tableNum (table number).

[0055] First, the function declaration of Java language is private StringBuilder sqlGenerator(StringBuilder sql, String modelId, String tableld, int tabNum); there is a global variable tableNum, the initial value is 0.

[0056] Step S302, tableld is used to query table field and table name. Specifically, the global variable tableNum is connected by character 't' to name this part, and the result is, for example:

[0057] select\n\tname,age\nfrom student t0.

[0058] Step S303, splice "select table field". The field information and table name information of the table are taken from the database by tableld, such as the fields id, name and age of the table employee, the select part is spliced, tabNum is used as indentation control, the initial value is 0, and the format of the generated sql is controlled, the number of initial \t is the same as tabNum each time, and when indentation is needed, \t of tabNum+1 number is used. The result is, for example:

[0059] select\n\tid,name,age\nfrom employee.

[0060] Step S304, splice "from table name".

[0061] Step S305, name the table as "t" + tableNum. Specifically, the global variable tableNum is connected by character 't' to name this part, and the result is, for example:

[0062] select\n\tname,age\nfrom student t0.

[0063] Step S306, use modelId (business model id) and tableld to query right table information. Specifically, the right connection table information is queried by modelId and tableld, such as the fields name and employeeId of the table department, the connection method is left join (the rest of the connection methods are the same), the first half of the connection sql is spliced, and the result is, for example:

[0064] select\n\tid,name,age\nfrom student t0\nleft join(\n.

[0065] Step S307, determine whether the right join table exists, if yes, execute step S309, if not, execute step S308.

[0066] Step S308, return the generated sql.

[0067] Step S309, determine the type of connection left, right, inner, full and field.

[0068] Step S310, splice the first half of the connection sql and the left bracket.

[0069] Step S311, call the sql generation process again with the current generated sql and the connection table id as parameters (sql, modelId, rightTableId, tabNum+1) to splice the return value.

[0070] Step S312, splice the second half of the connection sql and the right bracket.

[0071] Step S313, name the table as "t" + tableNum.

[0072] Step S314, tableNum++. In which, splice the second half of the connection sql, continue to name the connected subquery part according to the global variable tableNum, the result is as follows:

[0073] select\n\tid,name,age\nfrom student t0\nleft join

[0074] (\n\tselect\n\t\tname,employeeId\n\tfrom department t1)t2 on t0.id=t2.employeeId

[0075] Step S315, splice the connection condition. Specifically, if there are multiple connection conditions, use and to splice and process.

[0076] Step S316, return the generated sql.

[0077] According to the method for generating a query sql based on hive metadata provided in the embodiments of the present application, metadata information is acquired, the metadata information is labeled and stored in a preset mysql database to obtain initial metadata, based on the initial metadata, field lists of different tables are determined, the field lists of different tables are connected to generate a data table association relationship, the data table association relationship is stored in a preset subject domain as a data model, the preset subject domain is stored in a preset mysql database, target data relationships in the data model are acquired, a recursive algorithm is used to generate a sql template for querying the target data relationships, and the sql template is modified for development. Thus, the problem of low development efficiency caused by writing complex sql statements with the aid of big data components to extract data, the need to maintain the relationship between metadata and data tables, and the need for operators to write complex sql according to the relationship between tables is solved, the maintenance cost of administrators is reduced, and the development efficiency is improved.

[0078] Next, the device for generating a query sql based on hive metadata provided in the embodiments of the present application is described with reference to the accompanying drawings.

[0079] Figure 4 is a block schematic diagram of the device for generating a query sql based on hive metadata in the embodiments of the present application.

[0080] As shown in Figure 4 , the device 10 for generating a query sql based on hive metadata includes a labeling module 100, a storage module 200, and a generation module 300.

[0081] The labeling module 100 is configured to acquire metadata information and store the labeled metadata information in a preset mysql database to obtain initial metadata. The storage module 200 is configured to determine field lists of different tables based on the initial metadata, connect the field lists of different tables, generate a data table association relationship, store the data table association relationship in a preset subject domain as a data model, and store the preset subject domain in a preset mysql database. The generation module 300 is configured to acquire target data relationships in the data model and use a recursive algorithm to generate a sql template for querying the target data relationships, so as to modify the sql template for development.

[0082] Optionally, in some embodiments, the labeling module 100 is configured to acquire existing database and data table information through a hive metastore api, acquire metadata information in the data table information based on the existing database by using the hive metastore api and a hdfs api, label the metadata information based on preset labeling data, and store the labeled metadata information in a preset mysql database.

[0083] Optionally, in some embodiments, the metadata information comprises at least one of a table name, a table note, table field information, a hard disk space size where table original data is located, and a distributed file system HDFS space size where table data is located.

[0084] Optionally, in some embodiments, the preset annotation data comprises at least one of a table Chinese name, an update period, and a preset loading strategy.

[0085] Optionally, in some embodiments, the data model is a tree structure composed of a fact table and a plurality of dimension tables, wherein the fact table is a root node.

[0086] Optionally, in some embodiments, the data table association relationship comprises a first association relationship between the fact table and any one of the plurality of dimension tables, and a second association relationship between any two of the plurality of dimension tables.

[0087] It should be noted that the foregoing explanation and description of the embodiment of the device for generating a query sql based on hive metadata also applies to the embodiment of the method for generating a query sql based on hive metadata, which will not be described here.

[0088] According to the device for generating a query sql based on hive metadata provided in the embodiments of the present application, metadata information is obtained, the metadata information is annotated and stored in a preset mysql database to obtain initial metadata, based on the initial metadata, field lists of different tables are determined, the field lists of different tables are connected to generate a data table association relationship, the data table association relationship is stored in a preset subject domain as a data model, the preset subject domain is stored in a preset mysql database, a target data relationship in the data model is obtained, a recursive algorithm is used to generate a sql template for querying the target data relationship, and the sql template is modified for development. Thus, the problem of low development efficiency caused by writing complex sql statements to assist in data extraction by using big data components, the need to maintain metadata and relationships between data tables, and the need for operators to write complex sql according to relationships between tables is solved, the maintenance cost of administrators is reduced, and the development efficiency is improved.

[0089] Figure 5 The structure schematic diagram of the electronic device provided in the embodiments of the present application is shown in FIG. 1.

[0090] The vehicle can include a memory 501, a processor 502, and a computer program stored on the memory 501 and executable on the processor 502.

[0091] The processor 502 implements the method for generating a query sql based on hive metadata provided in the above embodiments when executing the program.

[0092] Further, the electronic device further comprises:

[0093] The communication interface 503 is configured to communicate between the memory 501 and the processor 502.

[0094] The memory 501 is configured to store a computer program executable in the processor 502.

[0095] The memory 501 can include a high-speed RAM memory, and can further include a non-volatile memory, for example, at least one disk memory.

[0096] If the memory 501, the processor 502 and the communication interface 503 are independently implemented, the communication interface 503, the memory 501 and the processor 502 can be connected through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For convenience of representation, Figure 5 Only one thick line is used in the figure, but it does not mean that there is only one bus or only one type of bus.

[0097] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can communicate with each other through an internal interface.

[0098] The processor 502 can be a Central Processing Unit (CPU), or an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0099] The embodiments of the present application further provide a computer readable storage medium, which stores a computer program, and the program is executed by the processor to implement the method for generating a query SQL based on hive metadata.

[0100] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms is not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or N embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the present specification and the features of the different embodiments or examples, without contradiction.

[0101] In addition, the terms "first", "second" are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "N" is at least two, for example, two, three, etc., unless otherwise specifically limited.

[0102] Any process or method descriptions in flow charts or otherwise described herein can be understood as representing code modules, segments, or portions of code that include one or more executable instructions for implementing the specified logical functions or steps, and the preferred embodiments of the present application include additional or fewer steps or methods, as appropriate or desired, and that the steps or methods represented in flow charts can be implemented in an order different than those described herein, such that the order of some steps can be different, including use of some steps in reverse order, and some steps can be performed simultaneously or with partial concurrence, particularly when benefit can be gained from doing so. The description herein of certain examples does not preclude additional implementations that can make use of more, or different, steps or methods.

[0103] It should be understood that portions of the present application can be realized with hardware, software, firmware or a combination thereof. In the above-described embodiments, the N steps or methods can be realized with software or firmware stored in a memory and executed by a suitable instruction execution system. As in another embodiment, if realized with hardware, any one or a combination of the following technologies known in the art can be used: discrete logic circuit with logic gate circuit for implementing logical functions on data signals, application specific integrated circuit with suitable combination logic gate circuit, programmable gate array, field programmable gate array, etc.

[0104] Those skilled in the art of the present technology can understand that all or part of the steps carried out by the above-mentioned embodiment method can be completed by a program instructing the relevant hardware, and the program can be stored in a computer readable storage medium. The program, when executed, includes one of the steps of the method embodiment or a combination thereof.

[0105] Although the embodiments of the present application have been shown and described above, it is understood that the above-described embodiments are exemplary and are not to be construed as limiting the present application, and that changes, modifications, substitutions and variations can be made by those skilled in the art without departing from the scope of the present application.

Claims

1. A method for generating a query sql based on hive metadata, characterized in that, The method comprises the following steps: obtaining metadata information, labeling the metadata information, and storing the labeled metadata information in a preset MySQL database to obtain initial metadata; based on the initial metadata, determining a field list of different tables, connecting the field list of different tables, generating a data table association relationship, storing the data table association relationship as a data model in a preset theme domain, and storing the preset theme domain in the preset MySQL database; and obtaining a target data relationship in the data model, and generating an SQL template for querying the target data relationship by using a recursive algorithm to modify the SQL template for development; the step of obtaining metadata information, labeling the metadata information, and storing the labeled metadata information in a preset MySQL database comprises the following steps: obtaining existing database and data table information through a hive metastore API; based on the existing database, obtaining the metadata information in the data table information by using the hive metastore API and a hdfs API; based on preset labeling data, labeling the metadata information, and storing the labeled metadata information in the preset MySQL database; the preset labeling data comprises at least one of a table Chinese name, an update period, and a preset loading strategy; the data model is a tree structure composed of a fact table and a plurality of dimension tables, wherein the fact table is a root node; the data table association relationship comprises a first association relationship between the fact table and any one of the plurality of dimension tables and a second association relationship between any two of the plurality of dimension tables. The metadata information comprises at least one of a table name, a table remark, table field information, a size of a hard disk space where original data of a table is located, and a size of a distributed file system HDFS space occupied by table data.

2. The method of claim 1, wherein, The method comprises the following steps:

3. An apparatus for generating a query SQL based on hive metadata, the apparatus comprising: a labeling module that obtains metadata information, labels the metadata information, and stores the labeled metadata information in a preset MySQL database to obtain initial metadata; a storage module that, based on the initial metadata, determines a field list of different tables, connects the field list of different tables, generates a data table association relationship, stores the data table association relationship as a data model in a preset theme domain, and stores the preset theme domain in the preset MySQL database; a generation module that obtains a target data relationship in the data model and generates an SQL template for querying the target data relationship by using a recursive algorithm to modify the SQL template for development; the labeling module is further configured to obtain existing database and data table information through a hive metastore API, obtain the metadata information in the data table information by using the hive metastore API and a hdfs API based on the existing database, label the metadata information based on preset labeling data, and store the labeled metadata information in the preset MySQL database; the preset labeling data comprises at least one of a table Chinese name, an update period, and a preset loading strategy; the data model is a tree structure composed of a fact table and a plurality of dimension tables, wherein the fact table is a root node; the data table association relationship comprises a first association relationship between the fact table and any one of the plurality of dimension tables and a second association relationship between any two of the plurality of dimension tables. The metadata information comprises at least one of a table name, a table remark, table field information, a size of a hard disk space where original data of a table is located, and a size of a distributed file system HDFS space occupied by table data. The data model is a tree structure composed of a fact table and a plurality of dimension tables, wherein the fact table is a root node; The data table association relationship includes a first association relationship between the fact table and any one of the plurality of dimension tables and a second association relationship between any two of the plurality of dimension tables.

4. An electronic device, comprising: comprising a memory, a processor; The processor executes the executable program code stored in the memory to run a program corresponding to the executable program code, so as to implement the method for generating a query sql based on hive metadata according to any one of claims 1-2.

5. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 4. The program is executed by the processor to implement the method for generating a query sql based on hive metadata according to any one of claims 1-2.

Images