Data query method and related equipment

Abstract

The invention discloses a data query method and related equipment. The data query method comprises the steps of determining database information of a queried database and basic information of a queried target object according to a query request; obtaining metadata of the database according to the database information; generating a model character string according to the metadata and a relationship file representing the dependency relationship between the metadata in the database; calling a query language supported by the database according to the model character string, the basic information and the query request, and generating a query statement for the database; and querying the database according to the query statement to obtain a query result of the target object. The data query method provided by the invention has high universality and can be suitable for data query in most of data storage engines.

Description

Technical Field

[0001] The present application relates to the technical field of data query, and in particular to a data query method and related equipment. Background Art

[0002] With the development of the big data industry, many types of data storage engines have emerged to meet the needs of various scenarios and businesses. For example: relational databases, non-relational databases, databases based on the Hadoop system, databases in the NewSQL direction, etc. These data storage engines have different design schemes, each with its own advantages, and can be applied to different business demand scenarios. However, due to different design schemes, the query methods of these databases are also different. For example: MongoDB currently uses MongoDB connectors to implement data queries. This method is very dependent on the data query engine Presto. Specifically, Presto parses the query request to generate a query plan and distributes the tasks in the plan to one or more workers for execution. In essence, it is still an indirect query implemented by Presto. It can be seen that the above method requires downloading the query engine corresponding to the database for installation and deployment and reasonable configuration. The cost of use is relatively high and the versatility is poor. Other databases also have similar query problems. Therefore, there is an urgent need for a highly versatile data query method that covers the full amount of data storage engines as much as possible. Summary of the invention

[0003] The present application provides a data query method and related equipment, which is highly versatile and applicable to data queries in most data storage engines.

[0004] In a first aspect, a data query method provided in an embodiment of the present application includes:

[0005] Determine database information of the queried database and basic information of the queried target object according to the query request;

[0006] Acquire metadata of the database according to the database information;

[0007] Generate a model string according to the metadata and a relationship file representing a dependency relationship between metadata in the database;

[0008] Invoke a query language supported by the database according to the model character string, the basic information and the query request to generate a query statement for the database;

[0009] The database is queried according to the query statement to obtain a query result of the target object.

[0010] It can be seen that this method obtains the metadata in the database by parsing the database information of the database being queried in the query request, and processes the metadata of the database based on the relationship file that characterizes the dependency relationship between the metadata in the database, and generates a model string containing the metadata in the database and the dependency relationship between the metadata. Then, the basic information of the target object being queried in the query request is parsed, and then according to the query request, the basic information of the target object being queried, and the model string, the query language supported by the database is called, and the query statement applicable to the database is regenerated, and the database is queried to obtain the query result of the target object. In this way, the query request is automatically converted into the query statement supported by the queried database, and the query of any database is realized. At the same time, the method has high versatility and can cover the full amount of data storage engines as much as possible.

[0011] In a second aspect, a data query device provided in an embodiment of the present application includes:

[0012] A determination module, used to determine the database information of the queried database and the basic information of the queried target object according to the query request;

[0013] An acquisition module, used for acquiring metadata of the database according to the database information;

[0014] A processing module, configured to generate a model string according to the metadata and a relationship file representing the dependency relationship between the metadata in the database, and to call a query language supported by the database according to the model string, the basic information and the query request to generate a query statement for the database;

[0015] The query module is used to query the database according to the query statement to obtain the query result of the target object.

[0016] A third aspect is an electronic device provided for an embodiment of the present application, comprising a processor, a memory, and a computer program or instructions stored on the memory, wherein the processor executes the computer program or instructions to implement the steps in the method designed in the first aspect above.

[0017] In a fourth aspect, a computer-readable storage medium is provided for an embodiment of the present application, wherein a computer program or instruction is stored therein, and when the computer program or instruction is executed, the steps in the method designed in the first aspect are implemented. For example, the computer program or instruction is executed by a processor.

[0018] In a fifth aspect, a computer program product provided by an embodiment of the present application includes a computer program or an instruction, wherein when the computer program or the instruction is executed, the steps in the method designed in the first aspect are executed. For example, the computer program or the instruction is executed by a processor.

[0019] The beneficial effects brought about by the technical solutions of the second to fifth aspects can be referred to the technical effects brought about by the technical solution of the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some implementation methods recorded in one or more of the present specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0021] Figure 1 is a schematic diagram of a data query system proposed in an embodiment of the present application;

[0022] Figure 2 A flowchart of a data query method proposed in an embodiment of the present application;

[0023] Figure 3 It is a schematic diagram of a relationship file in the form of a class object proposed in an implementation manner of the present application;

[0024] Figure 4 A flowchart of a method for generating a query statement for the database by calling a query language supported by the database according to the model character string, the target object information and the query request, proposed in an embodiment of the present application;

[0025] Figure 5 It is a functional module composition block diagram of a data query device proposed in an embodiment of the present application;

[0026] Figure 6 It is a structural schematic diagram of an electronic device proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0027] It should be understood that the terms "first", "second", etc. involved in the embodiments of the present application are used to distinguish different objects rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, software, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products, or devices.

[0028] The "implementation method" involved in the embodiments of the present application means that the specific features, structures or characteristics described in conjunction with the implementation method may be included in at least one implementation method of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same implementation method, nor is it an independent or alternative implementation method that is mutually exclusive with other implementation methods. It is explicitly and implicitly understood by those skilled in the art that the implementation methods described herein may be combined with other implementation methods.

[0029] The “and / or” in the implementation manner of the present application describes the association relationship of the associated objects, indicating that three types of relationships may exist.

[0030] In the implementation manner of the present application, the symbol “ / ” can indicate that the previously associated objects are in an “or” relationship.

[0031] "At least one item" or similar expressions in the embodiments of the present application refers to any combination of these items, including any combination of single items or plural items, and refers to one or more, and multiple refers to two or more.

[0032] In the embodiments of the present application, "higher than" can be expressed as the same concept as "greater than", "lower than" can be expressed as the same concept as "less than", "not lower than" can be expressed as the same concept as "higher than or equal to" or "greater than or equal to", and "not higher than" can be expressed as the same concept as "lower than or equal to" or "less than or equal to". In the present application, for the same scheme, "equal to" can be used in conjunction with "less than", and can also be used in conjunction with "greater than", but not used in conjunction with "less than" and "greater than" at the same time. When "equal to" is used in conjunction with "less than", it is applicable to the technical scheme adopted by "less than". When "equal to" is used in conjunction with "greater than", it is applicable to the technical scheme adopted by "greater than".

[0033] In the embodiments of the present application, the terms "of", "corresponding / relevant", "corresponding", "indicated", "associated", etc. may be used interchangeably.

[0034] In the implementation manner of this application, the words "associated", "corresponding", "corresponding", "is", "of", "for", "belonging to", "as", and "regarded as" can sometimes be used interchangeably.

[0035] The "connection" in the embodiments of the present application refers to various connection methods such as direct connection or indirect connection to achieve communication between devices, and there is no limitation on this.

[0036] The "network" in the implementation manner of the present application can be expressed as the same concept as the "system", and the communication system is the communication network.

[0037] The following is an explanation of the relevant contents, concepts, meanings, technical issues, technical solutions and beneficial effects involved in the implementation of the present application.

[0038] First, see Figure 1 , Figure 1 Schematic diagram of a data query system proposed in the embodiment of the present application. Figure 1 As shown, the model training system may include a terminal device, a query device and a database.

[0039] In this embodiment, a user may send a query request through a terminal device. After receiving the query request, the query device parses the query request to obtain the database information of the queried database and the basic information of the queried target object contained therein. The database information may be the name and / or type information of the queried database, and the basic information of the target object may be the data name, data value, data type, etc. of the queried target object. Then, the query device determines the target database to be queried according to the database information and obtains the metadata of the target database. Then, the query device generates a model string containing the metadata in the database and the dependency relationship between the metadata according to the metadata and the relationship file representing the dependency relationship between the metadata in the target database. Then, further according to the query request, the basic information of the target object to be queried, and the model string, the query language supported by the database is called to regenerate a query statement applicable to the target database. Finally, the query device connects to the target database, queries the target database according to the newly generated query statement, obtains the query result of the target object, and returns it to the terminal device.

[0040] Understandably, Figure 1 The forms and quantities of the terminal devices, query devices and databases shown in the figure are for illustrative purposes only and do not constitute limitations on the implementation modes of the present application.

[0041] In this embodiment, the terminal device and the query device may be a device with a communication function, which may be referred to as a terminal, user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal equipment, vehicle-mounted terminal equipment, industrial control terminal equipment, UE unit, UE station, mobile station, remote station, remote terminal equipment, mobile equipment, UE terminal equipment, wireless communication equipment, UE agent or UE device, etc. The terminal device may be fixed or mobile. It should be noted that the terminal device and the query device may support at least one wireless communication technology, such as LTE, new radio (NR), wideband code division multiple access (WCDMA), etc. For example, the terminal device and the query device can be a mobile phone, a tablet computer (pad), a desktop computer, a laptop computer, an all-in-one computer, a vehicle-mounted terminal, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal in industrial control, a wireless terminal in self driving, a wireless terminal in remote medical surgery, a wireless terminal in a smart grid, a wireless terminal in transportation safety, a wireless terminal in a smart city, a wireless terminal in a smart home, a cellular phone, a cordless phone, a session initiation protocol (SIP) phone, a wireless local loop (WLL) station, a personal digital assistant (PDA), a handheld device with wireless communication function, a computing device or other processing device connected to a wireless modem, a wearable device, a terminal device in a future mobile communication network, or a terminal device in a future evolved public mobile land network (PLMN), etc. In some implementations of the present application, the terminal device and the query device may also be devices with transceiver functions, such as a chip system, wherein the chip system may include a chip and may also include other discrete devices.

[0042] In this embodiment, the terminal device and the query device may also be a server, for example, an independent server, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. This application does not make specific limitations on this.

[0043] In this implementation, the terminal device and the query device may also be the same device.

[0044] In this embodiment, the database may also be a server, or a memory that provides data storage services, such as: a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, and this application does not make any specific limitations on this.

[0045] In this implementation, the database may also be a storage unit integrated in the terminal device and / or the query device, and may be a component of the terminal device and / or the query device.

[0046] Secondly, the data query method provided by the implementation mode of the present application can be applied to the scenario of querying any type of database using query requests in any language. Among them, the language may include: Structured Query Language (SQL), eXtensible Markup Language Query Language (XQuery), etc.; the database may include a relational database, such as: MySQL, PostgreSQL, MicrosoftSQL Server, etc., a NoSQL database, such as: MongoDB, Cassandra, Redis, etc., or a database of its type, and the present application does not limit this. Below, the data query method provided by the implementation mode of the present application will be described in detail by taking the query request based on the SQL language to query the MongoDB database as an example. The data query method in other scenarios is similar to the data query method in this scenario, and will not be repeated here.

[0047] See also Figure 2 , Figure 2 A flow chart of a data query method proposed in the embodiment of the present application, which is applied to Figure 1 The data query system shown in FIG. 1 can be implemented by a query device in the data query system. Figure 2 As shown, the method includes:

[0048] S201: Determine database information of the queried database and basic information of the queried target object according to the query request.

[0049] In this embodiment, the terminal device can determine the user's query intention, the target database to be queried, and the target object according to the user's operation, and then generate the query request based on the user's query intention, the target database to be queried, and the target object. Therefore, after receiving the query request, the query device can parse the query request and obtain the database information of the queried database (target database) included therein, such as: database name, and basic information of the queried target object, such as: data name, data value, data type, etc.

[0050] In this embodiment, taking querying the MongoDB database as an example, the query device can execute the data query method by deploying Calcite to query the MongoDB database. Calcite is simpler to deploy than Presto, with lower deployment costs, and has higher compatibility with most databases and wider applicability. Specifically, the official GitHub repository of Calcite can be git cloned locally, and the Calcite MongoDB submodule code can be copied to the local application project to deploy Calcite.

[0051] It should be understood that the above Calcite is only an example of implementing the data query method provided by this application, and does not limit this application. The data query method provided by this application can be integrated into a user terminal or a database, or deployed and implemented on an independent server or device, and this application does not limit this.

[0052] S202: Obtain metadata of the database according to the database information.

[0053] In this implementation, the metadata may be stored in the database via a metadata table of a table structure.

[0054] At present, the metadata of each database is usually stored in the database in the form of a configured JSON string. Specifically, for each collection in the database, one or more JSON strings need to be generated to record the metadata of the collection. Therefore, when a database contains a large number of collections, a large number of JSON strings will be configured accordingly. Configuring these JSON strings requires a lot of human resources, resulting in a lengthy and cumbersome configuration process. At the same time, a large number of JSON strings will take up storage space and are not easy to manage.

[0055] Based on this, in this embodiment, metadata is stored in the database through a metadata table of a table structure. In simple terms, a metadata table for storing metadata of the database is set in the target database, and then all metadata in the target database will be written into the metadata table for storage. Specifically, each record in the metadata table corresponds to metadata in a set of JSON strings in the database. Thus, metadata in a large number of JSON strings can be recorded through a metadata table, which is equivalent to replacing a large number of JSON strings with a metadata table. Then, there is no need to configure and generate JSON strings, and only this metadata table needs to be maintained to achieve the maintenance of all metadata in the database. Thus, the problem that the configuration process is lengthy and cumbersome due to too many data sets and the configuration of massive JSON strings, and the memory usage is too large, can be well solved. At the same time, all metadata in the database are stored in the metadata table. When querying the metadata of the database, only the metadata table needs to be obtained to obtain all metadata of the database. Compared with the method in the prior art that requires traversing the database to obtain all JSON strings, storing metadata through the metadata table can also improve the query efficiency of metadata.

[0056] For example, the metadata table for recording metadata in this implementation may be as shown in Table 1:

[0057] Table 1:

[0058] Field Name Field Type length Is it allowed to be empty? default value Notes Id int 11 N Primary key, auto-increment version_no varchar 20 N Version Number default_schema_name varchar 64 N schema_type varchar 20 N schema_name varchar 20 N schema_factory varchar 1000 N operand_host varchar 64 N operand_port varchar 10 N operand_database varchar 64 N operand_auth_mechanism varchar 20 N operand_username varchar 64 N operand_password varchar 64 N operand_auth_database varchar 64 N schema_ext_name varchar 64 N table_name varchar 64 N table_type varchar 10 N table_sql text N create_user varchar 100 Y Create User create_time datetime N Creation time update_time datetime N Update time Database Name varchar Database Version varchar Character Set varchar Privileges varchar Encryption varchar

[0059] Each row in Table 1 represents a metadata in the database, as well as attribute information or configuration information such as field name, field type, and length of the metadata.

[0060] At the same time, in this embodiment, the query device can set a related timed task to monitor the change message of the data set in the database regularly, and then dynamically refresh the metadata information of the database to ensure the accuracy of the metadata information. Alternatively, the query device can dynamically update the metadata information every time it queries to ensure the effect of this query.

[0061] Therefore, in this embodiment, the name of the database to be queried can be determined through the database information, and then the database with the name can be accessed to obtain its metadata table. Then, the metadata recorded in the metadata table can be extracted to obtain the metadata information of the database.

[0062] S203: Generate a model character string according to the metadata and the relationship file representing the dependency relationship between the metadata in the database.

[0063] In this embodiment, the relationship file is a file that records the dependency relationships between metadata in the database. These dependency relationships can be association relationships, inclusion relationships, etc. between data sets corresponding to metadata. The relationship file can be obtained by constructing class objects corresponding to data sets in the target database and assigning corresponding dependency relationships to each class of objects according to the dependency relationships between data sets.

[0064] For example, for Calcite, Figure 3 As shown, Figure 3 A schematic diagram of a relationship file is shown, which contains five class objects, namely: MongoCalciteModle, MongoCalciteSchema, MongoCalciteOperand, MongoCalciteSchemaExt and MongoCalciteTable, wherein the class object MongoCalciteModle corresponds to the data set MongoCalciteModle in Calcite, the class object MongoCalciteSchema corresponds to the data set MongoCalciteSchema in Calcite, the class object MongoCalciteOperand corresponds to the data set MongoCalciteOperand in Calcite, the class object MongoCalciteSchemaExt corresponds to the data set MongoCalciteSchemaExt in Calcite, and the class object MongoCalciteTable corresponds to the data set MongoCalciteTable in Calcite. Each class object contains the data name of the metadata of the data set corresponding to the class object.

[0065] In Calcite, there is a dependency relationship between the data set MongoCalciteModel and the data set MongoCalciteSchemaExt and the data set MongoCalciteSchema, there is a dependency relationship between the data set MongoCalciteSchemaExt and the data set MongoCalciteTable, and there is a dependency relationship between the data set MongoCalciteSchema and the data set MongoCalciteOperand. Based on this, create corresponding dependencies between the class objects corresponding to the above data sets, and you can get Figure 3 The relationship file shown.

[0066] In this embodiment, the metadata of each data set in the database can be filled into the class object corresponding to the data set in the relationship file according to the data name of each metadata, so as to materialize the abstract dependency relationship between the data sets in the original database, and then generate a model string based on the relationship file after filling in the metadata.

[0067] In this embodiment, when generating a model string, the metadata filled in the relationship file needs to meet certain conditions. Specifically, the metadata can be divided into multiple first metadata and multiple second metadata according to the data type of each metadata. Among them, the data type of each first metadata in the multiple first metadata is the first type, and the first type can be a file information type, that is, the metadata of this type is used to characterize the information related to the data stored in the database, for example: if the data set is a set of stored data, then the metadata of this type can be metadata such as data name, set name, table name, etc.; if the data set is a set of stored user information, then the metadata of this type can be metadata such as user name, password, creation time, etc. Since the relationship file is used to characterize the dependency relationship between the metadata in the corresponding database, strictly speaking, it is used to characterize the dependency relationship between the data sets corresponding to the metadata in the corresponding database. Therefore, each class object constructed in the relationship file corresponds to each data set in the database. Therefore, the metadata filled in each class object is the metadata of the corresponding data set, and the metadata of the data set are all file information types, that is, information characterizing the data stored in the database, for example: metadata of data name, set name, table name, etc., that is, metadata of the first type.

[0068] Take the metadata in Table 1 as an example, where the metadata table_name is the table name of the data table in the database, the metadata table_type is the type of the data table, the metadata create_user is the user name of the creating user, the metadata create_time is the creation time, the metadata operand_username is the user name of the operating user, and the metadata operan_password is the password of the operating user. These metadata are all metadata related to the data in the data set, so the above metadata are the first type of metadata. The metadata Database Name is the name of the database, the metadata DatabaseVersion is the version of the database, the metadata Character Set is the character set of the database, the metadata Privileges is the permissions of the database, and the metadata Encryption is the encryption information of the database. These metadata are not related to the data in the data set in the database, so they do not belong to the first type of metadata.

[0069] Based on this, in this implementation, metadata is first classified by data type, and metadata of the first type is screened out, so that when the metadata is subsequently filled into the relationship file, the corresponding metadata can be screened out more efficiently.

[0070] In this embodiment, the data type of the metadata can be obtained by querying the database through the field name of the metadata in the data table. In an optional implementation, the data type of each metadata can also be maintained in a separate data table, or maintained in the same data table as the metadata, so that the data type of each metadata can be quickly obtained when needed.

[0071] In this implementation, after determining a plurality of first metadata, each first metadata can be filled into a class object corresponding to the data set to which the metadata belongs in the relationship file according to the data name of each first metadata, and then the relationship file filled with metadata is sorted into a first string. Figure 3 Taking the relationship file shown in the figure as an example, the first string after sorting is as follows:

[0072] Database Structure:Modle'MongoCalciteModle'(defaultSchema:[],version:[],schemas:List <object>);Schema'MongoCalciteSchema'(factory:[],name:[],type:[],operand:MongoCalciteOperand);Operand'MongoCalciteOperand'(authDatabase:[],authMechanism:[],database:[],host:[],password:[],port:[],username:[]);Schema'MongoCalciteSchemaExt'(name:[],table:List <mongocalcitetable>);Table'MongoCalciteTable'(name:[],sql:[],database:[],type:[]).

[0073] because Figure 3 For a relational file without corresponding metadata, "[]" is used in the above string to replace the metadata. For example, if there is a metadata: 001, which belongs to the data set MongoCalciteSchema and its data name is name, then it will replace the "[]" corresponding to name after MongoCalciteSchema in the above string, as follows:

[0074] Database Structure:Modle'MongoCalciteModle'(defaultSchema:[],version:[],schemas:List <object>);Schema'MongoCalciteSchema'(factory:[],name:001,type:[],operand:MongoCalciteOperand);Operand'MongoCalciteOperand'(authDatabase:[],authMechanism:[],database:[],host:[],password:[],port:[],username:[]);Schema'MongoCalciteSchemaExt'(name:[],table:List <mongocalcitetable>);Table'MongoCalciteTable'(name:[],sql:[],database:[],type:[]).

[0075] Therefore, the related information of the data in each data set and the relationship between the data sets can be clearly seen through the above first string. For example, the set MongoCalciteSchema includes three metadata "factory:[],name:[],type:[]", and there is an operand relationship between it and the set MongoCalciteOperand. The set MongoCalciteOperand is used to store information about objects that can access the set MongoCalciteSchema.

[0076] Finally, the first string and the multiple second metadata are concatenated to obtain the model string containing the metadata in the database and the dependencies between the metadata. For example, assuming that the remaining multiple second metadata are metadata Database Name, metadata Database Version, metadata Character Set, metadata Privileges, and metadata Encryption, the model string is as follows:

[0077] Database Structure:Modle'MongoCalciteModle'(defaultSchema:[],version:[],schemas:List <object>);Schema'MongoCalciteSchema'(factory:[],name:[],type:[],operand:MongoCalciteOperand);Operand'MongoCalciteOperand'(authDatabase:[],authMechanism:[],database:[],host:[],password:[],port:[],username:[]);Schema'MongoCalciteSchemaExt'(name:[],table:List <mongocalcitetable>);Table'MongoCalciteTable'(name:[],sql:[],database:[],type:[]);Database Name;Database Version;Character Set;Privileges;Encryptio.

[0078] In this implementation, for Calcite, the dependency relationship presented in the form of a class object can be converted into a JSON string through a converter, and then passed into Calcite in an inline manner for use.

[0079] S204: Based on the model character string, basic information and query request, a query language supported by the database is called to generate a query statement for the database.

[0080] In this embodiment, if Figure 4 As shown, Figure 4 A flowchart of an implementation method of step S204 is shown below:

[0081] S401: Parse the query request to obtain query semantic information.

[0082] In this implementation, the query request may first be segmented to obtain a plurality of first segmented words.

[0083] Take the SQL-based query request as an example. If the query request is:

[0084] SELECT name,age FROM employees WHERE department='Sales'AND age>30;

[0085] After word segmentation, we can get the first participle: SELECT, name,,, age, FROM, employees, WHERE, department, =, 'Sales', AND, age, >, 30 and;.

[0086] Then, an abstract syntax tree can be constructed based on the multiple first word segments. An abstract syntax tree is a tree structure that expresses the syntax structure of the code. Each node represents a syntax structure, such as an expression or an operator.

[0087] By performing grammatical analysis on the above query request, it can be obtained that, based on the query request, the nodes in the abstract syntax tree are as follows:

[0088] Root node: SELECT statement;

[0089] Child nodes: name, age (indicates the columns to be selected);

[0090] Child node: FROM employees (indicates the data source);

[0091] Child node: WHERE department = 'Sales' AND age>30 (indicating filtering conditions).

[0092] The abstract syntax tree of the query request is as follows:

[0093]

[0094]

[0095] Then, the above abstract syntax tree can be converted into a relational algebra tree. A relational algebra tree is a tree structure that expresses relational algebra operations and is used to describe query operations of a relational database. For the above abstract syntax tree, when converted, the nodes in the relational algebra tree obtained are as follows:

[0096] Root node: σ (select operation), condition: department = 'Sales' AND age > 30;

[0097] Child nodes: π (projection operation), columns: name, age;

[0098] Child node: table scan operation, table: employees.

[0099] The relational algebra tree obtained after conversion is as follows:

[0100]

[0101] Finally, the query semantic information can be determined based on the above-mentioned relational algebraic tree. Exemplarily, the relational algebraic tree can be traversed first to determine the operation information and operation object of each node in the relational algebraic tree. Then, the operation information and operation object of each node are sorted according to the position and relationship of each node in the relational algebraic tree. Finally, semantic analysis is performed based on the sorted operation information and operation object of each node to obtain the query semantic information.

[0102] Specifically, the operation information and operation objects in each node in the above relational algebraic tree are as follows:

[0103] Root node: Operation information: "σ(select operation)"; Operation object: "Selection condition: department = 'Sales' AND age>30";

[0104] Child node: operation information: "π (projection operation)"; operation object: "column: name, age";

[0105] Subnode: Operation information: "Table scan operation"; Operation object: "Table: employees".

[0106] Then trace it back from the last child node to the root node to get the sorting:

[0107] "Table scan operation", "Table: employees", "π (projection operation)", "Column: name, age", "σ (selection operation)", "Selection condition: department = 'Sales' AND age>30"

[0108] Therefore, semantic analysis can be performed based on the above sorting to determine the operations, filter conditions, selected columns, data sources and other information involved in this query request, as follows:

[0109] Operation type: selection (σ), projection (π), table scan;

[0110] Filter condition: department = 'Sales' AND age > 30;

[0111] Selected columns: name, age;

[0112] Data source: employees table.

[0113] Finally, after sorting, we can get the query semantics:

[0114] Select the name and age columns from the employees table where the value of the department column is equal to Sales and the value of the age column is greater than 30.

[0115] S402: Determine a first dependency relationship of a target object corresponding to the basic information in a database according to the basic information query model character string.

[0116] In this embodiment, the model string records the association relationship, inclusion relationship, etc. between the data sets corresponding to each metadata in the database. Therefore, by querying the model string through the target object information, it is possible to determine which other data sets in the database the target object has a dependency relationship with, and then use the set of these dependencies as the first dependency of the target object in the database.

[0117] For example, if the target object is located in the database Figure 3 In the collection MongoCalciteSchema in the model string, it can be seen that there is an operand relationship between the collection MongoCalciteSchema where the target object is located and the collection MongoCalciteOperand, and the collection MongoCalciteOperand is used to store information about objects that can access the collection MongoCalciteSchema. The collection MongoCalciteSchema where the target object is located is also a database object (Schema) in the collection MongoCalciteModle. The first dependency relationship of the target object can be expressed as [MongoCalciteOperand:operand; MongoCalciteModle:Schema]. Of course, the dependency relationship can also be expressed in other ways, and this application does not limit this.

[0118] S403: Determine a query path of the target object in the database according to the first dependency relationship.

[0119] In this embodiment, after determining the first dependency of the target object in the database, multiple paths to access the target object can be determined based on these dependencies, that is, multiple first indexes of the target object in the database. At the same time, since different users have different access rights in the database, after determining multiple first indexes, the information of the queryer can be determined based on the query request, and then the query rights of the queryer for each first index can be determined based on the metadata of the database, and then the first indexes that the queryer has the right to query can be screened out to obtain multiple second indexes. Finally, the second index with the shortest index path is determined among the multiple second indexes, and its corresponding path is determined as the query path to achieve efficient query of the target object.

[0120] S404: Based on the query path, query semantics and basic information, the query language supported by the database is called to generate a query statement for the database.

[0121] In this embodiment, the query semantics can be divided into multiple query steps, and the query logic of each query step in the database and the execution order of the multiple query logics in the database are determined according to the query path and basic information. Then, according to the multiple query logics, multiple first execution statements corresponding to the multiple query logics are generated, and the multiple first execution statements are arranged according to the execution order of the corresponding query logics in the database to obtain second execution statements. Finally, the query language supported by the database is called, and a query statement for the database is generated according to the second execution statement.

[0122] Take MongoDB database, combined with the above query semantics: "Select the name and age columns from the employees table, if the value of the department column is equal to Sales and the value of the age column is greater than 30." as an example. First, split the query semantics into the following multiple query steps:

[0123] 1. Access the employees table;

[0124] 2. Get the values ​​of the name column and age column of the rows in the employees table that meet the conditions. The condition is that the value of the department column of the row is equal to Sales and the value of the age column is greater than 30.

[0125] Then, based on the query path, the query logic of the above two query steps in the database is determined respectively. Specifically, for the first access step, according to the query path, its query logic 1 in the database is: the collection name corresponding to the employees table in the database is "employees1", and accessing the collection "employees1" requires accessing the collection "text1". For the first access step, according to the query path, its query logic 2 in the database is: obtain the values ​​of the name column and age column of the rows that meet the conditions in the collection "employees1", and the condition is that the value of the department column of the row is equal to Sales and the value of the age column is greater than 30. Moreover, in the database, query logic 1 needs to be executed first to complete the access of employees1 before query logic 2 can be executed to obtain relevant data.

[0126] Therefore, for query logic 1, the first execution statement 1 in the database can be generated: access the set "text1", and access the set "employees1" according to the access condition. For query logic 2, the first execution statement 2 in the database can be generated: obtain the values ​​of the name column and the age column of the row that meets the condition in the set "employees1", and the condition is that the value of the department column of the row is equal to Sales and the value of the age column is greater than 30.

[0127] Then, the first execution statement 1 and the first execution statement 2 are arranged in the execution order of their corresponding query logics to obtain the following second execution statement:

[0128] Access the collection "text1", access the collection "employees1" according to the access conditions, and obtain the values ​​of the name column and age column of the rows in the collection "employees1" that meet the conditions. The conditions are that the value of the department column of the row is equal to Sales and the value of the age column is greater than 30.

[0129] Finally, the query language supported by the database is called, and a query statement for the database is generated according to the second execution statement. Taking the MongoDB database as an example, the following query statement can be obtained according to the query language supported by the database:

[0130] db.text1.find({ / *Access control conditions* / });

[0131] db.employees1.find(

[0132] {

[0133]

[0134] Among them, " / *Access control condition* / " is the condition for accessing the set "employees1" through the set "text1".

[0135] S205: Query the database according to the query statement to obtain the query result of the target object.

[0136] In this embodiment, after obtaining the query language, the query device can execute the query language in the database to query the target object in the database according to the query conditions in the query language and obtain the target object information that meets the query conditions as the query result.

[0137] For example, taking the query statement in step S404 as an example, after executing the query statement, the collection "text1" in the database is accessed to achieve access to the collection "employees1", and then the data in the collection "employees1" whose department column value is equal to Sales and whose age column value is greater than 30 is obtained, and the name column value and age column value of these data are output as query results.

[0138] In summary, this method obtains the metadata of the database by parsing the database information of the database requested to be queried in the query request, and processes the metadata of the database based on the relationship file that characterizes the dependency relationship between the metadata in the database, and generates a model string containing the metadata in the database and the dependency relationship between the metadata. Then, according to the query semantics of the query request, the target object information of the target object requested to be queried, and the model string, the query language supported by the database is called, and the query statement applicable to the database is regenerated to query the database. In this way, the dynamic conversion of the query request is realized, so that the query request can be automatically converted into a query statement applicable to the queried database, and a fast query of any database is realized. Then, this method has high versatility and can be applied to data queries in most data storage engines.

[0139] The above mainly introduces the scheme of the implementation method of the present application from the perspective of the method side. It can be understood that in order to realize the above functions, the query device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the implementation method disclosed in this article, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0140] The implementation method of the present application can divide the functional units of the terminal device according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one functional module. The above integrated module can be implemented in the form of hardware or in the form of a software program module. It should be noted that the division of modules in the implementation method of the present application is schematic and is only a logical function division, and there may be other division methods in actual implementation.

[0141] In the case of integrated modules, Figure 5 The data query device 500 includes a determination module 501 , an acquisition module 502 , a processing module 503 and a query module 504 .

[0142] In this implementation, the determination module 501, the acquisition module 502, the processing module 503 and the query module 504 may be a module for receiving and processing signals, information, etc. or determining a monitoring mechanism, and there is no specific limitation on this.

[0143] In this embodiment, the data query device 500 may further include a storage module for the computer program code or instructions executed by the data query device 500. The storage module may be a memory.

[0144] In this embodiment, the data query device 500 may be a chip or a chip module.

[0145] In this embodiment, the determination module 501, the acquisition module 502, the processing module 503 and the query module 504 can be integrated in the communication module. The communication module can be a communication interface, a transceiver, a transceiver circuit, etc.

[0146] In this implementation, the determination module 501 , the acquisition module 502 , the processing module 503 and the query module 504 may be integrated into a processor.

[0147] It should be noted that the processor can be a baseband processor, a baseband chip, a central processing unit (CPU), a general processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processing module can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of DSPs and microprocessors, etc.

[0148] In this embodiment, the data query device 500 is used to execute any step executed by the terminal device / chip / chip module, etc. in the above method implementation.

[0149] In specific implementation, the determination module 501, the acquisition module 502, the processing module 503 and the query module 504 are used to execute any step in the above method implementation, and when executing actions such as sending, other modules can be selectively called to complete the corresponding operation.

[0150] A determination module 501 is used to determine the database information of the queried database and the basic information of the queried target object according to the query request;

[0151] An acquisition module 502 is used to acquire metadata of the database according to the database information;

[0152] The processing module 503 is used to generate a model string according to the metadata and a relationship file representing the dependency relationship between the metadata in the database, and to call a query language supported by the database according to the model string, the basic information and the query request to generate a query statement for the database;

[0153] The query module 504 is used to query the database according to the query statement to obtain the query result of the target object.

[0154] In this implementation, the metadata is stored in the database via a metadata table of a table structure;

[0155] The acquiring metadata of the database according to the database information includes:

[0156] Acquire a metadata table of the database according to the database information;

[0157] The metadata recorded in the metadata table of the database is extracted to obtain the metadata of the database.

[0158] In this implementation, in terms of generating a model string according to the metadata and a relationship file representing a dependency relationship between metadata in the database, the processing module 503 is specifically configured to:

[0159] According to the first type, the metadata is screened to obtain a plurality of first metadata and a plurality of second metadata, wherein the data type of each first metadata is the first type;

[0160] According to the data name of each first metadata, fill each first metadata into a corresponding area in the relationship file to obtain a first character string;

[0161] The first character string and the plurality of second metadata are concatenated to obtain the model character string.

[0162] In this implementation, in terms of calling the query language supported by the database to generate a query statement for the database according to the model character string, the basic information and the query request, the processing module 503 is specifically used to:

[0163] Parsing the query request to obtain query semantic information;

[0164] Querying the model string according to the basic information to determine a first dependency relationship of the target object corresponding to the basic information in the database;

[0165] Determine a query path of the target object in the database according to the first dependency relationship;

[0166] According to the query path, the query semantics and the basic information, a query language supported by the database is called to generate a query statement for the database.

[0167] In this implementation, in determining the query path of the target object in the database according to the first dependency relationship, the processing module 503 is specifically used to:

[0168] Determine a plurality of first indexes of the target object in the database according to the first dependency relationship;

[0169] Determining the query authority of the query person for each of the multiple indexes according to the query request, and determining multiple second indexes from the multiple first indexes, wherein the query person has the query authority for each of the second indexes;

[0170] A path corresponding to a second index with the shortest index path among the plurality of second indexes is determined as the query path.

[0171] In this implementation, in parsing the query request to obtain query semantic information, the processing module 503 is specifically used to:

[0172] Performing word segmentation processing on the query request to obtain multiple first word segmentations;

[0173] Constructing an abstract syntax tree according to the plurality of first word participles;

[0174] Converting the abstract syntax tree into a relational algebra tree;

[0175] The query semantic information is determined according to the relational algebraic tree.

[0176] In this implementation, in determining the query semantic information according to the relational algebraic tree, the processing module 503 is specifically configured to:

[0177] Traversing the relational algebraic tree to determine operation information and operation objects of each node in the relational algebraic tree;

[0178] According to the position and relationship of each node in the relational algebraic tree, the operation information and operation objects of each node are sorted;

[0179] According to the sorted operation information and operation objects of each node, semantic analysis is performed to obtain the query semantic information.

[0180] In this implementation, in terms of calling the query language supported by the database according to the query path, the query semantics and the basic information to generate a query statement for the database, the processing module 503 is specifically used to:

[0181] Splitting the query semantics into multiple query steps;

[0182] Determine the query logic of each query step in the database and the execution order of multiple query logics in the database according to the query path and the basic information;

[0183] Generate a plurality of first execution statements according to the plurality of query logics, wherein the plurality of first execution statements correspond to the plurality of query logics one by one;

[0184] Arrange the multiple first execution statements according to the execution order of the multiple query logics in the database to obtain a second execution statement;

[0185] A query language supported by the database is called, and a query statement for the database is generated according to the second execution statement.

[0186] See also Figure 6 , Figure 6 6 is a schematic diagram of a structure of an electronic device proposed in an embodiment of the present application. The electronic device 600 may include a processor 610, a memory 620, and a communication bus for connecting the processor 610 and the memory 620.

[0187] Optionally, the memory 620 includes but is not limited to random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or portable read-only memory (CD-ROM), and the memory 620 is used to store the program code executed by the electronic device 600 and the transmitted data.

[0188] In this embodiment, the electronic device 600 further includes a communication interface for receiving and sending data.

[0189] In this implementation, the processor 610 may be one or more CPUs. When the processor 610 is a CPU, the CPU may be a single-core CPU or a multi-core CPU.

[0190] In this embodiment, the processor 610 can be a baseband chip, a chip, a CPU, a general processor, a DSP, an ASIC, an FPGA or other programmable logic devices, transistor logic devices, hardware components or any combination thereof.

[0191] In a specific implementation, the processor 610 in the electronic device 600 is used to execute the computer program or instruction 621 stored in the memory 620 to perform the following operations:

[0192] Determine database information of the queried database and basic information of the queried target object according to the query request;

[0193] Acquire metadata of the database according to the database information;

[0194] Generate a model string according to the metadata and a relationship file representing a dependency relationship between metadata in the database;

[0195] Invoke a query language supported by the database according to the model character string, the basic information and the query request to generate a query statement for the database;

[0196] The database is queried according to the query statement to obtain a query result of the target object.

[0197] It should be noted that Figure 6 The specific implementation of each operation in the implementation method can be found in the description of the method implementation method shown above, and will not be described in detail here.

[0198] The present application also provides a computer-readable storage medium storing a computer program or instructions, which implements the steps described in the above method implementation when executed.

[0199] The embodiment of the present application also provides a computer program product, including a computer program or instructions, which implement the steps described in the above method embodiment when executed.

[0200] It should be noted that, for the above-mentioned various embodiments, for the sake of simple description, they are all expressed as a series of action combinations. Those skilled in the art should be aware that the present application is not limited by the described order of actions, because some steps in the embodiments of the present application can be performed in other orders or simultaneously. In addition, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions, steps, modules or units involved are not necessarily required for the embodiments of the present application.

[0201] In the above-mentioned embodiments, the description of each embodiment of the present application has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0202] The steps of the method or algorithm described in the embodiments of the present application can be implemented in hardware or by executing software instructions by a processor. The software instructions can be composed of corresponding software modules, and the software modules can be stored in RAM, flash memory, ROM, EPROM, electrically erasable programmable read-only memory (electrically EPROM, EEPROM), registers, hard disks, mobile hard disks, read-only compact disks (CD-ROMs) or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in a terminal device or a management device. Of course, the processor and the storage medium can also be present in a terminal device or a management device as discrete components.

[0203] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiments of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital video disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0204] The modules / units included in the devices and products described in the above embodiments may be software modules / units or hardware modules / units, or may be partially software modules / units and partially hardware modules / units. For example, for the devices and products applied to or integrated in the chip, the modules / units included therein may all be implemented in the form of hardware such as circuits, or at least some of the modules / units may be implemented in the form of software programs, which run on the processor integrated inside the chip, and the remaining (if any) modules / units may be implemented in the form of hardware such as circuits; for the devices and products applied to or integrated in the chip module, the modules / units included therein may all be implemented in the form of hardware such as circuits, and different modules / units may be located in the same component (such as a chip, circuit module, etc.) or in different components of the chip module, or at least some of the modules / units may be implemented in the form of software programs. The software programs run on the processor integrated inside the chip, and the remaining (if any) modules / units may be implemented in the form of hardware such as circuits. It is implemented in the form of a software program, which runs on a processor integrated inside the chip module, and the remaining (if any) modules / units can be implemented in hardware such as circuits; for various devices and products applied to or integrated in the terminal equipment, the various modules / units contained therein can be implemented in hardware such as circuits, and different modules / units can be located in the same component (for example, chip, circuit module, etc.) or in different components in the terminal equipment, or, at least some modules / units can be implemented in the form of a software program, which runs on a processor integrated inside the terminal equipment, and the remaining (if any) modules / units can be implemented in hardware such as circuits.

[0205] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of the implementation methods of the present application. It should be understood that the above description is only the specific implementation method of the implementation method of the present application and is not intended to limit the protection scope of the implementation method of the present application. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the implementation methods of the present application should be included in the protection scope of the implementation methods of the present application.< / mongocalcitetable> < / object> < / mongocalcitetable> < / object> < / mongocalcitetable> < / object>

Claims

1. A data query method, characterized in that: The method comprises: Determine database information of the queried database and basic information of the queried target object according to the query request; Acquire metadata of the database according to the database information; Generate a model string according to the metadata and a relationship file representing a dependency relationship between metadata in the database; Invoke a query language supported by the database according to the model character string, the basic information and the query request to generate a query statement for the database; The database is queried according to the query statement to obtain a query result of the target object.

2. The method according to claim 1, characterized in that The metadata is stored in the database via a metadata table of a table structure; The acquiring metadata of the database according to the database information includes: Acquire a metadata table of the database according to the database information; The metadata recorded in the metadata table of the database is extracted to obtain the metadata of the database.

3. The method according to claim 1 or 2, characterized in that: The step of generating a model string according to the metadata and a relationship file representing the dependency relationship between the metadata in the database includes: According to the first type, the metadata is screened to obtain a plurality of first metadata and a plurality of second metadata, wherein the data type of each first metadata is the first type; According to the data name of each first metadata, fill each first metadata into a corresponding area in the relationship file to obtain a first character string; The first character string and the plurality of second metadata are concatenated to obtain the model character string.

4. The method according to any one of claims 1 to 3, characterized in that: The calling of the query language supported by the database according to the model character string, the basic information and the query request to generate a query statement for the database includes: Parsing the query request to obtain query semantic information; Querying the model string according to the basic information to determine a first dependency relationship of the target object corresponding to the basic information in the database; Determine a query path of the target object in the database according to the first dependency relationship; According to the query path, the query semantics and the basic information, a query language supported by the database is called to generate a query statement for the database.

5. The method according to claim 4, characterized in that The determining, according to the first dependency relationship, a query path of the target object in the database includes: Determine a plurality of first indexes of the target object in the database according to the first dependency relationship; Determining the query authority of the query person for each of the multiple indexes according to the query request, and determining multiple second indexes from the multiple first indexes, wherein the query person has the query authority for each of the second indexes; A path corresponding to a second index with the shortest index path among the plurality of second indexes is determined as the query path.

6. The method according to claim 4 or 5, characterized in that: The parsing of the query request to obtain query semantic information includes: Performing word segmentation processing on the query request to obtain multiple first word segmentations; Constructing an abstract syntax tree according to the plurality of first word participles; Converting the abstract syntax tree into a relational algebra tree; The query semantic information is determined according to the relational algebraic tree.

7. The method according to claim 6, characterized in that The determining the query semantic information according to the relational algebraic tree includes: Traversing the relational algebraic tree to determine operation information and operation objects of each node in the relational algebraic tree; According to the position and relationship of each node in the relational algebraic tree, the operation information and operation objects of each node are sorted; According to the sorted operation information and operation objects of each node, semantic analysis is performed to obtain the query semantic information.

8. The method according to claim 4, characterized in that The calling of a query language supported by the database according to the query path, the query semantics and the basic information to generate a query statement for the database includes: Splitting the query semantics into multiple query steps; Determine the query logic of each query step in the database and the execution order of multiple query logics in the database according to the query path and the basic information; Generate a plurality of first execution statements according to the plurality of query logics, wherein the plurality of first execution statements correspond to the plurality of query logics one by one; Arrange the multiple first execution statements according to the execution order of the multiple query logics in the database to obtain a second execution statement; A query language supported by the database is called, and a query statement for the database is generated according to the second execution statement.

9. A data query device, characterized in that: The device comprises: A determination module, used to determine the database information of the queried database and the basic information of the queried target object according to the query request; An acquisition module, used for acquiring metadata of the database according to the database information; A processing module, configured to generate a model string according to the metadata and a relationship file representing the dependency relationship between the metadata in the database, and to call a query language supported by the database according to the model string, the basic information and the query request to generate a query statement for the database; The query module is used to query the database according to the query statement to obtain the query result of the target object.

10. An electronic device comprising a processor, a memory and a computer program or instruction stored in the memory, characterized in that: The processor executes the computer program or instructions to implement the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program or instructions, and when the computer program or instructions are executed, the steps of the method according to any one of claims 1 to 8 are performed.

Images