A data query method and related device based on relational cluster database

Abstract

The present invention discloses a data query method and related device based on a relational cluster database, and relates to the field of computer technology. The method comprises: parsing a query statement in a query request based on an abstract syntax tree to obtain query task information and execution rule information; generating a query task queue based on the query task information, dynamically arranging the query task queue in sequence in combination with the execution rule information to obtain a query priority task queue; determining a corresponding target index based on the query priority task queue in combination with index performance analysis; obtaining a target hash index based on the target index, and determining a target query path using cost evaluation; performing data query in a relational cluster database using the target query path based on a storage engine to obtain target query data; and screening and optimizing the target query data based on client device permissions to obtain optimized target query data. The present invention enables data query in a relational cluster database to achieve a more ideal effect.

Description

Technical Field

[0001] The present invention mainly relates to the field of computer technology, and in particular to a data query method based on a relational cluster database and a related device. Background Art

[0002] With the continuous development of Internet technology, it is necessary to effectively store massive data, and currently, relational cluster databases are usually used to store relevant data. In the process of using a database, querying the data in the database is an important operation. At present, for data queries in relational cluster databases, parsing tables are usually used to parse query statements in query requests, but this method only supports some commonly used database languages ​​and requires multiple iterations, which affects the efficiency of statement parsing. Using indexes can quickly access relevant data in the database, but there is currently a lack of performance analysis of indexes for index selection, resulting in the determined indexes being unable to meet the demand for query efficiency improvement in big data scenarios. The query path is used to improve the efficiency and accuracy of data queries. Therefore, after determining the index, a query path is usually constructed. At present, for the generation of query paths, the performance data of the query path is usually determined only by deploying node queries, and the actual cost of the query path in query execution cannot be fully and accurately evaluated, resulting in difficulty in effectively improving the processing performance of query operations, so that the efficiency of the data query process fails to achieve the expected improvement effect. In addition, current data queries rarely support returning only query data with viewing permissions, and the security guarantee for data queries is unreliable, resulting in the failure of data queries in relational cluster databases to achieve the expected effect. Summary of the invention

[0003] The purpose of the present invention is to overcome the deficiencies of the prior art. The present invention provides a data query method and related devices based on a relational cluster database, which improves the efficiency of data query and enables data query of the relational cluster database to achieve a more ideal effect.

[0004] In order to solve the above technical problems, the present invention provides a data query method based on a relational cluster database, which is applied to a client device, a server device and a storage engine of the relational cluster database, wherein the server device is respectively connected to the client device and the storage engine of the relational cluster database in communication; the method comprises:

[0005] The server device receives the query request sent by the client device, and parses the query statement in the query request based on the abstract syntax tree to obtain query task information and execution rule information;

[0006] Generate a query task queue based on the query task information, dynamically arrange the query task queue in sequence in combination with the execution rule information, and obtain a query priority task queue;

[0007] Determine the corresponding target index based on the query priority task queue combined with index performance analysis;

[0008] Acquire a target hash index based on the target index, and determine a target query path based on the target hash index using cost evaluation;

[0009] Based on the storage engine, data query processing is performed in the relational cluster database using the target query path to obtain target query data;

[0010] The target query data is screened and optimized based on the client device authority to obtain optimized target query data, and the optimized target query data is transmitted to the client device.

[0011] Optionally, the query statement in the query request is parsed based on the abstract syntax tree to obtain query task information and execution rule information, including:

[0012] Acquire a syntax parser corresponding to the query statement, and parse the query statement into an abstract syntax tree based on the syntax parser;

[0013] Determine the nested relationship of the query statement, and determine the table information and field information in the query statement based on the abstract syntax tree and the nested relationship;

[0014] A corresponding complete syntax tree is generated based on the table information and the field information, and parsed based on the complete syntax tree to obtain query task information and execution rule information.

[0015] Optionally, a query task queue is generated based on the query task information, and the query task queue is dynamically arranged in sequence in combination with the execution rule information to obtain a query priority task queue, including:

[0016] Generate a query task queue using a Task structure and a TaskQueue structure based on the query task information;

[0017] The priority order in the query task queue is determined based on a preset priority sorting rule in combination with the execution rule information, and the query task queue is dynamically arranged in sequence based on the priority order to obtain a query priority task queue.

[0018] Optionally, the corresponding target index is determined based on the query priority task queue combined with index performance analysis, including:

[0019] Determine a number of corresponding initial composite indexes based on the query priority task queue using a preset index matching rule;

[0020] Calculate the index benefit of each initial composite index, and perform performance analysis on each initial composite index to obtain a performance analysis result of each initial composite index;

[0021] A corresponding target index is determined from among several initial composite indexes based on the index benefit and performance analysis result.

[0022] Optionally, a target hash index is obtained based on the target index, and a target query path is determined based on the target hash index using cost evaluation, including:

[0023] Obtaining an index type and a number of index key values ​​of a target index, and matching a target hash index based on the index type and the number of index key values;

[0024] Creating a plurality of query paths based on the target hash index;

[0025] Defining influencing factors of the query cost in the query path, and determining a query cost function based on the influencing factors;

[0026] Based on the random forest model, the performance data of each query path is optimized and updated to obtain several updated query paths;

[0027] Based on the query cost function, a plurality of update query paths are parallelized to obtain a parallelization result, and based on the parallelization result, a target query path is determined among the plurality of update query paths.

[0028] Optionally, data query processing is performed in the relational cluster database using the target query path based on the storage engine to obtain target query data, including:

[0029] A corresponding storage engine is determined based on the operator in the query statement, and a data query operation is performed in the relational cluster database based on the storage engine using the target query path to obtain target query data.

[0030] Optionally, the target query data is filtered and optimized based on the client device permissions to obtain optimized target query data, including:

[0031] Determine a user cluster role based on the client device permissions, and filter out query data with viewing permissions from the target query data based on the user cluster role;

[0032] Calling the extended processing function to integrate the query data with viewing authority to obtain integrated query data, and sorting and merging the integrated query data to obtain merged query data;

[0033] The combined query data is formatted and packaged to obtain optimized target query data.

[0034] In addition, the present invention also provides a data query device based on a relational cluster database, which is applied to a client device, a server device and a storage engine of the relational cluster database, wherein the server device is respectively connected to the client device and the storage engine of the relational cluster database in communication; the device comprises:

[0035] Query statement parsing module: used for the server device to receive the query request sent by the client device, and parse the query statement in the query request based on the abstract syntax tree to obtain query task information and execution rule information;

[0036] A query task queue arrangement module: used for generating a query task queue based on the query task information, dynamically arranging the query task queue in sequence in combination with the execution rule information, and obtaining a query priority task queue;

[0037] Index determination module: used to determine the corresponding target index based on the query priority task queue combined with index performance analysis;

[0038] A query path determination module: used to obtain a target hash index based on the target index, and determine a target query path based on the target hash index using a cost evaluation;

[0039] Data query module: used to perform data query processing in the relational cluster database based on the storage engine and using the target query path to obtain target query data;

[0040] Query data screening and optimization module: used to screen and optimize the target query data based on the client device permissions, obtain optimized target query data, and transmit the optimized target query data to the client device.

[0041] In addition, the present invention also provides a data query system based on a relational cluster database, the system comprising a client device, a server device and a storage engine of the relational cluster database, the server device being respectively communicatively connected to the client device and the storage engine of the relational cluster database, the system being configured to execute the above-mentioned data query method based on the relational cluster database.

[0042] In addition, the present invention also provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned data query method based on a relational cluster database.

[0043] In an embodiment of the present invention, the query statement in the query request is parsed based on the complete syntax tree generated by the abstract syntax tree, without multiple iterations, which can improve the efficiency of query statement parsing and avoid restrictions on the database language. The query task queue generated by the query task information is dynamically arranged in sequence in combination with the execution rule information, which can effectively improve the efficiency of query task execution. Based on the query priority task queue combined with the index performance analysis, the corresponding target index is determined, so that the determined index can meet the demand for query efficiency improvement in the big data scenario. Based on the target hash index, the target query path is determined by cost evaluation, which can comprehensively and accurately evaluate the actual cost of the query path in the query execution, improve the processing performance of the data query, and enable the efficiency of the data query process to be maintained in a better state. Based on the user cluster role determined by the client device authority, the target query data queried in the relational cluster database is screened and optimized to avoid data leakage problems, improve the security of data query, and enable the data query of the relational cluster database to achieve a more ideal effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 is a flow chart of a data query method based on a relational cluster database in an embodiment of the present invention;

[0046] Figure 2 is a flow chart of a data query method based on a relational cluster database in another embodiment of the present invention;

[0047] Figure 3 It is a schematic diagram of the structure of a data query system based on a relational cluster database in an embodiment of the present invention;

[0048] Figure 4 It is a schematic diagram of the structural composition of a data query device based on a relational cluster database in an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] Embodiment 1

[0051] See also Figure 1 , Figure 1 1 is a flow chart of a data query method based on a relational cluster database in an embodiment of the present invention, the method is applied to a client device, a server device and a storage engine of a relational cluster database, the server device is respectively connected to the client device and the storage engine of the relational cluster database in communication; the method comprises:

[0052] S11: The server device receives the query request sent by the client device, and parses the query statement in the query request based on the abstract syntax tree to obtain query task information and execution rule information;

[0053] In the specific implementation process of the present invention, the query statement in the query request is parsed based on the abstract syntax tree to obtain query task information and execution rule information, including: obtaining a syntax parser corresponding to the query statement, and parsing the query statement into an abstract syntax tree based on the syntax parser; determining the nesting relationship of the query statement, and determining the table information and field information in the query statement based on the abstract syntax tree and the nesting relationship; generating a corresponding complete syntax tree based on the table information and field information, and parsing based on the complete syntax tree to obtain query task information and execution rule information.

[0054] Specifically, the server device receives the query request sent by the client device through the query interface, and obtains the syntax parser corresponding to the query statement in the query request. Different query statements have different syntax parsers, such as PostgreSQL syntax parser and SQL syntax parser, etc. The server device determines the corresponding syntax parser according to the relevant type of the query statement in the received query request. Based on the syntax parser, the query statement is parsed into an abstract syntax tree. The syntax parser constructs an abstract syntax tree corresponding to the query statement. The abstract syntax tree is a tree representation method used to describe the syntax structure of program code. Each node of the syntax tree represents a syntax structure in the program code, which better avoids the restriction on the database language. Determine the nested relationship of the query statement. By parsing the outer query in the query statement, the sub-query position of the syntax structure in the query statement can be analyzed to obtain the nested relationship of the query statement. Based on the abstract syntax tree combined with the nested relationship, the table information and field information in the query statement are determined, and the hierarchical relationship between several branches related to the query statement in the abstract syntax tree is determined according to the nested relationship. According to the syntax node in each branch, the corresponding statement block is determined, and the hierarchical relationship between the statement blocks is determined according to the hierarchical relationship between the branches. According to the hierarchical relationship between the statement blocks, the corresponding table information and field information are extracted, thereby extracting more accurate table information and field information. The corresponding table information and field information are as follows: table name: test, alias: null, statement block: (select*form_test)t1, field: test_score. Based on the table information and field information, a corresponding complete syntax tree is generated, table nodes are determined according to the table information and field information, hierarchical relationships between table nodes are determined according to the hierarchical relationship between the table information and field information, and a complete syntax tree is constructed according to the hierarchical relationship between table nodes and the table nodes combined with the nested relationship. Based on the complete syntax tree, all nodes of the complete syntax tree are traversed to determine the statement type, execution target and index establishment point of the complete syntax tree, so as to obtain query task information and execution rule information, wherein the query task information includes information such as query target and type, and the execution rule information includes query priority information and query task level information, etc. Thus, the query statement in the query request is parsed without multiple iterations, which can improve the efficiency of query statement parsing, avoid restrictions on the database language, and improve the accuracy of query statement parsing.

[0055] S12: generating a query task queue based on the query task information, dynamically arranging the query task queue in sequence in combination with the execution rule information, and obtaining a query priority task queue;

[0056] In the specific implementation process of the present invention, a query task queue is generated based on query task information, and the query task queue is dynamically arranged in sequence in combination with execution rule information to obtain a query priority task queue, including: generating a query task queue based on the query task information using a Task structure and a TaskQueue structure; determining a priority order in the query task queue based on a preset priority sorting rule in combination with the execution rule information, and dynamically arranging the query task queue in sequence based on the priority order to obtain a query priority task queue.

[0057] Specifically, based on the query task information, a query task queue is generated using a Task structure and a TaskQueue structure. The Task structure is a key data structure used to describe a task process. The Task structure can be used to describe the corresponding process for the query task information. The TaskQueue structure is a template class for managing task queues. The TaskQueue structure can dynamically adjust the number of threads. Therefore, the use of the TaskQueue structure can adapt to different task processing requirements in the query task and improve the concurrent processing capabilities of related threads. The generated query task queue is not sorted, so the priority order in the query task queue is determined based on the preset priority sorting rules combined with the execution rule information. The preset priority sorting rules may be that cached and indexed queries are the first priority and cached query tasks are the second priority, etc. Cached and indexed refers to query information that has been entered in the past, and a corresponding ID index is generated based on the query information. The execution rule information includes query task level information. The priority order can be determined by combining the preset priority sorting rules with the execution rule information, and the query task queue is dynamically arranged in sequence based on the priority order. The dynamic arrangement of sequence is to arrange the query task queue in real time according to the priority order to obtain a query priority task queue, which can effectively improve the efficiency of query task execution.

[0058] S13: Determine a corresponding target index based on the query priority task queue combined with index performance analysis;

[0059] In the specific implementation process of the present invention, the corresponding target index is determined based on the query priority task queue combined with the index performance analysis, including: based on the query priority task queue, using the preset index matching rules to determine the corresponding several initial compound indexes; calculating the index benefit of each initial compound index, and performing performance analysis on each initial compound index to obtain the performance analysis result of each initial compound index; based on the index benefit and the performance analysis result, determining the corresponding target index among the several initial compound indexes.

[0060] Specifically, based on the query priority task queue, a corresponding number of initial composite indexes are determined using preset index matching rules, which include leftmost prefix rules and data coverage rules, etc. A candidate initial composite index set is determined according to the query priority task queue, and a number of initial composite indexes are determined in the candidate initial composite index set according to the preset index matching rules. A composite index refers to an index on multiple columns. A composite index requires less overhead during database operations and can replace multiple single indexes. The index benefit of each initial composite index is calculated, and the saved query time, reduced disk read and write data volume, and reduced memory usage of each initial composite index are calculated. The index benefit is calculated according to a preset ratio based on the calculated saved query time, reduced disk read and write data volume, and reduced memory usage. A performance analysis is performed on each initial composite index. The performance analysis includes index size analysis, data selectivity analysis, and index tree depth analysis. Index size analysis is to calculate the size of the space occupied by the initial composite index. The smaller the space occupied, the better the performance of the initial composite index. Data selectivity analysis is to calculate the number of all data indexed by the initial composite index in the process of querying the required data. If the number of obtained data is smaller, it means that the query speed of the initial composite index is faster, and it can determine the required data more quickly from less data. Index tree depth analysis is to analyze the tree depth corresponding to the initial composite index. The smaller the tree depth, the lower the query efficiency. The performance analysis result of each initial composite index is determined based on the index size analysis results, data selectivity analysis results, and index tree depth analysis results. Based on the index benefit and performance analysis results, a corresponding target index is determined from several initial composite indexes, and the index benefits and performance analysis results of each initial composite index are sorted. Under the condition of satisfying the index capacity restriction condition, the index benefit and performance analysis result of the selected target index are optimized. The index capacity restriction condition includes that the sum of the storage costs of each index does not exceed the preset index capacity restriction value, and then the required several target indexes are determined, so that the determined index can meet the demand for improving query efficiency in big data scenarios.

[0061] S14: obtaining a target hash index based on the target index, and determining a target query path based on the target hash index by using cost evaluation;

[0062] In the specific implementation process of the present invention, a target hash index is obtained based on the target index, and a target query path is determined based on the target hash index using cost evaluation, including: obtaining the index type and the number of index key values ​​of the target index, and matching the target hash index based on the index type and the number of index key values; creating a number of query paths based on the target hash index; defining influencing factors of the query cost in the query path, and determining a query cost function based on the influencing factors; optimizing and updating the performance data of each query path based on a random forest model to obtain a number of updated query paths; performing parallel operations on a number of updated query paths based on the query cost function to obtain parallel operation results, and determining a target query path among the number of updated query paths based on the parallel operation results.

[0063] Specifically, the index type and the number of index key values ​​of the target index are obtained, the corresponding index type and the corresponding index key values ​​are stored in the index configuration, and the target hash index is determined by using the leftmost match based on the index type and the number of index key values ​​of the target index. Several query paths are created based on the target hash index, and several query paths are created using the path storage hash table through the target hash index. Influencing factors of the query cost in the query path are defined, and the influencing factors may include the delay between nodes and the database scale, etc. The influencing factors can be defined according to the actual situation, and the query cost function is determined based on the influencing factors, and the query cost function is constructed according to the influencing factors and their corresponding weights. Based on the random forest model, the performance data of each query path is optimized and updated. The performance data of each query path, such as execution time, resource consumption and error rate, is collected to define a parameter space, which contains a combination set formed by several parameter configurations. Several combination sets are selected in the parameter space as configuration sample sets. All query paths perform query simulation operations according to the configured sample sets to obtain their query simulation operation data, that is, the performance data of the query operation performed according to the configuration sample sets. The parameter space is dynamically searched according to the obtained query simulation operation data. The dynamic search of the parameter space is performed through Bayesian optimization to obtain several updated query paths. Based on the query cost function, several update query paths are parallelized, and the parallel computing environment includes computing resource allocation, thread pool initialization, and process pool initialization. The query cost function is used as the target function of the parallel computing, the resource proportion of the historical query operation is obtained, and the operation result parameters of the historical query operation are counted. The path planning model is constructed through the resource proportion and the operation result parameters. The path planning model is used to perform parallel computing in combination with the target function and the parallel computing environment to obtain the parallel computing result, which is the query cost value, and the target query path is determined in several update query paths based on the parallel computing result. The update query path with the smallest query cost in several update query paths is determined as the target query path, which can realize the adaptive optimization of the query globally and better control the global query cost. At the same time, it can comprehensively and accurately evaluate the actual cost of the query path in the query execution, improve the processing performance of the data query, and keep the efficiency of the data query process in a better state.

[0064] S15: performing data query processing in a relational cluster database using the target query path based on the storage engine to obtain target query data;

[0065] In the specific implementation process of the present invention, data query processing is performed in a relational cluster database based on a storage engine using a target query path to obtain target query data, including: determining a corresponding storage engine based on an operator in a query statement, and performing a data query operation in a relational cluster database based on the storage engine using the target query path to obtain target query data.

[0066] Specifically, the corresponding storage engine is determined according to the operator in the query statement, and the corresponding storage engine can be determined according to the corresponding identifier in the operator. The storage engine includes a document database storage engine, a spatiotemporal database storage engine, and a time series database storage engine, etc. According to the storage engine, a data query operation is performed in the relational cluster database using the target query path to obtain the target query data, that is, the corresponding documents, spatiotemporal data, and pictures, etc. The relational cluster database is an overall database composed of at least two or more database servers. Each database server is equipped with a relational database. The relational cluster database can store more data than a general database, can support a larger amount of access, meet the storage and access requirements of the big data scenario, and transmit the target query data to the server device for subsequent data processing.

[0067] S16: Screening and optimizing the target query data based on the client device authority to obtain optimized target query data, and transmitting the optimized target query data to the client device.

[0068] In the specific implementation process of the present invention, the target query data is screened and optimized based on the client device permissions to obtain optimized target query data, including: determining the user cluster role based on the client device permissions, and filtering out the query data with viewing permissions in the target query data based on the user cluster role; calling the extended processing function to integrate the query data with viewing permissions to obtain integrated query data, and sorting and merging the integrated query data to obtain merged query data; formatting the merged query data to obtain optimized target query data.

[0069] Specifically, the user cluster role is determined based on the client device authority. The client device authority can be obtained in the authority database. The user cluster role corresponds to the client device authority. Different user cluster roles have different data with viewing authority. A query data is taken out from the target query data to determine whether the query data taken out has viewing authority in the client device. If the query data has viewing authority in the client device, the data is retained and added to the query result set. If the query data does not have viewing authority in the client device, the query data is removed from the query result. After all query data are judged, all query data with viewing authority for the client device can be obtained. Call the extended processing function to integrate the query data with viewing authority. The extended processing function is a custom data operation function. Integrating the query data with viewing authority is to integrate the same type of data to obtain integrated query data, and sort and merge the integrated query data. The sort and merge process is to merge the same data in the integrated query data, and then sort according to the corresponding time to obtain the merged query data. The combined query data is formatted and packaged, that is, packaged and processed according to a specific data format to obtain optimized target query data. The optimized target query data is transmitted to a client device for display.

[0070] In an embodiment of the present invention, the query statement in the query request is parsed based on the complete syntax tree generated by the abstract syntax tree, without multiple iterations, which can improve the efficiency of query statement parsing and avoid restrictions on the database language. The query task queue generated by the query task information is dynamically arranged in sequence in combination with the execution rule information, which can effectively improve the efficiency of query task execution. Based on the query priority task queue combined with the index performance analysis, the corresponding target index is determined, so that the determined index can meet the demand for query efficiency improvement in the big data scenario. Based on the target hash index, the target query path is determined by cost evaluation, which can comprehensively and accurately evaluate the actual cost of the query path in the query execution, improve the processing performance of the data query, and enable the efficiency of the data query process to be maintained in a better state. Based on the user cluster role determined by the client device authority, the target query data queried in the relational cluster database is screened and optimized to avoid data leakage problems, improve the security of data query, and enable the data query of the relational cluster database to achieve a more ideal effect.

[0071] Embodiment 2

[0072] See also Figure 2 , Figure 21 is a flow chart of a data query method based on a relational cluster database in another embodiment of the present invention, the method is applied to a client device, a server device and a storage engine of a relational cluster database, the server device is respectively connected to the client device and the storage engine of the relational cluster database in communication; the method comprises:

[0073] S201: The server device receives a query request sent by the client device, and parses a query statement in the query request based on an abstract syntax tree to obtain query task information and execution rule information;

[0074] S202: Generate a query task queue based on the query task information, dynamically arrange the query task queue in sequence in combination with the execution rule information, and obtain a query priority task queue;

[0075] S203: Determine a corresponding target index based on the query priority task queue combined with index performance analysis;

[0076] S204: Obtain the index type and the number of index key values ​​of the target index, and match the target hash index based on the index type and the number of index key values;

[0077] In the specific implementation process of the present invention, the index type and the number of index key values ​​of the target index are obtained, the corresponding index type and the corresponding index key value are stored in the index configuration, and the target hash index is determined by using the leftmost match based on the index type and the number of index key values ​​of the target index.

[0078] S205: Creating a plurality of query paths based on the target hash index;

[0079] In a specific implementation of the present invention, a plurality of query paths are created based on the target hash index, and a plurality of query paths are created by using a path storage hash table through the target hash index.

[0080] S206: defining influencing factors of the query cost in the query path, and determining a query cost function based on the influencing factors;

[0081] In the specific implementation process of the present invention, the influencing factors of the query cost in the query path are defined. The influencing factors may include the delay between nodes and the database scale, etc. The influencing factors can be defined according to the actual situation, and the query cost function is determined based on the influencing factors. The query cost function is constructed according to the influencing factors and their corresponding weights.

[0082] S207: optimizing and updating the performance data of each query path based on the random forest model to obtain a number of updated query paths;

[0083] In the specific implementation process of the present invention, the performance data of each query path is optimized and updated based on the random forest model, and the performance data of each query path, such as execution time, resource consumption and error rate, etc., are collected to define a parameter space, which includes a combination set formed by several parameter configurations. Several combination sets are selected in the parameter space as configuration sample sets, and all query paths perform query simulation operations according to the configured sample sets to obtain their query simulation operation data, that is, the performance data of the query operation performed according to the configuration sample sets. The parameter space is dynamically searched according to the obtained query simulation operation data, and the dynamic search of the parameter space is performed by Bayesian optimization to obtain several updated query paths.

[0084] S208: performing parallel operations on a plurality of update query paths based on the query cost function to obtain parallel operation results, and determining a target query path among the plurality of update query paths based on the parallel operation results;

[0085] In the specific implementation process of the present invention, a plurality of update query paths are parallelized based on the query cost function, and the parallel computing environment includes computing resource allocation, thread pool initialization, and process pool initialization. The query cost function is used as the target function of the parallel computing, the resource proportion of the historical query operation is obtained, and the operation result parameters of the historical query operation are counted. The path planning model is constructed by the resource proportion and the operation result parameters. The path planning model is used to perform parallel computing in combination with the target function and the parallel computing environment to obtain the parallel computing result, which is the query cost value, and the target query path is determined in the plurality of update query paths based on the parallel computing result. The update query path with the smallest query cost in the plurality of update query paths is determined as the target query path, which can realize the adaptive optimization of the query globally and better control the global query cost. At the same time, the actual cost of the query path in the query execution can be comprehensively and accurately evaluated, the processing performance of the data query is improved, and the efficiency of the data query process can be maintained in a better state.

[0086] S209: performing data query processing in a relational cluster database using the target query path based on the storage engine to obtain target query data;

[0087] S210: Determine a user cluster role based on client device permissions;

[0088] In a specific implementation of the present invention, the user cluster role is determined based on the client device permissions, and the client device permissions can be obtained in a permissions database.

[0089] S211: extracting a piece of query data from the target query data;

[0090] S212: judging whether the retrieved query data has viewing permission in the client device according to the user cluster role;

[0091] In the specific implementation process of the present invention, the user cluster role corresponds to the client device permission. Different user cluster roles have different viewing permissions. Therefore, it can be judged whether the retrieved query data has viewing permissions in the client device based on the user cluster role. If it is judged that the retrieved query data has viewing permissions in the client device, step S213 is entered. If it is judged that the retrieved query data has viewing permissions in the client device, the query data is discarded and the process goes to step S211 to continue to retrieve the next query data for judgment.

[0092] S213: If the retrieved query data has viewing permission in the client device, the query data is added to the query result set to obtain the query data with viewing permission;

[0093] S214: calling the extended processing function to integrate the query data with viewing authority to obtain integrated query data, and performing sorting and merging processing on the integrated query data to obtain merged query data;

[0094] S215: Perform format encapsulation processing on the combined query data to obtain optimized target query data, and transmit the optimized target query data to the client device.

[0095] In an embodiment of the present invention, the query statement in the query request is parsed based on the complete syntax tree generated by the abstract syntax tree, without multiple iterations, which can improve the efficiency of query statement parsing and avoid restrictions on the database language. The query task queue generated by the query task information is dynamically arranged in sequence in combination with the execution rule information, which can effectively improve the efficiency of query task execution. Based on the query priority task queue combined with the index performance analysis, the corresponding target index is determined, so that the determined index can meet the demand for query efficiency improvement in the big data scenario. Based on the target hash index, the target query path is determined by cost evaluation, which can comprehensively and accurately evaluate the actual cost of the query path in the query execution, improve the processing performance of the data query, and enable the efficiency of the data query process to be maintained in a better state. Based on the user cluster role determined by the client device authority, the target query data queried in the relational cluster database is screened and optimized to avoid data leakage problems, improve the security of data query, and enable the data query of the relational cluster database to achieve a more ideal effect.

[0096] Embodiment 3

[0097] See also Figure 3 , Figure 3It is a schematic diagram of the structural composition of a data query system based on a relational cluster database in an embodiment of the present invention, wherein the system comprises a client device 31, a server device 32 and a storage engine 33 of a relational cluster database, wherein the server device 32 is respectively communicated with the client device 31 and the storage engine 33 of the relational cluster database; the system is configured to execute any one of the data query methods based on a relational cluster database described in the embodiments.

[0098] In the specific implementation process of the present invention, the server device 32 establishes a communication connection with the client device 31 through the network, the client device 31 initiates a query request for the relational cluster database, and sends the query request to the server device 32. The server device 32 receives the query request sent by the client device 31, and processes the query request accordingly, converting the query request into a query operation corresponding to the storage engine 33 of the relational cluster database. The storage engine 33 of the relational cluster database executes the operation to obtain query data. The server device 32 processes the obtained query data and transmits the processed query data to the client device 31 for display. Figure 3 The system shown does not constitute a limitation of all components, and may include more or less components than shown, or combine certain components.

[0099] In the specific implementation process of the present invention, the specific implementation methods of the system items can be referred to the above embodiments, which will not be repeated here.

[0100] In an embodiment of the present invention, the query statement in the query request is parsed based on the complete syntax tree generated by the abstract syntax tree, without multiple iterations, which can improve the efficiency of query statement parsing and avoid restrictions on the database language. The query task queue generated by the query task information is dynamically arranged in sequence in combination with the execution rule information, which can effectively improve the efficiency of query task execution. Based on the query priority task queue combined with the index performance analysis, the corresponding target index is determined, so that the determined index can meet the demand for query efficiency improvement in the big data scenario. Based on the target hash index, the target query path is determined by cost evaluation, which can comprehensively and accurately evaluate the actual cost of the query path in the query execution, improve the processing performance of the data query, and enable the efficiency of the data query process to be maintained in a better state. Based on the user cluster role determined by the client device authority, the target query data queried in the relational cluster database is screened and optimized to avoid data leakage problems, improve the security of data query, and enable the data query of the relational cluster database to achieve a more ideal effect.

[0101] Embodiment 4

[0102] See also Figure 4 , Figure 4 1 is a schematic diagram of the structure of a data query device based on a relational cluster database in an embodiment of the present invention, wherein the device is applied to a client device, a server device and a storage engine of a relational cluster database, wherein the server device is respectively connected to the client device and the storage engine of the relational cluster database in communication; the device comprises:

[0103] Query statement parsing module 41: used for the server device to receive the query request sent by the client device, and parse the query statement in the query request based on the abstract syntax tree to obtain query task information and execution rule information;

[0104] The query task queue arrangement module 42 is used to generate a query task queue based on the query task information, and dynamically arrange the query task queue in sequence in combination with the execution rule information to obtain a query priority task queue;

[0105] Index determination module 43: used to determine the corresponding target index based on the query priority task queue combined with index performance analysis;

[0106] A query path determination module 44 is configured to obtain a target hash index based on the target index, and determine a target query path based on the target hash index using a cost evaluation;

[0107] Data query module 45: used to perform data query processing in the relational cluster database based on the storage engine and using the target query path to obtain target query data;

[0108] The query data screening and optimization module 46 is used to screen and optimize the target query data based on the client device authority, obtain optimized target query data, and transmit the optimized target query data to the client device.

[0109] In the specific implementation process of the present invention, the specific implementation method of the device item can refer to the implementation method of the above-mentioned method item, which will not be repeated here.

[0110] In an embodiment of the present invention, the query statement in the query request is parsed based on the complete syntax tree generated by the abstract syntax tree, without multiple iterations, which can improve the efficiency of query statement parsing and avoid restrictions on the database language. The query task queue generated by the query task information is dynamically arranged in sequence in combination with the execution rule information, which can effectively improve the efficiency of query task execution. Based on the query priority task queue combined with the index performance analysis, the corresponding target index is determined, so that the determined index can meet the demand for query efficiency improvement in the big data scenario. Based on the target hash index, the target query path is determined by cost evaluation, which can comprehensively and accurately evaluate the actual cost of the query path in the query execution, improve the processing performance of the data query, and enable the efficiency of the data query process to be maintained in a better state. Based on the user cluster role determined by the client device authority, the target query data queried in the relational cluster database is screened and optimized to avoid data leakage problems, improve the security of data query, and enable the data query of the relational cluster database to achieve a more ideal effect.

[0111] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, a data query method based on a relational cluster database in any of the above embodiments is implemented. The computer-readable storage medium includes, but is not limited to, any type of disk (including floppy disk, hard disk, optical disk, CD-ROM, and magneto-optical disk), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic card or optical card. That is, the storage device includes any medium that can store or transmit information in a readable form by a device (for example, a computer, a mobile phone), which can be a read-only memory, a disk or an optical disk, etc.

[0112] An embodiment of the present invention further provides a computer application program, which runs on a computer and is used to execute the data query method based on a relational cluster database of any one of the above embodiments.

[0113] In addition, the above is a detailed introduction to a data query method based on a relational cluster database and related devices provided in an embodiment of the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for a person skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A data query method based on a relational cluster database, characterized in that: The method is applied to a client device, a server device and a storage engine of a relational cluster database, wherein the server device is respectively connected to the client device and the storage engine of the relational cluster database in communication; the method comprises: The server device receives the query request sent by the client device, and parses the query statement in the query request based on the abstract syntax tree to obtain query task information and execution rule information; Generate a query task queue based on the query task information, dynamically arrange the query task queue in sequence in combination with the execution rule information, and obtain a query priority task queue; Determine the corresponding target index based on the query priority task queue combined with index performance analysis; Acquire a target hash index based on the target index, and determine a target query path based on the target hash index using cost evaluation; Based on the storage engine, data query processing is performed in the relational cluster database using the target query path to obtain target query data; Performing screening and optimization processing on the target query data based on the client device authority to obtain optimized target query data, and transmitting the optimized target query data to the client device; Among them, determining the corresponding target index based on the query priority task queue combined with the index performance analysis includes: determining several corresponding initial compound indexes based on the query priority task queue using preset index matching rules; calculating the index benefit of each initial compound index, calculating the saved query time, reduced disk read and write data volume and reduced memory usage by using each initial compound index, calculating the index benefit according to a preset ratio based on the calculated saved query time, reduced disk read and write data volume and reduced memory usage, and performing performance analysis on each initial compound index to obtain a performance analysis result of each initial compound index; determining the corresponding target index from several initial compound indexes based on the index benefit and performance analysis result, sorting according to the index benefit and performance analysis result of each initial compound index, and making the index benefit and performance analysis result of the selected target index optimal under the condition of satisfying the index capacity restriction condition, the index capacity restriction condition including that the sum of the storage costs of each index does not exceed the preset index capacity restriction value, and obtaining the corresponding target index; Obtain a target hash index based on the target index, and determine a target query path based on the target hash index using cost evaluation, including: obtaining the index type and the number of index key values ​​of the target index, and matching the target hash index based on the index type and the number of index key values; creating several query paths based on the target hash index; defining influencing factors of the query cost in the query path, and determining a query cost function based on the influencing factors; optimizing and updating the performance data of each query path based on a random forest model to obtain several updated query paths; performing parallel operations on several updated query paths based on the query cost function, using the query cost function as the target function of the parallel operation, obtaining the resource proportion of historical query operations, and counting the operation result parameters of the historical query operations, constructing a path planning model based on the resource proportion and the operation result parameters, performing parallel operations using the path planning model in combination with the target function and the parallel operation environment to obtain parallel operation results, and determining the target query path in several updated query paths based on the parallel operation results; The query task queue is dynamically and sequentially arranged in combination with the execution rule information to obtain a query priority task queue, including: determining the priority order in the query task queue based on a preset priority sorting rule in combination with the execution rule information, the preset priority sorting rule being that cached and indexed query tasks are the first priority and the query tasks being cached are the second priority, the execution rule information includes query task level information, and the query task queue is dynamically and sequentially arranged based on the priority order to obtain a query priority task queue.

2. The data query method based on the relational cluster database according to claim 1, characterized in that: Parse the query statement in the query request based on the abstract syntax tree to obtain query task information and execution rule information, including: Acquire a syntax parser corresponding to the query statement, and parse the query statement into an abstract syntax tree based on the syntax parser; Determine the nested relationship of the query statement, and determine the table information and field information in the query statement based on the abstract syntax tree and the nested relationship; A corresponding complete syntax tree is generated based on the table information and the field information, and parsed based on the complete syntax tree to obtain query task information and execution rule information.

3. The data query method based on the relational cluster database according to claim 1, characterized in that: Generate a query task queue based on the query task information, including: A query task queue is generated based on the query task information using a Task structure and a TaskQueue structure.

4. The data query method based on the relational cluster database according to claim 1, characterized in that: Based on the storage engine, data query processing is performed in the relational cluster database using the target query path to obtain the target query data, including: A corresponding storage engine is determined based on the operator in the query statement, and a data query operation is performed in the relational cluster database based on the storage engine using the target query path to obtain target query data.

5. The data query method based on the relational cluster database according to claim 1, characterized in that: The target query data is filtered and optimized based on the client device permissions to obtain optimized target query data, including: Determine a user cluster role based on the client device permissions, and filter out query data with viewing permissions from the target query data based on the user cluster role; Calling the extended processing function to integrate the query data with viewing authority to obtain integrated query data, and sorting and merging the integrated query data to obtain merged query data; The combined query data is formatted and packaged to obtain optimized target query data.

6. A data query device based on a relational cluster database, characterized in that: The apparatus is applied to a client device, a server device and a storage engine of a relational cluster database, wherein the server device is respectively connected to the client device and the storage engine of the relational cluster database in communication; the apparatus comprises: Query statement parsing module: used for the server device to receive the query request sent by the client device, and parse the query statement in the query request based on the abstract syntax tree to obtain query task information and execution rule information; A query task queue arrangement module: used for generating a query task queue based on the query task information, dynamically arranging the query task queue in sequence in combination with the execution rule information, and obtaining a query priority task queue; Index determination module: used to determine the corresponding target index based on the query priority task queue combined with index performance analysis; A query path determination module: used to obtain a target hash index based on the target index, and determine a target query path based on the target hash index using a cost evaluation; Data query module: used to perform data query processing in the relational cluster database based on the storage engine and using the target query path to obtain target query data; Query data screening and optimization module: used to screen and optimize the target query data based on the client device authority, obtain optimized target query data, and transmit the optimized target query data to the client device; Among them, determining the corresponding target index based on the query priority task queue combined with the index performance analysis includes: determining several corresponding initial compound indexes based on the query priority task queue using preset index matching rules; calculating the index benefit of each initial compound index, calculating the saved query time, reduced disk read and write data volume and reduced memory usage by using each initial compound index, calculating the index benefit according to a preset ratio based on the calculated saved query time, reduced disk read and write data volume and reduced memory usage, and performing performance analysis on each initial compound index to obtain a performance analysis result of each initial compound index; determining the corresponding target index from several initial compound indexes based on the index benefit and performance analysis result, sorting according to the index benefit and performance analysis result of each initial compound index, and making the index benefit and performance analysis result of the selected target index optimal under the condition of satisfying the index capacity restriction condition, the index capacity restriction condition including that the sum of the storage costs of each index does not exceed the preset index capacity restriction value, and obtaining the corresponding target index; Obtain a target hash index based on the target index, and determine a target query path based on the target hash index using cost evaluation, including: obtaining the index type and the number of index key values ​​of the target index, and matching the target hash index based on the index type and the number of index key values; creating several query paths based on the target hash index; defining influencing factors of the query cost in the query path, and determining a query cost function based on the influencing factors; optimizing and updating the performance data of each query path based on a random forest model to obtain several updated query paths; performing parallel operations on several updated query paths based on the query cost function, using the query cost function as the target function of the parallel operation, obtaining the resource proportion of historical query operations, and counting the operation result parameters of the historical query operations, constructing a path planning model based on the resource proportion and the operation result parameters, performing parallel operations using the path planning model in combination with the target function and the parallel operation environment to obtain parallel operation results, and determining the target query path in several updated query paths based on the parallel operation results; The query task queue is dynamically and sequentially arranged in combination with the execution rule information to obtain a query priority task queue, including: determining the priority order in the query task queue based on a preset priority sorting rule in combination with the execution rule information, the preset priority sorting rule being that cached and indexed query tasks are the first priority and the query tasks being cached are the second priority, the execution rule information includes query task level information, and the query task queue is dynamically and sequentially arranged based on the priority order to obtain a query priority task queue.

7. A data query system based on a relational cluster database, characterized in that: The system includes a client device, a server device and a storage engine of a relational cluster database. The server device is communicatively connected to the client device and the storage engine of the relational cluster database respectively. The system is configured to execute the data query method based on the relational cluster database as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on an electronic device, the electronic device executes the data query method based on a relational cluster database according to any one of claims 1 to 5.

Images