A data query method, device, electronic device and storage medium based on database

Abstract

The present invention discloses a data query method, device, electronic device and storage medium based on a database. It is applied to a coordination node, including: receiving a data query message sent by a client, generating a target execution plan based on the data query message; wherein the data query message includes an execution message; determining the type of a Portal object based on the execution message; when the type of the Portal object is an anonymous Portal, constructing a first message based on a first data volume and a target execution plan and sending it to a target data node, so that the target data node executes the target execution plan and returns the query results, and stops the query immediately until the number of query results returned meets the first data volume. This solution constructs a message according to the data volume and the execution plan when the type of the Portal object is an anonymous Portal, and controls the target data node to only query and return data that meets the data volume according to the message, thereby avoiding waste of database communication and memory resources and improving database performance.

Description

Technical Field

[0001] The present invention relates to the field of database technology, and in particular to a database-based data query method, device, electronic device and storage medium. Background Art

[0002] The overall performance of a database depends largely on the cost and efficiency of its query processing. During a distributed database data query, data flows through nodes including the client, Coordinator Node (CN), and Data Node (DN). The client sends a query statement to the coordinator node, which then sends the query plan or the original query statement to the data node. The data node returns the queried data to the coordinator node, which then processes the data and returns it to the client. However, the total number of query results returned by the data node can often far exceed the client's demand, resulting in a waste of database resources such as communication and memory. Summary of the Invention

[0003] The present invention provides a data query method, device, electronic device and storage medium based on a database to solve the problem of waste of resources such as database communication and memory.

[0004] According to one aspect of the present invention, a database-based data query method is provided, which is applied to a coordination node and includes:

[0005] Receive a data query message sent by a client, and generate a target execution plan based on the data query message; wherein the data query message includes an execution message;

[0006] Determine the type of the Portal object based on the execution message;

[0007] When the type of the Portal object is anonymous Portal, obtain a first data volume, and construct a first message based on the first data volume and the target execution plan; send the first message to the target data node, so that the target data node performs a data query based on the target execution plan in the first message and returns the query results, and stops the query immediately until the number of query results returned meets the first data volume.

[0008] Optionally, the data query message also includes a parsing message and a binding message; the parsing message includes a parameterized query statement, and the binding message includes parameter information corresponding to the query parameters in the parameterized query statement; generating a target execution plan based on the data query message includes: parsing the parameterized query statement in the parsing message to obtain a parsing result; generating an initial execution plan based on the parsing result through an execution plan pre-generation module; extracting the parameter information in the binding message, and binding the parameter information to the initial execution plan to generate a target execution plan.

[0009] Optionally, the parsing result includes at least the table name and the view name; after obtaining the parsing result, it also includes: performing permission verification on the table name and the view name through the access control list to obtain the verification result; if the verification result is that the verification passes, then continue to execute to generate the initial execution plan; if the verification result is that the verification fails, then terminate the query processing flow and generate exception information for exception reminder.

[0010] Optionally, determining the type of the Portal object based on the execution message includes: performing information extraction processing on the execution message to obtain a parameter value of the Portal object; if the parameter value of the Portal object is empty, determining that the type of the Portal object is an anonymous Portal; if the parameter value of the Portal object is non-empty, determining that the type of the Portal object is a named Portal.

[0011] Optionally, the method also includes: when the type of the Portal object is a named Portal, constructing a second message based on the target execution plan; sending the second message to the target data node, so that the target data node performs a data query based on the target execution plan in the second message, and caches the query results to the target storage space; when there is no second data volume in the execution message, extracting data that meets the default data volume from the target storage space and returning it to the data receiving end; continuing to receive new execution messages sent by the client until all query results in the target storage space are returned to the data receiving end.

[0012] Optionally, the method further includes: when there is a second data amount in the execution message, extracting data that meets the second data amount from the target storage space and returning the data to the data receiving end.

[0013] Optionally, the method further includes: when the type of the Portal object is a named Portal, receiving the execution message at least once; when the type of the Portal object is an anonymous Portal, receiving the execution message only once.

[0014] According to another aspect of the present invention, there is provided a data query device based on a database, comprising:

[0015] An execution plan determination module is configured to receive a data query message sent by a client and generate a target execution plan based on the data query message; wherein the data query message includes an execution message;

[0016] A type determination module, configured to determine the type of a Portal object based on an execution message;

[0017] The query result determination module is used to obtain a first data volume when the type of the Portal object is an anonymous Portal, and to construct a first message based on the first data volume and the target execution plan; the first message is sent to the target data node so that the target data node performs a data query based on the target execution plan in the first message and returns the query results, and the query is stopped immediately until the number of query results returned meets the first data volume.

[0018] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0019] at least one processor; and

[0020] a memory communicatively connected to at least one processor; wherein,

[0021] The memory stores a computer program that can be executed by at least one processor. The computer program is executed by the at least one processor so that the at least one processor can execute the database-based data query method of any embodiment of the present invention.

[0022] According to another aspect of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium stores computer instructions, which are used to enable a processor to implement the database-based data query method of any embodiment of the present invention when executed.

[0023] The technical solution of the embodiment of the present invention receives a data query message sent by a client and generates a target execution plan based on the data query message; wherein the data query message includes an execution message; generates execution logic according to the specific query requirements of the client, avoids meaningless data traversal, and improves resource utilization efficiency; can flexibly adapt to different query scenarios and quickly respond to the diverse query requirements of the client. The type of the portal object is determined based on the execution message; when the type of the portal object is an anonymous portal, a first data volume is obtained, and a first message is constructed based on the first data volume and the target execution plan; the first message is sent to the target data node, so that the target data node performs a data query based on the target execution plan in the first message and returns the query results, and stops the query immediately until the number of query results returned meets the first data volume. This achieves differentiated processing through type identification, constructs corresponding messages based on the characteristics of anonymous portals, and accurately controls the number of queries to avoid resource waste. It can ensure the target nature of data queries and reduce invalid queries through dynamic termination conditions, thereby improving query efficiency. At the same time, this mode can flexibly control the scale of data returned, effectively reducing network transmission pressure and system processing load while ensuring the integrity of data acquisition, and achieving the unity of data processing accuracy and efficiency. This solution constructs a message based on the data volume and execution plan when the Portal object type is anonymous Portal. Based on the message, the target data node is controlled to query and return only data that meets the data volume, ensuring that the query results of the target data node are completely consistent with the query data required by the client. This avoids the problem that the total amount of query results queried by the data node is far greater than the client's demand, thereby causing a waste of database communication and memory resources. This improves database query efficiency and helps improve the performance of distributed databases.

[0024] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 This is a flowchart of a database-based data query method provided by the first embodiment of the present invention;

[0027] Figure 2This is a flow chart of a database-based data query method provided by the second embodiment of the present invention;

[0028] Figure 3 This is a schematic diagram of the structure of a data query device based on a database provided in the third embodiment of the present invention;

[0029] Figure 4 It is a structural diagram of an electronic device for implementing the database-based data query method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] Example 1

[0033] Figure 1 This is a flowchart of a database-based data query method provided in Example 1 of the present invention. This embodiment is applicable to situations where data queries are performed. The method can be executed by a database-based data query device. The database-based data query device can be implemented in the form of hardware and / or software and can be integrated into a computer device.

[0034] It should be noted that during the query process in a distributed database system, data flows through nodes including the client, coordinator, and data nodes. The client can query data by sending PBE (Parse Bind and Execute) messages to the database system. PBE messages include parse, bind, and execute messages. PBE messages are a key communication mechanism for distributed query processing, primarily used for query plan distribution and execution coordination between the coordinator and data nodes. When the client sends an execute message, it indicates the amount of data to be returned. However, if the query statement in the parse message does not include a query data size limit clause, the amount of data retrieved by the data node far exceeds the client's request, resulting in wasted resources. In this embodiment, the messages between the coordinator and data nodes can be expanded to control the amount of data queried and returned by the data node, ensuring that the data node can accurately perform queries according to the data query requirements and avoiding excessive data overhead.

[0035] like Figure 1 As shown, the method includes:

[0036] S110: Receive a data query message sent by a client, and generate a target execution plan based on the data query message; wherein the data query message includes an execution message.

[0037] A data query message is a collection of information sent by a client to a database system or database cluster to request data. It includes at least an execution message, which triggers the database to execute a generated target execution plan to perform the corresponding operations on the database. A target execution plan is a specific execution plan generated after parsing and binding the received data query message. It defines the complete process from data query to data processing to result return.

[0038] Specifically, the coordination node receives data query messages from clients in real time and performs structured parsing of them according to a pre-set information parsing algorithm. The parsing results are then processed by an execution plan generation algorithm or optimizer to generate a target execution plan. For example, the information parsing algorithm can perform SQL (Structured Query Language) syntax parsing and validation on the data query messages. While ensuring the correctness of the statement structure, semantic analysis is performed to verify whether the parsed data table and column names contain user permissions. The query logic is then rewritten and optimized. The optimizer then generates multiple candidate physical execution plans based on statistical information, selecting between index scans and full table scans. Statistical information is core metadata in a database system that records the characteristics of data objects, describing the specific attributes and data distribution characteristics of objects such as tables, indexes, and columns. A pre-built cost model is used to calculate the resource consumption of each candidate physical execution plan and select the optimal plan. Finally, the plans are serialized into an executable structure to generate the target execution plan. The entire process relies on statistical information and cost estimates to achieve a balance between efficiency.

[0039] In this implementation, by performing structured parsing of data query messages, we can accurately understand client needs and reduce information bias; when generating an execution plan, we can optimize resource scheduling and execution order, improving data processing efficiency; and the integrated process design makes the conversion from query to execution more efficient, facilitates unified management and control of the system, enhances response speed and stability, and also provides a clear framework for subsequent performance optimization and exception handling.

[0040] Optionally, the data query message also includes a parsing message and a binding message; the parsing message includes a parameterized query statement, and the binding message includes parameter information corresponding to the query parameters in the parameterized query statement; generating a target execution plan based on the data query message includes: parsing the parameterized query statement in the parsing message to obtain a parsing result; generating an initial execution plan based on the parsing result through an execution plan pre-generation module; extracting the parameter information in the binding message, and binding the parameter information to the initial execution plan to generate a target execution plan.

[0041] Among them, the parsing message refers to the starting information for the interaction between the client and the database, carrying specific query instructions, including parameterized query statements, such as the common SELECT * FROM table_1, which is used to select all data from the table named table_1. The parsing message can also contain more complex query logic, such as query statements with conditional filtering, sorting, grouping and other clauses. The binding message refers to the storage of the actual parameter information corresponding to the placeholders in the parameterized query statement in the parsing message. It can also contain contextual information such as user permissions and data source configuration, which is used to associate abstract parameter placeholders with real data values ​​or execution conditions. The execution plan pre-generation module is a module that integrates the execution plan generation algorithm, which is used to parse and process data query statements to generate execution plans corresponding to data query statements.

[0042] Specifically, the coordination node receives a data query message consisting of a parsed message and a bound message. The parsed message includes a parameterized query statement, such as an SQL statement with placeholders, and the bound message includes the parameter values ​​corresponding to the placeholders. The coordination node then performs syntactic and semantic parsing on the parameterized statement in the parsed message to obtain a parsing result, which includes but is not limited to the operation type, table name, and query conditions. The coordination node then calls a preset execution plan pre-generation module to generate an initial execution plan based on the result. The coordination node then extracts the parameter information from the bound message and binds the parameter information to the corresponding placeholders in the initial execution plan, ultimately generating an executable target execution plan.

[0043] In this embodiment, by separating parameterized queries from parameter information, security risks such as SQL injection can be prevented. The step-by-step processing of parsing and binding makes plan generation more flexible. The method of pre-generating the initial plan and then binding the parameters can optimize the execution logic in advance and improve resource scheduling efficiency. At the same time, the parameterized design facilitates the reuse of the execution plan framework, reduces the overhead of repeated parsing, and enhances the standardization and efficiency of the system's query processing.

[0044] Optionally, the parsing result includes at least the table name and the view name; after obtaining the parsing result, it also includes: performing permission verification on the table name and the view name through the access control list to obtain the verification result; if the verification result is that the verification passes, then continue to execute to generate the initial execution plan; if the verification result is that the verification fails, then terminate the query processing flow and generate exception information for exception reminder.

[0045] Among them, Access Control List (ACL) specifically refers to a security mechanism for controlling access rights to resources. It is essentially a list composed of a set of rules that clearly stipulates the executable operations of the entity initiating the access request on the accessed object. The entity initiating the access request includes but is not limited to users, processes and devices, and the accessed object includes but is not limited to data tables and views. The executable operations include but are not limited to query operations, write operations, modify operations and delete operations. In this embodiment, the executable operation can specifically refer to the query operation. Permission verification is specifically used to verify the access rights of the entity initiating the access request, and ensure that the subject's access or operation on the object complies with the preset security rules. Its essence is to determine whether the subject has the legal authority to perform specific operations through a series of inspection processes, thereby preventing unauthorized access, data leakage or malicious operations.

[0046] Specifically, after parsing the parameterized query statement in the parsed message and obtaining the parsed results containing table and view names, the database's access control list is used to verify permissions on these table and view names to determine whether the user has access rights. If verification passes, the execution plan pre-generation module generates the initial execution plan. If verification fails, the query processing process is immediately terminated, and an exception message is generated as a reminder.

[0047] In this embodiment, by performing permission verification on tables and views before generating an execution plan, unauthorized access requests can be blocked at an early stage, effectively preventing unauthorized access to data and information leakage, and enhancing the data security of the system; the permission verification process is connected with the execution plan generation process, ensuring that only legitimate query requests will enter the subsequent processing link, avoiding the consumption of system resources by invalid operations, and at the same time, the timely feedback of abnormal information can also help users quickly locate permission problems, improving the reliability and usability of the system.

[0048] S120: Determine the type of the Portal object based on the execution message.

[0049] It should be noted that the binding message includes an object identifier for parsing the query statement in the message, namely the Portal object identifier. This identifier can be a meaningful string, such as "s_1." The Portal object identifier can be set according to actual needs and is not limited here. Setting the Portal object identifier facilitates the tracking and management of specific queries by the client and database. This is called a named portal, meaning the Portal object type is named Portal. Alternatively, the Portal object identifier can be an empty string, meaning an anonymous portal, which is suitable for temporary queries or queries that do not require a specific identifier. The Portal object, as the carrier of the execution plan, integrates dispersed query parameters, execution steps, and result caches into a unified logical unit. After the execution plan is bound to the Portal object, its lifecycle is strongly associated with the Portal object, facilitating centralized management of query tasks and providing a structured traceability path for subsequent execution optimization and troubleshooting. By assigning names to query statements, the client and database can more clearly organize and process multiple query requests, avoiding confusion.

[0050] Specifically, when an execution message is received, the execution message is parsed to determine a parsing result, wherein the parsing result includes at least information about the Portal object, and the type of the Portal object is determined based on the information about the Portal object. The information of the Portal object includes, but is not limited to, the Portal object name and the Portal object identifier. If the Portal object name or the Portal object identifier is a null value, the type of the Portal object is determined to be an anonymous Portal, and if the Portal object name or the Portal object identifier is not a null value, the type of the Portal object is determined to be a named Portal, and the corresponding information can be assigned to the Portal object.

[0051] In this embodiment, the type of the Portal object is determined by executing the message, which provides a clear basis for the subsequent personalized processing of different types of objects, improves the pertinence and efficiency of the data query system processing, makes the system more flexible and scalable, and facilitates the addition or adjustment of Portal object types according to business needs, thereby enhancing the system's adaptability to complex business scenarios.

[0052] Optionally, determining the type of the Portal object based on the execution message includes: performing information extraction processing on the execution message to obtain a parameter value of the Portal object; if the parameter value of the Portal object is empty, determining that the type of the Portal object is an anonymous Portal; if the parameter value of the Portal object is non-empty, determining that the type of the Portal object is a named Portal.

[0053] Specifically, first extract information from the execution message to obtain the parameter value of the Portal object. If the parameter value is empty, the Portal object is determined to be an anonymous Portal; if the parameter value is not empty, the Portal object is determined to be a named Portal.

[0054] In this embodiment, the Portal object type is determined by whether the parameter value is empty. The logic is concise and clear, and the type determination can be completed quickly, thereby improving processing efficiency. At the same time, the Portal objects are clearly divided into two categories: anonymous and named, which facilitates the system to execute differentiated processing strategies according to different types, enhances the flexibility and security of the system, and makes resource allocation and management more targeted.

[0055] S130. When the type of the Portal object is an anonymous Portal, obtain a first data volume, and construct a first message based on the first data volume and the target execution plan; send the first message to the target data node, so that the target data node performs a data query based on the target execution plan in the first message and returns the query results, and stops the query immediately until the number of query results returned meets the first data volume.

[0056] Among them, the first data volume specifically represents the target data volume that needs to be queried and obtained, and is set according to the query business requirements. A visual interface can be set on the client, and the user can set the corresponding first data volume according to the actual query requirements. The set first data volume can be stored in the execution message and sent to the coordination node, or the first data volume can be stored by setting a dedicated message, which is used to transmit the first data volume to the coordination node, so that the coordination node generates the corresponding first message when it receives the first data volume and the type of the Portal object is an anonymous portal. The first message specifically represents the message sent by the coordination node to the data node, and is specifically constructed based on the first data volume and the target execution plan when it is determined that the type of the Portal object is an anonymous portal. The target data node specifically represents the node that executes the target execution plan. It is the target node corresponding to the coordination node. The preset data sharding rules can be used to locate the shard where the target data is located. At the same time, the available data nodes are screened in combination with the data node status and load balancing strategy. The preset data sharding rules include but are not limited to hash-based sharding rules, range sharding rules, and label sharding rules; then the latest "data shard-data node" mapping relationship is obtained through the routing table or metadata service to dynamically adapt to changes in data distribution; for complex query scenarios, multiple target nodes need to be determined and coordinated for parallel execution, and finally the preset failover and retry mechanism is called to reroute tasks when the target node fails to ensure query reliability.

[0057] Specifically, when the Portal object type is anonymous, a preset first data volume is obtained. Based on the first data volume and the target execution plan, a first message containing the execution plan and data volume requirements is constructed. The first message is then sent to the target data node. Upon receiving the first message, the target data node performs a data query according to the target execution plan in the first message and immediately stops the query when the number of returned query results reaches the first data volume. This ensures that only data that meets the first data volume is queried, eliminating the need to query all data on the data node.

[0058] In this embodiment, by setting a limited data query volume for the anonymous portal, the scale of data query and return can be effectively controlled, excessive data acquisition due to anonymous access can be avoided, and data security can be guaranteed; the query mechanism with the first data volume as the termination condition can reduce invalid data processing, improve query efficiency, and reduce system resource consumption; by constructing a message containing the execution plan and data volume, the query operation of the data node is more targeted without the need to modify the execution plan, ensuring that while meeting the basic needs of anonymous access, accurate control of data access is achieved.

[0059] The technical solution of this embodiment is to generate a target execution plan based on the data query message by receiving a data query message sent by a client; wherein the data query message includes an execution message; determine the type of the Portal object based on the execution message; when the type of the Portal object is an anonymous Portal, obtain a first data volume, and construct a first message based on the first data volume and the target execution plan; send the first message to the target data node, so that the target data node performs a data query based on the target execution plan in the first message and returns the query results, and stops the query immediately until the number of returned query results meets the first data volume. This solution constructs a message based on the data volume and execution plan when the Portal object type is anonymous Portal, and controls the target data node according to the message to only query and return data that meets the data volume, thereby achieving differentiated processing through type identification to accurately control the number of queries and avoid resource waste. It can not only ensure the targeting of data queries, but also reduce invalid queries through dynamic termination conditions, ensuring that the query results of the target data node are completely consistent with the query data required by the client, and avoiding the problem that the total amount of query results queried by the data node is far greater than the client's demand, thereby causing waste of database communication and memory resources. At the same time, this model can flexibly control the scale of data return, effectively reduce network transmission pressure and system processing load while ensuring the integrity of data acquisition, improve database query efficiency, and help improve distributed database performance.

[0060] Example 2

[0061] Figure 2 This is a flowchart of a database-based data query method provided by the second embodiment of the present invention. The method of this embodiment is a further optimization of the method of the above embodiment. Optionally, when the type of the Portal object is a named Portal, a second message is constructed based on the target execution plan; the second message is sent to the target data node, so that the target data node performs data query based on the target execution plan in the second message, and caches the query results to the target storage space; when there is a second data volume in the execution message, data that meets the second data volume is extracted from the target storage space and returned to the data receiving end. Figure 2 As shown, the method includes:

[0062] S210: Receive a data query message sent by a client, and generate a target execution plan based on the data query message; wherein the data query message includes an execution message.

[0063] S220: Determine the type of the Portal object based on the execution message.

[0064] S230. When the type of the Portal object is named Portal, construct a second message based on the target execution plan; send the second message to the target data node, so that the target data node performs data query based on the target execution plan in the second message, and caches the query results to the target storage space.

[0065] Among them, the second information specifically represents the transmission information set when the type of the Portal object is a named Portal, and the second information consists of a target execution plan. It is used to drive the target data node to execute data queries and cache the results. The second message contains the specific content of the target execution plan, which serves as the instruction basis for the data node to perform query operations. After being sent to the target data node, it will trigger the target data node to execute the query as planned, and cache all the query results to the target storage space to achieve data reuse and improve query efficiency. It is a key information carrier for realizing personalized data processing and caching mechanisms for named portals. The target storage space specifically refers to a pre-set data cache space used to cache queried data.

[0066] Specifically, when the Portal object type is a named Portal, a second message is constructed according to the target execution plan, which contains complete execution plan information; then the second message is sent to the target data node. After receiving the second message, the target data node executes the data query operation according to the target execution plan in the message, and caches the query results to the target storage space.

[0067] In this embodiment, the caching mechanism for named portals can effectively utilize historical query results, reduce the overhead of repeated data queries, and improve the response speed of subsequent similar queries; accurately drive data node operations through target execution plans to ensure the consistency and efficiency of query logic; the design of caching to the target storage space facilitates unified management and reuse of data, while providing named portal users with a faster service experience and enhancing the system's support for personalized access.

[0068] S240. When the second data volume does not exist in the execution message, extract data that meets the default data volume from the target storage space and return it to the data receiving end; continue to receive new execution messages sent by the client until all query results in the target storage space are returned to the data receiving end.

[0069] Among them, the second data volume specifically represents the amount of data extraction explicitly specified in the execution message, which is used to instruct the system to extract data that meets this amount from the target storage space and return it to the data receiving end. The second data volume is a parameter that the client can customize when sending an execution message. When the second data volume exists in the execution message, the system will give priority to extracting data according to this specified amount, achieving precise control over the amount of data returned, and meeting the client's personalized needs for data acquisition scale in different scenarios. It is a key parameter used to define the scale of data extraction during data interaction. The default data specifically represents the amount of data obtained from the target storage space at a single time. It can be set in advance based on database performance and actual data query requirements, and is not limited here.

[0070] Specifically, when the second data volume is not specified in the execution message, the system will extract data that meets the default data volume from the target storage space and return it to the data receiving end. If all the query results of the data node have not been received, the client will continue to send execution messages to the coordination node and the data node to obtain all the query results. It can continuously receive new execution messages sent by the client and cyclically read the query results from the target storage space until all the query results in the target storage space are returned to the data receiving end.

[0071] In this embodiment, the setting of the default data volume ensures the standardization and stability of data return, avoiding data return confusion caused by unspecified quantity; the mechanism of continuously receiving new execution messages and cyclically returning data realizes the paginated and orderly return of large-scale query results, which can not only reduce the pressure of single data transmission, but also ensure the complete delivery of data; at the same time, the process does not require the client to clarify the data volume in advance, lowers the usage threshold, improves the system's usability and adaptability to different query scenarios, and makes the data return process more flexible and efficient.

[0072] Optionally, the method further includes: when there is a second data amount in the execution message, extracting data that meets the second data amount from the target storage space and returning the data to the data receiving end.

[0073] Specifically, determine whether there is a corresponding numerical value in the quantity setting field in the execution message. If there is a numerical value, assign the numerical value to the second data amount, that is, indicate that there is a second data amount in the execution message. When there is a second data amount in the execution message, the system directly extracts data that meets the second data amount from the target storage space and returns it to the data receiving end.

[0074] In this embodiment, data can be accurately extracted according to the quantity clearly specified in the execution message, avoiding the return of redundant data and improving data transmission efficiency; the client can flexibly control the amount of data obtained by setting the second data volume to meet personalized needs in different scenarios, such as reducing the amount of data transmission when the network bandwidth is limited, or accurately obtaining data when a specific scale is required; the method of directly extracting data according to the specified quantity is logically concise and has a fast processing speed, can quickly respond to client requests, and at the same time reduce system resource consumption, making data interaction more efficient and accurate.

[0075] Based on the above embodiment, the method further includes: when the type of the Portal object is a named Portal, receiving the execution message at least once; when the type of the Portal object is an anonymous Portal, receiving the execution message only once.

[0076] Specifically, if the Portal object type is a named Portal, the coordinator node receives at least one execute message. This means the coordinator node receives messages from the client as PBEEE*. The number of E messages depends on the number of queries and the amount of data returned in a single transaction. If the Portal object type is an anonymous Portal, the anonymous Portal cannot be executed repeatedly and receives only one execute message. This means the coordinator node receives messages from the client as PBE*.

[0077] The technical solution of this embodiment is to generate a target execution plan based on the data query message by receiving a data query message sent by the client; wherein the data query message includes an execution message; determine the type of the Portal object based on the execution message; when the type of the Portal object is a named Portal, construct a second message based on the target execution plan; send the second message to the target data node, so that the target data node performs a data query based on the target execution plan in the second message, and caches the query results to the target storage space; when the second data volume does not exist in the execution message, extract data that meets the default data volume from the target storage space and return it to the data receiving end; continue to receive new execution messages sent by the client until all query results in the target storage space are returned to the data receiving end. This solution achieves efficient scheduling and precise control of data queries through standardized execution plan generation, a named portal caching mechanism, and a flexible data return strategy. It can not only convert client requirements into unified execution logic, but also use cache to reduce duplication overhead. It can also adopt default batch or precise extraction strategies based on whether the quantity is specified. At the same time, it can achieve segmented delivery of large amounts of data by continuously receiving messages, improving system response speed and resource utilization. It can not only meet the client's clear demand for data volume, but also reduce the query pressure on data nodes through caching strategies, while optimizing network transmission efficiency. On the basis of ensuring real-time data response, it achieves efficient resource utilization and precise matching of data processing.

[0078] Example 3

[0079] Figure 3 This is a schematic diagram of the structure of a data query device based on a database provided by the third embodiment of the present invention. Figure 3 As shown, the device includes:

[0080] An execution plan determination module 310 is configured to receive a data query message sent by a client and generate a target execution plan based on the data query message; wherein the data query message includes an execution message;

[0081] A type determination module 320, configured to determine the type of the Portal object based on the execution message;

[0082] The query result determination module 330 is used to obtain a first data volume when the type of the Portal object is an anonymous Portal, and to construct a first message based on the first data volume and the target execution plan; the first message is sent to the target data node so that the target data node performs a data query based on the target execution plan in the first message and returns the query results, and the query is stopped immediately until the number of query results returned meets the first data volume.

[0083] The technical solution of this embodiment is to receive the data query message sent by the client through the execution plan determination module, and generate a target execution plan based on the data query message; wherein the data query message includes an execution message; the type determination module determines the type of the Portal object based on the execution message; when the type of the Portal object is an anonymous Portal, the query result determination module obtains a first data volume, and constructs a first message based on the first data volume and the target execution plan; the first message is sent to the target data node, so that the target data node performs a data query based on the target execution plan in the first message and returns the query results, and the query is stopped immediately until the number of returned query results meets the first data volume. This solution constructs a message based on the data volume and execution plan when the Portal object type is anonymous Portal, and controls the target data node according to the message to only query and return data that meets the data volume, thereby achieving differentiated processing through type identification to accurately control the number of queries and avoid resource waste. It can not only ensure the targeting of data queries, but also reduce invalid queries through dynamic termination conditions, ensuring that the query results of the target data node are completely consistent with the query data required by the client, and avoiding the problem that the total amount of query results queried by the data node is far greater than the client's demand, thereby causing waste of database communication and memory resources. At the same time, this model can flexibly control the scale of data return, effectively reduce network transmission pressure and system processing load while ensuring the integrity of data acquisition, improve database query efficiency, and help improve distributed database performance.

[0084] Based on the above embodiment, optionally, the data query message also includes a parsing message and a binding message; the parsing message includes a parameterized query statement, and the binding message includes parameter information corresponding to the query parameters in the parameterized query statement; the execution plan determination module 310 is specifically used to parse the parameterized query statement in the parsing message to obtain a parsing result; an initial execution plan is generated based on the parsing result through the execution plan pre-generation module; the parameter information in the binding message is extracted, and the parameter information is bound to the initial execution plan to generate a target execution plan.

[0085] Optionally, the parsing result includes at least the table name and the view name; after obtaining the parsing result, the execution plan determination module 310 is further specifically used to perform permission verification on the table name and the view name through the access control list to obtain a verification result; if the verification result is that the verification is passed, the execution continues to generate the initial execution plan; if the verification result is that the verification fails, the query processing process is terminated, and exception information is generated for exception reminder.

[0086] Optionally, the type determination module 320 is specifically used to perform information extraction processing on the execution message to obtain the parameter value of the Portal object. When the parameter value of the Portal object is empty, the type of the Portal object is determined to be an anonymous Portal. When the parameter value of the Portal object is non-empty, the type of the Portal object is determined to be a named Portal.

[0087] Optionally, the device is also used to construct a second message based on the target execution plan when the type of the Portal object is a named Portal; send the second message to the target data node so that the target data node performs a data query based on the target execution plan in the second message, and caches the query results to the target storage space; when there is no second data volume in the execution message, extract data that meets the default data volume from the target storage space and return it to the data receiving end; continue to receive new execution messages sent by the client until all query results in the target storage space are returned to the data receiving end.

[0088] Optionally, the device is further specifically configured to extract data that satisfies the second data amount from the target storage space and return the data to the data receiving end when there is a second data amount in the execution message.

[0089] Optionally, the device is further specifically configured to receive an execution message at least once when the type of the Portal object is a named Portal; and receive an execution message only once when the type of the Portal object is an anonymous Portal.

[0090] The database-based data query device provided in the embodiment of the present invention can execute the database-based data query method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0091] Example 4

[0092] Figure 4 1 is a schematic diagram of the structure of an electronic device provided in accordance with a fourth embodiment of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided for illustrative purposes only and are not intended to limit the implementation of the present inventions described and / or claimed herein.

[0093] like Figure 4As shown, electronic device 10 includes at least one processor 11 and memory, such as read-only memory (ROM) 12 and random access memory (RAM) 13, communicatively connected to at least one processor 11. The memory stores computer programs executable by the at least one processor. Processor 11 can perform various appropriate actions and processes based on the computer programs stored in ROM 12 or loaded from storage unit 18 into RAM 13. RAM 13 can also store various programs and data required for the operation of electronic device 10. Processor 11, ROM 12, and RAM 13 are interconnected via bus 14. An input / output (I / O) interface 15 is also connected to bus 14.

[0094] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0095] Processor 11 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any other suitable processor, controller, microcontroller, etc. Processor 11 executes the various methods and processes described above, such as a database-based data query method.

[0096] In some embodiments, the database-based data query method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the database-based data query method described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to execute the database-based data query method by any other appropriate means (e.g., by means of firmware).

[0097] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0098] The computer program for implementing the database-based data query method of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0099] Example 5

[0100] The fifth embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to cause a processor to execute a data query method based on a database, the method comprising:

[0101] Receive a data query message sent by a client, and generate a target execution plan based on the data query message; wherein the data query message includes an execution message;

[0102] Determine the type of the Portal object based on the execution message;

[0103] When the type of the Portal object is anonymous Portal, obtain a first data volume, and construct a first message based on the first data volume and the target execution plan; send the first message to the target data node, so that the target data node performs a data query based on the target execution plan in the first message and returns the query results, and stops the query immediately until the number of query results returned meets the first data volume.

[0104] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or apparatus. A computer-readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0105] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device that has: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0106] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0107] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0108] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0109] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A data query method based on a database, characterized in that: Applicable to the coordination node, including: Receive a data query message sent by a client, and generate a target execution plan based on the data query message; wherein the data query message includes an execution message; Determine the type of the Portal object based on the execution message; When the type of the Portal object is an anonymous Portal, a first data volume is obtained, and a first message is constructed based on the first data volume and the target execution plan; the first message is sent to a target data node, so that the target data node performs a data query based on the target execution plan in the first message and returns query results, and the query is stopped immediately until the number of returned query results meets the first data volume; The determining the type of the Portal object based on the execution message includes: Performing information extraction processing on the execution message to obtain a parameter value of the Portal object, and if the parameter value of the Portal object is empty, determining that the type of the Portal object is an anonymous Portal; if the parameter value of the Portal object is non-empty, determining that the type of the Portal object is a named Portal; The method further includes: when the type of the Portal object is a named Portal, constructing a second message based on the target execution plan; sending the second message to a target data node, so that the target data node performs a data query based on the target execution plan in the second message, and caches the query result in a target storage space; When the second data volume does not exist in the execution message, data that meets the default data volume is extracted from the target storage space and returned to the data receiving end; new execution messages sent by the client are continued to be received until all query results in the target storage space are returned to the data receiving end.

2. The method according to claim 1, characterized in that The data query message also includes a parsing message and a binding message; the parsing message includes a parameterized query statement, and the binding message includes parameter information corresponding to the query parameters in the parameterized query statement; Generating a target execution plan based on the data query message includes: Parsing the parameterized query statement in the parsed message to obtain a parsing result; generating an initial execution plan based on the parsing result by an execution plan pre-generation module; Parameter information in the binding message is extracted, and the parameter information is bound to the initial execution plan to generate a target execution plan.

3. The method according to claim 2, characterized in that The parsing result includes at least a table name and a view name; After obtaining the parsing results, it also includes: Performing permission verification on the table name and the view name through an access control list to obtain a verification result; If the verification result is that the verification is passed, the execution continues to generate the initial execution plan; if the verification result is that the verification fails, the query processing flow is terminated and exception information is generated for exception reminder.

4. The method according to claim 1, wherein The method also includes: In a case where the execution message contains a second data volume, data satisfying the second data volume is extracted from the target storage space and returned to the data receiving end.

5. The method according to claim 1, characterized in that The method also includes: When the type of the Portal object is a named Portal, the execution message is received at least once; when the type of the Portal object is an anonymous Portal, the execution message is received only once.

6. A data query device based on a database, characterized in that: include: An execution plan determination module, configured to receive a data query message sent by a client and generate a target execution plan based on the data query message; wherein the data query message includes an execution message; A type determination module, configured to determine the type of the Portal object based on the execution message; A query result determination module is configured to, when the type of the Portal object is an anonymous Portal, obtain a first data volume, construct a first message based on the first data volume and the target execution plan; send the first message to a target data node, so that the target data node performs a data query based on the target execution plan in the first message and returns query results, and immediately stop the query until the number of query results returned meets the first data volume; Wherein, the type determination module is specifically used to perform information extraction processing on the execution message to obtain a parameter value of the Portal object. If the parameter value of the Portal object is empty, the type of the Portal object is determined to be an anonymous Portal. If the parameter value of the Portal object is non-empty, the type of the Portal object is determined to be a named Portal. The device is also used to, when the type of the Portal object is a named Portal, construct a second message based on the target execution plan; send the second message to the target data node, so that the target data node performs a data query based on the target execution plan in the second message, and caches the query results to the target storage space; when there is no second data volume in the execution message, extract data that meets the default data volume from the target storage space and return it to the data receiving end; continue to receive new execution messages sent by the client until all query results in the target storage space are returned to the data receiving end.

7. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to execute the database-based data query method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the database-based data query method according to any one of claims 1 to 5 when executed.

Images