Database data query methods, devices, computer equipment, and storage media
By generating a primary key statistics table on each data node of the distributed database and utilizing GTID, the problem of low efficiency in traditional limit queries is solved, achieving efficient querying and data consistency.
Patent Information
- Application Number
- CN202311100010.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-29
- Publication Date
- 2025-11-14
- Estimated Expiration
- 2043-08-29
AI Technical Summary
In distributed databases, the efficiency of traditional limit queries drops significantly when dealing with tables containing millions of records, especially when updating massive amounts of data. Improving query efficiency has become an urgent problem to be solved.
By generating a primary key statistics table in each data node of the database, extracting the target primary key information and determining its existence, the amount of data extracted from the data nodes during queries is reduced. The primary key statistics table and the Global Transaction Identifier (GTID) are used to ensure query consistency.
It improves the query efficiency of distributed databases, reduces the amount of data extracted from data nodes, and ensures the consistency of query results, especially when dealing with add, delete, and modify operations.
Smart Images

Figure CN117112638B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of database technology, and in particular to a database data query method, apparatus, computer equipment and storage medium. Background Technology
[0002] Data querying refers to retrieving required data from a database. It is the most frequent and important operation in database operations, and the speed of query execution has become a major concern for database users and designers. Optimizing queries is essential to improving database system performance. Database query optimization has always been a complex issue, especially when dealing with large volumes of data, where improving efficiency and performance becomes a top priority.
[0003] When querying data, you might retrieve many records, but the user only needs a small portion. This necessitates limiting the number of results, hence the use of a `LIMIT` query. When a table has hundreds or thousands of records, efficiency isn't a primary concern with `LIMIT` queries, and the standard `LIMIT` syntax is sufficient. However, when the number of records reaches tens of thousands, especially millions, a standard `LIMIT` query will iterate through the data from the beginning. In a distributed database, this means retrieving data from all database nodes. Therefore, the efficiency of `LIMIT` queries decreases significantly with increasing offset.
[0004] Compared to the centralized storage of traditional databases, distributed databases face the challenge of optimizing queries and improving query efficiency when dealing with massive amounts of data that need updating. Summary of the Invention
[0005] In view of this, in order to solve the above-mentioned technical problems or some of the technical problems, the embodiments of the present invention provide a
[0006] Data query methods, devices, computer equipment, and storage media for various databases.
[0007] In a first aspect, embodiments of the present invention provide a database data query method, including:
[0008] When a query request for target data is received, the target primary key information corresponding to the target data is extracted from the query request;
[0009] Determine whether the primary key statistics table of the data nodes in the database contains the target primary key information, wherein the primary key statistics table includes the primary key information corresponding to each data node;
[0010] When the judgment result is yes, the target data corresponding to the target primary key information is extracted from the data node.
[0011] In one possible implementation, the primary key statistics table is generated in the following manner:
[0012] Generate a corresponding primary key for each piece of data in each data node of the database;
[0013] For each data node, the primary key within the current preset data range is counted at each preset data range interval to obtain the primary key information corresponding to each preset data range;
[0014] Generate a primary key statistics table for each data node by generating all the primary key information corresponding to each data node.
[0015] In one possible implementation, determining whether the primary key statistics table of the data node in the database contains the target primary key information includes:
[0016] Obtain the target data range corresponding to the target primary key information;
[0017] Determine if a primary key exists within the target data range of the primary key statistics table for the current data node;
[0018] When the judgment result is yes, it is determined that the primary key information of the target primary key is contained in the primary key statistics table of the current data node.
[0019] In one possible implementation, when multiple data nodes of the database contain the target primary key information, the step of extracting the target data corresponding to the target primary key information from the data nodes includes:
[0020] Determine the range of data containing the target primary key information in each of the data nodes;
[0021] For each of the data nodes, extract the target sub-data within the data range in sequence;
[0022] The target sub-data of all the data nodes is taken as the target data.
[0023] In one possible implementation, the primary key statistics table also includes the number of primary keys for each data node, and the step of using the target sub-data of all the data nodes as the target data includes:
[0024] Obtain the number of first primary keys for each of the data nodes;
[0025] The total number of primary keys in the database is determined based on the number of the first primary keys.
[0026] Obtain the number of second primary keys corresponding to the target primary key information;
[0027] When the total number of primary keys is greater than or equal to the number of the second primary keys, the target sub-data of all the data nodes is taken as the target data.
[0028] In one possible implementation, the method further includes:
[0029] When the total number of primary keys is less than the number of primary keys, the steps of generating a corresponding primary key for each data in each data node and generating primary key information corresponding to each preset data range of the data node are repeated to obtain new primary key information;
[0030] The primary key statistics table is updated based on the new primary key information.
[0031] In one possible implementation, the method further includes:
[0032] When the database receives a data processing operation, it updates the global transaction identifier corresponding to the primary key statistics table according to the data processing operation.
[0033] Determine the number of new primary keys corresponding to each data node after the data processing operation;
[0034] The primary key count of the data node is updated based on the new primary key count.
[0035] In a second aspect, embodiments of the present invention provide a database data query device, comprising:
[0036] The extraction module is used to extract the target primary key information corresponding to the target data from the query request when a query request for target data is received.
[0037] The judgment module is used to determine whether the primary key statistics table of the data nodes in the database contains the target primary key information, wherein the primary key statistics table includes the primary key information corresponding to each data node.
[0038] The extraction module is further configured to extract the data from the data node when the judgment result is yes.
[0039] The target data corresponding to the target primary key information.
[0040] Thirdly, embodiments of the present invention provide a computer device, including: a processor and a memory, wherein the processor is configured to execute a data query program of a database stored in the memory, so as to implement the data query method of the database described in any one of the first aspects above.
[0041] Fourthly, embodiments of the present invention provide a storage medium storing one or more programs, which can be executed by one or more processors to implement the database data query method described in any one of the first aspects.
[0042] The database data query scheme provided in this embodiment of the invention extracts the target primary key information corresponding to the data from the query request when a data query request is received; determines whether the primary key statistics table of the database data node contains the target primary key information, wherein the primary key statistics table includes the primary key information corresponding to each data node; and extracts the target data corresponding to the primary key information from the data node when the determination result is yes. Therefore, it is possible to add a primary key statistics table to the database node, and check the primary key statistics table first during querying, which can reduce the amount of data extracted from the database node and improve query efficiency. Attached Figure Description
[0043] Figure 1 A schematic flowchart of a database data query method provided in an embodiment of the present invention;
[0044] Figure 2 A flowchart illustrating another database data query method provided in an embodiment of the present invention;
[0045] Figure 3 A schematic diagram illustrating the principle of a database data query method provided in an embodiment of the present invention;
[0046] Figure 4 This is a schematic diagram of the structure of a database data query device provided in an embodiment of the present invention;
[0047] Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation
[0048] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0049] To facilitate understanding of the embodiments of the present invention, further explanations and descriptions will be provided below with reference to the accompanying drawings and specific embodiments. These embodiments do not constitute a limitation on the embodiments of the present invention.
[0050] Figure 1This is a flowchart illustrating a database data query method provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the method specifically includes:
[0051] S11. When a query request for target data is received, extract the target primary key information corresponding to the target data from the query request.
[0052] The data query method for a database provided in this invention is applied to a computer device, which may include, but is not limited to, a desktop computer, a server, a tablet computer, etc. Specifically, when performing a data query in the database, the method determines whether the primary key information corresponding to the data to be queried exists in the primary key statistics table of the data node. If it exists, the queried data is extracted from the data node. The database may be a distributed database (e.g., GoldenDB).
[0053] In this embodiment, when a limit query operation is required on the database, a query request sent by the client is received. The query request can be a limit query request used to query target data in the database. The query request is sent to each data node in the database, and the primary key corresponding to the target data is extracted from the query request as the target primary key information.
[0054] S12. Determine whether the primary key statistics table of the data nodes in the database contains the target primary key information. The primary key statistics table includes the primary key information corresponding to each data node.
[0055] In this embodiment, a primary key statistics table (id statistics table) corresponding to each data node in the database is pre-constructed. The primary key statistics table contains all primary key information corresponding to the current data node, and each primary key has corresponding data. The primary key statistics table can contain multiple primary key information entries, each corresponding to a range of data primary keys. For example, the primary key statistics table is named id, containing primary keys...
[0056] The information is: id1, id2... where id1 is a primary key with a range of 1000-2000, and id2 is a primary key with a range of 3000-4000. The number of primary keys in each range may be different. id1 may actually contain only 1000-1500 primary keys, and id2 may actually contain only 3200-3800 primary keys.
[0057] Specifically, the target primary key information contains the target range of the primary key corresponding to the data to be queried. When the query request is sent to the data node, the corresponding statistical range in the primary key statistics table of each data node is queried based on the target range to determine whether the target primary key information exists within the statistical range corresponding to the target range. If it exists, it is determined that the target primary key information is contained in the primary key statistics table of the current data node; otherwise, it is determined that the target primary key information is not contained in the primary key statistics table of the current data node.
[0058] S13. When the judgment result is yes, extract the target data corresponding to the target primary key information from the data node.
[0059] In this embodiment, when the judgment result is yes, the target data is located in the current data node according to the target primary key information. Each data node returns the located target data. The query is performed on each data node in turn and the target data is returned to obtain all the target data corresponding to the query request.
[0060] The database data query method provided in this embodiment of the invention extracts the target primary key information corresponding to the data from the query request when a data query request is received; determines whether the primary key statistics table of the data node in the database contains the target primary key information, wherein the primary key statistics table includes the primary key information corresponding to each data node; and extracts the target data corresponding to the primary key information from the data node when the determination result is yes. Therefore, it is possible to add a primary key statistics table to the database node, and during querying, first check the primary key statistics table; if the target primary key data exists in the primary key statistics table, then query the target data, which can reduce the amount of data extracted from the database node and improve query efficiency.
[0061] Figure 2 This is a flowchart illustrating another database data query method provided in an embodiment of the present invention, as shown below. Figure 2 As shown, the method specifically includes:
[0062] S21. Generate a corresponding primary key for each data item in each data node of the database;
[0063] For each data node, the primary key within the current preset data range is counted at each preset data range interval to obtain the primary key information corresponding to each preset data range; the primary key information corresponding to each data node is used to generate a primary key statistics table for the data node.
[0064] In this embodiment, a primary key statistics table is generated in advance for each data node in the database. The primary key statistics table contains the primary key corresponding to each data in the data node. The primary key within the statistical range of each data segment in the data node is a primary key information, and multiple primary key information forms the primary key statistics table.
[0065] Specifically, for each data node in the database, all primary keys corresponding to the data within each preset data range are counted to obtain the primary key information within each preset data range. The primary key information of all preset data ranges is used to generate the primary key statistics table for the current data node, with one primary key statistics table for each data node.
[0066] S22. When a query request for target data is received, extract the target primary key information corresponding to the target data from the query request.
[0067] In this embodiment, the process is similar to step S11; please refer to the following for details. Figure 1 The relevant descriptions are presented concisely and will not be elaborated upon here.
[0068] S23. Obtain the target data range corresponding to the target primary key information; determine whether there is a primary key in the target data range of the primary key statistics table of the current data node; when the determination result is yes, determine that the primary key statistics table of the current data node contains the target primary key information.
[0069] In this embodiment, the target primary key information includes the target data range corresponding to the target data to be queried. When performing a query operation in the database, the query request needs to be sent to each data node to check whether the primary key statistics of each data node contain the target primary key information.
[0070] Specifically, for the primary key statistics table of the current data node, obtain all primary key information that includes the target data range, determine whether a primary key exists within the primary key information, and when any primary key information contains a primary key, determine that a primary key exists within the target data range of the primary key statistics table of the current data node, and determine that the primary key statistics table of the current data node contains the target primary key information.
[0071] S24. When multiple data nodes in the database contain the target primary key information, determine the data range containing the target primary key information in each data node; sequentially, for each...
[0072] Each of the data nodes extracts target sub-data within the data range; the target sub-data of all the data nodes is used as the target data.
[0073] In this embodiment, when multiple data nodes contain the target primary key information, the range of data containing the target primary key information in each data node is determined. From this range, the data corresponding to the target primary key information is extracted as the target sub-data. The target sub-data in each data node is then used to generate the target data.
[0074] In one possible implementation, the primary key statistics table also includes the number of primary keys contained in each data node. Specifically, the number of primary keys contained in each primary key information is counted to obtain the number of primary keys in each preset data range.
[0075] The step of taking all data nodes' target sub-data as target data includes:
[0076] The primary key count is obtained from the primary key statistics table for each data node within the target data range. This count serves as the first primary key count, representing the total number of primary keys within the target data range corresponding to the target primary key information in the current data node. The total number of primary keys in the database is determined based on this first primary key count within each data node. This total number of primary keys represents the total number of primary keys within the target data range corresponding to the target primary key information in each data node.
[0077] Furthermore, the number of secondary primary keys corresponding to the target primary key information is obtained. The number of secondary primary keys represents the total number of primary keys corresponding to the target data to be queried. When the total number of primary keys is greater than or equal to the number of secondary primary keys, it indicates that the target data currently queried is correct, and the target sub-data of all data nodes can be used as the target data.
[0078] In one possible implementation, if the total number of primary keys is less than the number of secondary primary keys, it indicates that the target data being queried is incorrect. The error may be due to an error in the generated primary key statistics table. In this case, the steps of generating a corresponding primary key for each data in each data node, and generating primary key information for each preset data range of each data node, are repeated to obtain new primary key information. The primary key statistics table is then updated based on the new primary key information. The steps of generating the target data are then repeated for the updated primary key statistics table.
[0079] In one possible implementation, during the data query process, the database can receive...
[0080] Data processing operations can include, but are not limited to, client-side operations such as adding, deleting, and modifying data in the database. When a data processing operation is received, to ensure that the data in the database nodes is consistent with the primary key statistics table, the global transaction identifier GTID and MVCC corresponding to the primary key statistics table need to be updated according to the data processing operation; the number of new primary keys for each data node after the data processing operation needs to be determined; and the number of primary keys for the data nodes needs to be updated according to the number of new primary keys.
[0081] As an example, such as Figure 3 The diagram shown is a schematic representation of a database data query method provided in an embodiment of the present invention. Figure 3As shown, the method specifically includes: The database has n database nodes, named DB-1, DB-2...DB-n. Each data node corresponds to an id statistics table, containing primary key information (id1, id2...) for all data within its node. Each primary key (id1, id2...) contains primary keys within a specified range, as well as a count of the number of primary keys in each primary key, and also includes the GTID from the distributed database GoldenDB to ensure data consistency during queries. For example, the primary key information might include all primary keys within a specified range for that data node; for instance, id1 might contain primary keys with a count range of 1000-2000, and id2 might contain primary keys with a count range of 3000-4000.
[0082] Step 1: In the id statistics table, create primary key information for each data node within a preset data range, such as id1, id2... and count, the sum of the primary keys in each primary key information.
[0083] Step 2: The client sends a limit query request, providing the data node with the target primary key information corresponding to the limit query request;
[0084] Step 3: Distribute the target primary key information of the limit query request to each data node. Each data node calculates the number of primary keys within the target data range corresponding to the target primary key information in the id statistics table, obtaining the total number of primary keys corresponding to the target primary key information in the database, and determining the number of primary keys for the target primary key information. When the number of primary keys for the target primary key information does not exceed the total number of primary keys, the data corresponding to the target primary key information is extracted from each data node to obtain the target data corresponding to the query request. This avoids returning redundant data, saving a significant amount of resources. For example, each data node obtains the target primary key information of the limit query request, determines whether it has the target primary key information of the limit query request based on the primary key information in the id statistics table, and then calculates the primary key count in the id statistics table to ensure it does not exceed the primary key count of the target primary key information of the limit query request.
[0085] The target primary key information has a data range of 8000-10000. The preset data ranges for each data node are: id1: 8000-9000, id2: 9000-10000, id3: 10000-11000. Therefore, it is necessary to count the number of primary keys for id1 and id2 in each data node to obtain the total number of primary keys. If the number of primary keys in the target primary key information is less than the total number of primary keys, then the step of extracting the data corresponding to the target primary key information will be executed.
[0086] Step 4: Each data node returns the data corresponding to the primary key information requested by the limit query in the primary key information located.
[0087] When a client performs a limit query, other clients may perform CRUD operations on the database. This necessitates ensuring consistency between the database node data and the `id` statistics table, as well as the consistency of the retrieved data. Therefore, a trigger is set on the database node. When an CRUD operation occurs on the database node, the trigger modifies the primary key count in the corresponding primary key information of the `id` statistics table to ensure consistency between the database node data and the `id` statistics table. The trigger also modifies the GTID in the `id` statistics table to guarantee the consistency of the retrieved data. During client limit queries, data consistency can also be maintained through GoldenDB's distributed MVCC. This reduces the amount of data retrieved from the database node.
[0088] The database data query method provided in this invention generates a corresponding primary key for each data in each data node of the database; for each data node, the primary key within the current preset data range is statistically analyzed at preset data intervals to obtain primary key information corresponding to each preset data range; all primary key information corresponding to each data node is used to generate a primary key statistics table for the data node; when a query request for target data is received, the target primary key information corresponding to the target data is extracted from the query request; the target data range corresponding to the target primary key information is obtained; it is determined whether a primary key exists within the target data range of the primary key statistics table of the current data node; if the determination result is yes, it is determined that the primary key statistics table of the current data node contains the target primary key information; when multiple data nodes in the database contain the target primary key information, the data range containing the target primary key information in each data node is determined; target sub-data within the data range is extracted sequentially for each data node; and the target sub-data of all data nodes is used as the target data. Therefore, it is possible to add a primary key statistics table to the database data nodes, and check the primary key statistics table first during a limit query, which can reduce the amount of data extracted from the database nodes and improve the efficiency of limit queries. Adding GTID to the primary key statistics table and using MVCC to ensure data consistency during queries solves the problem of inconsistent query results when performing CRUD operations on the database.
[0089] Figure 4 This is a schematic diagram of the structure of a database data query device provided in an embodiment of the present invention, as shown below. Figure 4 As shown, the structure specifically includes:
[0090] Extraction module 41 is used to extract the target primary key information corresponding to the target data from the query request when a query request for target data is received;
[0091] The judgment module 42 is used to determine whether the primary key statistics table of the data nodes of the database contains the target primary key information, wherein the primary key statistics table includes the primary key information corresponding to each data node;
[0092] The extraction module 41 is further configured to extract the target data corresponding to the target primary key information from the data node when the judgment result is yes.
[0093] In one possible implementation, the generation module 43 is used to generate a corresponding primary key for each piece of data in each data node of the database;
[0094] The statistics module 44 is used to count the primary key within the current preset data range for each data node at preset data intervals, so as to obtain the primary key information corresponding to each preset data range.
[0095] The generation module is also used to generate a primary key statistics table for each data node from all the primary key information corresponding to each data node.
[0096] In one possible implementation, the acquisition module 45 is used to acquire the target data range corresponding to the target primary key information;
[0097] The judgment module is specifically used to determine whether a primary key exists within the target data range of the primary key statistics table of the current data node;
[0098] The determination module 46 is used to determine, when the determination result is yes, that the primary key statistics table of the current data node contains the target primary key information.
[0099] In one possible implementation, the determining module is further configured to determine the data range in each of the data nodes that contains the target primary key information;
[0100] The extraction module is also used to extract target sub-data within the data range for each data node in sequence;
[0101] The target sub-data of all the data nodes is taken as the target data.
[0102] In one possible implementation, the acquisition module is further configured to acquire the number of first primary keys for each of the data nodes;
[0103] The determining module is further configured to determine the total number of primary keys in the database based on the number of the first primary keys;
[0104] The acquisition module is further configured to acquire the number of second primary keys corresponding to the target primary key information;
[0105] When the total number of primary keys is greater than or equal to the number of the second primary keys, the target sub-data of all the data nodes is taken as the target data.
[0106] In one possible implementation, the generation module is further configured to repeatedly execute the steps of generating a corresponding primary key for each data in each data node and generating primary key information corresponding to each preset data range of the data node when the total number of primary keys is less than the number of the second primary keys, so as to obtain new primary key information;
[0107] The update module 47 is used to update the primary key statistics table based on the new primary key information.
[0108] In one possible implementation, the update module is further configured to update the global transaction identifier corresponding to the primary key statistics table according to the data processing operation when the database receives a data processing operation;
[0109] Determine the number of new primary keys corresponding to each data node after the data processing operation;
[0110] The primary key count of the data node is updated based on the new primary key count.
[0111] The database query device provided in this embodiment of the invention can be as follows: Figure 4 The database query device shown can perform, for example... Figure 1-3 All steps of the data query method in the Chinese database, thereby achieving Figure 1-3 For details on the technical effectiveness of the data query method shown in the database, please refer to [link / reference]. Figure 1-3 The relevant descriptions are presented concisely and will not be elaborated upon here.
[0112] Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Figure 5 The computer device 500 shown includes at least one processor 501, a memory 502, at least one network interface 504, and other user interfaces 503. The various components in the computer device 500 are coupled together via a bus system 505. It is understood that the bus system 505 is used to implement communication between these components. In addition to a data bus, the bus system 505 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 5 The general designated all buses as Bus System 505.
[0113] The user interface 503 may include a display, keyboard, or clicking device (e.g., mouse, trackball, touchpad, or touchscreen).
[0114] It is understood that the memory 502 in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory 502 described herein is intended to include, but is not limited to, these and any other suitable types of memory.
[0115] In some implementations, memory 502 stores elements, executable units or data structures, or subsets thereof, or extended sets thereof: operating system 5021 and application program 5022.
[0116] The operating system 5021 includes various system programs, such as the framework layer, core library layer, and driver layer, used to implement various basic business functions and handle hardware-based tasks. The application program 5022 includes various applications, such as a media player and a browser, used to implement various application functions. The program implementing the method of this embodiment can be included in the application program 5022.
[0117] In this embodiment of the invention, by calling the program or instructions stored in memory 502, specifically the program or instructions stored in application program 5022, processor 501 executes the method steps provided in each method embodiment, including, for example:
[0118] When a query request for target data is received, the target primary key information corresponding to the target data is extracted from the query request;
[0119] Determine whether the primary key statistics table of the data nodes in the database contains the target primary key information, wherein the primary key statistics table includes the primary key information corresponding to each data node;
[0120] When the judgment result is yes, the target data corresponding to the target primary key information is extracted from the data node.
[0121] In one possible implementation, a corresponding primary key is generated for each piece of data in each data node of the database;
[0122] For each data node, the primary key within the current preset data range is counted at each preset data range interval to obtain the primary key information corresponding to each preset data range;
[0123] Generate a primary key statistics table for each data node by generating all the primary key information corresponding to each data node.
[0124] In one possible implementation, the target data range corresponding to the target primary key information is obtained;
[0125] Determine if a primary key exists within the target data range of the primary key statistics table for the current data node;
[0126] When the judgment result is yes, it is determined that the primary key information of the target primary key is contained in the primary key statistics table of the current data node.
[0127] In one possible implementation, the data range containing the target primary key information in each of the data nodes is determined;
[0128] For each of the data nodes, extract the target sub-data within the data range in sequence;
[0129] The target sub-data of all the data nodes is taken as the target data.
[0130] In one possible implementation, the number of first primary keys for each of the data nodes is obtained;
[0131] The total number of primary keys in the database is determined based on the number of the first primary keys.
[0132] Obtain the number of second primary keys corresponding to the target primary key information;
[0133] When the total number of primary keys is greater than or equal to the number of the second primary keys, the target sub-data of all the data nodes is taken as the target data.
[0134] In one possible implementation, when the total number of primary keys is less than the number of the second primary keys, the steps of generating a corresponding primary key for each data in each data node and generating primary key information corresponding to each preset data range of the data node are repeated to obtain new primary key information.
[0135] The primary key statistics table is updated based on the new primary key information.
[0136] In one possible implementation, when the database receives a data processing operation, it updates the global transaction identifier corresponding to the primary key statistics table according to the data processing operation.
[0137] Determine the number of new primary keys corresponding to each data node after the data processing operation;
[0138] The primary key count of the data node is updated based on the new primary key count.
[0139] The methods disclosed in the above embodiments of the present invention can be applied to or implemented by processor 501. Processor 501 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 501 or by instructions in the form of software. The processor 501 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present invention can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software units in the decoding processor. The software units may be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 502. Processor 501 reads the information in memory 502 and, in conjunction with its hardware, completes the steps of the above method.
[0140] It is understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described herein, or combinations thereof.
[0141] For software implementation, the techniques described herein can be implemented by units that perform the functions described herein. The software code can be stored in memory and executed by a processor. The memory can be implemented in the processor or external to the processor.
[0142] The computer device provided in this embodiment may be as follows: Figure 5 The computer device shown can perform, for example Figure 1-3 All steps of the data query method in the Chinese database, thereby achieving Figure 1-3 For details on the technical effectiveness of the data query method shown in the database, please refer to [link / reference]. Figure 1-3 The relevant descriptions are presented concisely and will not be elaborated upon here.
[0143] This invention also provides a storage medium (computer-readable storage medium). This storage medium stores one or more programs. The storage medium may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as read-only memory, flash memory, hard disk, or solid-state drive; the memory may also include combinations of the above types of memory.
[0144] When one or more programs in the storage medium can be executed by one or more processors to implement the data query method of the database executed on the device side as described above.
[0145] The processor is used to execute a data query program for a database stored in memory, to implement the following steps of a database data query method executed on the device side:
[0146] When a query request for target data is received, the target primary key information corresponding to the target data is extracted from the query request;
[0147] Determine whether the primary key statistics table of the data nodes in the database contains the target primary key information, wherein the primary key statistics table includes the primary key information corresponding to each data node;
[0148] When the judgment result is yes, the target data corresponding to the target primary key information is extracted from the data node.
[0149] In one possible implementation, a corresponding primary key is generated for each piece of data in each data node of the database;
[0150] For each data node, the primary key within the current preset data range is counted at each preset data range interval to obtain the primary key information corresponding to each preset data range;
[0151] Generate a primary key statistics table for each data node by generating all the primary key information corresponding to each data node.
[0152] In one possible implementation, the target data range corresponding to the target primary key information is obtained;
[0153] Determine if a primary key exists within the target data range of the primary key statistics table for the current data node;
[0154] When the judgment result is yes, it is determined that the primary key information of the target primary key is contained in the primary key statistics table of the current data node.
[0155] In one possible implementation, the data range containing the target primary key information in each of the data nodes is determined;
[0156] For each of the data nodes, extract the target sub-data within the data range in sequence;
[0157] The target sub-data of all the data nodes is taken as the target data.
[0158] In one possible implementation, the number of first primary keys for each of the data nodes is obtained;
[0159] The total number of primary keys in the database is determined based on the number of the first primary keys.
[0160] Obtain the number of second primary keys corresponding to the target primary key information;
[0161] When the total number of primary keys is greater than or equal to the number of the second primary keys, the target sub-data of all the data nodes is taken as the target data.
[0162] In one possible implementation, when the total number of primary keys is less than the number of the second primary keys, the steps of generating a corresponding primary key for each data in each data node and generating primary key information corresponding to each preset data range of the data node are repeated to obtain new primary key information.
[0163] The primary key statistics table is updated based on the new primary key information.
[0164] In one possible implementation, when the database receives a data processing operation, it updates the global transaction identifier corresponding to the primary key statistics table according to the data processing operation.
[0165] Determine the number of new primary keys corresponding to each data node after the data processing operation;
[0166] The primary key count of the data node is updated based on the new primary key count.
[0167] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.
[0168] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.
[0169] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A data query method for a database, characterized in that, include: When a query request for target data is received, the target primary key information corresponding to the target data is extracted from the query request; Determine whether the primary key statistics table of the data nodes in the database contains the target primary key information, wherein the primary key statistics table includes the primary key information corresponding to each data node; When the judgment result is yes, extract the target data corresponding to the target primary key information from the data node; When multiple data nodes in the database contain the target primary key information, the step of extracting the target data corresponding to the target primary key information from the data nodes includes: determining the data range containing the target primary key information in each data node; sequentially extracting target sub-data within the data range for each data node; and using the target sub-data of all data nodes as the target data. The primary key statistics table also includes the number of primary keys for each data node. Taking the target sub-data of all data nodes as the target data includes: obtaining the first primary key count for each data node; determining the total number of primary keys in the database based on the first primary key count; obtaining the second primary key count corresponding to the target primary key information; and taking the target sub-data of all data nodes as the target data when the total number of primary keys is greater than or equal to the second primary key count. When the total number of primary keys is less than the number of the second primary keys, the steps of generating a corresponding primary key for each data in each data node and generating primary key information corresponding to each preset data range of the data node are repeated to obtain new primary key information; the primary key statistics table is updated according to the new primary key information.
2. The method according to claim 1, characterized in that, The primary key statistics table is generated in the following way: Generate a corresponding primary key for each piece of data in each data node of the database; For each data node, the primary key within the current preset data range is counted at each preset data range interval to obtain the primary key information corresponding to each preset data range; Generate a primary key statistics table for each data node by generating all the primary key information corresponding to each data node.
3. The method according to claim 1, characterized in that, The step of determining whether the primary key statistics table of the data node in the database contains the target primary key information includes: Obtain the target data range corresponding to the target primary key information; Determine if a primary key exists within the target data range of the primary key statistics table for the current data node; When the judgment result is yes, it is determined that the primary key information of the target primary key is contained in the primary key statistics table of the current data node.
4. The method according to claim 1, characterized in that, The method further includes: When the database receives a data processing operation, it updates the global transaction identifier corresponding to the primary key statistics table according to the data processing operation. Determine the number of new primary keys corresponding to each data node after the data processing operation; The primary key count of the data node is updated based on the new primary key count.
5. A data query device for a database, characterized in that, include: The extraction module is used to extract the target primary key information corresponding to the target data from the query request when a query request for target data is received. The judgment module is used to determine whether the primary key statistics table of the data nodes in the database contains the target primary key information, wherein the primary key statistics table includes the primary key information corresponding to each data node. The extraction module is further configured to extract the target data corresponding to the target primary key information from the data node when the judgment result is yes; The determination module is used to extract target data corresponding to the target primary key information from the data nodes when multiple data nodes of the database contain the target primary key information. Specifically, it is used to: determine the data range containing the target primary key information in each data node. The extraction module is further configured to sequentially extract target sub-data within the data range for each data node; and use the target sub-data of all data nodes as the target data; The acquisition module, which includes the primary key count of each data node in the primary key statistics table, specifically uses the step of taking the target sub-data of all data nodes as the target data to: acquire the first primary key count of each data node; The determining module is further configured to determine the total number of primary keys in the database based on the number of the first primary keys; The acquisition module is further configured to acquire the number of second primary keys corresponding to the target primary key information; When the total number of primary keys is greater than or equal to the number of the second primary keys, the target sub-data of all the data nodes is taken as the target data; The generation module is used to repeatedly execute the steps of generating a corresponding primary key for each data in each data node and generating primary key information corresponding to each preset data range of the data node when the total number of primary keys is less than the number of the second primary keys, so as to obtain new primary key information; The update module is used to update the primary key statistics table based on the new primary key information.
6. A computer device, characterized in that, include: A processor and a memory, the processor being configured to execute a data query program for a database stored in the memory, to implement the data query method for the database as described in any one of claims 1 to 4.
7. A storage medium, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the data query method of the database according to any one of claims 1 to 4.
Citation Information
Patent Citations
HBASE secondary indexing method and device based on Lucene and computer equipment
CN111125108A
Database table query method and device
CN113326285A