Parallel query method, device and related equipment of relational database

Abstract

The application provides a parallel query method and device of a relational database and related equipment, and relates to the technical field of big data. The parallel query method of the relational database is applied to a consumer stream thread. The method comprises the following steps: acquiring first address information from a first shared cyclic queue, wherein the first address information is address information of a first data block of the relational database in a memory block pool, a plurality of data blocks with different byte numbers are stored in the memory block pool, and the first data block is any one of the plurality of data blocks; acquiring the first data block from the memory block pool according to the first address information, wherein the first data block stores at least part of tuple data; and in the case that complete tuple data is acquired, releasing the first data block associated with the tuple data from the memory block pool. In this way, the cost of querying the tuple data in the first data block is reduced.

Description

Technical Field

[0001] This application relates to the field of big data technology, and in particular to a parallel query method, apparatus and related equipment for relational databases. Background Technology

[0002] With the continuous development of big data technology, its application in people's lives is becoming increasingly widespread. In today's data-driven world, with the continuous growth of business needs and the rapid increase in data volume, traditional relational database query methods often struggle to meet the requirements of efficient processing and rapid response to large amounts of data. Furthermore, the actual query process for tuple data in relational databases requires two memory allocations, two memory releases, and two memory copies, demonstrating the significant overhead of current relational database tuple queries. Summary of the Invention

[0003] This application provides a parallel query method, apparatus, and related equipment for relational databases to solve the problem of high overhead in querying tuple data in relational databases.

[0004] To solve the above problems, this application is implemented as follows:

[0005] In a first aspect, embodiments of this application provide a parallel query method for a relational database, applied to a consumer stream thread, wherein the consumer stream thread and the producer stream thread are used to perform parallel queries on the relational database, and the method includes:

[0006] Obtain first address information from the first shared circular queue. The first address information is the address information of the first data block of the relational database in the memory block pool. The memory block pool stores multiple data blocks with different byte counts. The first data block is any one of the multiple data blocks.

[0007] The first data block is obtained from the memory block pool according to the first address information, and the first data block stores at least a portion of the tuple data;

[0008] If the complete tuple data is obtained, the first data block associated with the tuple data is released to the memory block pool.

[0009] Secondly, embodiments of this application provide a parallel query method for a relational database, applied to a producer stream thread, wherein the producer stream thread and the consumer stream thread are used to perform parallel queries on the relational database, and the method includes:

[0010] A first data block from a relational database is obtained from a memory block pool, which stores multiple data blocks of different byte sizes. The first data block is any one of the multiple data blocks, and the first data block stores at least a portion of the tuple data.

[0011] At least a portion of the tuple data is stored in the first data block, and the first data block containing at least a portion of the tuple data is released to the memory block pool.

[0012] Send first address information to the first shared circular queue. The first address information is the address information of the first data block in the memory block pool.

[0013] Thirdly, embodiments of this application also provide a parallel query method for relational databases, including:

[0014] Get the first parameter of the consumer stream thread and the second parameter of the producer stream thread;

[0015] The target data block of the relational database is determined based on the first parameter and the second parameter;

[0016] The target data block is split or merged into multiple data blocks with different byte counts, and the multiple data blocks include the first data block;

[0017] The multiple data blocks are stored in a memory block pool, and the address information of the multiple data blocks in the memory block pool is stored in a first shared circular queue;

[0018] The consumer stream thread uses the parallel query method for relational databases described in the first aspect to perform parallel queries on the relational database, while the producer stream thread uses the parallel query method for relational databases described in the second aspect to perform parallel queries on the relational database.

[0019] Fourthly, embodiments of this application also provide a parallel query device for a relational database, applied to a consumer stream thread, wherein the consumer stream thread and the producer stream thread are used to perform parallel queries on the relational database. The parallel query device for the relational database includes:

[0020] The first acquisition module is used to acquire first address information from the first shared circular queue. The first address information is the address information of the first data block of the relational database in the memory block pool. The memory block pool stores multiple data blocks with different byte counts. The first data block is any one of the multiple data blocks.

[0021] The second acquisition module is used to acquire the first data block from the memory block pool according to the first address information, wherein the first data block stores at least a portion of the tuple data;

[0022] The first release module is used to release the first data block associated with the tuple data to the memory block pool when the complete tuple data is obtained.

[0023] Fifthly, embodiments of this application also provide a parallel query apparatus for a relational database, applied to a producer stream thread, wherein the producer stream thread and the consumer stream thread are used to perform parallel queries on the relational database, and the parallel query apparatus for the relational database includes:

[0024] The third acquisition module is used to acquire a first data block of the relational database from the memory block pool. The memory block pool stores multiple data blocks with different byte counts. The first data block is any one of the multiple data blocks. The first data block stores at least a portion of the tuple data.

[0025] The second release module is used to store at least a portion of the tuple data into the first data block and release the first data block containing at least a portion of the tuple data to the memory block pool.

[0026] The sending module is used to send first address information to the first shared circular queue, wherein the first address information is the address information of the first data block in the memory block pool.

[0027] Sixthly, embodiments of this application also provide a parallel query apparatus for a relational database, comprising:

[0028] The fourth acquisition module is used to acquire the first parameter of the consumer stream thread and the second parameter of the producer stream thread;

[0029] The first determining module is used to determine the target data block of the relational database based on the first parameter and the second parameter;

[0030] A splitting or merging module is used to split or merge the target data block into multiple data blocks with different byte counts, wherein the multiple data blocks include a first data block;

[0031] The storage module is used to store multiple data blocks into a memory block pool and to store the address information of the multiple data blocks in the memory block pool into a first shared circular queue;

[0032] The consumer stream thread uses the parallel query method for relational databases described in the first aspect to perform parallel queries on the relational database, while the producer stream thread uses the parallel query method for relational databases described in the second aspect to perform parallel queries on the relational database.

[0033] In a seventh aspect, embodiments of this application also provide an electronic device, including: a memory, a processor, and a program stored in the memory and executable on the processor; the processor is configured to read the program in the memory to implement the steps of the method described in the first aspect above, or to read the program in the memory to implement the steps of the method described in the second aspect above, or to read the program in the memory to implement the steps of the method described in the third aspect above.

[0034] Eighthly, embodiments of this application also provide a readable storage medium for storing a program, which, when executed by a processor, implements the steps of the method described in the first aspect above, or implements the steps of the method described in the second aspect above, or implements the steps of the method described in the third aspect above.

[0035] In a ninth aspect, embodiments of this application also provide a computer program product, including computer instructions, which, when executed by a processor, implement the steps in the methods described in the first, second, or third aspects above.

[0036] In this embodiment, the first address information is obtained from the first shared circular queue. This first address information is the address information of the first data block of the relational database in the memory block pool. The memory block pool stores multiple data blocks with different byte counts, and the first data block is any one of these multiple data blocks. The first data block is obtained from the memory block pool according to the first address information. The first data block stores at least a portion of the tuple data. When the complete tuple data is obtained, the first data block associated with the tuple data is released to the memory block pool. In this way, the step of obtaining the first data block only needs to be performed once, and the step of releasing the first data block associated with the tuple data only needs to be performed once when the complete tuple data is obtained. This reduces the overhead of querying the tuple data in the first data block. Attached Figure Description

[0037] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is one of the flowcharts illustrating the parallel query method for relational databases provided in the embodiments of this application;

[0039] Figure 2This is a second flowchart illustrating the parallel query method for relational databases provided in the embodiments of this application;

[0040] Figure 3 This is the third flowchart illustrating the parallel query method for relational databases provided in the embodiments of this application;

[0041] Figure 4 This is a schematic diagram of the memory block splitting provided in an embodiment of this application;

[0042] Figure 5 This is the fourth flowchart illustrating the parallel query method for relational databases provided in the embodiments of this application;

[0043] Figure 6 This is one of the structural schematic diagrams of the parallel query device for relational databases provided in the embodiments of this application;

[0044] Figure 7 This is a second schematic diagram of the structure of the parallel query device for relational databases provided in the embodiments of this application;

[0045] Figure 8 This is the third schematic diagram of the structure of the parallel query device for relational databases provided in the embodiments of this application;

[0046] Figure 9 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0047] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0048] The terms "first," "second," etc., used in the embodiments of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices. Additionally, the use of "and / or" in this application indicates at least one of the connected objects, such as A and / or B and / or C, representing seven possibilities: including A alone, B alone, C alone, and the presence of both A and B, both B and C, both A and C, and the presence of A, B, and C.

[0049] Please see Figure 1 , Figure 1 This is a flowchart illustrating the parallel query method for relational databases provided in this application embodiment. Figure 1 The parallel query method for relational databases shown can be executed by consumer stream threads, which, along with producer stream threads, are used to perform parallel queries on the relational database.

[0050] like Figure 1 As shown, the parallel query method for relational databases may include the following steps:

[0051] Step 101: Obtain the first address information from the first shared circular queue. The first address information is the address information of the first data block of the relational database in the memory block pool. The memory block pool stores multiple data blocks with different byte counts. The first data block is any one of the multiple data blocks.

[0052] The first shared circular queue can be shared between the consumer stream thread and the producer stream thread, meaning that both the consumer stream thread and the producer stream thread can obtain the address information of the above multiple data blocks in the memory block pool from the first shared circular queue.

[0053] It should be noted that the first shared circular queue can be used to store the address information of multiple data blocks in the memory block pool, while the memory block pool can be used to store multiple data blocks and the data stored in each data block.

[0054] Step 102: Obtain the first data block from the memory block pool according to the first address information. The first data block stores at least a portion of the tuple data.

[0055] Step 103: If the complete tuple data is obtained, release the first data block associated with the tuple data to the memory block pool.

[0056] In this context, the first data block associated with tuple data can be understood as: at least a portion of the tuple data is stored in the first data block.

[0057] In this embodiment, the number of consumer stream threads can be one or more. When there are multiple consumer stream threads, they can execute the various steps in this embodiment in parallel, which can improve the efficiency of the multiple consumer stream threads in obtaining tuple data, that is, improve the query efficiency of the multiple consumer stream threads in obtaining tuple data.

[0058] In this embodiment of the application, through steps 101 to 103, only one step of obtaining the first data block needs to be performed. And when the complete tuple data is obtained, the first data block associated with the tuple data is released to the memory block pool, that is, the step of releasing the first data block is performed once, thereby reducing the overhead of querying the tuple data in the first data block.

[0059] In other words, the embodiments of this application avoid the problem of frequent memory allocation and release during the distribution of tuple data, and speed up the execution of parallel queries.

[0060] In addition, in this embodiment of the application, when the consumer stream thread performs a query for tuple data, the memory resources consumed only depend on the number of storable bytes of the first data block. The number of storable bytes of the first data block is usually a fixed value and will not increase with the increase of the consumer stream thread or the producer stream thread, thereby saving memory resources.

[0061] It should be noted that the specific method for determining whether complete tuple data has been obtained is not limited here.

[0062] As an optional implementation, the first data block stores a target number of bytes, which represents the total number of bytes in the complete tuple data. Releasing the first data block to the memory block pool upon obtaining the complete tuple data includes:

[0063] Determine whether the number of bytes of tuple data stored in the first data block is equal to the target number of bytes;

[0064] If it is determined that the number of bytes of tuple data stored in the first data block is equal to the target number of bytes, the tuple data stored in the first data block is obtained, and it is determined that complete tuple data is obtained from the first data block.

[0065] If it is determined that complete tuple data has been obtained from the first data block, the first data block is released to the memory block pool.

[0066] In the first data block, the target byte count and tuple data can be stored adjacently.

[0067] In this embodiment of the application, when it is determined that the number of bytes of tuple data stored in the first data block is equal to the target number of bytes, it can be determined that the first data block stores complete tuple data. Therefore, after obtaining complete tuple data from the first data block, the first data block can be released to the memory block pool, that is, the first data block is released, so that the first data block can participate in the storage of other tuple data in the future, thereby improving the utilization rate of the first data block.

[0068] Optionally, when the consumer stream thread receives tuple data from the first data block, it first reads the target number of bytes of integer size from the first data block and stores it in an integer variable of expected length. This integer variable represents the number of bytes of the complete tuple data. When the number of bytes of tuple data stored in the first data block is equal to the target number of bytes, it can be determined that the first data block stores complete tuple data.

[0069] It should be noted that, optionally, after obtaining complete tuple data from the first data block, the consumer stream thread will exit the state of receiving tuple data from the first shared circular queue and enter the state of processing the aforementioned tuple data until the tuple data is processed, and then re-enter the state of receiving tuple data, that is, continue to read tuple data from the first shared circular queue.

[0070] As an optional implementation, after determining whether the number of bytes of tuple data stored in the first data block is equal to the target number of bytes, the method further includes:

[0071] If it is determined that the number of bytes of tuple data stored in the first data block is less than the target number of bytes, the tuple data stored in the first data block is obtained;

[0072] Release the first data block to the memory block pool and obtain the second data block from the memory block pool;

[0073] Obtain target tuple data from the second data block, wherein the target tuple data is the tuple data in the complete tuple data excluding the tuple data stored in the first data block.

[0074] In this embodiment of the application, when it is determined that the number of bytes of tuple data stored in the first data block is less than the target number of bytes, it indicates that the first data block does not store complete tuple data. Therefore, after obtaining the tuple data stored in the first data block, the second data block can be obtained from the memory block pool, and the target tuple data can be obtained from the second data block. In this way, it is guaranteed that complete tuple data can be obtained.

[0075] It should be noted that, optionally, when a second data block is not present in the memory block pool, the consumer stream thread will attempt to retrieve a data block from other shared circular queues besides the first shared circular queue to read the tuple data. When all shared circular queues are empty and the consumer stream thread has not read the complete tuple data, the consumer stream thread enters a blocked waiting state until it is awakened by the producer stream thread, and then re-attempts to retrieve a new data block from the shared circular queue before reading the tuple data.

[0076] As an optional implementation, obtaining the target tuple data from the second data block includes:

[0077] Obtain a cache memory block, wherein the number of bytes in the cache memory block is twice the number of bytes in the target memory block;

[0078] The target tuple data is obtained from the second data block and stored in the cache memory block.

[0079] In this embodiment, the target tuple data is stored in a cache memory block, and the number of bytes in the cache memory block is twice the number of bytes in the target tuple. This ensures that the target tuple data can be stored, thus enhancing the storage effect of the target tuple data.

[0080] It should be noted that, optionally, the tuple data obtained from the first data block can also be stored in the aforementioned cache memory block. In this way, both the tuple data obtained from the first data block and the target tuple data can be stored in the cache memory block, ensuring the integrity of the tuple data storage and facilitating subsequent retrieval of the complete tuple data.

[0081] Optionally, obtaining a cache memory block includes: obtaining a cache memory block when it is determined that no cache memory block has been requested for storing the target tuple data. This can ensure the accuracy of obtaining the cache memory block, avoid the phenomenon of repeated acquisition of cache memory blocks, and reduce the waste of storage resources caused by repeated acquisition of cache memory blocks.

[0082] Alternatively, if it is determined that a preset cached data block for storing the target tuple data has already been pre-acquired before acquiring the cached memory block, it is determined whether the number of bytes stored in the preset cached data block is greater than the number of bytes of the complete tuple data (i.e., the target number of bytes). If the number of bytes stored in the preset cached data block is less than the target number of bytes, the consumer stream thread will release the preset cached memory block and reacquire the cached memory block; if the number of bytes stored in the preset cached data block is greater than or equal to the target number of bytes, the preset cached memory block will continue to be used and the acquisition of cached memory blocks will stop.

[0083] It should be noted that, optionally, when the consumer stream thread obtains complete tuple data, it can continue to obtain the address information of new data blocks from the first shared circular queue to retrieve the data in the new data block. However, when there is no address information for new data blocks available in the first shared circular queue, the consumer stream thread will attempt to obtain the address information of data blocks from other shared circular queues besides the first shared circular queue to read the new tuple data. When there is no address information for data blocks in any shared circular queue, and the consumer stream thread has not read complete tuple data, the consumer stream thread enters a blocked waiting state until it is awakened by the producer stream thread, and then re-attempts to obtain the address information of new data blocks from the shared circular queue, and then reads the tuple data.

[0084] See Figure 2 , Figure 2 This application provides a parallel query method for a relational database, applied to a producer stream thread. The producer stream thread and consumer stream thread are used to perform parallel queries on the relational database. Figure 2 As shown, the method includes the following steps:

[0085] Step 201: Obtain the first data block of the relational database from the memory block pool. The memory block pool stores multiple data blocks with different byte counts. The first data block is any one of the multiple data blocks. The first data block stores at least a portion of the tuple data.

[0086] Step 202: Store at least a portion of the tuple data into the first data block, and release the first data block containing at least a portion of the tuple data to the memory block pool.

[0087] Step 203: Send the first address information to the first shared circular queue. The first address information is the address information of the first data block in the memory block pool.

[0088] It should be noted that the embodiments in this application are as follows: Figure 1 The illustrated embodiment applies to the corresponding embodiment of the producer stream thread. The difference between the two lies in the execution entity. Therefore, the technical features in the embodiments of this application can be found above. Figure 1 The relevant descriptions in the illustrated embodiments have the same beneficial technical effects.

[0089] In this embodiment, a first address information is sent to the first shared circular queue. The first address information is the address information of the first data block in the memory block pool. This allows the consumer stream thread to obtain the first address information from the first shared circular queue and then obtain the tuple data in the first data block, thereby improving the efficiency of the consumer stream thread in obtaining tuple data.

[0090] It should be noted that since the producer stream thread actively retrieves the first data block of the relational database from the memory block pool, the stronger the producer stream thread's production capacity, the more tuple data it produces, and the higher the frequency of retrieving the first data block, thus enabling it to acquire more first data blocks for tuple data transmission. Conversely, the weaker the producer stream thread's production capacity, the fewer tuple data it produces, and the less frequently it retrieves the first data block. In this way, the first data block occupied by the aforementioned producer stream thread can be allocated to other producer stream threads with strong production capacity, improving the utilization rate of the first data block and making the allocation of the first data block more reasonable.

[0091] As an optional implementation, obtaining the first data block of the relational database from the memory block pool includes:

[0092] Obtain the target number of bytes, which represents the total number of bytes in the complete tuple data;

[0093] The first data block is obtained from the memory block pool according to the target number of bytes, wherein the number of bytes in the first data block matches the target number of bytes, or the number of bytes in the first data block is less than the target number of bytes.

[0094] In this embodiment, by obtaining a target number of bytes and retrieving a first data block from the memory block pool based on that target number of bytes, if the number of bytes in the first data block matches the target number of bytes, it can be determined that the retrieved first data block can store complete tuple data. This avoids the need to use multiple data blocks to store complete tuple data separately, allowing the consumer stream thread to retrieve complete tuple data from the first data block at once, thus improving the efficiency of the consumer stream thread in retrieving complete tuple data. If the number of bytes in the first data block is less than the target number of bytes, it can be determined that the retrieved first data block does not store complete tuple data. Therefore, after retrieving a portion of the tuple data stored in the first database, a second data block can be retrieved, and the remaining portion of the tuple data can be retrieved from the second data block to obtain complete tuple data.

[0095] The above embodiments are illustrated below with a specific example.

[0096] 1. After producing a tuple, the producer stream thread first obtains the number of bytes in the tuple and saves it to an integer variable representing the length. Then, it copies the value of this integer variable byte by byte into the data block. If the producer stream thread does not have a usable data block, it can obtain a new data block from the memory block pool to copy the tuple data.

[0097] It should be noted that if the data block obtained by the producer stream thread does not have enough space to copy the tuple data, the producer stream thread will copy a portion of the data into the data block (for example, the data block can be the first data block). After filling the data block, the address information of the data block will be enqueued into the shared circular queue (for example, the shared circular queue can include the first shared circular queue), and a new data block will be requested from the memory block pool (for example, the data block can be the second data block). Then, the remaining part of the tuple data will be copied.

[0098] If the producer stream thread fails to request a new data block from the memory block pool, meaning the memory block pool has no available data blocks, it enters a blocked waiting state until the consumer stream thread returns the data block and the memory block pool has a new free data block. At this point, the memory block pool will notify and wake up the producer stream thread, and then retry copying the remaining bytes to the new memory.

[0099] 2. After successfully copying tuple data into the data block, the producer stream thread continues to copy tuple data into the data block. After obtaining the available space size of the current data block (i.e., the number of bytes that can be used to store data), the producer stream thread will compare it with the size of the tuple data that still needs to be copied (i.e., the number of bytes of tuple data).

[0100] If the available space in a data block is smaller than the size of the tuples that still need to be copied, then tuples of the same size as the available space in the data block will be copied into the data block. After the copy is complete, the producer stream thread will add this data block address to the shared circular queue, notify the consumer stream thread, and request a new data block from the memory block pool, then copy the remaining tuples into the new data block. If the producer stream thread fails to request a new data block from the memory block pool (i.e., the memory block pool has no available data blocks), it will enter a blocked waiting state until the consumer stream thread returns the data block and the memory block pool has a new free data block. At this point, the memory block pool will notify and wake up the producer stream thread, and then retry copying the remaining tuples into the new data block.

[0101] If the available space in the data block is greater than the size of the tuple data that still needs to be copied, then the tuple data that still needs to be copied will be copied into the data block all at once, completing the sending process of the tuple data. The producer stream thread will then exit the state of sending tuple data and enter the state of producing tuple data. After producing a new tuple data, it will re-enter the tuple sending state and send the new tuple data.

[0102] See Figure 3 , Figure 3 This application provides a parallel query method for relational databases, such as... Figure 3 As shown, the parallel query method for relational databases includes the following steps:

[0103] Step 301: Obtain the first parameter of the consumer stream thread and the second parameter of the producer stream thread;

[0104] Step 302: Determine the target data block of the relational database based on the first parameter and the second parameter;

[0105] Step 303: Split or merge the target data block into multiple data blocks with different byte counts, wherein the multiple data blocks include the first data block;

[0106] Step 304: Store the multiple data blocks into a memory block pool, and store the address information of the multiple data blocks in the memory block pool into a first shared circular queue;

[0107] The consumer stream thread adopts Figure 1 The parallel query method for relational databases in the illustrated embodiment performs parallel queries on the relational database, and the producer stream thread adopts... Figure 2 The parallel query method for relational databases in the illustrated embodiment performs parallel queries on the relational database.

[0108] The steps performed in this application embodiment can be understood as the initialization of data blocks. The execution subject of this application embodiment can be the memory block pool of the relational database or the data block free queue of the memory block pool of the relational database. It should be noted that the technical features in this application embodiment can be found above. Figure 1 The illustrated embodiments and Figure 2 The corresponding descriptions in the illustrated embodiments have corresponding technical effects, which will not be elaborated further here.

[0109] This application embodiment can allocate target data blocks based on the first parameter of the consumer stream thread and the second parameter of the producer stream thread, making the target data blocks more suitable for the needs of the consumer stream thread and the producer stream thread. This avoids the phenomenon of the target data block being too large or too small, resulting in a mismatch between the target data block and the needs of the consumer stream thread and the producer stream thread. At the same time, the storable bytes of the determined target data block can also be dynamically adjusted based on the first parameter of the consumer stream thread and the second parameter of the producer stream thread. The storable bytes can be understood as the size of the available memory.

[0110] It should be noted that the types of the first parameter of the consumer stream thread and the second parameter of the producer stream thread are not limited here. Optionally, the first parameter of the consumer stream thread may include the number of bytes of tuple data required by the consumer stream thread, and the second parameter of the producer stream thread may include the number of bytes of tuple data required by the producer stream thread. Alternatively, the first parameter may include the consumer stream thread's ability to process tuple data, and the second parameter may include the producer stream thread's ability to produce tuple data; these ability parameters may include processing speed, etc.

[0111] To split or merge a target data block into multiple data blocks with different byte counts, see the following description: The target data block can be split into multiple data blocks, or the target data block can be merged with other data blocks to obtain multiple different data blocks.

[0112] It should be noted that when the number of storable bytes of the target data block does not match the number of bytes of tuple data required by the consumer stream thread and the producer stream thread, the target data block can be split or merged based on the number of bytes of tuple data.

[0113] The above embodiment is illustrated below with a specific example. Specifically, it may include the following steps:

[0114] 1. Before starting the producer stream thread, allocate a data block with a storage capacity of M bytes (i.e., the target data block) in the memory block pool for a group of producer stream threads and consumer stream threads. The specific size of the target data block is not limited here and can be determined according to the size of the tuple data required by the producer stream thread and consumer stream thread. Then, split this data block into X groups of data blocks with a storage capacity of X bytes (e.g., decomposing 1MB of memory into four types: 64KB, 16KB, 4KB, and 1KB). The storage capacity of the different data blocks obtained by splitting them is in a linear proportional relationship.

[0115] 2. The process of splitting the target data block into X groups of data blocks starts from the data block with the largest storable byte size, as follows: First, the target data block is split into multiple data blocks with the largest storable byte size. Then, one-X of these blocks is recursively split into the next data block with the next largest storable byte size, until the data block with the smallest storable byte size is obtained.

[0116] For example, the data block is split into multiple 64KB blocks. Then, one-quarter of these 64KB blocks are further split into smaller 16KB blocks. This process is repeated until the smallest possible block size is reached. In a target data block with a 1MB capacity, this would result in 12 blocks of 64KB, 12 blocks of 16KB, 12 blocks of 4KB, and 16 blocks of 1KB.

[0117] 3. Enqueue data blocks with the same number of bytes to the corresponding free queue. These data blocks can also be referred to as memory blocks. For example... Figure 4 As shown, there are a total of 4 memory block free queues. These 4 queues form a memory block pool, which can allocate appropriate memory blocks to the producer stream thread and reclaim memory blocks from the consumer stream thread into the corresponding memory block free queue.

[0118] It should be noted that, Figure 4 The memory block in this embodiment can be understood as the first data block that has not yet stored tuple data in this application embodiment, and as follows Figure 4 As shown, there can be multiple memory blocks, and these memory blocks can be arranged in order. The memory block that is first in the arrangement can be called the head memory block, and the memory block that is last in the arrangement can be called the end memory block.

[0119] 4. When the producer stream thread sends tuple data, it requests a data block from the memory block pool based on the number of bytes in the tuple data to copy the tuple data. The memory block pool finds the data block free queue that matches the number of bytes of tuple data that the producer stream thread needs to send, obtains a free data block from the free data block queue, and then allocates the free data block to the producer stream thread to transmit the tuple data.

[0120] If there are no data blocks in the free queue of data blocks with a storage capacity of S bytes, the system will continuously try to find a data block from the free queue of data blocks with a storage capacity of one level smaller, until a free data block is found.

[0121] If no free data block is found up to the queue with the smallest storable bytes, an attempt will be made to obtain a free data block from the queue with the largest storable bytes. If no free data block is found up to the queue with the largest storable bytes (S), meaning there are no free data blocks in the data block pool, the producer stream thread will be notified that no data blocks are available. If a free data block is found, it will be split into smaller blocks, as described above, but split down to a storable byte size of S. The split data blocks are then added to the corresponding free data block queue, and a data block of size S is allocated to the producer stream thread.

[0122] For example, when S is 4KB, if a data block is found in the 64KB data block free queue, the data block is split into four 16KB data blocks. Then, one-quarter of the 16KB data block is split into 4KB data blocks. In the end, a 64KB data block will be split into three 16KB data blocks and four 4KB data blocks. Finally, a 4KB data block is returned to the producer stream thread.

[0123] 5. After the consumer stream thread obtains the first data block from the first shared circular queue, it dequeues the address of the first data block from the first shared circular queue, and then reads tuple data from the first data block until all the data in the first data block has been read. Finally, the first data block is added to the free data block queue with the corresponding number of storable bytes.

[0124] It should be noted that, for a more complete explanation of the above embodiments, please refer to [link / reference needed]. Figure 5 , Figure 5 A flowchart illustrating the parallel query method for relational databases provided in this application embodiment is available for further details. Figure 5 By optimizing the data distribution mechanism module of the relational database parallel query technology, the various steps in this application can reduce memory resource consumption during parallel query execution and accelerate the parallel execution speed of queries.

[0125] It should be noted that, Figure 5 The shared circular queue in the above embodiment can be understood as the first shared circular queue. Figure 5 The memory block in this embodiment can be understood as the first data block that has not yet stored tuple data in this application embodiment, while Figure 5 The data block in this embodiment can be understood as the first data block storing tuple data, the consumer thread can be understood as the consumer stream thread in the above embodiment, and the producer thread can be understood as the producer stream thread in the above embodiment, such as... Figure 5As shown, there can be multiple data blocks, and these data blocks can be arranged in order. The data block that is first in the arrangement can be called the head data block, and the data block that is last in the arrangement can be called the tail data block.

[0126] See Figure 6 , Figure 6 This is a structural diagram of the parallel query device for relational databases provided in the embodiments of this application. Figure 6 The parallel query device for the relational database shown is applied to the consumer stream thread, which, along with the producer stream thread, performs parallel queries on the relational database. The parallel query device 600 for the relational database includes:

[0127] The first acquisition module 601 is used to acquire first address information from the first shared circular queue. The first address information is the address information of the first data block of the relational database in the memory block pool. The memory block pool stores multiple data blocks with different byte counts. The first data block is any one of the multiple data blocks.

[0128] The second acquisition module 602 is used to acquire the first data block from the memory block pool according to the first address information, wherein the first data block stores at least a portion of the tuple data;

[0129] The first release module 603 is used to release the first data block associated with the tuple data to the memory block pool when the complete tuple data is obtained.

[0130] As an optional implementation, the first data block stores a target number of bytes, which represents the total number of bytes in the complete tuple data. The first release module 603 includes:

[0131] The first determining submodule is used to determine whether the number of bytes of tuple data stored in the first data block is equal to the target number of bytes;

[0132] The second determining submodule is used to obtain the tuple data stored in the first data block when it is determined that the number of bytes of tuple data stored in the first data block is equal to the target number of bytes, and to determine that complete tuple data has been obtained from the first data block.

[0133] The first release submodule is used to release the first data block to the memory block pool when it is determined that complete tuple data has been obtained from the first data block.

[0134] As an optional implementation, the first release module 603 further includes:

[0135] The first acquisition submodule is used to acquire the tuple data stored in the first data block when it is determined that the number of bytes of tuple data stored in the first data block is less than the target number of bytes.

[0136] The second acquisition submodule is used to release the first data block to the memory block pool and acquire the second data block from the memory block pool;

[0137] The third acquisition submodule is used to acquire target tuple data from the second data block. The target tuple data is the tuple data in the complete tuple data excluding the tuple data stored in the first data block.

[0138] As an optional implementation, the third acquisition submodule includes:

[0139] The first acquisition unit is used to acquire a cache memory block, wherein the number of bytes in the cache memory block is twice the number of bytes in the target memory block;

[0140] The second acquisition unit is used to acquire target tuple data from the second data block and store the target tuple data in the cache memory block.

[0141] The parallel query device 600 for relational databases can implement the embodiments of this application. Figure 1 The various processes in the method embodiments, and the ways to achieve the same beneficial effects, will not be repeated here to avoid repetition.

[0142] See Figure 7 , Figure 7 This is a structural diagram of the parallel query device for relational databases provided in the embodiments of this application. Figure 7 The parallel query device for the relational database shown is applied to the producer stream thread, which, along with the consumer stream thread, performs parallel queries on the relational database. The parallel query device 700 for the relational database includes:

[0143] The third acquisition module 701 is used to acquire a first data block of the relational database from the memory block pool. The memory block pool stores multiple data blocks with different byte counts. The first data block is any one of the multiple data blocks. The first data block stores at least a portion of the tuple data.

[0144] The second release module 702 is used to store at least a portion of the tuple data into the first data block and release the first data block containing at least a portion of the tuple data to the memory block pool.

[0145] The sending module 703 is used to send first address information to the first shared circular queue, wherein the first address information is the address information of the first data block in the memory block pool.

[0146] As an optional implementation, the third acquisition module 701 includes:

[0147] The fourth acquisition submodule is used to acquire the target number of bytes, which represents the total number of bytes in the complete tuple data;

[0148] The fifth acquisition submodule is used to acquire the first data block from the memory block pool according to the target number of bytes, wherein the number of bytes in the first data block matches the target number of bytes, or the number of bytes in the first data block is less than the target number of bytes.

[0149] The parallel query device 700 for relational databases can implement the embodiments of this application. Figure 2 The various processes in the method embodiments, and the ways to achieve the same beneficial effects, will not be repeated here to avoid repetition.

[0150] See Figure 8 , Figure 8 This is a structural diagram of a parallel query device for a relational database provided in an embodiment of this application. The parallel query device 800 for the relational database includes:

[0151] The fourth acquisition module 801 is used to acquire the first parameter of the consumer stream thread and the second parameter of the producer stream thread;

[0152] The first determining module 802 is used to determine the target data block of the relational database based on the first parameter and the second parameter;

[0153] The splitting or merging module 803 is used to split or merge the target data block into multiple data blocks with different byte counts, wherein the multiple data blocks include a first data block;

[0154] Storage module 804 is used to store multiple data blocks into a memory block pool and store the address information of the multiple data blocks in the memory block pool into a first shared circular queue;

[0155] The consumer stream thread adopts Figure 1 The parallel query method for relational databases in the illustrated embodiment performs parallel queries on the relational database, and the producer stream thread adopts... Figure 2 The parallel query method for relational databases described in the illustrated embodiment performs parallel queries on the relational database.

[0156] The parallel query device 800 for relational databases can implement the embodiments of this application. Figure 3 The various processes in the method embodiments, and the ways to achieve the same beneficial effects, will not be repeated here to avoid repetition.

[0157] This application also provides an electronic device. Please refer to [link to relevant documentation]. Figure 9 The electronic device may include a processor 901, a memory 902, and a program 9021 stored in the memory 902 and executable on the processor 901. When the electronic device is used in a consumer stream thread, the program 9021, when executed by the processor 901, can achieve... Figure 1 The steps in the corresponding method embodiments and the achievement of the same beneficial effects will not be elaborated here; when the electronic device is applied to the producer flow thread, program 9021 can be implemented when executed by processor 901. Figure 2 Any steps in the method embodiments shown, and the achievement of the same beneficial effects, will not be repeated here. When the electronic device is applied to the memory block pool of the relational database, program 9021 can be implemented when executed by processor 901. Figure 3 Any steps in the method embodiments shown, and the same beneficial effects achieved, will not be repeated here.

[0158] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by hardware related to program instructions, and the program can be stored in a readable medium. This application also provides a readable storage medium storing a computer program, which, when executed by a processor, can implement the above-described methods. Figure 1 , Figure 2 or Figure 3 Any step in the corresponding method embodiment can achieve the same technical effect, and will not be repeated here to avoid repetition.

[0159] The storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0160] This application also provides a computer program product, including computer instructions, which, when executed by a processor, can perform the above-described functions. Figure 1 , Figure 2 or Figure 3 Any step in the corresponding method embodiment can achieve the same technical effect, and will not be repeated here to avoid repetition.

[0161] The above description represents the preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles described in this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A parallel query method for a relational database, applied to a consumer stream thread, wherein the consumer stream thread and the producer stream thread are used to perform parallel queries on the relational database, characterized in that, The method includes: Obtain first address information from the first shared circular queue. The first address information is the address information of the first data block of the relational database in the memory block pool. The memory block pool stores multiple data blocks with different byte counts. The first data block is any one of the multiple data blocks. The first data block is obtained from the memory block pool according to the first address information, and the first data block stores at least a portion of the tuple data; If the complete tuple data is obtained, the first data block associated with the tuple data is released to the memory block pool; The first data block stores a target number of bytes, which represents the total number of bytes in the complete tuple data. Releasing the first data block to the memory block pool upon obtaining the complete tuple data includes: Determine whether the number of bytes of tuple data stored in the first data block is equal to the target number of bytes; If it is determined that the number of bytes of tuple data stored in the first data block is equal to the target number of bytes, the tuple data stored in the first data block is obtained, and it is determined that complete tuple data is obtained from the first data block. If it is determined that the complete tuple data has been obtained from the first data block, the first data block is released to the memory block pool; After determining whether the number of bytes of tuple data stored in the first data block is equal to the target number of bytes, the method further includes: If it is determined that the number of bytes of tuple data stored in the first data block is less than the target number of bytes, the tuple data stored in the first data block is obtained; Release the first data block to the memory block pool and obtain the second data block from the memory block pool; Obtain target tuple data from the second data block, wherein the target tuple data is the tuple data in the complete tuple data excluding the tuple data stored in the first data block.

2. The method according to claim 1, characterized in that, The step of obtaining the target tuple data from the second data block includes: Obtain a cache memory block, wherein the number of bytes in the cache memory block is twice the number of bytes in the target memory block; The target tuple data is obtained from the second data block and stored in the cache memory block.

3. A parallel query method for a relational database, applied to a producer stream thread, wherein the producer stream thread and the consumer stream thread are used to perform parallel queries on the relational database, characterized in that, The method includes: A first data block from a relational database is obtained from a memory block pool, which stores multiple data blocks of different byte sizes. The first data block is any one of the multiple data blocks, and the first data block stores at least a portion of the tuple data. At least a portion of the tuple data is stored in the first data block, and the first data block containing at least a portion of the tuple data is released to the memory block pool. Send first address information to the first shared circular queue, wherein the first address information is the address information of the first data block in the memory block pool; The step of retrieving the first data block of the relational database from the memory block pool includes: Obtain the target number of bytes, which represents the total number of bytes in the complete tuple data; The first data block is obtained from the memory block pool according to the target number of bytes, wherein the number of bytes in the first data block matches the target number of bytes, or the number of bytes in the first data block is less than the target number of bytes; Wherein, if the number of bytes in the first data block is less than the target number of bytes, the consumer stream thread is used to obtain target tuple data from the second data block, wherein the target tuple data is the tuple data in the complete tuple data excluding the tuple data stored in the first data block.

4. A parallel query method for relational databases, characterized in that, include: Get the first parameter of the consumer stream thread and the second parameter of the producer stream thread; The target data block of the relational database is determined based on the first parameter and the second parameter; The target data block is split or merged into multiple data blocks with different byte counts, and the multiple data blocks include the first data block; The multiple data blocks are stored in a memory block pool, and the address information of the multiple data blocks in the memory block pool is stored in a first shared circular queue; The consumer stream thread performs parallel queries on the relational database using the parallel query method for relational databases as described in any one of claims 1 to 2, and the producer stream thread performs parallel queries on the relational database using the parallel query method for relational databases as described in claim 3.

5. A parallel query device for a relational database, characterized in that, An application is provided to the consumer stream thread, which, together with the producer stream thread, performs parallel queries on a relational database. The parallel query device for the relational database includes: The first acquisition module is used to acquire first address information from the first shared circular queue. The first address information is the address information of the first data block of the relational database in the memory block pool. The memory block pool stores multiple data blocks with different byte counts. The first data block is any one of the multiple data blocks. The second acquisition module is used to acquire the first data block from the memory block pool according to the first address information, wherein the first data block stores at least a portion of the tuple data; The first release module is used to release the first data block associated with the tuple data to the memory block pool when the complete tuple data is obtained. The first data block stores a target number of bytes, which represents the total number of bytes in the complete tuple data. The first release module includes: The first determining submodule is used to determine whether the number of bytes of tuple data stored in the first data block is equal to the target number of bytes; The second determining submodule is used to obtain the tuple data stored in the first data block when it is determined that the number of bytes of tuple data stored in the first data block is equal to the target number of bytes, and to determine that complete tuple data has been obtained from the first data block. The first release submodule is used to release the first data block to the memory block pool when it is determined that complete tuple data has been obtained from the first data block; The first release module also includes: The first acquisition submodule is used to acquire the tuple data stored in the first data block when it is determined that the number of bytes of tuple data stored in the first data block is less than the target number of bytes. The second acquisition submodule is used to release the first data block to the memory block pool and acquire the second data block from the memory block pool; The third acquisition submodule is used to acquire target tuple data from the second data block. The target tuple data is the tuple data in the complete tuple data excluding the tuple data stored in the first data block.

6. A parallel query device for a relational database, characterized in that, An apparatus for parallel querying relational databases, applied to producer stream threads and consumer stream threads, comprising: The third acquisition module is used to acquire a first data block of the relational database from the memory block pool. The memory block pool stores multiple data blocks with different byte counts. The first data block is any one of the multiple data blocks. The first data block stores at least a portion of the tuple data. The second release module is used to store at least a portion of the tuple data into the first data block and release the first data block containing at least a portion of the tuple data to the memory block pool. The sending module is used to send first address information to the first shared circular queue, wherein the first address information is the address information of the first data block in the memory block pool; The third acquisition module includes: The fourth acquisition submodule is used to acquire the target number of bytes, which represents the total number of bytes in the complete tuple data; The fifth acquisition submodule is used to acquire the first data block from the memory block pool according to the target number of bytes, wherein the number of bytes in the first data block matches the target number of bytes, or the number of bytes in the first data block is less than the target number of bytes; Wherein, if the number of bytes in the first data block is less than the target number of bytes, the consumer stream thread is used to obtain target tuple data from the second data block, wherein the target tuple data is the tuple data in the complete tuple data excluding the tuple data stored in the first data block.

7. A parallel query device for a relational database, characterized in that, include: The fourth acquisition module is used to acquire the first parameter of the consumer stream thread and the second parameter of the producer stream thread; The first determining module is used to determine the target data block of the relational database based on the first parameter and the second parameter; A splitting or merging module is used to split or merge the target data block into multiple data blocks with different byte counts, wherein the multiple data blocks include a first data block; The storage module is used to store multiple data blocks into a memory block pool and store the address information of the multiple data blocks in the memory block pool into a first shared circular queue; The consumer stream thread performs parallel queries on the relational database using the parallel query method for relational databases as described in any one of claims 1 to 2, and the producer stream thread performs parallel queries on the relational database using the parallel query method for relational databases as described in claim 3.

8. An electronic device, comprising: A memory, a processor, and a program stored in the memory and executable on the processor; characterized in that the processor is configured to read the program in the memory to implement the steps in the parallel query method for a relational database as described in any one of claims 1 to 2; or, to implement the steps in the parallel query method for a relational database as described in claim 3; or, to implement the steps in the parallel query method for a relational database as described in claim 4.

9. A readable storage medium for storing a program, characterized in that, When the program is executed by the processor, it implements the steps in the parallel query method for relational databases as described in any one of claims 1 to 2; or, implements the steps in the parallel query method for relational databases as described in claim 3; or, implements the steps in the parallel query method for relational databases as described in claim 4.

10. A computer program product, characterized in that, The method includes computer instructions that, when executed by a processor, implement the steps in the parallel query method for a relational database as described in any one of claims 1 to 2; or, when executed by a processor, the computer instructions implement the steps in the parallel query method for a relational database as described in claim 3; or, when executed by a processor, the computer instructions implement the steps in the parallel query method for a relational database as described in claim 4.

Images