Parallel query method and device of relational database and related equipment

Abstract

The invention provides a parallel query method and device of a relational database and related equipment, and relates to the technical field of big data, the parallel query method of the relational database is applied to a consumer streamline thread, and the method comprises the following steps: obtaining first address information from a first shared circular queue, the first address information is address information of a first data block of the relational database in a memory block pool, a plurality of data blocks with different byte numbers are stored in the memory block pool, and the first data block is any one of the plurality of data blocks; obtaining the first data block from the memory block pool according to the first address information, wherein the first data block stores at least part of data of tuple data; and releasing the first data block associated with the tuple data to the memory block pool under the condition that the complete tuple data is obtained. In this way, the overhead of querying the tuple data in the first data block is reduced.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of big data technology, and in particular, to a parallel query method, device and related equipment for a relational database. Background Art

[0002] With the continuous development of big data technology, the application scope of big data technology in people's lives is also becoming more and more extensive. In today's data-driven world, with the continuous growth of business requirements and the rapid increase in the amount of data, traditional relational database query methods often struggle to meet the requirements of efficient processing and rapid response of large amounts of data. In the actual query process of tuple data in a relational database, two memory applications, two memory releases, and two memory copies need to be performed. It can be seen that the current overhead of querying tuple data in a relational database is relatively large. Summary of the Invention

[0003] The embodiments of the present application provide a parallel query method, device and related equipment for a relational database to solve the problem of relatively large overhead in currently querying tuple data in a relational database.

[0004] To solve the above problems, the present application is implemented as follows: In a first aspect, the embodiments of the present application provide a parallel query method for a relational database, which is applied to a consumer stream thread. The consumer stream thread and the producer stream thread are used for parallel query of a relational database. The method includes: Obtain first address information from a first shared circular queue. The first address information is the address information of a first data block of a relational database in a memory block pool. Multiple data blocks with different byte counts are stored in the memory block pool, and the first data block is any one of the multiple data blocks; Obtain the first data block from the memory block pool according to the first address information. The first data block stores at least part of the tuple data; When complete tuple data is obtained, release the first data block associated with the tuple data to the memory block pool.

[0005] In a second aspect, the embodiments of the present application provide a parallel query method for a relational database, which is applied to a producer stream thread. The producer stream thread and the consumer stream thread are used for parallel query of a relational database. The method includes: Obtain a first data block of a relational database from a memory block pool. Multiple data blocks with different byte counts are stored in the memory block pool, and the first data block is any one of the multiple data blocks. The first data block stores at least part of the tuple data; Store at least part of the tuple data in the first data block, and release the first data block storing at least part of the tuple data to the memory block pool; Send first address information to the first shared circular queue, where the first address information is the address information of the first data block in the memory block pool.

[0006] In a third aspect, an embodiment of the present application further provides a parallel query method for a relational database, including: Obtain a first parameter of a consumer stream thread and a second parameter of a producer stream thread; Determine a target data block of the relational database according to the first parameter and the second parameter; Split or merge the target data block into multiple data blocks with different byte counts, and the multiple data blocks include a first data block; Store the multiple data blocks in the memory block pool, and store the address information of the multiple data blocks in the memory block pool in the first shared circular queue; Wherein, the consumer stream thread performs parallel query of the relational database by using the parallel query method of the relational database in the first aspect, and the producer stream thread performs parallel query of the relational database by using the parallel query method of the relational database in the second aspect.

[0007] In a fourth aspect, an embodiment of the present application further provides a parallel query device for a relational database, which is applied to a consumer stream thread. The consumer stream thread and the producer stream thread are used for parallel query of the relational database. The parallel query device for the relational database includes: A first acquisition module, configured to acquire first address information from a first shared circular queue, where the first address information is the address information of a first data block of a relational database in a memory block pool, and the memory block pool stores multiple data blocks with different byte counts, and the first data block is any one of the multiple data blocks; A second acquisition module, configured to acquire the first data block from the memory block pool according to the first address information, and the first data block stores at least part of the tuple data; A first release module, configured to release the first data block associated with the tuple data to the memory block pool when complete tuple data is acquired.

[0008] In a fifth aspect, an embodiment of the present application further provides a parallel query device for a relational database, which is applied to a producer stream thread. The producer stream thread and the consumer stream thread are used for parallel query of the relational database. The parallel query device for the relational database includes: A third acquisition module, configured to acquire a first data block of a relational database from a memory block pool, where multiple data blocks with different byte counts are stored in the memory block pool, the first data block is any one of the multiple data blocks, and the first data block stores at least part of the tuple data; A second release module, configured to store at least part of the tuple data into the first data block, and release the first data block storing at least part of the tuple data to the memory block pool; A sending module, configured to send first address information to a first shared circular queue, where the first address information is the address information of the first data block in the memory block pool.

[0009] In a sixth aspect, an embodiment of the present application further provides a parallel query device for a relational database, including: A fourth acquisition module, configured to acquire a first parameter of a consumer stream thread and a second parameter of a producer stream thread; A first determination module, configured to determine a target data block of the relational database according to the first parameter and the second parameter; A splitting or merging module, configured to split or merge the target data block into multiple data blocks with different byte counts, and the multiple data blocks include a first data block; A storage module, configured to store the multiple data blocks into the memory block pool, and store the address information of the multiple data blocks in the memory block pool into the first shared circular queue; Wherein, the consumer stream thread performs parallel query of the relational database by using the parallel query method of the relational database in the first aspect, and the producer stream thread performs parallel query of the relational database by using the parallel query method of the relational database in the second aspect.

[0010] In a seventh aspect, an embodiment of the present application further provides an electronic device, including: a memory, a processor, and a program stored on the memory and executable on the processor; the processor is configured to read the program in the memory to implement the steps in the method in the foregoing first aspect, or, configured to read the program in the memory to implement the steps in the method in the foregoing second aspect, or, configured to read the program in the memory to implement the steps in the method in the foregoing third aspect.

[0011] In an eighth aspect, an embodiment of the present application further provides a readable storage medium, configured to store a program, where the program, when executed by a processor, implements the steps in the method in the foregoing first aspect, or, implements the steps in the method in the foregoing second aspect, or, implements the steps in the method in the foregoing third aspect.

[0012] In a ninth aspect, an embodiment of the present application further provides a computer program product, including computer instructions which, when executed by a processor, implement the steps in the method described in the first aspect, the second aspect, or the third aspect above.

[0013] In an embodiment of the present application, since the first address information is obtained from the first shared circular queue, where the first address information is the address information of the first data block of the relational database in the memory block pool, and multiple data blocks with different byte counts are stored in the memory block pool, and the first data block is any one of the multiple data blocks; the first data block is obtained from the memory block pool according to the first address information, and the first data block stores at least part of the tuple data; in the case of obtaining complete tuple data, the first data block associated with the tuple data is released to the memory block pool. In this way, only one step of obtaining the first data block needs to be executed, and in the case of obtaining complete tuple data, the first data block associated with the tuple data is released to the memory block pool, that is, one step of releasing the first data block is executed, thereby reducing the overhead of querying the tuple data in the first data block. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for description in the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0015] Figure 1 is one of the flow diagrams of the parallel query method for a relational database provided by an embodiment of the present application; Figure 2 is another flow diagram of the parallel query method for a relational database provided by an embodiment of the present application; Figure 3 is yet another flow diagram of the parallel query method for a relational database provided by an embodiment of the present application; Figure 4 is the splitting diagram of the memory block provided by an embodiment of the present application; Figure 5 is yet another flow diagram of the parallel query method for a relational database provided by an embodiment of the present application; Figure 6 is one of the structural diagrams of the parallel query device for a relational database provided by an embodiment of the present application; Figure 7 is another structural diagram of the parallel query device for a relational database provided by an embodiment of the present application; Figure 8It is the third schematic structural diagram of the parallel query device for the relational database provided by the embodiment of the present application; Figure 9 It is the schematic structural diagram of the electronic device provided by the embodiment of the present application. Detailed implementation manners

[0016] Next, the technical solutions in the embodiments of the present application will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the protection scope of the present application.

[0017] The terms "first", "second", etc. in the embodiments of the present application are used to distinguish similar objects and do not necessarily describe a specific order or sequence. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily limit to those clearly listed steps or units, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices. In addition, the use of "and / or" in the present application represents at least one of the connected objects. For example, A and / or B and / or C represents seven situations including A alone, B alone, C alone, A and B existing together, B and C existing together, A and C existing together, and A, B, and C existing together.

[0018] Please refer to Figure 1 , Figure 1 It is the schematic flowchart of the parallel query method for the relational database provided by the embodiment of the present application. Figure 1 The parallel query method for the relational database shown can be executed by a consumer stream thread, and the consumer stream thread and the producer stream thread are used for parallel query of the relational database.

[0019] As Figure 1 shown, the parallel query method for the relational database may include the following steps: Step 101, obtain first address information from a first shared circular queue, where the first address information is the address information of a first data block of the relational database in a memory block pool, and the memory block pool stores multiple data blocks with different byte counts, and the first data block is any one of the multiple data blocks.

[0020] Among them, the first shared circular queue can be shared between the consumer stream thread and the producer stream thread, that is, both the consumer stream thread and the producer stream thread can obtain the address information of the above multiple data blocks in the memory block pool from the first shared circular queue.

[0021] It should be noted that the first shared circular queue can be used to store the address information of multiple data blocks in the memory block pool, and the memory block pool can be used to store multiple data blocks and the data stored in each data block.

[0022] Step 102: Obtain the first data block from the memory block pool according to the first address information, where the first data block stores at least part of the tuple data.

[0023] Step 103: When the complete tuple data is obtained, release the first data block associated with the tuple data to the memory block pool.

[0024] Among them, the first data block associated with the tuple data can be understood as: at least part of the tuple data is stored in the first data block.

[0025] Among them, the number of consumer stream threads in the embodiments of the present application can be one or multiple. When the number of consumer stream threads is multiple, the above multiple consumer stream threads can execute each step in the embodiments of the present application in parallel. In this way, the efficiency of the above multiple consumer stream threads to obtain tuple data can be improved, that is, the query efficiency of the tuple data of the above multiple consumer stream threads is improved.

[0026] In the embodiments of the present application, through steps 101 to 103, only one step of obtaining the first data block needs to be executed, and when the complete tuple data is obtained, the first data block associated with the tuple data is released to the memory block pool, that is, one step of releasing the first data block is executed, thereby reducing the overhead of querying the tuple data in the first data block.

[0027] That is, the embodiments of the present application avoid the problem of frequent memory application and release during the distribution of tuple data, and accelerate the execution of parallel queries.

[0028] In addition, in the embodiments of the present application, when the consumer stream thread executes the query of the tuple data, the memory resources consumed only depend on the number of bytes that the first data block can store, and the number of bytes that the first data block can store is usually a fixed value and does not increase with the increase of the consumer stream thread or the producer stream thread, thereby saving memory resources.

[0029] It should be noted that the specific method for determining whether the complete tuple data is obtained is not limited herein.

[0030] As an optional implementation manner, the first data block stores a target number of bytes, and the target number of bytes is used to represent the number of bytes of the complete tuple data. When the complete tuple data is obtained, releasing the first data block to the memory block pool includes: Determine whether the number of bytes of the tuple data stored in the first data block is equal to the target number of bytes; In the case where it is determined that the number of bytes of the tuple data stored in the first data block is equal to the target number of bytes, obtain the tuple data stored in the first data block, and determine that complete tuple data is obtained from the first data block; In the case where it is determined that complete tuple data is obtained from the first data block, release the first data block to the memory block pool.

[0031] Among them, in the first data block, the target number of bytes and the tuple data can be stored adjacent to each other.

[0032] In the embodiments of the present application, when it is determined that the number of bytes of the tuple data stored in the first data block is equal to the target number of bytes, it can be determined that there is complete tuple data stored in the first data block. Therefore, after complete tuple data is obtained from the first data block, the first data block can be released to the memory block pool, that is, the first data block is released, so that the first data block can participate in the storage of other subsequent tuple data, improving the utilization rate of the first data block.

[0033] Optionally, when the consumer flow thread receives tuple data from the first data block, first read the target number of bytes of integer size from the first data block and save it into an integer variable of the expected length. This integer variable represents the number of bytes of the complete tuple data. When the number of bytes of the tuple data stored in the first data block is equal to the target number of bytes, it can be determined that there is complete tuple data stored in the first data block.

[0034] It should be noted that, optionally, after complete tuple data is obtained from the first data block, the consumer flow thread will exit the state of receiving tuple data from the first shared circular queue and enter the state of processing the above-mentioned tuple data until the tuple data is processed, and then enter the state of receiving tuple data again, that is, continue to read tuple data from the first shared circular queue.

[0035] As an optional implementation manner, after determining whether the number of bytes of the tuple data stored in the first data block is equal to the target number of bytes, the method further includes: In the case where it is determined that the number of bytes of the tuple data stored in the first data block is less than the target number of bytes, obtain the tuple data stored in the first data block; Release the first data block to the memory block pool, and obtain a second data block from the memory block pool; Obtain target tuple data from the second data block, where the target tuple data is the tuple data other than the tuple data stored in the first data block in the complete tuple data.

[0036] In the embodiments of the present application, when it is determined that the number of bytes of the tuple data stored in the first data block is less than the target number of bytes, it indicates that the first data block does not store complete tuple data. Therefore, after obtaining the tuple data stored in the first data block, a second data block can be obtained from the memory block pool, and the target tuple data can be obtained from the second data block, so as to ensure that complete tuple data can be obtained.

[0037] It should be noted that, optionally, when there is no second data block in the memory block pool, the consumer stream thread will attempt to obtain a data block from other shared circular queues except the first shared circular queue to read the tuple data. When there is no data block in all shared circular queues and the consumer stream thread has not read out complete tuple data, the consumer stream thread enters a blocked waiting state until it is notified and awakened by the producer stream thread, and then attempts to obtain a new data block from the shared circular queue again, and then reads the tuple data.

[0038] As an optional implementation manner, obtaining the target tuple data from the second data block includes: Obtaining a cache memory block, where the number of bytes of the cache memory block is twice the target number of bytes; Obtaining the target tuple data from the second data block and storing the target tuple data in the cache memory block.

[0039] In the embodiments of the present application, the target tuple data is stored in the cache memory block, and the number of bytes of the cache memory block is twice the target number of bytes, so as to ensure that the target tuple data can be stored, that is, the storage effect of the target tuple data is enhanced.

[0040] It should be noted that, optionally, the tuple data obtained from the first data block can also be stored in the above cache memory block. In this way, the tuple data obtained from the first data block and the target tuple data can be stored in the cache memory block, ensuring the integrity of the storage of the tuple data, so as to facilitate subsequent invocation of the above complete tuple data.

[0041] Optionally, obtaining the cache memory block includes: obtaining the cache memory block when it is determined that no cache data block for storing the target tuple data has been applied. In this way, the accuracy of obtaining the cache memory block can be ensured, avoiding the phenomenon of repeated acquisition of the cache memory block caused by obtaining multiple cache memory blocks, and reducing the phenomenon of waste of storage resources caused by repeated acquisition of the cache memory block.

[0042] Additionally, optionally, when it is determined that a preset cache data block for storing target tuple data has been pre-fetched before fetching the cache memory block, it is determined whether the stored byte count of the preset cache data block is greater than the byte count of the complete tuple data (i.e., the target byte count). If the stored byte count of the preset cache data block is less than the target byte count, the consumer stream thread will release the preset cache memory block and re-fetch the cache memory block; if the stored byte count of the preset cache data block is greater than or equal to the target byte count, the preset cache memory block will continue to be used and the fetching of the cache memory block will stop.

[0043] It should be noted that, optionally, when the consumer stream thread obtains the complete tuple data, it can continue to obtain the address information of the new data block from the first shared circular queue for obtaining the data in the new data block. However, when there is no address information of the new data block available in the first shared circular queue, the consumer stream thread will attempt to obtain the address information of the data block from other shared circular queues except the first shared circular queue to read the new tuple data. When there is no address information of the data block in all shared circular queues and the consumer stream thread has not read out the complete tuple data, the consumer stream thread enters the blocked waiting state until it is notified and awakened by the producer stream thread, and then attempts to obtain the address information of the new data block from the shared circular queue again, and then reads the tuple data.

[0044] See Figure 2 , Figure 2 A parallel query method for a relational database provided by an embodiment of the present application, which is applied to a producer stream thread. The producer stream thread and the consumer stream thread are used for parallel query of the relational database. As Figure 2 shown, the method includes the following steps: Step 201: Obtain a first data block of the relational database from the memory block pool. Multiple data blocks with different byte counts are stored in the memory block pool, and the first data block is any one of the multiple data blocks. The first data block stores at least part of the tuple data.

[0045] Step 202: Store at least part of the tuple data in the first data block, and release the first data block storing at least part of the tuple data to the memory block pool.

[0046] Step 203: Send first address information to the first shared circular queue. The first address information is the address information of the first data block in the memory block pool.

[0047] It should be noted that the embodiment of the present application is Figure 1The embodiments shown are applied to corresponding embodiments of the producer stream thread. The difference between the two lies in the different execution entities. Therefore, the technical features in the embodiments of the present application can be referred to the relevant descriptions in the above Figure 1 shown embodiments and have the same beneficial technical effects.

[0048] In the embodiments of the present application, the first address information is sent to the first shared circular queue. The first address information is the address information of the first data block in the memory block pool. In this way, the consumer stream thread can obtain the first address information from the first shared circular queue, and then obtain the tuple data in the first data block, improving the efficiency of the consumer stream thread to obtain tuple data.

[0049] It should be noted that since the producer stream thread actively obtains the first data block of the relational database from the memory block pool, the stronger the production capacity of the producer stream thread, the more tuple data is produced, and the higher the frequency of obtaining the first data block. Therefore, more first data blocks can be obtained to transmit tuple data; while the weaker the production capacity of the producer stream thread, the fewer tuple data is produced, and the lower the frequency of obtaining the first data block. In this way, the first data blocks occupied by the above producer stream thread can be allocated to other producer stream threads with strong production capacity, improving the utilization rate of the first data block and making the allocation of the first data block more reasonable.

[0050] As an optional implementation manner, the obtaining of the first data block of the relational database from the memory block pool includes: Obtaining a target number of bytes, where the target number of bytes is used to represent the number of bytes of the complete tuple data; Obtaining the first data block from the memory block pool according to the target number of bytes, where the number of bytes of the first data block matches the target number of bytes, or the number of bytes of the first data block is less than the target number of bytes.

[0051] In the embodiments of the present application, by obtaining the target number of bytes and obtaining the first data block from the memory block pool according to the target number of bytes, if the number of bytes of the first data block matches the target number of bytes, it can be determined that the obtained first data block can store the complete tuple data, avoiding the need to use multiple data blocks to separately store the complete tuple data, so that the consumer stream thread can obtain the complete tuple data from the first data block at one time, that is, improving the efficiency of the consumer stream thread to obtain the complete tuple data. If the number of bytes of the first data block is less than the target number of bytes, it can be determined that the obtained first data block does not store the complete tuple data. Therefore, after obtaining part of the tuple data stored in the first database, the second data block can be obtained, and the remaining part of the tuple data can be continuously obtained from the second data block, so as to obtain the complete tuple data.

[0052] The following uses a specific embodiment to illustrate the above embodiments.

[0053] 1. After the producer flow thread produces a tuple of data, it first obtains the number of bytes of the tuple data, saves the number of bytes of the tuple data into an integer variable representing the length, and then copies the value of this integer variable byte by byte into the data block. If the producer flow thread does not have an available data block, it can obtain a new data block from the memory block pool for copying the tuple data.

[0054] It should be noted that if the data block obtained by the producer flow thread does not have enough space to copy the tuple data, the producer flow thread copies a part of the data into the data block (for example: the data block can be the first data block). After filling the data block, it enqueues the address information of the data block into the shared circular queue (for example: the shared circular queue can include the first shared circular queue), and applies to the memory block pool for a new data block (for example: the data block can be the second data block) and then copies the remaining part of the tuple data.

[0055] If the producer flow thread fails to apply for a new data block from the memory block pool, that is, the memory block pool does not have an available data block, it enters the blocked waiting state until the consumer flow thread returns the data block and the memory block pool has a new free data block. At this time, the memory block pool will notify and wake up the producer flow thread, and then retry to copy the remaining bytes into the new memory.

[0056] 2. After the producer flow thread successfully copies the tuple data into the data block, it continues to copy the tuple data into the data block. After the producer flow thread obtains the available space size of the current data block (that is, the number of bytes that can be used to store data), it will compare it with the size of the tuple data that still needs to be copied (that is, the number of bytes of the tuple data).

[0057] If the available space of the data block is less than the size of the tuple data that still needs to be copied, it copies the tuple data with the same size as the available space of the data block into the data block. After the copy is completed, the producer flow thread will add the address of this data block to the shared circular queue, notify the consumer flow thread at the same time, and apply to the memory block pool for a new data block to copy the remaining tuple data into the new data block again. If the producer flow thread fails to apply for a new data block from the memory block pool, that is, the memory block pool does not have an available data block, it enters the blocked waiting state until the consumer flow thread returns the data block and the memory block pool has a new free data block. At this time, the memory block pool will notify and wake up the producer flow thread, and then retry to copy the remaining tuple data into the new data block.

[0058] If the available space in the data block is greater than the size of the tuple data that still needs to be copied, then the tuple data that still needs to be copied is copied into the data block at once to complete the transmission process of the tuple data. The producer stream thread will exit the state of transmitting tuple data and enter the state of producing tuple data. Until a new tuple data is produced, it will enter the tuple transmission state again to transmit the new tuple data.

[0059] See Figure 3 , Figure 3 which is a parallel query method for a relational database provided by an embodiment of the present application. As Figure 3 shown, the parallel query method for a relational database includes the following steps: Step 301, obtain a first parameter of the consumer stream thread and a second parameter of the producer stream thread; Step 302, determine a target data block of the relational database according to the first parameter and the second parameter; Step 303, split or merge the target data block into multiple data blocks with different byte numbers, and the multiple data blocks include a first data block; Step 304, store the multiple data blocks in a memory block pool, and store the address information of the multiple data blocks in the memory block pool in a first shared circular queue; Wherein, the consumer stream thread uses the Figure 1 parallel query method for a relational database in the embodiment shown to perform parallel query of the relational database, and the producer stream thread uses the Figure 2 parallel query method for a relational database in the embodiment shown to perform parallel query of the relational database.

[0060] Wherein, the steps executed in the embodiment of the present application can be understood as the initialization of the data block, and the execution subject of the embodiment of the present application can be the memory block pool of the relational database or the data block free queue of the memory block pool of the relational database. It should be noted that the technical features in the embodiment of the present application can refer to the corresponding descriptions in the Figure 1 embodiment shown above and Figure 2 the embodiment shown, and have corresponding technical effects, which will not be elaborated herein.

[0061] The embodiments of the present application can allocate target data blocks according to the first parameter of the consumer stream thread and the second parameter of the producer stream thread, so that the target data blocks better meet the requirements of the consumer stream thread and the producer stream thread, and avoid the phenomenon that the target data blocks are too large or too small in memory, resulting in the mismatch between the target data blocks and the requirements of the consumer stream thread and the producer stream thread. At the same time, the storable byte count of the determined target data blocks can also be dynamically adjusted according to the first parameter of the consumer stream thread and the second parameter of the producer stream thread, and the storable byte count can be understood as the size of the available memory.

[0062] It should be noted that the types of the first parameter of the consumer stream thread and the second parameter of the producer stream thread are not limited here. Optionally, the first parameter of the consumer stream thread may include the byte count of the tuple data required by the consumer stream thread, and the second parameter of the producer stream thread may include the byte count of the tuple data required by the producer stream thread. Alternatively, the first parameter may include the capability parameter of the consumer stream thread for processing tuple data, and the second parameter may include the capability parameter of the producer stream thread for producing tuple data. The above capability parameters may include processing rate and the like.

[0063] The splitting or merging of the target data block into multiple data blocks with different byte counts can be seen in the following description: the target data block can be split into multiple data blocks, or alternatively, the target data block can be merged with other data blocks to obtain multiple different data blocks.

[0064] It should be noted that when the storable byte count of the target data block does not match the byte count of the tuple data required by the consumer stream thread and the producer stream thread, the target data block can be split or merged according to the byte count of the tuple data.

[0065] The above embodiments will be illustrated by a specific embodiment below. Specifically, it may include the following steps: 1. Before starting the producer stream thread, apply for a data block with a storable byte count of M (i.e., the target data block, and the size of the target data block is not specifically limited here and can be determined according to the size of the tuple data required by the producer stream thread and the consumer stream thread) in the memory block pool for a group of producer stream threads and consumer stream threads, and then split this data block into X groups of data blocks with storable byte counts (such as decomposing 1MB of memory into four types: 64KB, 16KB, 4KB, and 1KB). The storable byte counts of the different split data blocks are in a linear proportional relationship.

[0066] 2. The process of splitting the above target data block into X groups of data blocks starts from the data block with the maximum storable bytes, as follows: First, split the target data block entirely into multiple data blocks with the maximum storable bytes, and then recursively split one-Xth of them into data blocks with the next storable bytes until the data blocks with the minimum storable bytes are split out.

[0067] For example, split it into multiple data blocks with a storable byte count of 64 KB, and then split one-fourth of the data blocks with a storable byte count of 64 KB again into data blocks with a smaller storable byte count, that is, 16 KB data blocks; finally, keep repeating this process until the data blocks with the smallest storable byte count are split. When the storable byte count of the target data block is 1 MB, it will be split into 12 data blocks with a storable byte count of 64 KB, 12 data blocks with a storable byte count of 16 KB, 12 data blocks with a storable byte count of 4 KB, and 16 data blocks with a storable byte count of 1 KB.

[0068] 3. Enqueue the data blocks with the same storable byte count into the corresponding data block free queue, and the data blocks can also be referred to as memory blocks. As Figure 4 shown, there are a total of 4 memory block free queues, and these 4 queues form a memory block pool, which can allocate appropriate memory blocks to the producer stream thread and recycle the memory blocks from the consumer stream thread into the corresponding data block free queue.

[0069] It should be noted that Figure 4 the memory blocks in can be understood as the first data blocks that have not stored tuple data in the embodiments of the present application, and as Figure 4 shown in, the number of memory blocks can be multiple, and multiple memory blocks can be arranged in order. The memory block in the first position can be referred to as the head memory block, and the memory block in the last position can be referred to as the end memory block.

[0070] 4. When the producer stream thread sends tuple data, it applies to the memory block pool for a data block to copy the tuple data according to the byte count of the tuple data. The memory block pool finds the data block free queue with a storable byte count matching the tuple data according to the byte count of the tuple data that the producer stream thread needs to send, obtains an idle data block from this data block free queue, and then allocates this idle data block to the producer stream thread to transmit the tuple data.

[0071] If there is no data block in the data block free queue with a storable byte count (assumed to be S), it will keep trying to find a data block from the data block free queue with a smaller storable byte count until an idle data block is found.

[0072] If there is no free data block even in the free data block queue with the smallest number of storable bytes, an attempt will be made to obtain a free data block from the free data block queue with the largest number of storable bytes. If no free data block is found until the free queue one level larger than S, that is, there is no free data block in the data block pool, the producer stream thread will be notified that no data block is available. If a free data block is found, the data block will be split. The splitting process can refer to the above splitting process, but here it is split until the storable byte count reaches S. Then the split data blocks are added to the corresponding free data block queues, and a data block of size S is allocated to the producer stream thread.

[0073] For example: when S is 4KB, if a data block is found in the 64KB free data block queue, the data block is split into 4 data blocks of 16KB. Then, one quarter of it, that is, a 16KB data block, is further split into 4KB data blocks. Eventually, a 64KB data block will be split into 3 data blocks of 16KB and 4 data blocks of 4KB, and then a 4KB data block is returned to the producer stream thread.

[0074] 5. After the consumer stream thread obtains the first data block from the first shared circular queue, it dequeues the address of the first data block from the first shared circular queue, and then reads tuple data from the first data block until all the data in the first data block is read. Then, the first data block is added to the free data block queue with the corresponding storable byte count.

[0075] It should be noted that, for a more complete illustration of the above embodiments, refer to Figure 5 , Figure 5 which is the flowchart of the parallel query method for the relational database provided by the embodiments of the present application. Specifically, refer to Figure 5 for each step. In this way, the embodiments of the present application can optimize the data distribution mechanism module for the relational database parallel query technology, reduce the memory resource occupation during the parallel query execution process, and accelerate the parallel query execution speed.

[0076] It should be noted that, Figure 5 the shared circular queue in Figure 5 can be understood as the first shared circular queue in the above embodiments, Figure 5 the memory block in Figure 5As shown, the number of data blocks can be multiple, and the multiple data blocks can be arranged in sequence. The data block arranged in the first position can be called the data block of the head, and the data block arranged in the last position can be called the data block of the end.

[0077] See Figure 6 , Figure 6 is a structural diagram of a parallel query device for a relational database provided by an embodiment of the present application. Figure 6 The parallel query device of the relational database shown is applied to a consumer stream thread. The consumer stream thread and the producer stream thread are used for parallel query of the relational database. The parallel query device 600 of the relational database includes: A first acquisition module 601, configured to acquire first address information from a first shared circular queue. The first address information is the address information of a first data block of the relational database in a memory block pool. Multiple data blocks with different byte counts are stored in the memory block pool, and the first data block is any one of the multiple data blocks; A second acquisition module 602, configured to acquire the first data block from the memory block pool according to the first address information. The first data block stores at least part of the tuple data; A first release module 603, configured to release the first data block associated with the tuple data to the memory block pool when complete tuple data is acquired.

[0078] As an optional implementation manner, a target byte count is stored in the first data block. The target byte count is used to represent the byte count of the complete tuple data. The first release module 603 includes: A first determination sub-module, configured to determine whether the byte count of the tuple data stored in the first data block is equal to the target byte count; A second determination sub-module, configured to acquire the tuple data stored in the first data block and determine that complete tuple data is acquired from the first data block when it is determined that the byte count of the tuple data stored in the first data block is equal to the target byte count; A first release sub-module, configured to release the first data block to the memory block pool when it is determined that complete tuple data is acquired from the first data block.

[0079] As an optional implementation manner, the first release module 603 further includes: A first acquisition sub-module, configured to acquire the tuple data stored in the first data block when it is determined that the byte count of the tuple data stored in the first data block is less than the target byte count; A second acquisition sub-module, configured to release the first data block to the memory block pool and acquire a second data block from the memory block pool; A third acquisition sub-module, configured to acquire target tuple data from the second data block, where the target tuple data is the tuple data in the complete tuple data except the tuple data stored in the first data block.

[0080] As an alternative implementation, the third acquisition sub-module includes: A first acquisition unit, configured to acquire a cached memory block, where the number of bytes of the cached memory block is twice the number of target bytes; A second acquisition unit, configured to acquire target tuple data from the second data block and store the target tuple data in the cached memory block.

[0081] The parallel query device 600 of the relational database can implement each process in the method embodiments of this application Figure 1 and achieve the same beneficial effects. To avoid repetition, details are not described here again.

[0082] Refer to Figure 7 , Figure 7 which is a structural diagram of the parallel query device of the relational database provided in the embodiments of this application. Figure 7 The parallel query device of the relational database shown is applied to a producer stream thread, and the producer stream thread and a consumer stream thread are used for parallel query of the relational database. The parallel query device 700 of the relational database includes: A third acquisition module 701, configured to acquire a first data block of the relational database from a memory block pool, where multiple data blocks with different numbers of bytes are stored in the memory block pool, the first data block is any one of the multiple data blocks, and the first data block stores at least part of the tuple data; A second release module 702, configured to store at least part of the tuple data in the first data block and release the first data block storing at least part of the tuple data to the memory block pool; A sending module 703, configured to send first address information to a first shared circular queue, where the first address information is the address information of the first data block in the memory block pool.

[0083] As an alternative implementation, the third acquisition module 701 includes: A fourth acquisition sub-module, configured to acquire a target number of bytes, where the target number of bytes is used to represent the number of bytes of the complete tuple data; The fifth acquisition sub-module is configured to acquire the first data block from the memory block pool according to the target number of bytes, where the number of bytes of the first data block matches the target number of bytes, or the number of bytes of the first data block is less than the target number of bytes.

[0084] The parallel query device 700 of the relational database can implement each process in the method embodiments of this application Figure 2 and achieve the same beneficial effects. To avoid repetition, details are not described here again.

[0085] See Figure 8 , Figure 8 which is a structural diagram of the parallel query device of the relational database provided in the embodiments of this application. The parallel query device 800 of the relational database includes: A fourth acquisition module 801, configured to acquire a first parameter of a consumer stream thread and a second parameter of a producer stream thread; A first determination module 802, configured to determine a target data block of the relational database according to the first parameter and the second parameter; A splitting or merging module 803, configured to split or merge the target data block into multiple data blocks with different numbers of bytes, and the first data block is included in the multiple data blocks; A storage module 804, configured to store the multiple data blocks in a memory block pool, and store the address information of the multiple data blocks in the memory block pool in a first shared circular queue; wherein, the consumer stream thread uses Figure 1 the parallel query method of the relational database in the embodiment shown in Figure 2 to perform parallel query of the relational database, and the producer stream thread uses

[0086] The parallel query device 800 of the relational database can implement each process in the method embodiments of this application Figure 3 and achieve the same beneficial effects. To avoid repetition, details are not described here again.

[0087] The embodiments of this application further provide an electronic device. Please refer to Figure 9 , the electronic device may include a processor 901, a memory 902, and a program 9021 stored in the memory 902 and executable on the processor 901. When the electronic device is applied to the consumer stream thread, when the program 9021 is executed by the processor 901, it can implement Figure 1 any step in the corresponding method embodiment and achieve the same beneficial effects, which are not described here again; when the electronic device is applied to the producer stream thread, when the program 9021 is executed by the processor 901, it can implement Figure 2Any steps in the method embodiments shown and the same beneficial effects achieved will not be elaborated here. When the electronic device is applied to the memory block pool of a relational database, when the program 9021 is executed by the processor 901, it can achieve Figure 3 Any steps in the method embodiments shown and the same beneficial effects achieved will not be elaborated here.

[0088] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the program can be stored in a readable medium. The embodiments of the present application also provide a readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it can implement the above Figure 1 、 Figure 2 or Figure 3 any steps in the corresponding method embodiments, and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0089] The storage medium mentioned above, such as a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disc, etc.

[0090] The embodiments of the present application also provide a computer program product, including computer instructions. When the computer instructions are executed by a processor, it can implement the above Figure 1 、 Figure 2 or Figure 3 any steps in the corresponding method embodiments, and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0091] The above is the preferred implementation manner of the embodiments of the present application. It should be noted that for those of ordinary skill in the art, without departing from the principle described in the present application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present application.

Claims

1. A parallel query method for a relational database, which is applied to a consumer stream thread. The consumer stream thread and a producer stream thread are used for parallel query of the relational database, and is characterized in that, The method includes: Obtain first address information from a first shared circular queue, where the first address information is the address information of a first data block of a relational database in a memory block pool, multiple data blocks with different byte counts are stored in the memory block pool, and the first data block is any one of the multiple data blocks; Obtain the first data block from the memory block pool according to the first address information, and the first data block stores at least part of the tuple data; When complete tuple data is obtained, release the first data block associated with the tuple data to the memory block pool.

2. The method according to claim 1, characterized in that, The first data block stores a target byte count, where the target byte count is used to represent the byte count of the complete tuple data. When complete tuple data is obtained, releasing the first data block to the memory block pool includes: Determine whether the byte count of the tuple data stored in the first data block is equal to the target byte count; When it is determined that the byte count of the tuple data stored in the first data block is equal to the target byte count, obtain the tuple data stored in the first data block, and determine that complete tuple data is obtained from the first data block; When it is determined that complete tuple data is obtained from the first data block, release the first data block to the memory block pool.

3. The method according to claim 2, wherein After determining whether the byte count of the tuple data stored in the first data block is equal to the target byte count, the method further includes: When it is determined that the byte count of the tuple data stored in the first data block is less than the target byte count, obtain the tuple data stored in the first data block; Release the first data block to the memory block pool, and obtain a second data block from the memory block pool; Obtain target tuple data from the second data block, where the target tuple data is the tuple data in the complete tuple data other than the tuple data stored in the first data block.

4. The method according to claim 3, wherein The obtaining the target tuple data from the second data block includes: Obtain a cache memory block, where the byte count of the cache memory block is twice the target byte count; Obtain the target tuple data from the second data block, and store the target tuple data in the cache memory block.

5. A parallel query method for a relational database, which is applied to a producer stream thread. The producer stream thread and the consumer stream thread are used for parallel query of the relational database, and is characterized in that The method includes: Obtain a first data block of a relational database from a memory block pool, multiple data blocks with different byte counts are stored in the memory block pool, the first data block is any one of the multiple data blocks, and the first data block stores at least part of the tuple data; Store at least part of the tuple data in the first data block, and release the first data block storing at least part of the tuple data to the memory block pool; Send the first address information to a first shared circular queue, where the first address information is the address information of the first data block in the memory block pool.

6. The method according to claim 5, characterized in that, The obtaining the first data block of a relational database from a memory block pool includes: Obtain a target byte count, where the target byte count is used to represent the byte count of the complete tuple data; Obtain the first data block from the memory block pool according to the target number of bytes, where the number of bytes of the first data block matches the target number of bytes, or the number of bytes of the first data block is less than the target number of bytes.

7. A parallel query method for a relational database, characterized in that, Including: Obtain the first parameter of the consumer stream thread and the second parameter of the producer stream thread; Determine the target data block of the relational database according to the first parameter and the second parameter; Split or merge the target data block into multiple data blocks with different numbers of bytes, and the first data block is included in the multiple data blocks; Store the multiple data blocks into the memory block pool, and store the address information of the multiple data blocks in the memory block pool into the first shared circular queue; Wherein, the consumer stream thread performs parallel query of the relational database by using the parallel query method of the relational database described in any one of claims 1 to 4, and the producer stream thread performs parallel query of the relational database by using the parallel query method of the relational database described in claim 5 or 6.

8. A parallel query device for a relational database, characterized in that Applied to a consumer stream thread, the consumer stream thread and the producer stream thread are used for parallel query of a relational database. The parallel query device of the relational database includes: A first acquisition module, configured to acquire first address information from a first shared circular queue, where the first address information is the address information of the first data block of the relational database in the memory block pool, and the memory block pool stores multiple data blocks with different numbers of bytes, and the first data block is any one of the multiple data blocks; A second acquisition module, configured to acquire the first data block from the memory block pool according to the first address information, and the first data block stores at least part of the tuple data; A first release module, configured to release the first data block associated with the tuple data to the memory block pool when complete tuple data is acquired.

9. A parallel query device for a relational database, characterized in that, Applied to a producer stream thread, the producer stream thread and the consumer stream thread are used for parallel query of a relational database. The parallel query device of the relational database includes: A third acquisition module, configured to acquire the first data block of the relational database from the memory block pool, where the memory block pool stores multiple data blocks with different numbers of bytes, and the first data block is any one of the multiple data blocks, and the first data block stores at least part of the tuple data; A second release module, configured to store at least part of the tuple data into the first data block, and release the first data block storing at least part of the tuple data to the memory block pool; A sending module, configured to send first address information to the first shared circular queue, where the first address information is the address information of the first data block in the memory block pool.

10. A parallel query device for a relational database, characterized in that, Including: A fourth acquisition module, configured to acquire the first parameter of the consumer stream thread and the second parameter of the producer stream thread; A first determination module, configured to determine the target data block of the relational database according to the first parameter and the second parameter; A splitting or merging module, configured to split or merge the target data block into multiple data blocks with different numbers of bytes, and the first data block is included in the multiple data blocks; A storage module, configured to store the multiple data blocks into a memory block pool, and store the address information of the multiple data blocks in the memory block pool into a first shared circular queue; Wherein, the consumer flow thread performs a parallel query of the relational database by using the parallel query method of the relational database according to any one of claims 1 to 4, and the producer flow thread performs a parallel query of the relational database by using the parallel query method of the relational database according to claim 5 or 6.

11. An electronic device, comprising: A memory, a processor, and a program stored on the memory and executable on the processor; characterized in that the processor is configured to read the program in the memory to implement the steps in the parallel query method of the relational database according to any one of claims 1 to 4; or, to implement the steps in the parallel query method of the relational database according to any one of claims 5 to 6, or to implement the steps in the parallel query method of the relational database according to claim 7.

12. A readable storage medium for storing a program, characterized in that, When the program is executed by the processor, it implements the steps in the parallel query method of the relational database according to any one of claims 1 to 4; or, it implements the steps in the parallel query method of the relational database according to any one of claims 5 to 6, or it implements the steps in the parallel query method of the relational database according to claim 7.

13. A computer program product, characterized in that, It includes computer instructions, when the computer instructions are executed by the processor, it implements the steps in the parallel query method of the relational database according to any one of claims 1 to 4, or when the computer instructions are executed by the processor, it implements the steps in the parallel query method of the relational database according to any one of claims 5 to 6, or when the computer instructions are executed by the processor, it implements the steps in the parallel query method of the relational database according to claim 7.

Images