A data acceleration method, system and storage medium in a large load scenario

Abstract

The application discloses a data acceleration method and system in a large-load scene and a storage medium, and comprises the following steps: a tree model is established, the comparison tree model comprises a tree for comparison, a global flag bit is arranged in the tree, and a flag bit is arranged in a non-leaf node; to-be-processed data is sequentially inserted into each empty leaf node of the tree, parent nodes are used as comparison sites for pairwise comparison, large data is left in the comparison site nodes, smaller data continues to participate in comparison in the direction of the parent nodes, after comparison, the smallest data is output from the root node to outside the tree, the leaf node where the smallest data is located is emptied, and the global flag bit of the tree is updated according to the flag bit of the output data; and the next to-be-processed data is transmitted into the only empty leaf node of the tree. Through flag bit tournament sorting, only an initial insertion stage and an evaluation stage are entered once at the beginning of sorting, and a stage is entered once when the to-be-processed data is exhausted, so that the algorithm will not cause performance decline due to frequent stage switching.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a data acceleration method, system and storage medium in a large-load scenario. BACKGROUND

[0002] In modern society, the sharp increase in data volume causes the data managed by a database management system to increase substantially. The time required for performing database operations also becomes longer accordingly. In order to improve user experience, some database systems begin to use dedicated hardware to accelerate database operations. The dedicated hardware used to accelerate database operations includes GPUs, FPGAs and the like.

[0003] As an important part of a database query, a join operation consumes a large amount of time in the execution process, and acceleration thereof can shorten the execution time of many database queries. According to the research status of accelerating join operations on FPGAs, current research work mainly focuses on the acceleration of merge joins and hash joins. In the research on using FPGAs to accelerate basic operations of databases, the main accelerated operations studied include predicate evaluation, sorting, projection, join and clustering operations and the like. However, when the data stream length in a data table is greater than the width byte of a data stream transmitted in an FPGA, the transmission time is increased. In this scenario, multiple processing units cannot be used to process the new data simultaneously. This scenario is a large-load scenario. In the large-load scenario, the FPGA needs a sorting module suitable for this application scenario to save hardware resources on the FPGA and improve the sortable sequence length. SUMMARY

[0004] In view of the above defects or deficiencies, the present application aims to provide a data acceleration method, system and storage medium in a large-load scenario.

[0005] To achieve the above object, the technical scheme of the present application is as follows:

[0006] A data acceleration method in a large-load scenario, comprising:

[0007] 1) constructing a comparison tree model, wherein the comparison tree model comprises a tree for comparison, a global flag bit is arranged in the tree, and a flag bit is arranged in a non-leaf node;

[0008] 2) inserting to-be-processed data into a leaf node in the comparison tree model, comparing, outputting the smallest data, and updating the global flag bit according to the flag bit of the output data;

[0009] 3) inserting new data and setting a flag bit of the new data;

[0010] 4) According to the flag bit, the data in the tree is compared with each other, the minimum data after this comparison is output, and the global flag bit is updated according to the flag bit of the output minimum data, and when the data to be sorted is sorted, the sorted data set is output.

[0011] The global flag bit and the flag bit are 1 bit, and the values of the global flag bit and the flag bit are 0 or 1.

[0012] The global flag bit and the flag bit are 1 bit, and the values of the global flag bit and the flag bit are 0 or 1.

[0013] In the initial insertion stage, the data to be processed is inserted into the empty leaf node of the tree in turn, the global flag bit is initialized to 0 at the beginning, and the flag bit of the data added is 0.

[0014] After each data is inserted into the leaf node, if its parent node is empty, the data is inserted into its parent node at the same time, if its parent node is not empty, 1 data comparison is performed in its parent node, the larger data is inserted into the node, and the smaller data continues to participate in comparison in the direction of the root node, finally, the data with the larger value is left in the root node, and the data with the smallest value is output.

[0015] According to the flag bit of the minimum data output, the value of the global flag bit is set to the value of the flag bit of the inverse bit.

[0016] The insertion of the new data and the setting of the flag bit of the new data specifically include:

[0017] The insertion of the new data and the setting of the flag bit of the new data specifically include:

[0018] The insertion of the new data and the setting of the flag bit of the new data specifically include:

[0019] After the new data is inserted, the continuous sorting stage is entered, and the data of two adjacent leaf nodes in the tree is compared, first the flag bit is compared, when the flag bits of two adjacent leaf nodes are not equal, the flag bit is equal to the global flag bit, which is large data, and the flag bit is not equal to the global flag bit, which is small data, the large data is inserted into the parent node, and the small data continues to be compared upward until the output data; when the flag bits of two adjacent leaf nodes are equal, the data is directly compared, the large data is inserted into the parent node, and the small data continues to be compared upward until the output data.

[0020] When the data to be sorted is sorted, the flushing stage is entered, and the input virtual key is used to empty the whole tree.

[0021] The input virtual key to empty the whole tree is specifically:

[0022] The virtual key is represented by a flag bit-1, and by inserting the virtual key, all the remaining data in the tree is output, and the sorting is completed.

[0023] A data acceleration system in a large load scenario, modeling unit, processing unit, insertion unit, and output unit, wherein:

[0024] The modeling unit is used to build a comparison tree model, the comparison tree model includes a tree for comparison, and a global flag bit is arranged in the tree, and the flag bit is arranged in the non-leaf node;

[0025] The processing unit is used to insert the data to be processed into the leaf node of the comparison tree model, and compare, output the smallest data, and update the global flag bit according to the flag bit of the output data;

[0026] The insertion unit is used to insert new data and set the flag bit of the new data;

[0027] The output unit is used to compare the data in the tree according to the flag bit, output the smallest data after comparison, update the global flag bit according to the flag bit of the output smallest data, and output the sorted data set after the data to be sorted is sorted.

[0028] A non-transitory computer readable storage medium, the non-transitory computer readable storage medium stores computer instructions, the computer instructions are used to make the computer execute the method according to the above.

[0029] Compared with the prior art, the beneficial effects of the present application are:

[0030] The present application provides a data acceleration method in a large load scenario, which is suitable for a large load scenario on FPGA, and through the flag bit tournament sorting algorithm, the data processing is always running in the continuous sorting stage, only once initial insertion stage and evaluation stage are entered at the beginning of sorting, and only once flushing stage is entered when the data to be sorted is exhausted. This design makes the algorithm not cause performance degradation due to frequent stage switching. At the same time, only 1 bit is attached to each data as a flag bit, so that the flag bit tournament sorting algorithm does not need to use multiple bits as additional coloring space for each data like the colored tournament sorting algorithm, so it is more hardware resource-saving, and the length of the sorted sequence is longer. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1is a flowchart of a data acceleration method in a large load scenario of the present application;

[0032] Figure 2 is an example illustration of the initial insertion stage, the evaluation stage, and the continuous sorting stage of the flag bit tournament sorting method of the present application;

[0033] Figure 3 is an example illustration of the continuous sorting stage of the flag bit tournament sorting method of the present application;

[0034] Figure 4 is an example illustration of the flushing stage of the flag bit tournament sorting method of the present application;

[0035] Figure 5 is a block diagram structure of the system of the present application. DETAILED DESCRIPTION

[0036] The present application will be described in detail below with reference to the drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, and not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative labor fall within the protection scope of the present application.

[0037] For data to be sorted, there are two cases: large load and not large load. For data with large load, a data stream with a certain width usually needs multiple clock cycles to transmit one data into the sorting module. The present application refers to this scenario as a large load scenario. For data with not large load, a data stream with a certain width can transmit multiple data into the sorting module in parallel each time, and the sorting module can use multiple processing units to process multiple new data at the same time. The present application refers to this scenario as a high parallel scenario. The present application mainly introduces data acceleration in a large load scenario in detail. In a database system, there is often a case where each data in a data table carries a large load. Therefore, a data stream with a width of p in an FPGA needs multiple clock cycles to transmit one data into a module. For example, each data in a data table is 400 bytes long, and on a data stream with a width of 16 bytes, 25 clock cycles are needed to transmit one data into a hardware module. In this scenario, multiple processing units cannot be used to process this new data at the same time. This scenario is referred to as a large load scenario in this paper. In a large load scenario, the FPGA needs a sorting module suitable for this application scenario to save hardware resources on the FPGA and improve the length of the sortable sequence. To meet this demand, the present application proposes a flag bit tournament sorting algorithm, which is implemented as a hardware module on an FPGA. The hardware implementation of the algorithm ensures the throughput while making the average length of the output sequence longer, reducing the consumption of hardware resources.

[0038] In order to accelerate the merge join operation on FPGA, a sorting module for accelerating the sorting phase is needed. In the present application, the sorting module is implemented by the tournament sorting method. Its function is to sort the input data into several ordered sequences. Then these ordered sequences need to be merged into a large ordered table. Therefore, the proposed flag tournament sorting method will be implemented as a sorting module embedded in the merge join architecture to accelerate the sorting phase of the merge join.

[0039] As shown in Figure 1 The present application provides a data acceleration method in a large load scenario, comprising:

[0040] S1, a comparison tree model is constructed, the comparison tree model comprises a tree for comparison, a global flag is arranged in the tree, and a flag is arranged in a non-leaf node

[0041] When the comparison tree model is constructed, the tree for comparison is first established, the global flag is arranged in the tree, the flag is arranged in the non-leaf node of the tree, and the initial value of the global flag and the flag is defined as 0; initially, 1 bit is attached to each data as a flag, and the flag is non-0 or 1. At the same time, a global flag is defined, which is also 1 or 0. The global flag indicates that when the algorithm needs to insert a key violation, it needs to attach a bit to it. For example, the last number output from the algorithm is 50, and then the number 25 is inserted into the algorithm, then 25 is the key violation, and the flag attached to the number 25 is equal to the global flag; if 75 is inserted into the algorithm, since 75 is greater than 50, 75 is a normal key, and then the flag attached to 75 is the inverse of the global flag, that is, if the global flag is 0, 1 should be attached; if the global flag is 1, 0 should be attached.

[0042] The defined global flag also has another function, that is, to indicate which flag of the number will be larger. For example, if the global flag is 1, the number with flag 1 is larger than the number with flag 0 in the comparison process of the tree. In addition, the global flag will change, and the algorithm will detect the flag of the output number every time the data is output from the tree. If the flag of the output number is 0, the global flag is set to 1; if the flag of the output number is 1, the global flag is set to 0.

[0043] In the tree, when two data are compared with each other, their flags are compared first, and the data with the flag equal to the global flag is larger. If the flags of the two data are the same, the data with the larger number is larger.

[0044] S2, the data to be processed is inserted into the leaf node of the comparison tree model, and comparison is performed, the smallest data is output, and the global flag is updated according to the flag of the output data;

[0045] Specifically, comprising:

[0046] In the initial insertion stage, the data to be processed is inserted into the empty leaf node of the tree in turn, and the global flag is initialized to 0 and the flag of the added data is 0 at the beginning;

[0047] After each data is inserted into the leaf node, if its parent node is empty, the data is inserted into its parent node at the same time, if its parent node is not empty, the data is compared in its parent node once, the larger data is inserted into the node, and the smaller data continues to participate in comparison towards the root node direction, finally, the data with larger value is left in the root node, and the data with the smallest value is outputted;

[0048] According to the flag of the outputted minimum data, the value of the global flag is set to the value of the inverted flag.

[0049] The insertion of the new data sets the flag of the new data, and specifically includes:

[0050] The insertion of the new data and the output of the minimum data are compared, when the inserted new data is greater than the outputted minimum data, the flag of the inserted new data is 1; when the inserted new data is smaller than the minimum data, the flag of the inserted new data is 0.

[0051] S3, inserting new data and setting the flag of the new data;

[0052] S4, comparing the data in the tree according to the flag, outputting the minimum data after the comparison, updating the global flag according to the flag of the outputted minimum data, and outputting the sorted data set after the sorting of the data to be sorted is completed.

[0053] The comparison of the data in the tree according to the flag, the output of the minimum data after the comparison, and the updating of the global flag according to the flag of the outputted minimum data include:

[0054] After the insertion of the new data, the continuous sorting stage is entered, the data of two adjacent leaf nodes in the tree is compared, first, the flags are compared, when the flags of two adjacent leaf nodes are not equal, the flag is equal to the global flag, which is the large data, the flag is not equal to the global flag, which is the small data, the large data is inserted into the parent node, and the small data continues to be compared upwards until the output data; when the flags of two adjacent leaf nodes are equal, the data is directly compared, the large data is inserted into the parent node, and the small data continues to be compared upwards until the output data.

[0055] The above steps are repeated until all data is inserted, and the sorted data set is outputted after the sorting of the data to be sorted is completed.

[0056] When the data to be sorted is sorted, the flushing stage is entered, and the dummy key is inputted to empty the whole tree.

[0057] The input of the dummy key to empty the whole tree is specifically:

[0058] The dummy key is represented by a flag bit -1, and by inserting the dummy key, all the remaining data in the tree is outputted, and the sorting is completed.

[0059] Exemplarily, the data set {49, 38, 65, 97, 76, 68, 50, 80, 15, 23} is inserted as an example for illustration.

[0060] As shown in Figure 2 , the middle is an example of the flag bit tournament sorting method. In the figure, the data set {49, 38, 65, 97, 76, 68, 50, 80, 15, 23} is inserted into the algorithm and waits for sorting.

[0061] Figure 2 In (a), the algorithm enters the initial insertion stage. Initially, the global flag bit is initialized to 0, and the data set {49, 38, 65, 97} is inserted into the tree. Since the current global flag bit is 0, the flag bit added to the data is also 0. Each data performs 1 data comparison at its parent node after being inserted into the leaf node. As shown in (a), 49 with a flag bit of 0 is greater than 38 with a flag bit of 0, so 49 is placed in the non-leaf node, and 38 continues to compare upwards; 65 with a flag bit of 0 is less than 97 with a flag bit of 0, so 97 is placed in the non-leaf node, and 65 continues to compare upwards.

[0062] In Figure 2 (b), the algorithm enters the evaluation stage. In the empty non-leaf nodes in the tree, data comparison is performed from bottom to top according to the above comparison method. As shown in Figure 2 (b), 49 with a flag bit of 0 is compared with 38, and 38 with a flag bit of 0 is outputted. The global flag bit is set to 1 of the inverse of 0.

[0063] Then, the continuous sorting stage is entered. The data set {76, 68, 50, 80, 15, 23} is inserted. Among them, {76, 68, 80} are normal keys, and the others are violation keys. The sorted data set {38, 49, 65, 68, 76, 80} is outputted. These processes are shown in Figure 2 (c), (d), (e), (f), and (g) in

[0064] As shown in Figure 2(c) As shown, the flag bit 0 of 76 is inserted into the empty leaf node where 38 was. At the parent node, the flag bit 0 of 76 is compared with 49, 76 is put in the node, and 49 is promoted to be compared with the flag bit 0 of 65. 65 is put in the root node, and the flag bit 0 of 49 is output.

[0065] As shown in (a), the data set {7, 33, 44} is inserted into the tree. The flag bit of 7 is 0, and the flag bit of 33 is 0. The flag bit of 44 is 1. Figure 3 As shown in (b), the flag bit of 7 is compared with the flag bit of 33. The flag bit of 44 is compared with the flag bit of 7. The flag bit of 33 is compared with the flag bit of 44. Finally, the flag bit of 7 is compared with the flag bit of 44. The flag bit of 44 is output. Figure 3 As shown in (c), the flag bit 0 of 76 is inserted into the empty leaf node where 38 was. At the parent node, the flag bit 0 of 76 is compared with 49, 76 is put in the node, and 49 is promoted to be compared with the flag bit 0 of 65. 65 is put in the root node, and the flag bit 0 of 49 is output. Figure 3 As shown in (d), the flag bit of 49 is compared with the flag bit of 65. The flag bit of 65 is compared with the flag bit of 49. Finally, the flag bit of 49 is output.

[0066] As shown in (e), the flag bit of 49 is compared with the flag bit of 65. The flag bit of 65 is compared with the flag bit of 49. Finally, the flag bit of 49 is output. Figure 3 As shown in (b) (c) (d) (e) (f), the data set {7, 55, 44, 33, 85} is inserted into the tree. Among them, {7, 44, 33} is a normal key, and the flag bit is marked as 0. The others are violation keys, and the flag bit is marked as 1.

[0067] As shown in (f), when the input data is completed, the algorithm enters the flushing stage, and the process of inputting the virtual key to empty the whole tree. Figure 4 As shown in (g), when the sorting of the data to be sorted is completed, the remaining data in the tree needs to be output. The method is to insert a virtual key to empty the data. Here, the virtual key is represented by the flag bit equal to -1 of the global flag bit. In the (a) subgraph, there is data {7, 33, 44} in the tree. By inserting a virtual key, the data set {7, 33, 44} is output in turn.

[0068] Figure 4 As shown in (h), when the sorting of the data to be sorted is completed, the remaining data in the tree needs to be output. The method is to insert a virtual key to empty the data. Here, the virtual key is represented by the flag bit equal to -1 of the global flag bit. In the (a) subgraph, there is data {7, 33, 44} in the tree. By inserting a virtual key, the data set {7, 33, 44} is output in turn.

[0069] Algorithm verification:

[0070] The flag bit tournament sorting method does not need to use multiple bits as an additional coloring space for each data like the colored tournament sorting algorithm. Instead, the new algorithm only needs to reserve 1 bit for each data, which saves FPGA hardware resources and reduces resource consumption. Let the algorithm consume memory M data to store data in leaf nodes, and M index to save the index of the data in non-leaf nodes. In addition, the colored tournament sorting algorithm needs to save the coloring bit of each data, which consumes memory M color , and the flag bit tournament sorting algorithm needs to save the flag bit of each data, which consumes memory M flag . Therefore, the total memory consumption M TOTAL_COLOR of the colored tournament sorting algorithm and the memory consumption M TOTAL_FLAG of the flag bit tournament sorting algorithm are as follows: ​

[0071] (1);

[0072] (2);

[0073] (3);

[0074] According to the formula (1), (2), (3), we can get:

[0075] (4);

[0076] On the other hand, the colored tournament sort needs to be cycled in the initial insertion stage, the evaluation stage, the continuous sorting stage, and the flushing stage. And each time the algorithm enters the initial insertion stage and the evaluation stage, the algorithm cannot output data. This affects the hardware module throughput to some extent, although it can be reduced by increasing the number of coloring bits, but it will increase the consumption of hardware resources. In contrast, the flag bit tournament sort proposed in the application only needs to enter the initial insertion stage and the evaluation stage once at the start of the algorithm, and enter the flushing stage once when the to-be-sorted data is exhausted. It runs in the continuous sorting stage at other times, and does not need to be cycled between stages, so it will not affect the performance.

[0077] Because data can be output in the continuous sorting stage and the flushing stage, but cannot be output in the initial insertion stage and the evaluation stage. So, let the time consumption of outputting data be Time output_data , the time consumption in the initial insertion stage and the evaluation stage be Time init . At the same time, let the time consumption of the flag bit tournament sort be Time flag , and the time consumption of the colored tournament sort be Time color . Then the time consumption of the colored tournament sort algorithm is:

[0078] (5);

[0079] That is, the time consumption of the colored tournament sort algorithm is equal to the sum of the time of outputting data and the time in the initial insertion stage and the evaluation stage. In contrast, since the flag bit tournament sort algorithm only enters the initial insertion stage and the evaluation stage once, when the to-be-sorted table is very long, the time consumed by the initial insertion stage and the evaluation stage can be ignored, so we have:

[0080] (6);

[0081] Therefore, when the length of the to-be-sorted table is the same and long enough, according to the formula (5) and (6), we can get:

[0082] (7) ;

[0083] Therefore, when the length of the table to be sorted is the same and long enough, the time consumption of the flag bit tournament sorting algorithm is less.

[0084] Let the throughput of the colored tournament sorting algorithm be T color , and the throughput of the flag bit tournament sorting algorithm be T flag . Because the product of time and module throughput is equal to the length of the table to be sorted, assuming that the length of the table to be sorted of the colored tournament sorting algorithm is equal to that of the flag bit tournament sorting algorithm, then:

[0085] (8) ;

[0086] According to formulas (7) and (8), it can be obtained that:

[0087] (9) ;

[0088] Therefore, when the length of the table to be sorted is very long, so that the colored tournament sorting algorithm enters the initial insertion stage and the evaluation stage multiple times, the performance of the flag bit tournament sorting algorithm is better.

[0089] In addition, for the flag bit tournament sorting algorithm which can sort data of length N at least, every logN clock cycles, a data is output from the algorithm. However, the colored tournament sorting algorithm and the original tournament sorting algorithm need NlogN clock cycles to output the next data because the algorithm suddenly enters the initial insertion stage and the evaluation stage. Therefore, the frequency of outputting data of the flag bit tournament sorting algorithm is more stable.

[0090] The tournament sorting algorithm is a tree structure sorting algorithm, which inputs a data to be sorted every time and outputs a sorted data, and the minimum sequence length sorted by the algorithm is related to the number of leaf nodes of the tree. The FPGA hardware implementation of the tournament sorting algorithm only needs a comparison unit, so it is very resource-saving.

[0091] In software, the hardware design of FPGA uses HLS (High Level Synthesis) technology to synthesize C / C++ language into RTL (Register-transfer Level) level hardware language. Under the Windows 10 system, the FPGA module with sorting, merging, connection, function selection and the like is designed by using Vivado HLS. Among them, Vivado is the FPGA development environment used in this paper. The overall design is created by using Vivado 2017.4, various hardware modules are integrated into an overall hardware architecture, and are synthesized, implemented, and finally made into a bit stream file to be downloaded to the FPGA.

[0092] The software system runs under the Linux system, and is connected with the FPGA hardware system through PCI-E to transmit data. The FPGA driver is modified from the official example driver provided by Xilinx.

[0093] In hardware, the specific configuration is as follows:

[0094] (1) CPU: Intel(R) Core i7-4790k CPU @4.0GHz.

[0095] (2) FPGA development kit: Xilinx VCU118 (the FPGA therein is XCVU9P flga-2104-2l-e)

[0096] (3) RAM: 16GB DDR3 RAM.

[0097] (4) Hard disk: 1T Hard Disk Drive.

[0098] As shown in Figure 5 The application provides a data acceleration system in a large load scenario, characterized by comprising an initial unit 1, a processing unit 2, an insertion unit 3, and an output unit 4, wherein:

[0099] The initial unit 1 is used to establish a tree for comparison, set a global flag bit for the tree, set a flag bit for the non-leaf node of the tree, and define the initial value of the global flag bit and the flag bit as 0;

[0100] The processing unit 2 is used to insert the data to be processed into the leaf node of the tree for comparison, output the smallest data, and update the global flag bit according to the flag bit of the output data;

[0101] The insertion unit 3 is configured to insert new data, set a flag of the new data, and compare the data in the tree with each other according to the flag, output the minimum data after the comparison, and update a global flag according to the flag of the minimum data.

[0102] The output unit 4 is configured to cyclically execute step 3) until all data are inserted, and output the sorted data set after the data to be sorted are sorted.

[0103] A non-transitory computer readable storage medium, characterized in that the non-transitory computer readable storage medium stores computer instructions, the computer instructions are used to make the computer execute the data acceleration method in a large load scenario according to any one of the above embodiments.

[0104] The exemplary embodiments of the present disclosure also provide a computer program product, comprising a computer program, wherein the computer program is used to make the computer execute the method according to the embodiments of the present disclosure when the computer program is executed by the processor of the computer.

[0105] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the above. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disks read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0106] As used in the present disclosure, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., magnetic discs, optical disks, memory, programmable logic devices (PLDs)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal that provides machine instructions and / or data to a programmable processor.

[0107] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0108] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0109] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0110] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer programs or instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present disclosure are performed. The computer can be a general purpose computer, a special purpose computer, a computer network, a terminal, a user equipment or other programmable apparatus. The computer programs or instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer programs or instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center through wired or wireless manner. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center and the like integrated with one or more available media. The available media can be a magnetic medium, for example, a floppy disk, a hard disk, a magnetic tape; can also be an optical medium, for example, a digital video disc (digital video disc, DVD); can also be a semiconductor medium, for example, a solid state disk (solid state drive, SSD).

[0111] It is obvious for those skilled in the art that the above specific examples are only preferred solutions of the present application, and therefore any improvement or change made by those skilled in the art to some parts of the present application still embodies the principles of the present application and achieves the purposes of the present application, and all of them belong to the protection scope of the present application.

Claims

1. A data acceleration method for high-load scenarios, characterized in that, include: 1) In a high-load scenario where the load carried by each data in the data table is such that multiple clock cycles are required to transmit one data into the hardware module on a data stream with a width of p, a comparison tree model is constructed. The comparison tree model includes a tree for comparison. The tree is set with a global flag bit, and the non-leaf nodes are set with flag bits. The global flag bit and the initial value of the flag bit are defined as 0. The global flag bit and the flag bit are 1 bit, and the value of the global flag bit and the flag bit is 0 or 1. 2) Insert the data to be processed into the leaf node of the comparison tree model, compare them, output the smallest data, and update the global flag according to the flag of the output data. According to the flag of the smallest data output, set the value of the global flag to the value of the inverse of the flag. 3) Insert new data and set a flag for the new data; The process of inserting new data and setting a flag for the new data specifically includes: The new data to be inserted is compared with the smallest output data. If the new data to be inserted is greater than the smallest output data, then the flag bit of the new data to be inserted is equal to the flag bit of the smallest data. If the new data to be inserted is less than the smallest data, then the flag bit of the new data to be inserted is the inverted bit of the flag bit of the smallest data. 4) Based on the flag bits, compare the data within the tree, output the minimum data after the comparison, update the global flag bits based on the flag bits of the output minimum data, and output the sorted dataset after the data to be sorted is sorted. The process of comparing data within the tree based on flag bits, outputting the minimum data after the comparison, and updating the global flag bit based on the flag bit of the output minimum data includes: After inserting new data, the continuous sorting phase begins. Within the tree, data from each pair of adjacent leaf nodes are compared. First, the flag bits are compared. If the flag bits of each pair of adjacent leaf nodes are not equal, then the flag bit equals the global flag bit, indicating a large data value. If the flag bit is not equal to the global flag bit, then the small data value is inserted into the parent node, and the small data value continues to be compared upwards until the data is output. If the flag bits of each pair of adjacent leaf nodes are equal, then the data size is directly compared, the large data value is inserted into the parent node, and the small data value continues to be compared upwards until the data is output.

2. The data acceleration method under high load scenarios according to claim 1, characterized in that, Step 2) involves inserting the data to be processed into the leaf nodes of the tree for comparison, outputting the smallest data, and updating the global flag based on the flag of the output data. 3.1 Entering the initial insertion phase, the data to be processed is inserted into the empty leaf nodes of the tree in sequence. Initially, the global flag is initialized to 0, and the flag for data addition is also 0. 3.2 After each data point is inserted into a leaf node, if its parent node is empty, the data is also inserted into its parent node. If its parent node is not empty, a data comparison is performed once in its parent node. The larger data point is inserted into that node, and the smaller data point continues to participate in the comparison towards the root node. Finally, the data with the larger value remains in the root node, and the data with the smallest value is output. 3.3 Based on the flag of the smallest output data, set the global flag value to the value that is the inverse of the flag.

3. The data acceleration method under high load scenarios according to claim 1, characterized in that, Once the data to be sorted is sorted, the flushing phase begins. Enter a virtual key to empty the entire tree.

4. The data acceleration method under high load scenarios according to claim 3, characterized in that, The input of the virtual key to empty the entire tree specifically means: The virtual key is represented by a flag bit of -1. By inserting the virtual key, all the remaining data in the tree is output, and the sorting is completed.

5. A data acceleration system for high-load scenarios, characterized in that, include: The unit comprises a modeling unit (1), a processing unit (2), an insertion unit (3), and an output unit (4), wherein: The modeling unit (1) is used to construct a comparison tree model in a scenario where the load carried by each data in the data table is so heavy that multiple clock cycles are required to transmit one data into the hardware module in a data stream with a width of p. The comparison tree model includes a tree for comparison, and the tree is set with a global flag bit and the non-leaf nodes are set with flag bits. The processing unit (2) is used to insert the data to be processed into the leaf node of the comparison tree model, compare it, output the smallest data, update the global flag bit according to the flag bit of the output data, and set the global flag bit value to the value opposite to the flag bit according to the flag bit of the smallest data output. The insertion unit (3) is used to insert new data and set the flag bit of the new data; The output unit (4) is used to compare the data in the tree with each other according to the flag bit, output the minimum data after the comparison, update the global flag bit according to the flag bit of the output minimum data, and output the sorted dataset after the data to be sorted is sorted. The process of comparing data within the tree based on flag bits, outputting the minimum data after the comparison, and updating the global flag bit based on the flag bit of the output minimum data includes: After inserting new data, the continuous sorting phase begins. Within the tree, data from each pair of adjacent leaf nodes are compared. First, the flag bits are compared. If the flag bits of each pair of adjacent leaf nodes are not equal, then the flag bit equals the global flag bit, indicating a large data value. If the flag bit is not equal to the global flag bit, then the small data value is inserted into the parent node, and the small data value continues to be compared upwards until the data is output. If the flag bits of each pair of adjacent leaf nodes are equal, then the data size is directly compared, the large data value is inserted into the parent node, and the small data value continues to be compared upwards until the data is output.

6. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions for causing the computer to perform the method according to any one of claims 1 to 4.

Images