A data sorting system based on merge sort algorithm to realize variable-scale data

Abstract

The present application discloses a data sorting system for implementing a variable-scale quantity based on a merge sort algorithm. The data sorting system is used to merge and sort the original data in an external storage module. The acceleration module in the system is used to group and sort the acquired original data by using a data comparison method, and record it as ordered subsequence data; the data output module is used to output the ordered subsequence data to the external storage module; the merge module is used to acquire the ordered subsequence data in the external storage module, and adopt a loop method to merge the acquired ordered subsequence data, and record the merged data as merged sorted data; the data output module is also used to output the merged sorted data to the external storage module. Through the technical solution in the present application, it is possible to quickly sort large amounts of data in a limited local storage space, reduce resource consumption, and improve data sorting speed.

Description

Technical Field

[0001] The present application relates to the technical field of data processing, and in particular to a data sorting system for implementing a variable-scale quantity based on a merge sort algorithm. Background Art

[0002] Data sorting is a fundamental data processing problem in scenarios such as data processing, image processing, and machine learning. Statistics show that 25% to 50% of all computer tasks involve data sorting. Furthermore, with the advancement of technologies like machine learning and artificial intelligence, the requirements for data sorting speed and quantity are increasing.

[0003] Compared to software, using FPGA for hardware acceleration can significantly improve data sorting speed. However, when conventional FPGA hardware circuits (such as FPGA sorters) sort large quantities of disordered raw data, they cannot allocate a large amount of storage space within the chip to store this data. Therefore, when comparing the data to be sorted, RAM or FIFO are usually used for temporary storage.

[0004] In this data comparison mode of the FPGA hardware circuit, continuous read and write control of the storage module is involved, which not only increases the control logic complexity of the FPGA hardware circuit data sorting implementation, but also requires additional time in the process of continuous reading and writing data, reducing the efficiency of data sorting. Summary of the Invention

[0005] The purpose of this application is to use FPGA hardware circuits and data read and write control methods to quickly sort large quantities of data within limited local storage space, reduce resource consumption, and increase data sorting speed.

[0006] The technical solution of the present application is: to provide a data sorting system for implementing a variable-scale quantity based on a merge sort algorithm, the data sorting system is used to merge and sort the original data in an external storage module, the system includes: an acceleration module, a merging module and a data output module; the acceleration module is used to group and sort the acquired original data by using a data comparison method, and record the data as ordered subsequence data; the data output module is used to output the ordered subsequence data to the external storage module; the merging module is used to acquire the ordered subsequence data in the external storage module, and adopt a cyclic method to perform a merge operation on the acquired ordered subsequence data, and record the merged data as merged sorted data; the data output module is also used to output the merged sorted data to the external storage module.

[0007] In any of the above technical solutions, the system further includes: a control module and a cache module; the control module is used to determine the starting address, ending address and number of merging layers of the read and write data based on the received configuration information; the cache module is used to obtain the original data from the external storage module based on the starting address and ending address, and send the original data to the acceleration module.

[0008] In any of the above technical solutions, further, the acceleration module includes: a RAM array, a pipeline module and a merge tree module; a RAM array is provided on each side of the pipeline module, and a ten-stage pipeline is provided in the pipeline module. The pipeline module is used to group and sort the acquired original data by using data comparison and encoding sorting of the lookup table; the merge tree module is used to merge the grouped and sorted data to obtain ordered subsequence data.

[0009] In any of the above technical solutions, further, the acceleration module is also used to obtain original data according to the buffer size of the RAM array.

[0010] In any of the above technical solutions, further, the RAM array includes a first RAM and a second RAM, the first RAM and the second RAM contain 8-bit registers, and the pipeline module is configured as follows: Step 11, grouping the data obtained from the first RAM in pairs in order, and comparing the size of the data in each group, and according to the order of each group, storing the data with larger values ​​in the group in the odd-bit registers of the second RAM, and storing the data with smaller values ​​in the group in the even-bit registers of the second RAM, wherein the number of bits of the registers in the first RAM and the second RAM is numbered sequentially starting from 0; Step 12, grouping the data in the odd-bit registers and even-bit registers of the second RAM in pairs, and comparing the size of the data in each group after grouping, multiplexing the first RAM, and storing the data with larger values ​​in the group in the first register of the first RAM. In the register, the data with smaller values ​​in the group is stored in the second register of the first RAM, wherein the first register is the first, second, fifth and sixth registers, and the second register is the third, fourth, seventh and eighth registers; step 13, multiplexing the second RAM, comparing the sizes of the data in the second and third registers and the sixth and seventh registers in the first RAM respectively, storing the data with larger values ​​in the second and sixth registers of the second RAM, and storing the data with smaller values ​​in the third and seventh registers of the second RAM, and writing the data in the first, fourth, fifth and eighth registers of the first RAM in sequence into the first, fourth, fifth and eighth registers of the second RAM; step 14, using the coding sorting method of the lookup table, encoding, searching and decoding the data in the second RAM in sequence to generate grouped and sorted data.

[0011] In any of the above technical solutions, further, the control module is also used to read the ordered subsequence data of any length from the external storage module in sequence in a cyclic manner, and send the obtained ordered subsequence data of any length to the cache module, wherein the length of the data obtained next time is twice the length of the data obtained last time; the cache module is also used to control the ordered subsequence data of any length obtained by the module to be sent to the merging module.

[0012] In any of the above technical solutions, further, when the length of the ordered subsequence data read for the first time by the control module is less than the total length of the ordered subsequence data, the control module is further configured to: when it is determined that the total length of the merge-sorted data output by the data output module is equal to the total length of the ordered subsequence data, determine the data write-back starting address, wherein the data write-back starting address is used to determine the storage location of the merge-sorted data in the external storage module.

[0013] The beneficial effects of this application are:

[0014] The technical solution in the present application sets up a cache module, an acceleration module and a merging module, and combines the data output module and the control module, uses the underlying acceleration module to preprocess the disordered original data, and uses a ten-stage pipeline sorting and merging FIFO tree method to quickly obtain an ordered subsequence of fixed length, complete the preprocessing, and reduce the overall sorting time. It enables the use of FPGA hardware circuits to quickly sort large amounts of disordered original data in limited local storage space, thereby reducing resource consumption and improving data sorting speed.

[0015] In the preferred implementation of the present application, a RAM array, a pipeline module and a merge tree module are set in the acceleration module, and a ten-stage pipeline method is used to merge and sort the data. The RAM array in the acceleration module is used to store the data for this comparison in eight registers. The eight stored data are encoded, decoded, and retrieved through a pipeline process. Therefore, in the process of sorting 4 rows of 8, there is no need to transport the data. The final data sorting information can be obtained through the encoding information. Only one reading and writing is required, which reduces the time consumed by the overall sorting. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The advantages of the above and / or additional aspects of the present application will become apparent and readily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0017] Figure 1 This is a schematic block diagram of a data sorting system for a variable size based on a merge sort algorithm according to an embodiment of the present application;

[0018] Figure 2is a schematic diagram of the circuit structure of an acceleration module according to an embodiment of the present application;

[0019] Figure 3 is a schematic diagram of a ten-stage pipeline workflow according to an embodiment of the present application;

[0020] Figure 4 10-stage pipeline schematic diagram for fully sorting 8 data according to an embodiment of the present application;

[0021] Figure 5 4 is a schematic diagram of a 4-row 8 encoding process according to an embodiment of the present application;

[0022] Figure 6 It is a schematic diagram of the merging module workflow according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to more clearly understand the above-mentioned objectives, features and advantages of the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features therein can be combined with each other in the absence of conflict.

[0024] In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present application is not limited to the specific embodiments disclosed below.

[0025] like Figure 1 As shown, this embodiment provides a data sorting system 100 for implementing a variable-scale quantity based on a merge sort algorithm. The data sorting system 100 is used to merge and sort the original data in the external storage module 200. The data sorting system 100 includes: an acceleration module 40, a merging module 20, and a data output module 50; the acceleration module 40 is used to group and sort the acquired original data by using a data comparison method, and record the data as ordered subsequence data; the data output module 50 is used to output the ordered subsequence data to the external storage module 200; the merging module 20 is used to acquire the ordered subsequence data in the external storage module 200, and in a cyclic manner, perform a merge operation on the acquired ordered subsequence data, and record the merged data as merged sorted data; the data output module 50 is also used to output the merged sorted data to the external storage module 200.

[0026] Furthermore, the system also includes: a control module 10 and a cache module 30; the control module 10 is used to determine the starting address, ending address and number of merging layers of the read and write data based on the received configuration information; the cache module 30 is used to obtain the original data from the external storage module 200 based on the starting address and ending address, and send the original data to the acceleration module 40.

[0027] Specifically, the control module 10 calculates the data read and write address range, including the start address and end address of the read and write data, based on the received start signal. The start signal provides the size of the data to be sorted, the start address of the data to be sorted, the start address of the free storage space, and the sorting order.

[0028] After the data sorting system 100 is started, the control module 10 sends a data packet request signal to coordinate the operations of the various modules. It also outputs a control signal to configure the control state machine of the merging module 20 and control the writing and unwriting of data from the external storage module 200 to the data sorting system 100.

[0029] The merging module 20 is composed of two FIFOs, A and B, a comparator, and a configurable FIFO read state machine, which controls the reading of data and the request for data from the buffer module 30. It should be noted that the configuration of the state machine can be set according to the specific hardware implementation, and this embodiment is not limited thereto.

[0030] Buffer module 30 consists of two FIFOs, CA and CB, of a certain depth. Based on the relevance of merge sort data, this minimizes multiple reads from external storage module 200 while ensuring continuous comparison of data between two subsequences, improving the smoothness of data comparison within the overall data sorting system 100. Buffer module 30 can cache two sets of data, awaiting data requests from merge module 20 and acceleration module 40, enabling the transmission of related data.

[0031] The acceleration module 40 uses a combination of pipeline and merge FIFO tree to group and sort the acquired original data, and completes the fast sorting of the underlying subsequence.

[0032] When the comparison result data reaches a certain packet length, the data output module 50 requests the control module 10 to send a packet to the external memory to complete the data merging and sorting.

[0033] This embodiment is now described by taking the merge sorting of 100 trillion unordered floating-point numbers as an example.

[0034] After the data sorting system 100 receives the start signal, the control module 10 determines the start address, end address, and number of merge levels (iterations) for reading and writing data according to the received start signal, and completes the initialization configuration of all comparators.

[0035] After all modules are initialized, the cache module 30 requests data from the control module 10, and then the control module 10 sends the request to the external storage module 200. According to the length N of the sorted data provided by the start signal, the number of iterations M to be implemented by the merge algorithm is determined.

[0036]

[0037] According to the formula Rounding up, it takes 20 iterations for the merge module 20 to complete the sorting of 1 million data.

[0038] In this embodiment, the minimum subsequence length K of the acceleration module 40 is set to 128, and the first sequence sorting of the original data requires a total of That is, 1 million original data need to be sorted 7813 times by the acceleration module 40 to complete the ordered subsequence required by the merging module 20.

[0039] Afterwards, the control module 10 triggers the acceleration module 40 to work. The acceleration module 40 takes out the data to be sorted from the cache module 30 FIFO according to the RAM buffer size determined by its own structure, and completes a set of data sorting through the coding sorting technology based on the lookup table.

[0040] like Figure 2 As shown, this embodiment also provides an implementation method of an acceleration module, and the acceleration module 40 includes: a RAM array, a pipeline module and a merge tree module; a RAM array is provided on both sides of the pipeline module, and a ten-stage pipeline is provided in the pipeline module. The pipeline module is used to group and sort the acquired original data by using data comparison and encoding sorting of the lookup table; the merge tree module is used to merge the grouped and sorted data to obtain ordered subsequence data.

[0041] Specifically, the acceleration module 40 is provided with four 8*8 RAM arrays, two of which receive data sent from the cache, and the other two RAM arrays store a sequence of size 8 output by the sorted pipeline in rows.

[0042] Through the ten-stage pipeline in the pipeline module, any 8 data are fully sorted. The pipeline uses a coding sorting technology based on a lookup table to achieve full order from a subsequence of size 4 to 8 data. The ten-stage pipeline workflow diagram is as follows Figure 3 .

[0043] The root FIFO data input of the FIFO tree in the merge tree module comes from the RAM array that stores the sorting results of the pipeline module. The data is merged along the trunk, and finally the two data packets are merged. The specific merging method is not repeated here.

[0044] After the cache module 30 completes a certain amount of data buffering, it waits for the read request of the acceleration module 40. The acceleration module 40 reads 8 data at a time. Figure 4 .

[0045] Furthermore, the acceleration module 40 is further configured to obtain original data according to the buffer size of the RAM array.

[0046] Furthermore, the RAM array includes a first RAM and a second RAM, the first RAM and the second RAM contain 8-bit registers, and the pipeline module is configured as follows:

[0047] Step 11: Group the data obtained from the first RAM into pairs in order of precedence, and compare the size of the data in each group. According to the order of the groups, the data with larger values ​​in the group are stored in the odd-numbered registers of the second RAM, and the data with smaller values ​​in the group are stored in the even-numbered registers of the second RAM. The bits of the registers in the first RAM and the second RAM are numbered sequentially starting from 0.

[0048] Specifically, the RAM array includes four 8*8 RAMs. Each time data is compared, eight data can enter the pipeline module in parallel.

[0049] When performing the first data comparison, the first two RAMs are recorded as the first RAM, which is used to receive data sent by the cache module 30; the last two RAMs are recorded as the second RAM, which stores the data sorted by the pipeline module in RAM by row so as to enter the corresponding FIFO.

[0050] Each RAM is 8*8 and can actually store 64 data. Each time, eight data are fed into the pipeline module, the module sorts them, and outputs an ordered sequence of 8, which enters a FIFO. This is executed eight times in total, generating eight subsequences of 8, which are then fed into the FIFO. Similarly, the following part can also generate eight subsequences of 8. Finally, an ordered subsequence of 128 is generated by merging the FIFO trees.

[0051] The above process is actually two eight-time pipeline operations. Each pipeline sorts 8 data. Using multiple identical modules to work simultaneously can use more resources to reduce the sorting time.

[0052] Therefore, it should be noted that the size and quantity of RAM in the RAM array can be configured according to actual needs.

[0053] In addition, since the number of bits of the registers in the first RAM and the second RAM is numbered sequentially starting from 0, the odd-numbered registers in the first RAM and the second RAM are R0, R2, R4, and R6, and the even-numbered registers are R1, R3, R5, and R7, totaling 8 bits.

[0054] In this embodiment, in the first stage of sorting (1 row, 2 stages), the eight unordered data (original data) provided by the cache module 30 are set to 5, 0, 3, 7, 9, 6, 1, and 4, and are stored in the R0 register (i.e., the first RAM). The eight data are divided into four groups in order from top to bottom, and the sizes of the two groups are compared. The larger data is written to the odd-numbered registers of the R1 register (i.e., the second RAM), and the smaller data is written to the even-numbered registers of the R1 register.

[0055] For example: compare the data in register R00 with that in register R01, write the data 5 in register R00 into register R10, and write the data 0 in register R01 into register R11; compare the data in register R02 with that in register R03, write the data 3 in register R02 into register R13, and write the data 7 in register R02 into register R12; and so on, the data in register R1 are 5, 0, 7, 3, 9, 6, 4, and 1 respectively. The data in register R1 can form four groups of two ordered subsequences, namely (5, 0), (7, 3), (9, 6), and (4, 1).

[0056] Step 12: Group the data in the odd-numbered registers and the even-numbered registers of the second RAM into pairs, compare the sizes of the grouped data, reuse the first RAM, store the data with larger values ​​in the first register of the first RAM, and store the data with smaller values ​​in the second register of the first RAM. The first register is the first, second, fifth, and sixth bit registers, and the second register is the third, fourth, seventh, and eighth bit registers.

[0057] Step 13, multiplexing the second RAM, comparing the data in the second and third registers and the sixth and seventh registers in the first RAM, storing the data with larger values ​​in the second and sixth registers of the second RAM, and storing the data with smaller values ​​in the third and seventh registers of the second RAM, and writing the data in the first, fourth, fifth, and eighth registers of the first RAM into the first, fourth, fifth, and eighth registers of the second RAM in sequence;

[0058] Specifically, the second stage of sorting (2 rows, 4 stages) requires two cycles. In the first cycle, the maximum values ​​(R10 and R12, R14 and R16) and minimum values ​​(R11 and R13, R15 and R17) of the four groups of two subsequences are compared, and the comparison results of the maximum values ​​are stored in sequence in the first two registers corresponding to the next level of the pipeline, namely the R20 and R21 registers, and the R24 and R25 registers in the reused first RAM. The comparison results of the minimum values ​​are stored in sequence in the last two registers, namely the R22 and R23 registers, and the R26 and R27 registers in the reused first RAM.

[0059] The second cycle compares the values ​​of the two middle registers, that is, compares the R21 and R22 registers, and the R25 and R26 registers, and stores the comparison results in the corresponding two registers of the next level pipeline in order, that is, the large data is placed in the R31 and R35 registers in the reused second RAM, and the small data is placed in the R32 and R36 registers in the reused second RAM. The remaining R2-level registers (reused first RAM) write the data into the R3-level register (reused second RAM) in the corresponding order.

[0060] For example, in the first cycle, the values ​​in registers R10(5) and R12(7) are compared (5 < 7), and the value R12(7) is written into register R20, and the data in register R10(5) is written into register R21, and so on. In the second cycle, the values ​​in registers R21(5) and R22(3) are compared (5 > 3), and the value R21(5) is written into register R31, and the data in register R22(3) is written into register R32, and so on. Finally, two groups of four ordered sequences are obtained and stored in register R3, with the subsequences being (7, 5, 3, 0) and (9, 6, 4, 1) respectively.

[0061] Step 14: using the coding and sorting method of the lookup table, the data in the second RAM is encoded, searched, and decoded in sequence to generate grouped and sorted data.

[0062] Specifically, the third stage of sorting (4 rows, 8 stages) consists of encoding, searching, and decoding, requiring a total of 6 cycles. The first 4 cycles encode the arrangement of the two subsequences. The search stage uses the encoding result of the previous cycle as an address to retrieve the sequence ordering information pre-stored in the ROM. The decoding stage merges the two subsequences based on the retrieved sequence ordering information. Each register corresponding to the T9 cycle has an 8-to-1 data selector in front of it. The data input of the data selector is connected to all registers corresponding to the T8 cycle. The selector's select signal is connected to three bits of the ROM output. Based on the encoded information, the data of the corresponding register is written to the specified register, completing a sort of length 8.

[0063] There are 70 kinds of sequence sorting information for merging two subsequences of size 4. The sequence sorting information is encoded and stored in the ROM. The encoding process is as follows Figure 5 shown. In the first cycle, according to the comparison results of the maximum and minimum values of the two subsequences, the branches are determined and recorded, and the two groups of registers participating in the comparison in the next cycle are determined; in the following two cycles, the branches are determined and recorded according to the comparison results of the selected registers, and the registers participating in the comparison in the next cycle are determined; in the fourth cycle, the encoding result is obtained, and some sequences can obtain the encoding result in the second or third cycle.

[0064] As Figure 4 shown, for the two subsequences 7530 and 9641 corresponding to the T3 cycle, the values corresponding to registers R0 to R7 are 7, 5, 3, 0, 9, 6, 4, 1 respectively. Each branch has four possibilities, as Figure 5 shown. For example, there are four possibilities for the relationship between R0 and R4, and R3 and R7, which are R0 > R4, R3 < R7; R0 > R4, R3 > R7; R0 < R4, R3 < R7; R0 < R4, R3 > R7. These four possibilities are encoded as 00, 01, 10, 11 in sequence. The register branch for recording the branch is 6 bits and is composed of three segments. The specific encoding process is as follows:

[0065] (1) R0 is 7, R4 is 9, R3 is 0, R7 is 1, R0 < R4, R3 < R7, branch = 6'b10xxxx, and the two groups of registers participating in the comparison in the next cycle are R0R5 and R2R7;

[0066] (2) R0 is 7, R5 is 6, R2 is 3, R7 is 1, R0 > R5, R2 > R7, branch = 6'b1001xx, and the group of registers participating in the comparison in the next cycle is R1R5 and R2R6;

[0067] (3) R1 is 5, R5 is 6, R2 is 3, R6 is 4, R1 < R5, R2 < R6, branch = 6'b100110, and the two groups of registers participating in the comparison in the next cycle are R1R6;

[0068] (4) R1 is 5, R6 is 4, R1 > R6, and the corresponding sequence sorting information is 40516273. The sequence sorting information corresponds to registers R0 to R7 in step S13. According to the sequence information, the data is taken out from the corresponding numbered registers in sequence to complete the sorting of these eight data. The eight sorted data are sent into a FIFO in the FIFO tree and wait for merging.

[0069] The data to be sorted is sorted once through a 10-stage pipeline sorting method, and the sorted data is sent to the merge FIFO tree to continue the sorting operation. The merge FIFO tree has four layers, using two-way parallel operation. Each layer stores 128 data and finally outputs an ordered subsequence of length 128. Each pair of FIFOs is controlled and managed by a dedicated state machine. Since each FIFO stores an ordered subsequence, the output of the FIFO can be compared and the data can be stored in the FIFO of the next layer in sequence. And so on, the sorting operation of the data by the merge FIFO tree is completed. Finally, an ordered subsequence of length 128 is obtained, which serves as a minimum ordered subsequence of the merge module 20. Each subsequence obtained is output and written into the specified storage area in order to complete the sorting of all data in the original sequence.

[0070] This embodiment provides a RAM array, a pipeline module, and a merge tree module in the acceleration module 40, and adopts a ten-stage pipeline method to merge and sort the data. The RAM array in the acceleration module 40 is used to store the data for this comparison in eight registers. The eight stored data are encoded, decoded, and retrieved through a pipeline process. As a result, in the process of sorting 4 rows of 8, there is no need to move the data, and only one reading and writing is required, which reduces the time consumed by the overall sorting.

[0071] In addition, using the buffer module 30 to buffer the data in the external memory module 200 (DDR) also reduces the read and write control of the external memory module 200 from a certain perspective, and at the same time, improves the continuity of data comparison.

[0072] In this embodiment, the control module 10 is further configured to sequentially read the ordered subsequence data of any length from the external storage module 200 in a cyclic manner, and send the obtained ordered subsequence data of any length to the cache module 30, wherein the length of the data obtained next time is twice the length of the data obtained previously;

[0073] The cache module 30 is further configured to control the sending of ordered subsequence data of any length acquired by the control module 10 to the merging module 20 .

[0074] Specifically, after all the data to be sorted are sorted by the acceleration module 40, multiple ordered subsequences of length K (K=128) will be generated. The control module 10 reads the data of length n in the ordered subsequence data from the external storage module 200 to the cache module 30, waiting for the merge module 20 to read the sort.

[0075] Therefore, the data involved in sorting by the merging module 20 changes each time. If the length of the current sorting sequence is n, the length of the sequence in the next sorting round is 2n. The length of the data compared by the merging module 20 in each round will be twice that of the previous one.

[0076] Furthermore, when the length n of the ordered subsequence data read for the first time by the control module 10 is smaller than the total length K of the ordered subsequence data, the control module 10 is further configured to:

[0077] When it is determined that the total length of the merge sorted data output by the data output module 50 is equal to the total length of the ordered subsequence data, the data write back start address is determined, wherein the data write back start address is used to determine the storage location of the merge sorted data in the external storage module 200.

[0078] like Figure 6 As shown, the first step of the overall sorting is completed by the acceleration module 40, dividing all the disordered data into 7813 ordered subsequences, and then the control module 10, cache module 30, merge module 20 and output module are enabled to complete 20 iterative operations.

[0079] Since the depth of each FIFO in the data sorting system 100 is fixed, when the amount of data cached in the output module reaches the packet length of 128, the control module 10 determines to write the data back to the starting address, and the external storage module 200 responds and sends the data packet.

[0080] It should be noted that the external storage is read and written in an orderly manner through reasonable control methods to meet the data needs of the internal buffer FIFO. Considering the extreme case that may occur during the ordered subsequence comparison process, that is, the maximum value of comparison sequence 1 is less than the minimum value of comparison sequence 2, in this case, it is necessary to read out the entire comparison sequence 2 first, and then read out the data of comparison sequence 1 in sequence to form a new ordered sequence. The specific process is not repeated here.

[0081] The above describes the technical solution of the present application in detail in conjunction with the accompanying drawings. The present application proposes a data sorting system based on a merge sort algorithm to implement a variable-scale quantity. The data sorting system is used to merge and sort the original data in an external storage module. The system includes: an acceleration module, a merge module, and a data output module; the acceleration module is used to group and sort the acquired original data by data comparison, and record them as ordered subsequence data; the data output module is used to output the ordered subsequence data to the external storage module; the merge module is used to acquire the ordered subsequence data in the external storage module, and adopt a cyclical method to merge the acquired ordered subsequence data, and record the merged data as merged sorted data; the data output module is also used to output the merged sorted data to the external storage module. Through the technical solution in the present application, it is possible to quickly sort large quantities of data in a limited local storage space, reduce resource consumption, and improve data sorting speed.

[0082] The steps in this application can be adjusted in order, combined, and deleted according to actual needs.

[0083] The units in the device of the present application can be combined, divided and deleted according to actual needs.

[0084] Although the present application is disclosed in detail with reference to the accompanying drawings, it should be understood that these descriptions are merely exemplary and are not intended to limit the application of the present application. The scope of protection of the present application is defined by the appended claims and may include various modifications, alterations and equivalents made to the invention without departing from the scope and spirit of the present application.

Claims

1. A data sorting system for implementing a scalable amount of data based on a merge sort algorithm, wherein the data sorting system is used to merge sort the original data in an external storage module, and is characterized in that: The system includes: an acceleration module (40), a merging module (20) and a data output module (50); The acceleration module (40) is used to group and sort the acquired original data by data comparison, and record them as ordered subsequence data; The data output module (50) is used to output the ordered subsequence data to the external storage module; The merging module (20) is used to obtain the ordered subsequence data in the external storage module, and perform a merging operation on the obtained ordered subsequence data in a circular manner, and record the merged data as merged sorted data; The data output module (50) is further used to output the merge sorted data to the external storage module; The acceleration module (40) includes: a RAM array, a pipeline module and a merge tree module, the RAM array includes a first RAM and a second RAM located on both sides of the pipeline module, the first RAM and the second RAM contain 8-bit registers, the pipeline module is provided with ten stages of pipelines, and is configured as follows: Step 11: Group the data obtained from the first RAM into pairs in order of precedence, and compare the size of the data in each group. According to the order of the groups, the data with larger values ​​in the group are stored in the odd-numbered registers of the second RAM, and the data with smaller values ​​in the group are stored in the even-numbered registers of the second RAM. The number of bits of the registers in the first RAM and the second RAM is sequentially numbered starting from 0. Step 12: Group the data in the odd-bit register and the even-bit register of the second RAM into pairs, compare the sizes of the grouped data, reuse the first RAM, store the data with larger values ​​in the group in the first register of the first RAM, and store the data with smaller values ​​in the group in the second register of the first RAM, wherein the first register is the first, second, fifth, and sixth bit registers, and the second register is the third, fourth, seventh, and eighth bit registers; Step 13: Multiplexing the second RAM, comparing the data in the second and third registers and the sixth and seventh registers of the first RAM, storing the data with larger values ​​in the second and sixth registers of the second RAM, and storing the data with smaller values ​​in the third and seventh registers of the second RAM, and writing the data in the first, fourth, fifth, and eighth registers of the first RAM into the first, fourth, fifth, and eighth registers of the second RAM in sequence; Step 14, using the coding and sorting method of the lookup table, the data in the second RAM is sequentially encoded, searched and decoded to generate grouped and sorted data; specifically, The three stages of encoding, searching, and decoding require six cycles. The first four cycles encode the arrangement of the two subsequences. The searching stage uses the encoding result of the previous cycle as the address to retrieve the sequence order information pre-stored in the ROM. The decoding stage merges the two subsequences based on the retrieved sequence order information. The data of the corresponding register is written to the specified register according to the encoding information, completing a sequence order of length 8. Among them, there are 70 kinds of sequence order information for merging two groups of subsequences of size 4, which are encoded and stored in the ROM. The merge tree module is used to merge the grouped and sorted data to obtain the ordered subsequence data.

2. The data sorting system for implementing scalable quantities based on the merge sort algorithm according to claim 1, characterized in that: The system further comprises: a control module (10) and a cache module (30); The control module (10) is used to determine the starting address, ending address and number of merging layers of read and write data according to the received configuration information; The cache module (30) is used to obtain the original data from the external storage module according to the start address and the end address, and send the original data to the acceleration module (40).

3. The data sorting system for implementing scalable quantities based on the merge sort algorithm according to claim 1, characterized in that: A RAM array is provided on both sides of the pipeline module. The pipeline module is used to group and sort the acquired original data by using data comparison and coding sorting of a lookup table.

4. The data sorting system for implementing scalable quantities based on the merge sort algorithm according to claim 3, characterized in that: The acceleration module (40) is further used to obtain the original data according to the buffer size of the RAM array.

5. The data sorting system for implementing scalable quantities based on the merge sort algorithm according to claim 2, characterized in that: The control module (10) is further configured to sequentially read ordered subsequence data of any length from the external storage module in a cyclic manner, and send the obtained ordered subsequence data of any length to the cache module (30), wherein the length of the data obtained next time is twice the length of the data obtained last time; The cache module (30) is also used to send the ordered subsequence data of any length acquired by the control module (10) to the merging module (20).

6. The data sorting system for implementing scalable quantities based on the merge sort algorithm according to claim 5, characterized in that: The length of the ordered subsequence data read for the first time by the control module (10) is less than the total length of the ordered subsequence data, The control module (10) is further configured to: When it is determined that the total length of the merge sorted data output by the data output module (50) is equal to the total length of the ordered subsequence data, the data is written back to the starting address. The data write-back start address is used to determine the storage location of the merge sort data in the external storage module.

Images