Data compression device and data compression method

Abstract

Provided is a data compression device (10), wherein a division unit (15b) divides a sorted integer string for each prescribed concurrent processing number. A generation unit (15c) calculates the difference between adjacent elements in each of divided arrays to generate difference arrays. A determination unit (15d) determines the number of bits required for expressing the maximum value of elements in each of the generated difference arrays. A compression unit (15e) compresses the sorted integer string by expressing the elements of each of the difference arrays with the determined number of bits and combining same.

Description

Data compression device and data compression method

[0001] The present invention relates to a data compression device and a data compression method.

[0002] Conventionally, differential compression is known as a representative method for compressing a sorted integer sequence. Also known are methods for efficiently compressing differential arrays (see Non-Patent Documents 1 and 2) and methods for high-speed decompression in a Single Instruction Multiple Data (SIMD) execution model (see Non-Patent Documents 3 and 4).

[0003] Larry H. Thiel and H. S. Heaps. 1972, "PROGRAM DESIGN FOR RETROSPECTIVE SEARCHES ON LARGE DATA BASES", Information Storage and Retrieval 8, 1 (1972), pp.1-20Vo Ngoc Anh and Alistair Moffat. 2010, "Index compression using 64-bit words", Software: Practice and Experience 40, 2 (2010), pp.131-147Daniel Lemire, Nathan Kurz, and Christoph Rupp. 2018, "STREAM-VBYTE: Faster byte-oriented integer compression", Information Processing Letters 130 (2018), pp.1-6Andrew Trotman. 2014, "Compression, SIMD, and Postings Lists", In Proceedings of the 2014 Australasian Document Computing Symposium. ACM, New York, NY, 50

[0004] However, conventional techniques can result in slow processing speeds when compressed data is decompressed using a graphics processing unit (GPU). For example, conventional compression techniques are based on the assumption that data is decompressed using sequential processing that extracts integers in order, starting from the first integer in a sequence. Therefore, even if compressed data is decompressed using a GPU that excels in parallel processing using a large number of computing cores, there is a problem of low parallelism and slow processing speeds.

[0005] The present invention has been made in view of the above, and has an object to enable high-throughput decompression processing of compressed data on a GPU.

[0006] In order to solve the above-mentioned problems and achieve the object, the data compression device of the present invention is characterized by having a division unit that divides a sorted integer sequence into a predetermined number of simultaneous processes; a generation unit that calculates the difference between adjacent elements in each divided array to generate a difference array; an identification unit that identifies the number of bits required to express the maximum value of the elements in each generated difference array; and a compression unit that compresses the sorted integer sequence by representing the elements of each difference array with the identified number of bits and combining them.

[0007] According to the present invention, high-throughput decompression processing of compressed data becomes possible on a GPU.

[0008] FIG. 1 is a diagram for explaining an overview of a data compression device according to this embodiment. FIG. 2 is a schematic diagram illustrating a general configuration of a data compression device according to this embodiment. FIG. 3 is a diagram for explaining data compression processing. FIG. 4 is a diagram for explaining data compression processing. FIG. 5 is a diagram for explaining processing by a decompression unit. FIG. 6 is a diagram for explaining processing by a decompression unit. FIG. 7 is a flowchart showing a data compression processing procedure. FIG. 8 is a diagram illustrating an example of a computer that executes a data compression program.

[0009] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to this embodiment. In addition, in the description of the drawings, the same parts are designated by the same reference numerals.

[0010] [Outline of Data Compression Apparatus] Fig. 1 is a diagram for explaining an outline of a data compression apparatus according to this embodiment. As illustrated in Figs. 1(a) and 1(b), the data compression apparatus according to this embodiment compresses target data, which is a sorted integer sequence D of length N. In doing so, the data compression apparatus compresses the target data using a method that allows efficient decompression by a GPU.

[0011] The GPU decompresses the compressed data through parallel processing of N threads. Specifically, as shown in Fig. 1(c), the i-th thread obtains the i-th part D[i] of D and performs its own processing (0≦i≦N−1).

[0012] Although each thread of the GPU is processed independently, when information within another thread needs to be obtained, that information is shared. When sharing among all threads, information is shared in global memory or shared memory, but this results in a large number of reads and writes to memory, slowing down processing speed. Therefore, in this embodiment, the range of information sharing is limited, and values ​​stored in registers are shared only within that range, enabling high-speed processing.

[0013] Many GPUs employ the SIMT (Single Instruction Multiple Thread) execution model, in which multiple threads execute a single instruction. In SIMT, a set of a predetermined number of threads executes a single instruction simultaneously. Hereinafter, this set of threads is referred to as a Warp, and the number of threads in a Warp is referred to as the Warp Size.

[0014] Furthermore, an instruction for sharing register values ​​among threads within a Warp is called Warp Shuffle. In other words, Warp Shuffle is an instruction that limits the scope of sharing the values ​​stored in the registers to within the Warp. As described above, Warp Shuffle enables faster processing than information sharing using global memory and shared memory.

[0015] Therefore, the data compression device compresses data in units of Warp, and when decompressing the compressed data, it is possible to share registers only among threads within the Warp, thereby enabling high-speed parallel processing on the GPU and high-throughput decompression processing.

[0016] [Configuration of Data Compression Device] Fig. 2 is a schematic diagram illustrating the general configuration of a data compression device according to this embodiment. Figs. 3 and 4 are diagrams for explaining the data compression process. As illustrated in Fig. 2, a data compression device 10 according to this embodiment is implemented by a general-purpose computer such as a personal computer, and includes an input unit 11, an output unit 12, a communication control unit 13, a storage unit 14, and a control unit 15.

[0017] The input unit 11 is realized by input devices such as a keyboard and a mouse, and inputs various instruction information such as a command to start processing to the control unit 15 in response to input operations by an operator. The output unit 12 is realized by a display device such as a liquid crystal display, a printing device such as a printer, etc. For example, the output unit 12 displays the results of a data compression process described below.

[0018] The communication control unit 13 is realized by a NIC (Network Interface Card) or the like, and controls communication between the control unit 15 and external devices via telecommunication lines such as a LAN (Local Area Network) or the Internet. For example, the communication control unit 13 controls communication between the control unit 15 and a management device or the like that manages various types of information.

[0019] The storage unit 14 is realized by a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 14 stores in advance the processing program that operates the data compression device 10 and data used during execution of the processing program, or temporarily stores the data each time processing is performed. The storage unit 14 may be configured to communicate with the control unit 15 via the communication control unit 13.

[0020] The control unit 15 is realized using a CPU (Central Processing Unit), a GPU, an NP (Network Processor), an FPGA (Field Programmable Gate Array), or the like, and executes a processing program stored in memory. As a result, the control unit 15 functions as an acquisition unit 15a, a division unit 15b, a generation unit 15c, an identification unit 15d, a compression unit 15e, and a decompression unit 15f, as illustrated in FIG. 2 , to perform data compression processing. Note that these functional units may be implemented individually or in part on different hardware. For example, the decompression unit 15f may be implemented on hardware realized using a GPU, but may not necessarily be implemented on the same hardware as the other functional units. The control unit 15 may also include other functional units.

[0021] The acquiring unit 15a acquires a sorted integer sequence, which is a sequence of integers to be processed that has been sorted in ascending or descending order. For example, the acquiring unit 15a acquires the sorted integer sequence from a management device that manages various types of information, via the input unit 11 or the communication control unit 13. The acquiring unit 15a may store the acquired sorted integer sequence in the storage unit 14.

[0022] For example, the acquiring unit 15a acquires a sorted integer sequence D illustrated in Fig. 3. Fig. 3 illustrates an example of an integer sequence D sorted in ascending order, such as 0, 1, 3, 4, . . .

[0023] The dividing unit 15b divides the sorted integer sequence D into a predetermined number of simultaneous processes. For example, as shown in FIG. 3(1), the dividing unit 15b divides the acquired sorted integer sequence D into Warp Sizes. In the example shown in FIG. 3(1), Warp Size=4, and the array D is divided into four integers. ｊ (j=0, 1, 2, ...). １ = [6, 9, 17, 19] is shown as an example.

[0024] The generation unit 15c generates each divided array D ｊ In this case, the difference between adjacent elements is calculated to obtain the difference array D ｊ For example, in FIG. 3(2), D １ Difference array D１ '=[3, 8, 2] is shown as an example.

[0025] The identification unit 15d identifies each generated differential array D ｊ ', the maximum value n of the elements ｊ The number of bits required to represent b ｊ Identify the ｊ is calculated by the following formula (1).

[0026]

[0027] For example, in the example shown in FIG. １ '=Maximum value n among elements in [3, 8, 2] ｊ = Number of bits b required to represent 8 １ =4 has been identified.

[0028] The compression unit 15e compresses each differential array D ｊ The elements of ' are specified by the number of bits b ｊ For example, in the example shown in FIG. 3(4), D １ By expressing each element of ' with 4 bits, D １ ' is compressed. Specifically, D １ Each element of '=[3, 8, 2] is expressed as [0011, 1000, 0010], and the elements are further combined to be expressed as a bit string [001110000010].

[0029] Furthermore, the compression unit 15e compresses each array D ｊ Hereinafter, this array will be referred to as Warp heads. ｊ Hereinafter, this is called Warp diffs.

[0030] In this way, the compression unit 15e derives the Warp heads and the Warp diffs to compress the sorted integer sequence D. The compression unit 15e also outputs the Warp heads and the Warp diffs, which are data obtained by compressing the sorted integer sequence D, to a predetermined destination via the output unit 12 or the communication control unit 13.

[0031] When performing the decompression process described later on the compressed data, the compression unit 15e derives comp bits and offsets, which will be described below, as support arrays so that each array divided for each Warp can be extracted independently.

[0032] That is, the compression unit 15e compresses each array D ｊ Difference array D ｊ The number of bits b required to represent the maximum value of the elements of ｊ In addition, the compression unit 15e derives an array (hereinafter referred to as comp bits) storing the compressed D ｊ The offsets are derived from the compressed D ｊ Each difference array D ｊ ' indicates the position corresponding to '.

[0033] 4 shows the array lengths of the Warp heads, Warp diffs, comp bits, and offsets when the array length of the input sorted integer sequence D is N and the Warp Size is s. As shown in FIG. 4, the array length of each array is expressed by a ceiling function.

[0034] FIG. 4 also shows, as an example, examples of arrays in the case where s=4 and D=[0, 1, 3, 4, 6, 9, 17, 19]. ０ =[0,1,3,4],D １ = [6, 9, 17, 19].

[0035] Therefore, as shown in FIG. 4, Warp heads ｊ In other words, in the expansion process described later, this is an array storing the elements of D handled by the leading thread of each Warp, and is [0, 6].

[0036] Warp diffs also reduces the compressed D ｊ', that is, an array that stores the difference values ​​of the elements of D that are handled by threads other than the first thread of each Warp in the expansion process, and is formed by combining [1, 2, 1, 3, 8, 2] as a bit string.

[0037] Also, comp bits are the bits of each array D ｊ Difference array D ｊ The number of bits b required to represent the maximum value of the elements of ｊ That is, the array storing the number of bits used to express the difference value for each Warp, which is [2, 4].

[0038] Also, offsets are the compressed D ｊ This is an array that stores information indicating from which bit the bit string of ' is stored, that is, an array that stores the offset for each Warp in Warp diffs, and is [0, 6]. ０ The offset of D １ The offset of ' is b ０ ×3=6.

[0039] Returning to the explanation of FIG. 2, the expansion unit 15f ｊ 'Number of bits required for representation b ｊ and each difference array D in the compressed sorted integer sequence D ｊ The expansion unit 15f expands the compressed sorted integer sequence using the positions corresponding to the support arrays comp_bits and offsets. Specifically, the expansion unit 15f expands the compressed data using the support arrays comp_bits and offsets.

[0040] At this time, the decompression unit 15f decompresses the compressed sorted integer sequence D by sharing register values ​​among a predetermined number of threads that perform simultaneous processing. That is, the decompression unit 15f performs decompression processing of the compressed data using Warp Shuffle. Specifically, the i-th thread i restores the i-th element D[i] of D.

[0041] 5 and 6 are diagrams for explaining the processing of the expansion unit. First, Fig. 5(a) shows an example of compressed data. Based on the support array comp_bits and offsets, the expansion unit 15f acquires the elements to be processed by each thread, as shown in Fig. 5(b) (1).

[0042] That is, the expansion unit 15f causes the leading thread of each Warp to acquire the element it is responsible for from Warp heads, and the other threads acquire the difference value from Warp diff. At this time, the expansion unit 15f identifies each element using the support array comp bits and offsets. In Figure 5 (1), for example, the leading thread 4 of the second Warp acquires 6 from Warp heads, and threads 5 to 7 acquire elements 3, 8, and 2 from Warp diff.

[0043] In addition, in each Warp, as shown in Fig. 5(2), the elements acquired by the threads in the range of each Warp are shared using Warp shuffle. Then, the expansion unit 15f performs D ｊ Restore.

[0044] Fig. 6 illustrates a detailed unfolding process in the range of the second Warp illustrated in Fig. 5. As shown in Fig. 6, in the unfolding unit 15f, the leading thread 4 obtains 6 from Warp heads, and threads 5 to 7 obtain elements 3, 8, and 2 from Warp diffs. Then, for example, thread 7 obtains the values ​​6, 3, and 8 of the threads preceding it in the same Warp using warp shuffle, and adds them all together to obtain 19.

[0045] In this way, the expansion unit 15f uses Warp Shuffle to pass the value that each thread has in the same Warp to the next thread. ２By repeating this process (Warp Size) times, the necessary information is transmitted to all threads in the same Warp, and the desired value can ultimately be obtained. Furthermore, by using Warp Shuffle to limit the scope in which the values ​​stored in the register are shared to within the Warp, faster processing is possible compared to sharing information among all threads using global memory or shared memory.

[0046] [Data Compression Process] Next, the data compression process performed by the data compression device 10 according to this embodiment will be described with reference to Fig. 7. Fig. 7 is a flowchart showing the procedure of the data compression process. The flowchart in Fig. 7 starts, for example, when the user performs an operation input to instruct the start of the process.

[0047] First, the acquisition unit 15a acquires the sorted integer sequence D to be processed. Then, the division unit 15b divides the acquired sorted integer sequence D into an array D for each Warp Size. ｊ (Step S1).

[0048] The generation unit 15c generates each divided array D ｊ In this case, the difference between adjacent elements is calculated to obtain the difference array D ｊ ' is generated (step S2).

[0049] The identification unit 15d identifies each generated differential array D ｊ ', the maximum value n of the elements ｊ The number of bits required to represent b ｊ (Step S3).

[0050] The compression unit 15e compresses each differential array D ｊ The elements of ' are specified by the number of bits b ｊ and combine them to derive Warp heads and Warp diffs. The sorted integer sequence D is compressed (step S4). The compression unit 15e then outputs Warp heads and Warp diffs, which are data obtained by compressing the sorted integer sequence D, to a predetermined destination (step S5). This completes the data compression process.

[0051] [Effects] As described above, in the data compression device 10 of this embodiment, the dividing unit 15b divides the sorted integer sequence into a predetermined number of simultaneous processes. The generating unit 15c calculates the differences between adjacent elements in each divided array to generate a difference array. The identifying unit 15d identifies the number of bits required to represent the maximum value of the elements in each generated difference array. The compressing unit 15e compresses the sorted integer sequence by representing the elements of each difference array using the identified number of bits and combining them.

[0052] In this way, the data compression device 10 compresses data using Warp Size, which makes it possible to utilize Warp Shuffle to take advantage of the parallelism of the GPU and compress data at a high compression rate so that high-throughput decompression processing of the compressed data is possible.

[0053] Data compressed in this way is effective, for example, for processing large sparse matrices. A sparse matrix is ​​a matrix in which most of the elements are zero, and is used in various real-world applications, such as graph theory and machine learning. Because sparse matrices in their original form consume more memory than necessary, they are often converted into formats such as CSR (Compressed Sparse Row). In CSR, the non-zero elements in each row of a sparse matrix are stored as a sorted integer sequence.

[0054] In recent years, there have been many applications in which GPUs are used as accelerators for computations, such as machine learning, and there is an increasing need to process data of sorted integer sequences on GPUs. Therefore, the data compression process by the data compression device 10 of this embodiment enables processing of large amounts of data, and is effectively applied to systems and applications that require sparse matrix computations.

[0055] The expansion unit 15f expands the compressed sorted integer sequence using the identified number of bits of each difference array and the position of each difference array in the compressed sorted integer sequence.

[0056] In this case, the expansion unit 15f expands the compressed sorted integer sequence by sharing register values ​​among a predetermined number of threads that perform simultaneous processing.

[0057] This allows the data compression device 10 to take advantage of the parallelism of the GPU to perform high-throughput decompression processing of compressed data.

[0058] [Program] A program written in a computer-executable language may be created to execute the processes performed by the data compression device 10 according to the above embodiment. In one embodiment, the data compression device 10 can be implemented by installing a data compression program that executes the data compression process as package software or online software on a desired computer. For example, by executing the data compression program on an information processing device, the information processing device can function as the data compression device 10. The information processing device referred to here includes desktop and notebook personal computers. Other examples of information processing devices include mobile communication terminals such as smartphones, mobile phones, and PHS (Personal Handyphone Systems), as well as slate terminals such as PDAs (Personal Digital Assistants). The functions of the data compression device 10 may also be implemented on a cloud server.

[0059] 8 is a diagram showing an example of a computer that executes a data compression program. The computer 1000 includes, for example, a memory 1010, a CPU 1020, a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0060] The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012. The ROM 1011 stores a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1031. The disk drive interface 1040 is connected to a disk drive 1041. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1041. The serial port interface 1050 is connected to a mouse 1051 and a keyboard 1052, for example. The video adapter 1060 is connected to a display 1061, for example.

[0061] Here, the hard disk drive 1031 stores, for example, an OS (Operating System) 1091, an application program 1092, a program module 1093, and program data 1094. The various pieces of information described in the above embodiments are stored in the hard disk drive 1031 or the memory 1010, for example.

[0062] The data compression program is stored in the hard disk drive 1031 as a program module 1093 in which instructions to be executed by the computer 1000 are written. Specifically, the program module 1093 in which each process executed by the data compression device 10 described in the above embodiment is written is stored in the hard disk drive 1031.

[0063] Furthermore, data used for information processing by the data compression program is stored as program data 1094, for example, in the hard disk drive 1031. Then, the CPU 1020 reads the program module 1093 and the program data 1094 stored in the hard disk drive 1031 into the RAM 1012 as necessary, and executes each of the above-described procedures.

[0064] The program module 1093 and program data 1094 related to the data compression program are not limited to being stored in the hard disk drive 1031, and may be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1041. Alternatively, the program module 1093 and program data 1094 related to the data compression program may be stored in another computer connected via a network such as a LAN or a WAN (Wide Area Network), and read by the CPU 1020 via the network interface 1070.

[0065] Although the present invention has been described above as an embodiment, the present invention is not limited to the description and drawings that form part of the disclosure of the present invention. In other words, other embodiments, examples, and operational techniques that can be made by those skilled in the art based on the present invention are all included in the scope of the present invention.

[0066] REFERENCE SIGNS LIST 10 Data compression device 11 Input unit 12 Output unit 13 Communication control unit 14 Storage unit 15 Control unit 15a Acquisition unit 15b Dividing unit 15c Generating unit 15d Identifying unit 15e Compressing unit 15f Decompressing unit

Claims

1. A data compression device comprising: a division unit that divides a sorted integer sequence into a predetermined number of simultaneous processes; a generation unit that calculates the difference between adjacent elements in each divided array to generate a difference array; a specification unit that specifies the number of bits required to express the maximum value of the elements in each generated difference array; and a compression unit that compresses the sorted integer sequence by expressing and combining the elements of each difference array using the specified number of bits.

2. The data compression device according to claim 1, further comprising an expansion unit that expands the compressed sorted integer sequence using the number of bits of each difference array and the position corresponding to each difference array in the compressed sorted integer sequence.

3. The data compression device according to claim 2, wherein the expansion unit expands the compressed sorted integer sequence by sharing register values ​​among a predetermined number of threads that perform simultaneous processing.

4. A data compression method executed by a data compression device, comprising: a division step of dividing a sorted integer sequence into a predetermined number of simultaneous processes; a generation step of calculating the difference between adjacent elements in each divided array to generate a difference array; a specification step of specifying the number of bits required to express the maximum value of the elements in each generated difference array; and a compression step of compressing the sorted integer sequence by expressing the elements of each difference array using the specified number of bits and combining them.

Images