Key-Value Storage and Data Prefetching Method Based on Caffe Application Features

Abstract

A key-value storage and data prefetching method based on Caffe application features, which relates to the field of computer storage, includes: setting the data block size in the SST file according to the bandwidth size of different storage devices; when image data is written into the key-value storage system, converting the image data into key-value pairs and storing them in the in-memory table; if the capacity of the in-memory table reaches the preset threshold, converting the image data into an immutable in-memory table and flushing it to the disk, and organizing it into an SST file; wherein, each SST file contains multiple data blocks for storing key-value pair data; when performing the data prefetching operation, matching the read disk I / O unit size with the bandwidth of the storage device; at the same time, when the image resolution is greater than the preset resolution, adopting a dual-thread parallel execution strategy in which one thread is responsible for reading and parsing key-value pairs and the other thread performs data format conversion; when reading key-value pairs, adopting a global out-of-order reading mechanism. The present invention improves the data prefetching efficiency and the test accuracy of the model.

Description

Technical Field

[0001] The present invention relates to the field of computer storage, and particularly to a key-value storage and data prefetching method based on Caffe application characteristics. Background Art

[0002] Modern data storage design is driven by three basic trends: 1) applications are storing more and more data; 2) many data perform more insert operations than read operations during their lifetime; 3) the popularity of cloud-based data management has promoted the development of immutable systems. Combining these trends, a large number of applications choose to adopt key-value storage systems based on the LSM tree in their storage layer, such as LevelDB, RocksDB, and Cassandra. The main idea of the LSM tree is to maintain a batch of random writes in the memory buffer. When the buffer is full, these key-value pairs are flushed to the storage device in the form of files to achieve high write throughput. In the storage device, the LSM tree maintains a hierarchical structure, and users can add levels and components as needed. Different from traditional indexes that perform in-place updates, the LSM tree performs out-of-place updates, converting multiple random disk I / Os into sequential disk I / Os, and continuously reorganizing data through merge operations to achieve better space utilization.

[0003] Data reading in the LSM tree includes point query and range query operations. The search order of the point query is similar to the data write order. First, search in the memory table, then search in the immutable memory table, and finally search layer by layer from layer L0 to L6 until the required data is found or the data does not exist. Compared with the point query, the range query operation is more complex. First, read the first data block of all SST files in layer L0 and the first SST file in layers L1 to L6 into the memory, and build iterator pointers for them with the memory table and the immutable memory table, pointing to the first data of each file. Then create the minimum pointer and the current pointer. Then start from the pointer pointing to the memory table, compare with the minimum pointer layer by layer in order. During each comparison, the minimum pointer always points to the data with a smaller key. After the comparison is over, the current pointer points to the minimum pointer, and the user reads the data pointed to by the current pointer. The comparison process is repeated until all the data within the query range is read.

[0004] The deep learning framework Caffe is used for model training and adopts LevelDB as its underlying storage engine. Due to the sequential data storage characteristic of LevelDB, image datasets are stored in the key-value storage system in key order (i.e., the write order). During model training, Caffe executes in parallel with two threads. One thread is used for the forward and backward passes of the model, and the other thread is used for data prefetching. During the data prefetching process, the processing of each image data mainly includes three steps: reading key-value pairs, parsing key-value pairs, and converting the data format. Reading key-value pairs means reading a key-value pair from LevelDB. Parsing key-value pairs means deserializing the value from a string into a Datum object. Converting the data format means performing data conversion on the Datum object, such as mirroring, scaling, cropping, etc. Since the disk I / O unit of LevelDB is only one data block (64MB), and the size of the key-value pairs corresponding to image data is relatively large. For example, the key-value pair size corresponding to a 28*28 pixel image is 3KB. Therefore, only a small amount of data can be read in one disk I / O operation, and multiple disk I / O operations need to be performed for one round of model training. And Caffe uses a global range query operation to read all data. After building the iterator pointer, it is necessary to compare the key values level by level starting from the memory table. However, because the smallest key value is always at the highest level, multiple pointer comparisons are required for each key-value pair reading operation. In the worst case, 12 comparisons are required for one read operation. At the beginning of each training round, Caffe will call the SeektoFirst function of LevelDB to reposition the pointer to the first data of each level and read the data in key order. Therefore, during the model training process, the order of input training instances within each round is the same. The model may use this input pattern as a method to reduce training loss, which is more likely to cause overfitting and thus lead to poor test accuracy. At the same time, because the three steps of reading key-value pairs, parsing key-value pairs, and converting the data format are executed serially, only one of the I / O resources and CPU resources can be utilized at the same time, resulting in serious resource waste and increasing the waiting time during the data prefetching process. Summary of the Invention

[0005] The main objective of the present invention is to overcome the above-mentioned defects in the prior art, and propose a key-value storage and data prefetching method based on the application characteristics of Caffe, which improves the utilization rate of computer system resources, reduces the data prefetching time, and inputs training instances in different orders in each round of Caffe training the model, reduces model overfitting, improves the model test accuracy, and realizes the improvement of model training performance.

[0006] The present invention adopts the following technical solutions:

[0007] A key-value storage and data prefetching method based on Caffe application features, applied to a key-value storage system, including:

[0008] Set the data block size in the sorted string table (SST) file according to the bandwidth of different storage devices;

[0009] When image data is written into the key-value storage system, convert the image data into key-value pairs and save them in the memory table; if the capacity of the memory table reaches the preset threshold, convert the image data into an immutable memory table and flush it to the disk, and organize it into an SST file; wherein, each SST file contains multiple data blocks for storing key-value pair data;

[0010] When performing data prefetching operations, match the size of the disk I / O unit read with the bandwidth of the storage device; at the same time, when the image resolution is greater than the preset resolution, adopt a two-thread parallel execution strategy where one thread is responsible for reading and parsing key-value pairs and the other thread performs data format conversion; when reading key-value pairs, adopt a global out-of-order reading mechanism;

[0011] The implementation process of the global out-of-order reading mechanism is as follows:

[0012] Read the first data block in all SST files in the L0 layer into the memory;

[0013] Use a random sorter to randomly sort the SST files in each level L1 to L6 of the LSM tree and record them in a random order table, and then load the first data block of the first SST file in each level L1 to L6 in the random order table into the memory, and create corresponding iterator pointers and current pointers for all data blocks in the memory; the iterator pointers point to the first key-value pair in each level of data blocks in turn, and the current pointer loops to point to the iterator pointer; among them, the iterator pointers correspond to the numbers 1 to n;

[0014] Caffe reads according to the data pointed to by the current pointer; when all the data in an SST is read, read the next SST file in the same level from the random order table and continue to perform the reading operation until all the data in the key-value storage system is read;

[0015] For the next round of data reading, reconstruct the iterator and the random order table to ensure that the input instance order is different in each round.

[0016] Preferably, one of the two threads is the first thread and the other is the second thread; the implementation process of the two-thread parallel execution strategy is as follows:

[0017] After the first thread finishes reading and parsing image i, the data will be written into the first buffer, and then the second thread will convert the data in the first buffer. Meanwhile, the first thread starts to read and parse image i + 1 and writes it into the second buffer. After the second thread finishes processing the data of image i, it starts to convert the data in the second buffer. The two buffers are used alternately, and a mutex is used to ensure safe access to the buffers.

[0018] Preferably, both the first buffer and the second buffer are located in the memory, used to temporarily store the image data read and parsed by the first thread, and the data is cyclically allocated between the two buffers through an index variable j. When a thread is using a certain buffer, a mutex is used to ensure that other threads cannot modify the data during the access to this buffer.

[0019] Preferably, the random sorter is located in the memory, used to read the index information of all SST files in layers L1 to L6, and shuffle the order of the files in each layer through a random number generation mechanism.

[0020] Preferably, the random order table is located in the memory, including two fields: the layer number and the file order. Among them, the layer number is used to record the numbers of layers L1 to L6, and the file order is used to save the file order of each layer after random sorting.

[0021] Preferably, the preset resolution is 64×64 pixels.

[0022] Preferably, the data prefetch operation specifically includes:

[0023] S101, the first thread sends a read request to the key-value storage system, and proceeds to S102;

[0024] S102, determine whether there are a memory table and an immutable memory table. If so, proceed to S103; otherwise, proceed to S104;

[0025] S103, create table iterators for the memory table and the immutable memory table. After creation, proceed to S104;

[0026] S104, create table iterators for all SST files in L0, and read the first data block of all files into the memory. After reading, proceed to S105;

[0027] S105, read the index information of all SST files in layers L1 to L6 respectively. The random sorter randomly sorts the file order of layers L1 to L6 and writes it into the random order table. After writing, proceed to 106;

[0028] S106. Create hierarchical iterators for L1 to L6 layers according to the file order in the random order table, read the first data block in the first SST file of each layer into memory, and then proceed to S107 after the reading;

[0029] S107. Merge all table iterators and hierarchical iterators into a global range query iterator, create n iterator pointers and a current pointer, with the iterator pointers pointing to data blocks in sequence and the current pointer pointing to the first iterator pointer, and then proceed to S108 after the pointing;

[0030] S108. The first thread reads the key-value pair pointed to by the current pointer, parses it and writes it into the buffer, and then proceeds to S109 after the writing;

[0031] S109. The second thread performs format conversion on the data in the buffer and writes it into an array. At the same time, the first thread executes the Next operation and the iterator pointer moves backward, and then proceeds to S110 after the movement;

[0032] S110. Determine whether there is still unread data in the system. If so, proceed to S111; otherwise, proceed to S114;

[0033] S111. Point the current pointer to the next iterator pointer with unread data, and then proceed to S112 after the pointing;

[0034] S112. Determine whether the number of data in the array reaches a batch size. If so, proceed to S113; otherwise, proceed to S108;

[0035] S113. Forward the array to other layers in the model and create a new array, and then proceed to S108 after the creation;

[0036] S114. Determine whether it is still necessary to perform the next round of data reading. If so, proceed to S101; otherwise, proceed to S115;

[0037] S115. The data prefetch is completed, and the global range query iterator is deleted.

[0038] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0039] (1) By adjusting the data block size, the present invention enables disk I / O operations to more effectively utilize the computer system bandwidth; at the same time, by adopting dual-thread parallel execution for data prefetch, it avoids the CPU idle waiting during the reading operation and the disk I / O idle waiting during the data conversion execution, thereby realizing the full utilization of computer system resources and improving the utilization rate of computer system resources;

[0040] (2) By increasing the data block size, the present invention enables more data to be read in one disk I / O operation, reducing the number of disk I / O operations. In addition, by using two threads to execute the data prefetch operation in parallel and parallelizing the read and parse steps with the data conversion steps, the time for data prefetch is significantly shortened. Meanwhile, by using an iterator pointer to sequentially read the data, unnecessary pointer comparisons are avoided, further improving the data prefetch efficiency.

[0041] (3) By randomly selecting SST files and using an iterator pointer to sequentially read the data, the present invention realizes the random input of data during the Caffe training model, thereby enhancing the stability of the model, reducing the risk of overfitting, and ultimately improving the test accuracy of the model. Brief Description of the Drawings

[0042] Figure 1 is the system structure diagram of the key-value storage and data prefetch method based on Caffe application features of the present invention;

[0043] Figure 2 is the schematic diagram of the data prefetch process of the key-value storage and data prefetch method based on Caffe application features of the present invention;

[0044] Figure 3 is the schematic diagram of out-of-order reading of the key-value storage and data prefetch method based on Caffe application features of the present invention. Detailed Embodiments

[0045] The following further elaborates the present invention in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and not to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms also fall within the scope defined by the appended claims of this application.

[0046] The key-value storage and data prefetch method based on Caffe application features proposed by the present invention includes the following steps.

[0047] First, bandwidth tests are performed on different storage devices, and the data block size in the sorted string table (SST) file is set according to the bandwidth size. When image data is written into the key-value storage system, it is first converted into key-value pairs and saved in the memory table. If the capacity of the memory table reaches the preset threshold, it is converted into an immutable memory table and flushed to the disk, and organized into an SST file. Each SST file contains multiple data blocks for storing key-value pair data. When performing the data prefetch operation, the size of the disk I / O unit read is matched with the bandwidth of the storage device, so as to make full use of the system bandwidth and transfer more data within the same transmission time. At the same time, the data prefetch operation adopts a dual-thread parallel execution strategy: one thread is responsible for reading and parsing key-value pairs, and the other thread performs data format conversion. Specifically, when the first thread (thread 1) finishes reading and parsing image i, the data is written into the first buffer (buffer 1), and then the second thread (thread 2) converts the data in the first buffer; at the same time, the first thread starts to read and parse image i + 1 and writes it into the second buffer (buffer 2). After the second thread finishes processing the data of image i, it starts to convert the data in the second buffer. The two buffers are used alternately, and a mutex lock is used to ensure safe access to the buffers. When reading key-value pairs, a global out-of-order reading mechanism is adopted. First, the first data block in all SST files in layer L0 is read into the memory. Then, a random sorter is used to randomly sort the SST files in each level from L1 to L6 in the LSM tree and record them in the random order table. Then, the first data block of the first SST file in each level from L1 to L6 in the random order table is loaded into the memory, and iterator pointers (1 - n) and a current pointer are created for all data blocks in the memory. The iterator pointer points to the first key-value pair in each level of data blocks in turn, and the current pointer loops to point to the iterator pointer. Caffe reads according to the data pointed to by the current pointer when training the model. After all the data in an SST is read, the next SST file in the same level is read from the random order table, and the reading operation continues until all the data in the key-value storage system is read. For the next round of data reading, the iterator and the random order table are reconstructed to ensure that the order of input instances is different in each round. Finally, in order to adapt to image data sets of different resolution sizes, an adaptive optimization mechanism is adopted. When the image resolution is less than 64×64 pixels, the dual-thread parallel strategy is not used; otherwise, the dual-thread parallel execution is enabled for data prefetching.

[0048] It should be noted that Caffe is a deep learning framework for image classification and image segmentation, supporting the design of CNN, RCNN, LSTM, and fully connected neural networks. Since the inventive point of the present invention involves the key-value storage system called by Caffe and the way of reading data, as long as the model can be deployed and trained in Caffe, the present invention can achieve out-of-order reading. Specifically, the model can be LeNet, ResNet50, CaffeNet, CIFAR10_full, etc.

[0049] See Figure 1 As shown, it is the system structure diagram of a key-value storage and data prefetching method based on Caffe application features proposed by the present invention, including a series of memory and disk components, such as a random sorter, a random order table, a buffer, an iterator pointer, a current pointer, an SST file, and data blocks. When model training is carried out, data prefetching is required to improve the model training efficiency. The data prefetching step includes four steps: 1) The first thread (Thread 1) initiates a read key-value pair operation on the key-value storage system; 2) Construct a global range query iterator, read the first data block of all SST files in layer L0 into the memory, sort the SST files in layers L1 to L6 using the random sorter, then read the first data block of the sorted first SST file in each layer into the memory, and then create an iterator pointer and a current pointer, and the current pointer loops to point to the iterator pointer; 3) Return the key-value pair data pointed to by the current pointer, and the first thread parses it and writes it into the buffer (Buffer 1 and Buffer 2); 4) The second thread (Thread 2) performs an operation to convert the data format of the data parsed by the first thread (Thread 1). At the same time, the first thread (Thread 1) continues to execute the operation of reading and parsing the next key-value pair data.

[0050] See Figure 2 As shown, it is the data prefetching flowchart of the key-value storage and data prefetching method based on Caffe application features, including the following steps.

[0051] 101, The first thread sends a read request to the key-value storage system, and goes to 102.

[0052] 102, Determine whether there is a memory table and an immutable memory table. If so, go to 103, otherwise go to 104.

[0053] 103, Create table iterators for the memory table and the immutable memory table. After creation, go to 104.

[0054] 104, Create table iterators for all SST files in L0, and read the first data block of all files into the memory. After reading, go to 105.

[0055] 105. Read the index information of all SST files in layers L1 to L6 respectively. The random sorter randomly sorts the file order of layers L1 to L6 and writes it into the random order table. After writing, go to 106.

[0056] 106. According to the file order in the random order table, create a hierarchical iterator for layers L1 to L6, and read the first data block in the first SST file of each layer into memory. After reading, go to 107.

[0057] 107. Merge all table iterators and hierarchical iterators into a global range query iterator, and create n iterator pointers and a current pointer. The iterator pointers point to the data blocks in turn, and the current pointer points to iterator pointer 1 (i.e., the first iterator pointer). After pointing, go to 108.

[0058] 108. The first thread reads the key-value pair pointed to by the current pointer, parses it and writes it into the buffer. After writing, go to 109.

[0059] 109. The second thread converts the data in the buffer into a format and writes it into an array. At the same time, the first thread performs the Next operation, and the iterator pointer moves backward. After moving, go to 110.

[0060] 110. Judge whether there is still unread data in the system. If so, go to 111, otherwise go to 114.

[0061] 111. Point the current pointer to the next iterator pointer with unread data. After pointing, go to 112.

[0062] 112. Judge whether the number of data in the array reaches a batch size. If so, go to 113, otherwise go to 108.

[0063] 113. Forward the array to other layers in the model and create a new array. After creating, go to 108.

[0064] 114. Judge whether it is still necessary to perform the next round of data reading. If so, go to 101, otherwise go to 115.

[0065] 115. The data prefetch is completed, and the global range query iterator is deleted.

[0066] See Figure 3 As shown in the figure, it is a schematic diagram of disordered reading of the key-value storage and data prefetch method based on the application characteristics of Caffe.

[0067] Figure 3Shows the process of randomly reading model training instances in two training rounds of the Caffe training model. In the first round, after being processed by the random sorter, the file orders of layers L1 to L6 are: L1 (SST13, SST11, SST12), ……, L6 (SST2, SST4, SST3, SST1). According to this order, the data block ranges read into memory include: u130 - u134, u140 - u144, u120 - u124, ……, u10 - u14. Among them, u130 - u134 and u140 - u144 belong to layer L0. In the initial stage of training, the first data blocks of all SST files in layer L0 need to be loaded into memory. Thus, the input order of model training instances in the first round is: 130, 140, 120, ……, 10, and then successively 131, 141, 121, ……, 11, and so on. In the second round, after being processed by the random sorter, the file orders of layers L1 to L6 become: L1 (SST11, SST13, SST12), ……, L6 (SST3, SST1, SST2, SST4). The corresponding data block ranges read include: u130 - u134, u140 - u144, u100 - u104, ……, u20 - u24. Therefore, the input order of model training instances in the second round is: 130, 140, 100, ……, 20, and then successively 131, 141, 101, ……, 21, and so on. If the model training includes multiple rounds, the operations in each round are the same as the above method.

[0068] The above is only the specific implementation manner of the present invention, but the design concept of the present invention is not limited thereto. Any non - substantial modification of the present invention using this concept shall fall within the scope of infringement of the protection scope of the present invention.

Claims

1. A key-value storage and data pre-fetching method based on Caffe application characteristics, applied in a key-value storage system, characterized in that: include: Set the data block size in the sort string table SST file according to the bandwidth size of different storage devices; When image data is written into the key-value storage system, the image data is converted into key-value pairs and saved in the memory table. If the capacity of the memory table reaches the preset threshold, the image data is converted into an immutable memory table and refreshed to the disk, organized into an SST file. Each SST file contains multiple data blocks for storing key-value pair data. When performing data pre-fetch operations, the disk I / O unit size to be read is matched with the bandwidth of the storage device. At the same time, when the image resolution is greater than the preset resolution, a dual-thread parallel execution strategy is adopted, in which one thread is responsible for reading and parsing key-value pairs and the other thread performs data format conversion. When reading key-value pairs, a global out-of-order reading mechanism is adopted. The implementation process of the global out-of-order reading mechanism is as follows: Read the first data block of all SST files in the L0 layer into memory; Use a random sorter to randomly sort the SST files of each level from L1 to L6 in the LSM tree and record them in the random order table. Then load the first data block of the first SST file of each level from L1 to L6 in the random order table into the memory, and create corresponding iterator pointers and current pointers for all data blocks in the memory; the iterator pointer points to the first key-value pair in the data block of each level in turn, and the current pointer points to the iterator pointer cyclically; among them, each iterator pointer corresponds to number 1~n; Caffe reads data according to the data pointed to by the current pointer; when all data in an SST is read, the next SST file of the same level is read from the random order table, and the reading operation continues until all data in the key-value storage system is read; For the next round of data reading, rebuild the iterator and random order table to ensure that the order of input instances in each round is different; One of the two threads is a first thread, and the other is a second thread; the implementation process of the dual-thread parallel execution strategy is as follows: When the first thread completes reading and parsing image i, the data will be written into the first buffer, and then the second thread will convert the data in the first buffer; at the same time, the first thread starts to read and parse image i+1 and write it into the second buffer, and the second thread starts to convert the data in the second buffer after processing the data of image i; the two buffers are used alternately, and the mutex lock is used to ensure the safe access to the buffer.

2. The key-value storage and data pre-fetching method based on Caffe application features according to claim 1, characterized in that: The first buffer and the second buffer are both located in the memory, and are used to temporarily store the image data read and parsed by the first thread, and the data is cyclically allocated between the two buffers through the index variable j. When a thread is using a buffer, a mutex lock is used to ensure that other threads cannot modify the data during the access of the buffer.

3. The key-value storage and data pre-fetching method based on Caffe application features according to claim 1, characterized in that: The random sorter is located in the memory and is used to read the index information of all SST files in the L1 to L6 layers, and disrupt the order of files in each layer through a random number generation mechanism.

4. The key-value storage and data pre-fetching method based on Caffe application features according to claim 1, characterized in that: The random order table is located in the memory and includes two fields: level number and file order; wherein the level number is used to record the numbers of the layers L1 to L6, and the file order is used to save the order of files of each level after random sorting.

5. The key-value storage and data pre-fetching method based on Caffe application features according to claim 1, characterized in that: The preset resolution is 64×64 pixels.

6. The key-value storage and data pre-fetching method based on Caffe application features according to claim 1, characterized in that: The data pre-fetching operation specifically includes: S101, the first thread sends a read request to the key-value storage system, to S102; S102, determine whether there is a memory table and an immutable memory table, if so, go to S103, otherwise, go to S104; S103, create table iterators for the memory table and the immutable memory table, and then go to S104; S104, create table iterators for all SST files in L0, and read the first data block of all files into memory, and then go to S105; S105, respectively read the index information of all SST files in layers L1 to L6, the random sorter randomly sorts the order of files in layers L1 to L6, and writes them into the random order table, and then goes to 106; S106, create a level iterator for L1 to L6 layers according to the file order in the random order table, and read the first data block in the first SST file of each layer into the memory, and then go to S107; S107, merge all table iterators and level iterators into a global range query iterator, and create n iterator pointers and a current pointer, the iterator pointers point to the data blocks in sequence, the current pointer points to the first iterator pointer, and then points to S108; S108, the first thread reads the key-value pair pointed to by the current pointer, parses it and writes it into the buffer, and then goes to S109; S109, the second thread converts the format of the data in the buffer and writes it into the array. At the same time, the first thread executes the Next operation, and the iterator pointer moves backward, and then moves to S110; S110, determine whether there is any unread data in the system, if so, go to S111, otherwise go to S114; S111, point the current pointer to the next iterator pointer with unread data, and then point to S112; S112, determine whether the number of data in the array reaches a batch size, if so, go to S113, otherwise go to S108; S113, forward pass the array to other layers in the model, and create a new array, and then go to S108; S114, determine whether the next round of data reading needs to be performed, if so, go to S101, otherwise go to S115; S115, data pre-fetching is completed, and the global range query iterator is deleted.

Images