Methods, apparatus and storage media for acquiring data

Abstract

This application discloses a method, apparatus, and storage medium for acquiring data, belonging to the field of storage. The method is applied to a computing device included in a data management system, which further includes a storage device for storing data blocks. The index information of the data blocks indicates multiple first data ranges. Each data block includes data located within the multiple first data ranges. The method includes: acquiring a second data range to be queried; reading the data block from the storage device, wherein the index information of the data block indicates an intersection between the first data ranges and the second data range; and acquiring data located within the second data range from the data block. This application can improve the performance of data acquisition.

Description

Technical Field

[0001] This application relates to the field of storage, and in particular to a method, apparatus and storage medium for acquiring data. Background Technology

[0002] A data management system includes computing devices and storage devices. The storage devices store data blocks, each containing multiple data items. The computing devices can retrieve data from the storage devices. In implementation, the computing devices can receive query requests, which include a range of data to be queried. They can then read a data block from the storage device and retrieve the data within that range from that data block.

[0003] For example, suppose the query request includes a range of values ​​greater than or equal to 4 and less than or equal to 10. After receiving the query request, the computing device reads a data block from the storage device and queries the data block for values ​​greater than or equal to 4 and less than or equal to 10.

[0004] The data block read by the computing device from the storage device may not include data within the query range, resulting in the inability to retrieve data within the query range from the data block, thus reducing the performance of data retrieval. Summary of the Invention

[0005] This application provides a method, apparatus, and storage medium for acquiring data, thereby improving the performance of data acquisition. The technical solution is as follows:

[0006] In a first aspect, this application provides a method for acquiring data. The method is applied to a computing device included in a data management system. The data management system also includes a storage device for storing data blocks. Index information of the data blocks is used to indicate multiple first data ranges. Each data block includes data located within the multiple first data ranges. In the method, a second data range to be queried is acquired; a data block is read from the storage device, where the index information of the data block indicates that there is an intersection between the first and second data ranges; and data located within the second data range is acquired from the data block.

[0007] After obtaining the second data range to be queried, the index information of the data block is retrieved. Since the index information is used to indicate multiple first data ranges, and the data block includes data located within multiple first data ranges, if there is an intersection between the first and second data ranges, the data block is read from the storage device. The presence of an intersection between the first and second data ranges indicates a high probability that the data block includes data located within the second data range, and thus a high probability of retrieving data within the second data range from that data block, thereby improving data retrieval performance.

[0008] In one possible implementation, multiple first data ranges correspond to multiple sub-data blocks. For each sub-data block, the sub-data block includes data located within the first data range corresponding to the sub-data block. The process involves: acquiring multiple sub-data blocks and multiple first data ranges; merging the multiple sub-data blocks to obtain a data block; acquiring index information for the data block based on the multiple first data ranges; and storing the data block and the index information for the data block in a storage device. Because multiple first data ranges of multiple sub-data blocks are acquired, and the index information for the data block is acquired based on the multiple first data ranges, the index information can indicate the multiple first data ranges.

[0009] In another possible implementation, multiple first data ranges correspond to multiple sub-data blocks. For each sub-data block, the sub-data block includes data located within its corresponding first data range. This sub-data block includes multiple first data of a first type and multiple second data of a second type. The first data range corresponding to this sub-data block includes both a first sub-data range and a second sub-data range. The multiple first data are located within the first sub-data range, and the multiple second data are located within the second sub-data range. Since the first data range corresponding to this sub-data block includes both first and second data of the first type and second type respectively, when retrieving data, the system aims to ensure that the same row of the retrieved data block includes both first and second data located within the second data range, thereby improving the accuracy of the retrieved data.

[0010] In another possible implementation, the multiple first data ranges include multiple first sub-data ranges and multiple second sub-data ranges. The second data ranges include third sub-data ranges of a first type and fourth sub-data ranges of a second type. The index information of the data block indicates that there is an intersection between the first data ranges and the second data ranges among the multiple first data ranges, including: an intersection between the first sub-data range and the third sub-data range of the target sub-data block among the multiple first sub-data ranges; and an intersection between the second sub-data range and the fourth sub-data range of the target sub-data block among the multiple second sub-data ranges. The multiple sub-data blocks include the target sub-data block. This maximizes the accuracy of data acquisition by ensuring that the same row of the acquired data block includes data located within the third sub-data range and data located within the fourth sub-data range.

[0011] In another possible implementation, the index information includes a bitmap comprising multiple bits, each bit corresponding to a multiple third data range. At least one of the third data ranges intersects with multiple first data ranges. The value of at least one bit corresponding to at least one third data range is a first bit value, and the values ​​of the bits in the bitmap other than the at least one bit are second bit values. Here, the bitmap has a smaller data size, thus reducing the amount of index information and the resources required to store it.

[0012] In another possible implementation, the index information includes multiple first data ranges. This simplifies the implementation of the index information, making it more flexible and convenient to obtain.

[0013] In another possible implementation, the bitmap comprises multiple bitmaps corresponding to multiple sub-data blocks. Multiple first data ranges correspond to multiple sub-data blocks, and the bitmap corresponding to each sub-data block includes multiple bits. These bits correspond to multiple third data ranges. At least one of the third data ranges intersects with the first data range corresponding to the sub-data block. The value of at least one bit corresponding to at least one third data range is a first bit value, and the values ​​of all bits in the bitmap corresponding to the sub-data block, excluding the at least one bit, are second bit values. This enriches the implementation methods of the bitmap and improves the flexibility of bitmap acquisition.

[0014] In another possible implementation, for each sub-data block, the first data range corresponding to the sub-data block includes a first sub-data range of a first type and a second sub-data range of a second type, and the bitmap corresponding to the sub-data block includes a first sub-bitmap of a first type and a second sub-bitmap of a second type.

[0015] The first sub-bitmap includes multiple bits, and the multiple bits in the first sub-bitmap correspond to multiple third data ranges. The value of at least one bit in the first sub-bitmap corresponding to at least one third data range that intersects with the first sub-data range is the first bit value, and the values ​​of the bits in the first sub-bitmap other than at least one bit are the second bit values.

[0016] The second sub-bitmap includes multiple bits, and the multiple bits in the second sub-bitmap correspond to multiple third data ranges. The value of at least one bit in the second sub-bitmap corresponding to at least one third data range that intersects with the second sub-data range is the first bit value, and the values ​​of the bits in the second sub-bitmap other than at least one bit are the second bit values.

[0017] Since the bitmap corresponding to this sub-data block includes a first sub-bitmap of the first type and a second sub-bitmap of the second type, this aims to ensure that the same row of the acquired data block includes data within the third sub-data range and data within the fourth sub-data range, thereby improving the accuracy of the acquired data.

[0018] In another possible implementation, the index information includes a bitmap. A creation request is received, comprising a first value, a second value, and the number of bits, where the first value is less than the second value. Based on the first value, the second value, and the number of bits, the bitmap is obtained. This allows bitmaps to be created on demand, producing bitmaps with structures that meet specific requirements, thus improving the flexibility of bitmap creation.

[0019] In another possible implementation, the storage device is also used to store index information, and the index information is read from the storage device. This eliminates the need to store the index information on the computing device, reducing the resource consumption of the computing device.

[0020] Secondly, this application provides an apparatus for acquiring data, used to perform the method in the first aspect or any possible implementation of the first aspect. Specifically, the apparatus includes units for performing the method in the first aspect or any possible implementation of the first aspect.

[0021] Thirdly, this application provides a computing device cluster, the computing device cluster including at least one computing device, each computing device including a processor and a memory;

[0022] The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method of the first aspect or any possible implementation thereof.

[0023] Fourthly, this application provides a computer program product containing instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method of the first aspect or any possible implementation thereof.

[0024] Fifthly, this application provides a computer-readable storage medium including computer program instructions that, when executed by a cluster of computing devices, perform the method of the first aspect or any possible implementation thereof.

[0025] In a sixth aspect, this application provides a chip including a memory and a processor, wherein the memory is used to store computer instructions, and the processor is used to call and execute the computer instructions from the memory to perform the method in the first aspect or any possible implementation thereof. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the structure of a data management system provided in an embodiment of this application;

[0027] Figure 2 This is a schematic diagram of a data merging method provided in an embodiment of this application;

[0028] Figure 3 This is a flowchart of a data storage method provided in an embodiment of this application;

[0029] Figure 4 This is a schematic diagram of a bitmap provided in an embodiment of this application;

[0030] Figure 5 This is a schematic diagram of another bitmap provided in an embodiment of this application;

[0031] Figure 6 This is a flowchart of a method for acquiring data provided in an embodiment of this application;

[0032] Figure 7 This is a schematic diagram of a data acquisition device provided in an embodiment of this application;

[0033] Figure 8 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0034] Figure 9 This is a schematic diagram of a cluster structure for acquiring data provided in an embodiment of this application;

[0035] Figure 10 This is a schematic diagram of another cluster structure for acquiring data provided in an embodiment of this application. Detailed Implementation

[0036] The following is an introduction to the relevant concepts involved in this application:

[0037] The first data range is the data range of the sub-data blocks, which is the data range obtained based on the minimum and maximum data included in the sub-data blocks. The sub-data blocks include data that are located within the first data range.

[0038] The second data range is the range of data to be queried.

[0039] The third data range is the data range corresponding to each bit in the bitmap.

[0040] See Figure 1This application provides a data management system 100, which includes a computing device 101 and a storage device 102. The computing device 101 can communicate with the storage device 102, store data in the storage device 102, and retrieve data from the storage device 102.

[0041] Optionally, the computing device 101 may also perform other operations on the storage device 102, such as controlling and / or managing the storage device 102.

[0042] Optionally, computing device 101 may be a physical device such as a server or terminal device, or it may be a service running on data management system 100. For example, if computing device 101 is a service, it may be a virtual instance running on data management system 100, which is used to run the service. This virtual instance may include one or more of the following: virtual machine, container, process, or thread, etc.

[0043] Optionally, storage device 102 may be a data storage engine or other database system. Optionally, storage device 102 may include one or more data tables or files, where data tables are used to store data blocks and files are used to store data blocks.

[0044] For example, storage device 102 may be a relational storage device, which includes one or more data tables for storing data blocks. As another example, storage device 102 may be an object storage system, which includes one or more object files for storing data blocks.

[0045] Alternatively, the data management system 100 may be a database system with a log-structured merge tree (LSM-Tree), a b+tree (a type of balanced multi-way search tree), or a balanced k-dimensional balanced tree (BKD-tree) structure.

[0046] In some embodiments, the process of the computing device 101 storing data to the storage device 102 can be as follows: the computing device 101 can receive multiple data blocks, which are referred to as multiple sub-data blocks for ease of explanation. Each time the computing device 101 receives a sub-data block, it stores that sub-data block in the storage device 102.

[0047] Then, computing device 101 reads multiple sub-data blocks from storage device 102, merges these sub-data blocks into a single data block, obtains the data range corresponding to the data block based on the minimum and maximum data included in the data block, and stores the mapping relationship between the data block and its corresponding data range in storage device 102. Optionally, the multiple sub-data blocks can be deleted from storage device 102.

[0048] In some embodiments, the amount of data in the merged data block is not large. The computing device 101 continues to receive multiple sub-data blocks, and upon receiving each sub-data block, stores it in the storage device 102. Then, it reads the multiple sub-data blocks from the storage device 102, merges them into a single data block, and stores the mapping relationship between the data block and its corresponding data range in the storage device 102. Optionally, the multiple sub-data blocks can be deleted from the storage device 102.

[0049] When multiple data blocks are merged, i.e., when multiple data blocks are stored in storage device 102, computing device 101 can read these multiple data blocks from storage device 102, merge the multiple data blocks into a target data block with a larger data volume, obtain the data range corresponding to the target data block based on the minimum and maximum data included in the target data block, and store the mapping relationship between the target data block and the data range corresponding to the target data block in storage device 102. Optionally, the multiple data blocks can be deleted from storage device 102.

[0050] For example, see Figure 2 The computing device 101 sequentially receives sub-data blocks 1, 2, 3, and 4 to be stored, and stores them in the storage device 102. Sub-data block 1 includes data within the range of 1-3, where 1-3 represents a range greater than or equal to 1 and less than or equal to 3. Sub-data block 1 includes multiple data values ​​greater than or equal to 1 and less than or equal to 3; that is, the minimum data value in sub-data block 1 is 1, and the maximum data value is 3. Similarly, sub-data block 2 includes data within the range of 2-4, sub-data block 3 includes data within the range of 4-6, and sub-data block 4 includes data within the range of 7-8.

[0051] Computing device 101 reads sub-data blocks 1, 2, 3, and 4 from storage device 102. Sub-data blocks 1, 2, 3, and 4 are merged into a single data block 9, where the minimum data value is 1 and the maximum data value is 8. Based on the minimum data value 1 and the maximum data value 8 of data block 9, the data range corresponding to data block 9 is determined to be 1-8. Then, the mapping relationship between data block 9 and the corresponding data range 1-8 can be stored in storage device 102, and sub-data blocks 1, 2, 3, and 4 can be deleted from storage device 102.

[0052] The computing device 101 sequentially receives sub-data blocks 5, 6, 7, and 8 to be stored, and sequentially stores the received sub-data blocks 5, 6, 7, and 8 into the storage device 102. Sub-data block 5 includes data located within data range 2-3, sub-data block 6 includes data located within data range 2-4, sub-data block 7 includes data located within data range 4-9, and sub-data block 8 includes data located within data range 21-22.

[0053] Computing device 101 reads sub-data blocks 5, 6, 7, and 8 from storage device 102. Sub-data blocks 5, 6, 7, and 8 are merged into a single data block 10, where data block 10 contains a minimum data value of 2 and a maximum data value of 22. Based on the minimum data value of 2 and the maximum data value of 22 contained in data block 10, the data range corresponding to data block 10 is determined to be 2-22. Then, the mapping relationship between data block 10 and the corresponding data range 2-22 can be stored in storage device 102, and sub-data blocks 5, 6, 7, and 8 can be deleted from storage device 102.

[0054] Since data blocks 9 and 10 may have relatively small amounts of data, computing device 101 can read data blocks 9 and 10 from storage device 102 and merge them into a single data block 11. Data block 11 contains a minimum value of 1 and a maximum value of 22. Based on these values, the data range corresponding to data block 11 is 1-22. Then, the mapping relationship between data block 11 and its corresponding data range 1-22 can be stored in storage device 102, and data blocks 9 and 10 can be deleted from storage device 102.

[0055] In some embodiments, the process of computing device 101 obtaining data from storage device 102 can be as follows: computing device 101 can receive a query request, the query request including a range of data to be queried. It can obtain the data range corresponding to a data block from storage device 102. If there is an intersection between the data range corresponding to the data block and the range of data to be queried, it can read the data block from storage device 102 and obtain data within the range of data to be queried from the data block.

[0056] This data block is obtained by merging multiple sub-data blocks. The data range of this data block is the range between the minimum and maximum data values ​​it includes. This range is relatively long, which may result in the data block not including a large amount of data within this range. Therefore, this data block may not include data within the query range, leading to the inability to retrieve data within the query range from this data block. In other words, computing device 101 reads extra data blocks from storage device 102 that do not actually contain data within the query range, resulting in a decrease in data retrieval performance.

[0057] For example, see Figure 2 For data block 11, data block 11 is obtained by merging sub-data blocks 1, 2, 3, 4, 5, 6, 7, and 8. The data range corresponding to data block 11 is the range between the minimum data 1 and the maximum data 22 included in the data block. The interval length of this data range is relatively large. This data block does not include data greater than 9 and less than 21.

[0058] Suppose that computing device 101 receives a query request containing a range of data greater than or equal to 10 and less than or equal to 20, i.e., the range of data to be queried is 10-20. Computing device 101 retrieves data block 11 from storage device 102, corresponding to a data range of 1-22. This data range (1-22) overlaps with the range of data to be queried (10-20), so data block 11 is read from storage device 102. However, since data block 11 does not actually contain data greater than 9 and less than 21, computing device 101 cannot retrieve data within the range of data to be queried (10-20) from data block 11. Therefore, computing device 101 reads an extra data block 11 from storage device 102 that does not actually contain data within the range of data to be queried (10-20), resulting in a performance degradation in data retrieval.

[0059] In some embodiments, storage device 102 may be a time-series storage device, and the data stored in storage device 102 has the following characteristics.

[0060] Feature 1: Adjacent data for the same indicator are relatively consistent. For the same indicator of the same object, the state of data coming in at adjacent time periods is "continuous," relatively consistent, and with small fluctuations.

[0061] For example, suppose the object is a vehicle and the index is the vehicle's speed. The speed of the same vehicle does not change much in two consecutive seconds, and the speed in those two seconds is continuous. For a column of data in storage device 102 used to store the speed of the vehicle, the column of data includes the speed of the vehicle at different timestamps, and the speed stored in the column of data changes continuously.

[0062] Feature 2: Query patterns are relatively uniform in the monitoring and maintenance field. Users primarily use these methods in the maintenance and monitoring areas, resulting in relatively standardized and fixed query statements and conditions.

[0063] For example, when querying abnormal speeds, abnormal speeds on highways are typically less than 80 km / h or greater than 120 km / h. Therefore, when querying abnormal speeds, the range of data to be queried in each query request received by computing device 101 is typically for speeds less than 80 km / h or greater than 120 km / h. In other words, the range of data to be queried is usually less than 80 km / h and / or greater than 120 km / h.

[0064] The above-described process of storing and retrieving data often results in the computing device 101 reading data blocks from the storage device 102 that do not actually fall within the range of the data to be queried, leading to a decrease in data retrieval performance. To solve this problem, the embodiments of this application store or retrieve data through the following embodiments.

[0065] See Figure 3 This application provides a method 300 for storing data, which can be applied to... Figure 1 The data management system 100 shown. The method 300 includes the following process.

[0066] Step 301: The computing device acquires multiple sub-data blocks and multiple first data ranges corresponding to the multiple sub-data blocks.

[0067] For each of the plurality of sub-data blocks, the first data range corresponding to the sub-data block is the range between the minimum and maximum data included in the sub-data block.

[0068] The computing device can receive storage requests, which include one or more sub-blocks of data to be stored.

[0069] In some embodiments, after receiving a storage request, the computing device may first save the sub-data blocks included in the storage request to the storage device. The computing device may receive multiple storage requests and save the sub-data blocks included in the multiple received storage requests to the storage device.

[0070] Optionally, when the number of multiple sub-data blocks stored in the storage device exceeds a number threshold or the total data volume of the multiple sub-data blocks stored in the storage device exceeds a data volume threshold, the computing device can retrieve the multiple sub-data blocks stored in the storage device and retrieve the multiple first data ranges corresponding to the multiple sub-data blocks.

[0071] Optionally, the computing device can periodically merge sub-data blocks into larger data blocks. That is, when the time elapsed since the last merging of sub-data blocks reaches a period length, the computing device retrieves the stored sub-data blocks from the storage device and retrieves the multiple first data ranges corresponding to the multiple sub-data blocks. The multiple sub-data blocks are the sub-data blocks received by the computing device in the current period.

[0072] In some embodiments, the computing device may include memory, and upon receiving a storage request, the computing device may first store the sub-data blocks included in the storage request in local memory. The computing device may receive multiple storage requests and store the sub-data blocks included in the multiple received storage requests in local memory.

[0073] Optionally, when the number of sub-data blocks stored in local memory exceeds a number threshold or the total data volume of the sub-data blocks stored in local memory exceeds a data volume threshold, the computing device can obtain multiple sub-data blocks stored in local memory and multiple first data ranges corresponding to the multiple sub-data blocks.

[0074] Optionally, the computing device can periodically store data in local memory into the storage device. That is, when the time elapsed since the last time the computing device stored data into the storage device reaches a period length, it obtains multiple sub-data blocks stored in local memory and multiple first data ranges corresponding to the multiple sub-data blocks. These multiple sub-data blocks are the sub-data blocks received by the computing device in the current period.

[0075] Optionally, for each sub-data block included in the storage request, the storage request may include a first data range corresponding to each sub-data block, and the computing device can obtain the first data range corresponding to each sub-data block from the storage request. Alternatively, the storage request may not include the first data range corresponding to each sub-data block, and the computing device obtains the first data range corresponding to each sub-data block based on the minimum and maximum data included in each sub-data block.

[0076] In some embodiments, for any sub-data block, the sub-data block may include one type of data, and the sub-data block corresponds to a first data range. Alternatively, the sub-data block includes multiple types of data, and the first data range corresponding to the sub-data block includes multiple sub-data ranges of the multiple types.

[0077] For example, suppose the sub-data block includes multiple first data of a first type and multiple second data of a second type, and the first data range corresponding to the sub-data block includes a first sub-data range and a second sub-data range, with the multiple first data located within the first sub-data range and the multiple second data located within the second sub-data range.

[0078] Next, referring to the example shown in Table 1, the computing device receives a storage request, which includes a first sub-data block and a second sub-data block. The first sub-data block consists of five rows and two columns of data, with one column representing speed and the other representing current. That is, the first sub-data block includes two types of data: speed and current. The first data range corresponding to the first sub-data block includes a first sub-data range of 100-130 for speed and a second sub-data range of 0.6-0.9 for current.

[0079] The second sub-data block also includes five rows and two columns of data, with one column representing speed and the other representing current. In other words, the second sub-data block contains two types of data: speed and current. The first data range corresponding to the second sub-data block includes a speed range of 80-85 and a current range of 0.2-0.3.

[0080] Table 1

[0081]

[0082] Step 302: The computing device merges the multiple sub-data blocks to obtain a data block.

[0083] In step 302, the computing device may merge the plurality of sub-data blocks to obtain a data block, such that the data block includes the plurality of sub-data blocks. Alternatively, the computing device may compress the plurality of sub-data blocks, merge the compressed plurality of sub-data blocks to obtain a data block, such that the data block includes the plurality of sub-data blocks.

[0084] For example, in the example shown in Table 1 above, the computing device obtains a first sub-data block and a second sub-data block, and merges the first sub-data block and the second sub-data block to obtain a single data block.

[0085] Step 303: The computing device obtains the index information of the data block based on the plurality of first data ranges, the index information being used to indicate the plurality of first data ranges.

[0086] In step 303, the following three types of index information are listed. Of course, in addition to the three types of index information listed below, there may be other types of index information, which will not be listed and described one by one in this embodiment of the application.

[0087] In the first case, the index information includes multiple first data ranges corresponding to the multiple sub-data blocks, so that the index information is used to indicate the multiple first data ranges.

[0088] Optionally, the first data range corresponding to each sub-data block may include one or more sub-data ranges. That is, the index information includes one or more sub-data ranges corresponding to each sub-data block.

[0089] For example, suppose each sub-data block includes data of a first type and data of a second type, and the first data range corresponding to the sub-data block includes a first sub-data range of the first type and a second sub-data range of the second type. Therefore, the multiple first data ranges include multiple first sub-data ranges of the first type and multiple second sub-data ranges of the second type, that is, the index information includes multiple first sub-data ranges of the first type and multiple second sub-data ranges of the second type.

[0090] For example, in the example shown in Table 1 above, the first data range corresponding to the first sub-data block includes the first sub-data range of speed (100-130) and the second sub-data range of current (0.6-0.9). The first data range corresponding to the second sub-data block includes the first sub-data range of speed (80-95) and the second sub-data range of current (0.2-0.3). Therefore, for the data block obtained by merging the first and second sub-data blocks, the index information of this data block includes the first sub-data range of speed (80-95) and the first sub-data range of speed (100-130), and the second sub-data range of current (0.2-0.3) and the second sub-data range of current (0.6-0.9).

[0091] For any data range (sub-data range) included in the index information, the data range needs to be represented using one or two threshold values, each threshold value occupies N bytes, where N is an integer greater than 1. A data range needs to occupy N or 2*N bytes. Since the index information of this data block includes multiple data ranges, the index information occupies multiple bytes, requiring a large amount of storage resources to store the index information.

[0092] For example, a data range of 100-130 needs to be represented using threshold values ​​of 100 and 130. Assuming each threshold requires 8 bytes, a data range requires 16 bytes. In the example shown in Table 1 above, the index information includes four sub-data ranges: the first sub-data range for speed (80-85) and the second sub-data range for current (100-130), and the third sub-data range for current (0.2-0.3) and the fourth sub-data range for current (0.6-0.9). This index information requires 64 bytes of storage resources, which is somewhat wasteful.

[0093] The second method involves indexing multiple bitmaps corresponding to the multiple sub-data blocks. For each sub-data block, the corresponding bitmap includes multiple bits, each bit corresponding to a third data range. At least one of the third data ranges intersects with the first data range corresponding to the sub-data block. The value of at least one bit corresponding to the at least one third data range is the first bit value, and the values ​​of the bits in the bitmap other than the at least one bit are the second bit values.

[0094] The index information includes multiple bitmaps corresponding to multiple sub-data blocks, so that the index information is used to indicate the multiple first data ranges.

[0095] Optionally, the sub-data block may include multiple types of data, and the bitmap corresponding to the sub-data block may include multiple sub-bitmaps of multiple types. The first data range corresponding to the sub-data block may include multiple sub-data ranges of multiple types. For each type, the sub-bitmap of that type includes multiple bits, which correspond one-to-one with multiple third data ranges. At least one of the multiple third data ranges intersects with the sub-data range of that type. The value of at least one bit corresponding to the at least one third data range is the first bit value, and the values ​​of the bits in the sub-bitmap of that type other than the at least one bit are the second bit values. Therefore, the index information includes multiple sub-bitmaps corresponding to each sub-data block in the multiple sub-data blocks.

[0096] Optionally, the first bit is 1 and the second bit is 0. Or, the first bit is 0 and the second bit is 1.

[0097] For example, suppose each sub-data block includes data of a first type and data of a second type, and the first data range of the sub-data block includes a first sub-data range of the first type and a second sub-data range of the second type. The bitmap of the sub-data block includes a first sub-bitmap of the first type and a second sub-bitmap of the second type.

[0098] The first sub-bitmap of the first type includes multiple bits, each bit corresponding to a multiple third data range. At least one of the multiple third data ranges intersects with the first sub-data range of the first type. The value of at least one bit corresponding to the at least one third data range is the first bit value, and the values ​​of the bits in the first sub-bitmap of the first type other than the at least one bit are the second bit values.

[0099] The second sub-bitmap of the second type includes multiple bits, each bit corresponding to a multiple third data range. At least one of the multiple third data ranges intersects with the second sub-data range of the second type. The value of at least one bit corresponding to the at least one third data range is the first bit value, and the values ​​of the bits in the second sub-bitmap of the second type other than the at least one bit are the second bit values.

[0100] Therefore, each sub-data block corresponds to a first sub-bitmap of type 1 and a second sub-bitmap of type 2. In other words, the index information includes multiple first sub-bitmaps and multiple second sub-bitmaps corresponding to multiple sub-data blocks.

[0101] For example, in the example shown in Table 1 above, the first data range corresponding to the first sub-data block includes a first sub-data range of velocity (100-130) and a second sub-data range of current (0.6-0.9). See also Figure 4 The first sub-data block includes a first sub-bit map a for velocity and a second sub-bit map a for current. The first sub-bit map a comprises eight bits, each corresponding to one of the eight third data ranges. The second sub-bit map a comprises eight bits, each corresponding to one of the eight third data ranges.

[0102] The third data range a11 corresponding to the first bit of the first sub-bitmap a is less than 60; the third data range a12 corresponding to the second bit of the first sub-bitmap a is greater than or equal to 60 and less than 70 (represented by [60, 70)); the third data range a13 corresponding to the third bit of the first sub-bitmap a is greater than or equal to 70 and less than 80 (represented by [70, 80)); the third data range a14 corresponding to the fourth bit of the first sub-bitmap a is greater than or equal to 80 and less than 90 (represented by [80, 90)); the first sub-bitmap a... The third data range a15 corresponding to the fifth bit is greater than or equal to 90 and less than 100 (represented by [90, 100)). The third data range a16 corresponding to the sixth bit of the first sub-bitmap a is greater than or equal to 100 and less than 110 (represented by [100, 110)). The third data range a17 corresponding to the seventh bit of the first sub-bitmap a is greater than or equal to 110 and less than 120 (represented by [110, 120)). The third data range a18 corresponding to the eighth bit of the first sub-bitmap a is greater than or equal to 120.

[0103] The third data ranges that intersect with the first sub-data range 100-130 are the third data range a16, the third data range a17, and the third data range a18. The sixth bit corresponding to the third data range a16, the seventh bit corresponding to the third data range a17, and the eighth bit corresponding to the third data range a18 of the first sub-bitmap a are all the first bit value 1. The first bit, the second bit, the third bit, the fourth bit, and the fifth bit of the first sub-bitmap a are all the second bit value 0.

[0104] The third data range a21 corresponding to the first bit of the second sub-bitmap a is less than 0.2; the third data range a22 corresponding to the second bit of the second sub-bitmap a is greater than or equal to 0.2 and less than 0.3 (represented as [0.2, 0.3)); the third data range a23 corresponding to the third bit of the second sub-bitmap a is greater than or equal to 0.3 and less than 0.4 (represented as [0.3, 0.4)); the third data range a24 corresponding to the fourth bit of the second sub-bitmap a is greater than or equal to 0.4 and less than 0.5 (represented as [0.4, 0.5)); the... The third data range a25 corresponding to the fifth bit of the second sub-bitmap a is greater than or equal to 0.5 and less than 0.6 (represented by [0.5, 0.6)). The third data range a26 corresponding to the sixth bit of the second sub-bitmap a is greater than or equal to 0.6 and less than 0.7 (represented by [0.6, 0.7)). The third data range a27 corresponding to the seventh bit of the second sub-bitmap a is greater than or equal to 0.7 and less than 0.8 (represented by [0.7, 0.8)). The third data range a28 corresponding to the eighth bit of the second sub-bitmap a is greater than or equal to 0.8.

[0105] The third data ranges that intersect with the second sub-data range of 0.6-0.9 are the third data range a26, the third data range a27, and the third data range a28. The sixth bit corresponding to the third data range a26, the seventh bit corresponding to the third data range a27, and the eighth bit corresponding to the third data range a28 of the second sub-bitmap a are all the first bit value 1. The first, second, third, fourth, and fifth bits of the second sub-bitmap a are all the second bit value 0.

[0106] The first data range corresponding to the second sub-data block includes the first sub-data range for velocity (80-85) and the second sub-data range for current (0.2-0.3). See also Figure 4 The second sub-data block includes a first sub-bitmap b for velocity and a second sub-bitmap b for current. The first sub-bitmap b comprises eight bits, each corresponding to one of the eight third data ranges. The second sub-bitmap b comprises eight bits, each corresponding to one of the eight third data ranges.

[0107] The third data range b11 corresponding to the first bit of the first sub-bitmap b is less than 60; the third data range b12 corresponding to the second bit of the first sub-bitmap b is greater than or equal to 60 and less than 70 (represented by [60, 70)); the third data range b13 corresponding to the third bit of the first sub-bitmap b is greater than or equal to 70 and less than 80 (represented by [70, 80)); the third data range b14 corresponding to the fourth bit of the first sub-bitmap b is greater than or equal to 80 and less than 90 (represented by [80, 90)); the first sub-bitmap b... The third data range b15 corresponding to the fifth bit is greater than or equal to 90 and less than 100 (represented by [90, 100)). The third data range b16 corresponding to the sixth bit of the first sub-bitmap b is greater than or equal to 100 and less than 110 (represented by [100, 110)). The third data range b17 corresponding to the seventh bit of the first sub-bitmap b is greater than or equal to 110 and less than 120 (represented by [110, 120)). The third data range b18 corresponding to the eighth bit of the first sub-bitmap b is greater than or equal to 120.

[0108] The third data range that intersects with the first sub-data range 80-85 is the third data range b14. The value of the fourth bit of the first sub-bit map b corresponding to the third data range b14 is the first bit value 1. The values ​​of the first bit, second bit, third bit, fifth bit, sixth bit, seventh bit and eighth bit of the first bit map b are all the second bit value 0.

[0109] The third data range b21 corresponding to the first bit of the second sub-bitmap b is less than 0.2; the third data range b22 corresponding to the second bit of the second sub-bitmap b is greater than or equal to 0.2 and less than 0.3 (represented as [0.2, 0.3)); the third data range b23 corresponding to the third bit of the second sub-bitmap b is greater than or equal to 0.3 and less than 0.4 (represented as [0.3, 0.4)); the third data range b24 corresponding to the fourth bit of the second sub-bitmap b is greater than or equal to 0.4 and less than 0.5 (represented as [0.4, 0.5)); the... The third data range b25 corresponding to the fifth bit of the second sub-bitmap b is greater than or equal to 0.5 and less than 0.6 (represented by [0.5, 0.6)). The third data range b26 corresponding to the sixth bit of the second sub-bitmap b is greater than or equal to 0.6 and less than 0.7 (represented by [0.6, 0.7)). The third data range b27 corresponding to the seventh bit of the second sub-bitmap b is greater than or equal to 0.7 and less than 0.8 (represented by [0.7, 0.8)). The third data range b28 corresponding to the eighth bit of the second sub-bitmap b is greater than or equal to 0.8.

[0110] The third data ranges that intersect with the second sub-data range 0.2-0.3 are the third data range b22 and the third data range b23. The value of the second bit corresponding to the third data range b22 and the value of the third bit corresponding to the third data range b23 in the second sub-bitmap b are both the first bit value 1. The values ​​of the first, fourth, fifth, sixth, seventh and eighth bits of the second bitmap b are all the second bit value 0.

[0111] Therefore, in this example, for the data block obtained by merging the first sub-data block and the second sub-data block, the index information of the data block includes the first sub-bitmap a and the second sub-bitmap a corresponding to the first sub-data block, and the first sub-bitmap b and the second sub-bitmap b corresponding to the second sub-data block.

[0112] In the second type of indexing information, one bitmap corresponds to one sub-data range, and one bitmap typically occupies one or two bytes. Usually, the number of bytes occupied by one bitmap is less than the number of bytes occupied by a sub-data range. This reduces the storage resources required to store the second type of indexing information compared to the first type. As shown in the example in Table 1 above, the first sub-bitmap a, the second sub-bitmap a, the first sub-bitmap b, and the second sub-bitmap b each occupy one byte. That is, the second type of indexing information includes these four sub-bitmaps, occupying a total of 4 bytes, compared to the 64 bytes occupied by the first type of indexing information, thus reducing the storage resources required to store the second type of indexing information.

[0113] The second type of index information includes one or more sub-bitmaps corresponding to each sub-data block. The number of sub-bitmaps is large, which increases the storage resources required to store the sub-bitmaps.

[0114] Optionally, the structure of the bitmap (sub-bitmap) can be pre-configured in the computing device, or the structure of the bitmap (sub-bitmap) can be created according to the user's needs.

[0115] For example, a user can send a creation request to a computing device. The creation request includes a first value, a second value, and a number M of bits, where the first value is less than the second value. The computing device receives the creation request and, based on the first value, the second value, and M, obtains a bitmap.

[0116] Optionally, in implementation, the computing device configures M third data ranges based on the first value, the second value, and M, and configures a bitmap including M bits, wherein the M bits correspond one-to-one with the M third data ranges.

[0117] Optionally, for the M third data ranges, one third data range less than the first value can be configured, and another third data range greater than or equal to the second value can be configured. Based on M-2 and the difference between the second value and the first value, M-2 third data ranges are obtained.

[0118] Optionally, the creation request can be CREATE INDEX index1(min, max, M) ON table(column), where min is the first value, max is the second value, and column is a column in the data table table, used to indicate the data type.

[0119] For example, regarding the reference Figure 4 In the first sub-bitmap a, the computing device receives a creation request as CREATE INDEX index1(60, 120, 8) ON table1(column1). table1 is the aforementioned table 1, column1 is the first column in the data table table1, representing the data type as speed, index1 is the sub-bitmap of speed, the first value is 60, the second value is 120, M is 8, M-2 is 6, and the difference between the second value and the first value is 60.

[0120] The computing device can be configured with a third data range a11 less than 60, and a third data range a18 greater than or equal to 120. Based on 6 and the difference 60, six third data ranges are obtained. These six third data ranges are a12 greater than or equal to 60 and less than 70, a13 greater than or equal to 70 and less than 80, a14 greater than or equal to 80 and less than 90, a15 greater than or equal to 90 and less than 100, a16 greater than or equal to 100 and less than 110, and a17 greater than or equal to 110 and less than 120.

[0121] The configuration includes an 8-bit first sub-bitmap a. The first bit of the first sub-bitmap a corresponds to the third data range a11, the second bit of the first sub-bitmap a corresponds to the third data range a12, the third bit of the first sub-bitmap a corresponds to the third data range a13, the fourth bit of the first sub-bitmap a corresponds to the third data range a14, the fifth bit of the first sub-bitmap a corresponds to the third data range a15, the sixth bit of the first sub-bitmap a corresponds to the third data range a16, the seventh bit of the first sub-bitmap a corresponds to the third data range a17, and the eighth bit of the first sub-bitmap a corresponds to the third data range a18.

[0122] The third type includes a bitmap. For multiple first data ranges corresponding to multiple sub-data blocks, the bitmap includes multiple bits, each bit corresponding to a multiple third data range. At least one of the multiple third data ranges intersects with the multiple first data ranges. The value of at least one bit corresponding to the at least one third data range is a first bit value, and the values ​​of the bits in the bitmap other than the at least one bit are second bit values.

[0123] Therefore, the index information includes a bitmap, which is used to record the plurality of first data ranges, so that the index information is used to indicate the plurality of first data ranges.

[0124] For each of the multiple sub-data blocks, if the sub-data block contains data of one type, obtain a third type of index information. Alternatively, if the data used for indexing in the sub-data block is of one type, obtain a third type of index information. For example, if the sub-data block contains multiple key-value pairs, the first data range corresponding to the sub-data block is the key range, and the type of the data used for indexing is key, then obtain the third type of index information.

[0125] Next, referring to the example shown in Table 2, the first sub-data block consists of five rows and one column of data, with each column representing a speed. In other words, the first sub-data block contains only one type of data. The first data range corresponding to the first sub-data block is the speed range of 100-130.

[0126] The second sub-data block also consists of five rows and one column of data, with each column representing a speed. In other words, the second sub-data block contains only one type of data. The first data range corresponding to the second sub-data block is the speed range of 80-85.

[0127] Table 2

[0128]

[0129]

[0130] In the example shown in Table 2 above, the bitmap in the obtained index information includes eight bits, which correspond one-to-one with eight third data ranges. The third data range 11 corresponding to the first bit of the bitmap is a range less than 60; the third data range 12 corresponding to the second bit of the bitmap is a range greater than or equal to 60 and less than 70 (represented by [60, 70)); the third data range 13 corresponding to the third bit of the bitmap is a range greater than or equal to 70 and less than 80 (represented by [70, 80)); the third data range 14 corresponding to the fourth bit of the bitmap is a range greater than or equal to 80 and less than 90 (represented by [80, 90)); the fifth bit of the bitmap... The third data range 15 corresponding to the bitmap is greater than or equal to 90 and less than 100 (represented by [90, 100)). The third data range 16 corresponding to the sixth bit of the bitmap is greater than or equal to 100 and less than 110 (represented by [100, 110)). The third data range 17 corresponding to the seventh bit of the bitmap is greater than or equal to 110 and less than 120 (represented by [110, 120)). The third data range 18 corresponding to the eighth bit of the bitmap is greater than or equal to 120.

[0131] See Figure 5The third data ranges that intersect with the first data range 100-130 corresponding to the first sub-data block are the third data range 16, the third data range 17, and the third data range 18. The third data range that intersects with the first data range 80-85 corresponding to the second sub-data block is the third data range 14. The values ​​of the fourth bit corresponding to the third data range 14, the sixth bit corresponding to the third data range 16, the seventh bit corresponding to the third data range 17, and the eighth bit corresponding to the third data range 18 in the bitmap are all the first bit value 1, and the values ​​of the first bit, the second bit, the third bit, and the fifth bit in the bitmap are all the second bit value 0.

[0132] The third type of index information includes a bitmap, thus reducing the storage resources required to store the third type of index information.

[0133] Step 304: The computing device saves the mapping relationship between the data block and the index information in the storage device.

[0134] In some embodiments, the computing device may also store the data block in a storage device and store the mapping relationship between the identification information of the data block and the index information locally on the computing device.

[0135] By repeating steps 301-304 above, multiple data blocks and their index information can be obtained. Alternatively, multiple data blocks and their index information can be saved to the storage device, and the mapping relationship between the identification information and the index information of the multiple data blocks can be stored locally on the computing device.

[0136] In this embodiment, when merging multiple sub-data blocks into one data block, the index information of the data block is obtained. This index information is used to indicate multiple first data ranges. These multiple first data ranges replace the data range corresponding to the data block. Since the data range corresponding to the data block is the range between the minimum and maximum data in the data block, the interval length of the data range corresponding to the data block is much larger than the length of each first data range. Therefore, when retrieving data based on the data range to be queried, the accuracy of the determination result is improved by using the data range to be queried and the multiple first data ranges to determine whether the data block includes data within the data range to be queried. This improves the accuracy of the determination result and encourages the reading of data blocks from the storage device that include data within the data range to be queried, thereby reducing the number of data blocks that do not include data within the data range to be queried, thus improving the performance of data retrieval.

[0137] See Figure 6This application provides a method 600 for acquiring data, which can be applied to... Figure 1 The data management system 100 shown. The method 600 includes the following process.

[0138] Step 601: The computing device obtains the second data range to be queried.

[0139] In step 601, the computing device may receive a query request that includes a second range of data to be queried.

[0140] In some embodiments, the second data range includes multiple sub-data ranges of various types. For example, the second data range includes a third sub-data range of a first type and a fourth sub-data range of a second type.

[0141] Next, using the example shown in Table 1 above, the first type is speed, and the second type is current. The computing device receives a first query request, which includes a second data range to be queried. The second data range includes a third sub-data range of speed (105-115) and a fourth sub-data range of current (0.6-0.7). The third sub-data range 105-115 represents values ​​greater than or equal to 105 and less than or equal to 115, and the fourth sub-data range 0.6-0.7 represents values ​​greater than or equal to 0.6 and less than or equal to 0.7.

[0142] For example, the computing device receives a second query request, which includes a second data range to be queried. The second data range includes a third sub-data range of speed (90-95) and a fourth sub-data range of current (0.3-0.5). The third sub-data range (90-85) represents values ​​greater than or equal to 90 and less than or equal to 95, and the fourth sub-data range (0.3-0.5) represents values ​​greater than or equal to 0.3 and less than or equal to 0.5.

[0143] Step 602: The computing device obtains the index information of the target data block, which is a data block stored in the storage device. The index information is used to indicate multiple first data ranges. The target data block includes data located within the multiple first data ranges.

[0144] In step 602, the storage device stores a mapping relationship between target data blocks and their index information, and the computing device can read the index information of the target data blocks stored in the storage device. Alternatively,

[0145] The computing device stores a mapping relationship between the identification information of the target data block and the index information of the target data block, and the computing device reads the index information of the target data block from the local machine.

[0146] The target data block comprises multiple sub-data blocks, and the target data block is adopted... Figure 3 The index information of the target data block stored in the storage device by the method 300 shown is obtained by using... Figure 3 The method shown in 300 is used to obtain the result.

[0147] Step 603: The computing device obtains multiple first data ranges indicated by the index information based on the index information.

[0148] In some embodiments, the index information is a first type of index information, which includes a plurality of first data ranges, the plurality of first data ranges being data ranges corresponding to a plurality of sub-data blocks.

[0149] Optionally, each sub-data block includes multiple types of data, and the first data range of each sub-data block includes multiple sub-data ranges of multiple types. For example, each sub-data block includes data of a first type and data of a second type, and the multiple first data ranges include multiple first sub-data ranges of the first type and multiple second sub-data ranges of the second type.

[0150] For example, in the example shown in Table 1 above, the plurality of first data ranges include a first sub-data range of speed of 80-85 and a first sub-data range of speed of 100-130, and a second sub-data range of current of 0.2-0.3 and a second sub-data range of current of 0.6-0.9.

[0151] In some embodiments, the index information is a second type of index information, which includes multiple bitmaps corresponding to multiple sub-data blocks, and a first data range of each sub-data block is obtained based on the bitmap corresponding to each sub-data block.

[0152] For each sub-data block, which includes multiple types of data, the corresponding bitmap includes multiple sub-bitmaps of multiple types. Based on these multiple sub-bitmaps, the range of multiple sub-data of these multiple types is obtained, that is, the range of multiple sub-data corresponding to the sub-data block is obtained.

[0153] For example, the sub-data block includes data of a first type and data of a second type. The bitmap corresponding to the sub-data block includes a first sub-bitmap of the first type and a second sub-bitmap of the second type. Based on the first and second sub-bitmaps, the first sub-data range of the first type and the second sub-data range of the second type are obtained. That is, the first and second sub-data ranges corresponding to the sub-data block are obtained.

[0154] For example, see Figure 4The index information includes bitmaps corresponding to the first and second sub-data blocks. The bitmaps corresponding to the first sub-data block include a first sub-bitmap of velocity (a) and a second sub-bitmap of current (a). The bitmaps corresponding to the second sub-data block include a first sub-bitmap of velocity (b) and a second sub-bitmap of current (b). Based on the first sub-bitmap (a), the first sub-data range A11 of velocity is greater than or equal to 100. Based on the second sub-bitmap (a), the second sub-data range A12 of current is greater than or equal to 0.6. In other words, the first sub-data ranges A11 and A12 of the first sub-data block are obtained. Based on the first sub-bitmap (b), the first sub-data range A21 of velocity is greater than or equal to 80 and less than 90. Based on the second sub-bitmap (b), the second sub-data range A22 of current is greater than or equal to 0.2 and less than 0.4. In other words, the first sub-data ranges A21 and A22 of the second sub-data block are obtained.

[0155] In some embodiments, the index information is a third type of index information, which includes a bitmap, and a plurality of first data ranges are obtained based on the bitmap. The plurality of first data ranges are the data ranges corresponding to bits in the bitmap whose values ​​are first bit values.

[0156] For example, see Figure 5 The index information includes a bitmap. Based on this bitmap, multiple first data ranges corresponding to the speed can be obtained. These multiple first data ranges are the data ranges corresponding to the bits in the bitmap whose values ​​are the first bit values. Specifically, these multiple first data ranges are: a data range greater than or equal to 80 and less than 90 (15), a data range greater than or equal to 100 and less than 110 (16), a data range greater than or equal to 110 and less than 120 (17), and a data range greater than or equal to 120 (18).

[0157] Step 604: If there is an intersection between the first data range and the second data range among the multiple first data ranges, the computing device reads the target data block stored in the storage device.

[0158] In some embodiments, the index information is either a first type of index information or a second type of index information, and the target data block includes multiple types of data. For example, the target data block includes data of a first type and data of a second type, the plurality of first data ranges include a plurality of first sub-data ranges of the first type and a plurality of second sub-data ranges of the second type, and the second data ranges include a third sub-data range of the first type and a fourth sub-data range of the second type. When there is an intersection between the first and third sub-data ranges of the target sub-data block within the plurality of first sub-data ranges, and an intersection between the second and fourth sub-data ranges of the target sub-data block within the plurality of second sub-data ranges, the computing device reads the target data block stored in the storage device, and the target data block includes the target sub-data block.

[0159] For example, in the example shown in Table 1 above, the index information is the first type of index information, and the multiple first data ranges include the first sub-data range of speed 80-85 and the first sub-data range of speed 100-130, the second sub-data range of current 0.2-0.3 and the second sub-data range of current 0.6-0.9.

[0160] Suppose a computing device receives a first query request. The second data range in the first query request includes a third sub-data range of speed (105-115) and a fourth sub-data range of current (0.6-0.7). The first sub-data range of speed (100-130) and the third sub-data range of speed (105-115) of the first sub-data block intersect, as do the second sub-data range of current (0.6-0.9) and the fourth sub-data range of current (0.6-0.7) of the first sub-data block. This indicates that the first sub-data block may contain the same row containing speed within the third sub-data range of speed (105-115) and current within the fourth sub-data range of current (0.6-0.7). Therefore, the computing device reads a target data block from the storage device that includes the first sub-data block.

[0161] Now assume the computing device receives a second query request. The second data range in the second query request includes a third sub-data range of speed (90-95) and a fourth sub-data range of current (0.3-0.5). The first sub-data range of speed (100-130) and the third sub-data range (90-95) of the first sub-data block do not intersect. Similarly, the second sub-data range of current (0.6-0.9) and the fourth sub-data range (0.3-0.5) of the first sub-data block do not intersect. The first sub-data range of speed (80-85) and the third sub-data range (90-95) of the second sub-data block do not intersect. However, the second sub-data range of current (0.2-0.3) and the fourth sub-data range (0.3-0.5) of the second sub-data block do intersect. Therefore, the computing device will not read data blocks containing the first and second sub-data blocks from the storage device, thus reducing the number of data blocks that do not include data within the second data range that need to be read from the storage device.

[0162] For example, this index information is the second type of index information. Multiple first data ranges include a first sub-data range A11 (greater than or equal to 100) and a first sub-data range A21 (greater than or equal to 80 and less than 90) for speed, and a second sub-data range A12 (greater than or equal to 0.6) and a second sub-data range A22 (greater than or equal to 0.2 and less than 0.4) for current. The first sub-data ranges A11 and A12 are two sub-data ranges corresponding to the first sub-data block, and the first sub-data ranges A21 and A22 are two sub-data ranges corresponding to the second sub-data block.

[0163] Suppose a computing device receives a first query request. The second data range in the first query request includes a third sub-data range of speed (105-115) and a fourth sub-data range of current (0.6-0.7). The first sub-data range A11 (greater than or equal to 100) of speed in the first sub-data block intersects with the third sub-data range of 105-115, and the second sub-data range A12 (greater than or equal to 0.6) of current in the first sub-data block intersects with the fourth sub-data range of 0.3-0.7. This indicates that the first sub-data block may contain the same row containing speed within the third sub-data range of 105-115 and current within the fourth sub-data range of 0.6-0.7. Therefore, the computing device reads a target data block from the storage device that includes the first sub-data block.

[0164] Now assume the computing device receives a second query request. The second data range in the second query request includes a third sub-data range of speed (90-95) and a fourth sub-data range of current (0.3-0.5). The first sub-data range A11 (greater than or equal to 100) of the speed of the first sub-data block does not intersect with the third sub-data range of 90-95. The second sub-data range A12 (greater than or equal to 0.6) of the current of the first sub-data block does not intersect with the fourth sub-data range of 0.3-0.5. The first sub-data range A21 (greater than or equal to 80 and less than 90) of the speed of the second sub-data block does not intersect with the third sub-data range of 90-95. The second sub-data range A22 (greater than or equal to 0.2 and less than 0.4) of the current of the second sub-data block intersects with the fourth sub-data range of 0.3-0.5. Therefore, the computing device will not read data blocks including the first and second sub-data blocks from the storage device, thereby reducing the number of data blocks that do not include data within the second data range that need to be read from the storage device.

[0165] In some embodiments, the index information is a third type of index information, which includes a bitmap. Multiple first data ranges are obtained based on the bitmap, where each first data range corresponds to a bit in the bitmap that has a value of a first bit. If there is an intersection between the first data ranges and second data ranges among the multiple first data ranges, the computing device reads the target data block from the storage device.

[0166] Next, taking the example shown in Table 2 above, the third type of index information is: Figure 5 The bitmap shown. Based on this bitmap, multiple first data ranges corresponding to the speed can be obtained. These multiple first data ranges are: a data range 15 greater than or equal to 80 and less than 90; a data range 16 greater than or equal to 100 and less than 110; a data range 17 greater than or equal to 110 and less than 120; and a data range 18 greater than or equal to 120.

[0167] Assume the query request received by the computing device includes a second data range of 105-115, meaning the second data range is greater than or equal to 105 and less than or equal to 115. Since data ranges 16 and 17 both intersect with the second data range of 105-115, the computing device reads the target data block from the storage device, which includes the data in Table 2 above.

[0168] In some embodiments, the storage device stores a mapping relationship between the target data block and the index information, and the computing device reads the target data block that has a mapping relationship with the index information from the storage device.

[0169] In some embodiments, the computing device stores a mapping relationship between the identification information of the target data block and the index information. The computing device obtains the identification information of the target data block that has a mapping relationship with the index information. Based on the identification information of the target data block, the computing device reads the target data block from the storage device.

[0170] Repeating steps 602-604 above may read one or more target data blocks from the storage device.

[0171] Step 605: The computing device retrieves data located within the second data range from the target data block.

[0172] In step 605, the computing device may decompress the target data block and obtain data within the second data range from the decompressed target data block.

[0173] For example, the target data block might be as shown in Table 1 above. The computing device retrieves multiple rows of data from the target data block shown in Table 1. Each row includes the speed within the third sub-data range of 105-115 and the current within the fourth sub-data range of 0.6-0.7. For instance, it might retrieve two rows of data, row numbers 4 and 5, from the first sub-data block.

[0174] In this embodiment, the index information of the data block stored in the storage device is used to indicate multiple first data ranges, and the data block includes data located within these multiple first data ranges. When a second data range to be queried is obtained, multiple first data ranges of the target data block can be obtained based on the index information of the target data block in the storage device. Thus, when retrieving data based on the second data range, determining whether the data block includes data located within the second data range based on the second data range and the multiple first data ranges improves the accuracy of the determination result, thereby maximizing the reading of data blocks from the storage device that include data located within the second data range, thus improving data retrieval performance. For example, if the second data range is located within the interval between two adjacent first data ranges, and the second data range does not intersect with the two first data ranges, it is determined that the data block does not include data located within the second data range, and the computing device will not read the data block from the storage device, thereby reducing the number of data blocks read from the storage device and improving data retrieval performance.

[0175] See Figure 7 This application provides a data acquisition device 700, which is located in a data management system. The device 700 can be... Figure 1 The computing devices in the data management system shown Figure 3 The computing device or method 300 shown Figure 6The computing device of method 600 shown. The data management system also includes a storage device for storing data blocks, the index information of which is used to indicate a plurality of first data ranges, the data blocks including data located within the plurality of first data ranges, and the apparatus 700 includes:

[0176] Processing unit 701 is used to obtain the second data range to be queried;

[0177] The reading unit 702 is used to read data blocks from the storage device, wherein the index information of the data blocks indicates that there is an intersection between the first data range and the second data range among multiple first data ranges;

[0178] The processing unit 701 is also used to obtain data located within the second data range from the data block.

[0179] Optionally, for details of how processing unit 701 obtains the second data range to be queried, please refer to [link to relevant documentation]. Figure 6 The relevant content in step 601 of method 600 shown will not be described in detail here.

[0180] Optionally, for details of the implementation process of the read unit 702 reading data blocks from the storage device, please refer to [link to relevant documentation]. Figure 6 The relevant content in step 604 of method 600 shown will not be described in detail here.

[0181] Optionally, for details of how processing unit 701 obtains data within the second data range from the data block, please refer to [link to relevant documentation]. Figure 6 The relevant content in step 605 of method 600 shown will not be described in detail here.

[0182] Optionally, multiple first data ranges correspond to multiple sub-data blocks. For each sub-data block, the sub-data block includes data located within the first data range corresponding to the sub-data block. The processing unit 701 is further configured to:

[0183] Retrieve multiple sub-data blocks and multiple first data ranges;

[0184] Merge multiple sub-data blocks to obtain a data block;

[0185] Index information of data blocks is obtained based on multiple first data ranges;

[0186] Store data blocks and their index information on the storage device.

[0187] Optionally, for details on how processing unit 701 acquires multiple sub-data blocks and multiple first data ranges, please refer to [link to relevant documentation]. Figure 3 The relevant content in step 301 of method 300 shown will not be described in detail here.

[0188] Optionally, for details on how processing unit 701 merges multiple sub-data blocks to obtain a data block, please refer to [link to relevant documentation]. Figure 3 The relevant content in step 302 of method 300 shown will not be described in detail here.

[0189] Optionally, for details of the implementation process of processing unit 701 obtaining index information of data blocks based on multiple first data ranges, please refer to [link to relevant documentation]. Figure 3 The relevant content in step 303 of method 300 shown will not be described in detail here.

[0190] Optionally, for details of the process by which processing unit 701 stores data blocks and index information of the data blocks in the storage device, please refer to [link to relevant documentation]. Figure 3 The relevant content in step 304 of method 300 shown will not be described in detail here.

[0191] Optionally, multiple first data ranges correspond to multiple sub-data blocks. For each sub-data block, the sub-data block includes data located within the first data range corresponding to the sub-data block. The sub-data block includes multiple first data of a first type and multiple second data of a second type. The first data range corresponding to the sub-data block includes a first sub-data range and a second sub-data range. Multiple first data are located within the first sub-data range, and multiple second data are located within the second sub-data range.

[0192] Optionally, the plurality of first data ranges include a plurality of first sub-data ranges and a plurality of second sub-data ranges, wherein the second data ranges include a third sub-data range of a first type and a fourth sub-data range of a second type;

[0193] The index information of the data block indicates that among multiple first data ranges, there is an intersection between the first data range and the second data range, including:

[0194] There is an intersection between the first sub-data range and the third sub-data range, which contain the target sub-data block, and there is an intersection between the second sub-data range and the fourth sub-data range, which contain the target sub-data block, in multiple first sub-data ranges. The multiple sub-data blocks include the target sub-data block.

[0195] Optionally, the index information includes a bitmap, which comprises multiple bits corresponding to multiple third data ranges. At least one of the third data ranges intersects with multiple first data ranges. The value of at least one bit corresponding to at least one third data range is a first bit value, and the values ​​of the bits in the bitmap other than the at least one bit are second bit values; or...

[0196] The index information includes multiple first data ranges.

[0197] Optionally, the index information includes a bitmap, and the device 700 further includes a receiving unit 703;

[0198] The receiving unit 703 is used to receive a creation request, which includes a first value, a second value, and the number of multiple bits, wherein the first value is less than the second value;

[0199] The processing unit 701 is also used to obtain a bitmap based on the first value, the second value, and the number of bits.

[0200] Optionally, for details of the implementation process of receiving the creation request by receiving unit 703, please refer to [link / reference]. Figure 3 The relevant content in step 303 of method 300 shown will not be described in detail here.

[0201] Optionally, the processing unit 701 obtains the bitmap based on the first value, the second value, and the number of bits. (See the detailed implementation process for further details.) Figure 3 The relevant content in step 303 of method 300 shown will not be described in detail here.

[0202] Optionally, the storage device is also used to store index information, and the reading unit 702 is also used to read the index information from the storage device.

[0203] Optionally, for details on how the reading unit 702 reads index information from the storage device, please refer to [link to relevant documentation]. Figure 6 The relevant content in step 602 of method 600 shown will not be described in detail here.

[0204] The processing unit 701, reading unit 702, and receiving unit 703 can all be implemented in software or in hardware. For example, the implementation of processing unit 701 will be described below. Similarly, the implementation of reading unit 702 and receiving unit 703 can refer to the implementation of processing unit 701.

[0205] As an example of a software functional unit, processing unit 701 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the aforementioned computing instance may be one or more. For example, processing unit 701 may include code running on multiple hosts / virtual machines / containers. The multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one or more geographically proximate data centers. Typically, a region may include multiple AZs.

[0206] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0207] As an example of a hardware functional unit, processing unit 701 may include at least one computing device, such as a server. Alternatively, processing unit 701 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0208] The processing unit 701 includes multiple computing devices that can be distributed in the same region or in different regions. Similarly, the processing unit 701 includes multiple computing devices that can be distributed in the same Availability Zone (AZ) or in different AZs. Likewise, the processing unit 701 includes multiple computing devices that can be distributed in the same Virtual Private Cloud (VPC) or in multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0209] It should be noted that, in other embodiments, the processing unit 701 can be used to execute any step in the acquisition method, the reading unit 702 can be used to execute any step in the data acquisition method, and the receiving unit 703 can be used to execute any step in the acquisition method. The steps implemented by the processing unit 701, the reading unit 702, and the receiving unit 703 can be specified as needed. By implementing different steps in the data acquisition method through the processing unit 701, the reading unit 702, and the receiving unit 703, all functions of the data acquisition device 700 can be realized.

[0210] In this embodiment, after the processing unit obtains the second data range to be queried, it obtains the index information of the data block. Since the index information is used to indicate multiple first data ranges, and the data block includes data located within multiple first data ranges, if there is an intersection between the first and second data ranges, the reading unit reads the data block from the storage device. The presence of an intersection between the first and second data ranges indicates a high probability that the data block includes data located within the second data range, and thus a high probability that the processing unit will retrieve data within the second data range from the data block, thereby improving data retrieval performance.

[0211] See Figure 8 This application provides a computing device 800. For example, the computing device 800 may be... Figure 1 The device in the data management system shown, or the computing device 800, can be... Figure 3 Method 300 or shown Figure 6 The devices, etc., in method 600 shown.

[0212] like Figure 8 As shown, the computing device 800 includes a bus 802, a processor 804, a memory 806, and a communication interface 808. The processor 804, the memory 806, and the communication interface 808 communicate with each other via the bus 802. The computing device 800 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the computing device 800.

[0213] The 802 bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 8 The bus 802 may be represented by a single line, but this does not mean that there is only one bus or one type of bus. The bus 802 may include a path for transmitting information between various components of the computing device 800 (e.g., processor 804, memory 806, communication interface 808).

[0214] Processor 804 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0215] Memory 806 may include volatile memory, such as random access memory (RAM). Memory 806 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0216] See Figure 8 The memory 806 stores executable program code, and the processor 804 executes the executable program code to implement the following respectively. Figure 7 The processing unit 701, reading unit 702, and receiving unit 703 in the illustrated device 700 function to implement the method provided in any of the above embodiments. That is, the memory 806 stores instructions for executing the method provided in any of the above embodiments. Alternatively,

[0217] The communication interface 808 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the computing device 800 and other devices or communication networks.

[0218] This application also provides a data acquisition cluster. The data acquisition cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0219] like Figure 9 As shown, the data acquisition cluster includes at least one computing device 800. The memory 806 of one or more computing devices 800 in the data acquisition cluster may store the same instructions for performing the data acquisition method provided in any of the above embodiments.

[0220] In some possible implementations, the memory 806 of one or more computing devices 800 in the data acquisition cluster may also store partial instructions for executing the data acquisition method described above. In other words, a combination of one or more computing devices 800 can jointly execute instructions for performing the method provided in any of the above embodiments.

[0221] The memory 806 in different computing devices 800 within a computing device cluster can store different instructions, each used for execution such as... Figure 7 This describes some of the functions of the data acquisition device 700 shown. That is, the instructions stored in the memory 806 of different computing devices 800 can implement the functions of one or more units among the processing unit 701, the reading unit 702, and the receiving unit 703.

[0222] In some possible implementations, one or more computing devices in the data acquisition cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc. Figure 10 One possible implementation is shown. For example... Figure 10 As shown, the two computing devices 800A and 800B are connected via a network. Specifically, they are connected to the network through the communication interfaces in each computing device.

[0223] In this type of possible implementation, the memory 806 in the computing device 800A stores the execution of, for example... Figure 7 The instructions for the processing unit 701 and the reading unit 702 in the illustrated embodiment are shown. Meanwhile, the memory 806 in the computing device 800B stores instructions for performing operations such as... Figure 7 Instructions for the function of the receiving unit 703 in the illustrated embodiment.

[0224] Figure 10The connection method between the computing device clusters shown may take into account the data acquisition method provided in this application (e.g., large-scale data storage and / or data transmission and reception), and therefore consider delegating the functions implemented by the receiving unit 703 to the computing device 800B.

[0225] It should be understood that Figure 10 The functions of the computing device 800A shown can also be performed by multiple computing devices 800. Similarly, the functions of the computing device 800B can also be performed by multiple computing devices 800.

[0226] This application also provides another data acquisition cluster. The connection relationships between the computing devices in this data acquisition cluster can be similarly referenced. Figure 10 The connection method of the data acquisition cluster is different in that the memory 806 of one or more computing devices 800 in the data acquisition cluster can store the same instructions for executing the data acquisition method provided in any of the above embodiments.

[0227] In some possible implementations, the memory 806 of one or more computing devices 800 in the data acquisition cluster may also store partial instructions for executing the methods provided in any of the above embodiments. In other words, a combination of one or more computing devices 800 can jointly execute instructions for performing the methods provided in any of the above embodiments.

[0228] This application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to perform the methods provided in any of the above embodiments.

[0229] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium capable of being stored by a computing device, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct a computing device to perform the method provided in any of the above embodiments.

[0230] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0231] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A method for acquiring data, characterized in that, The method is applied to a computing device included in a data management system, the data management system further including a storage device for storing data blocks, the index information of the data blocks for indicating a plurality of first data ranges, the data blocks including data located within the plurality of first data ranges, the method comprising: Get the second data range to be queried; The data block is read from the storage device, and the index information of the data block indicates that there is an intersection between the first data range and the second data range among the plurality of first data ranges; Obtain data located within the second data range from the data block.

2. The method as described in claim 1, characterized in that, The plurality of first data ranges correspond to plurality of sub-data blocks. For each sub-data block, the sub-data block includes data located within the first data range corresponding to the sub-data block. The method further includes: Obtain the plurality of sub-data blocks and the plurality of first data ranges; The multiple sub-data blocks are merged to obtain the data block; The index information of the data block is obtained based on the plurality of first data ranges; The data block and its index information are stored in the storage device.

3. The method as described in claim 1 or 2, characterized in that, The plurality of first data ranges correspond to a plurality of sub-data blocks. For each sub-data block, the sub-data block includes data located within the first data range corresponding to the sub-data block. The sub-data block includes a plurality of first data of a first type and a plurality of second data of a second type. The first data range corresponding to the sub-data block includes a first sub-data range and a second sub-data range. The plurality of first data are located within the first sub-data range, and the plurality of second data are located within the second sub-data range.

4. The method as described in claim 3, characterized in that, The plurality of first data ranges include a plurality of first sub-data ranges and a plurality of second sub-data ranges, wherein the second data ranges include a third sub-data range of the first type and a fourth sub-data range of the second type; The index information of the data block indicates that there is an intersection between the first data range and the second data range among the plurality of first data ranges, including: The first sub-data range containing the target sub-data block in the plurality of first sub-data ranges intersects with the third sub-data range, and the second sub-data range containing the target sub-data block in the plurality of second sub-data ranges intersects with the fourth sub-data range, wherein the plurality of sub-data blocks include the target sub-data block.

5. The method according to any one of claims 1-4, characterized in that, The index information includes a bitmap, which comprises multiple bits. These multiple bits correspond to multiple third data ranges. At least one of the multiple third data ranges intersects with the multiple first data ranges. The value of at least one bit corresponding to the at least one third data range is a first bit value, and the values ​​of the bits in the bitmap other than the at least one bit are second bit values; or... The index information includes the plurality of first data ranges.

6. The method as described in claim 5, characterized in that, The index information includes a bitmap, and the method further includes: A creation request is received, the creation request including a first value, a second value, and the number of the plurality of bits, wherein the first value is less than the second value; The bitmap is obtained based on the first value, the second value, and the number of bits.

7. The method according to any one of claims 1-6, characterized in that, The storage device is also used to store the index information, and the method further includes: Read the index information from the storage device.

8. A device for acquiring data, characterized in that, The device is located in a data management system, which further includes a storage device for storing data blocks. Index information of the data blocks is used to indicate multiple first data ranges. Each data block includes data located within the multiple first data ranges. The device includes: The processing unit is used to obtain the second data range to be queried; A reading unit is configured to read the data block from the storage device, wherein the index information of the data block indicates that there is an intersection between the first data range and the second data range among the plurality of first data ranges; The processing unit is further configured to obtain data located within the second data range from the data block.

9. The apparatus as claimed in claim 8, characterized in that, The plurality of first data ranges correspond to a plurality of sub-data blocks. For each sub-data block, the sub-data block includes data located within the first data range corresponding to the sub-data block. The processing unit is further configured to: Obtain the plurality of sub-data blocks and the plurality of first data ranges; The multiple sub-data blocks are merged to obtain the data block; The index information of the data block is obtained based on the plurality of first data ranges; The data block and its index information are stored in the storage device.

10. The apparatus as claimed in claim 8 or 9, characterized in that, The plurality of first data ranges correspond to a plurality of sub-data blocks. For each sub-data block, the sub-data block includes data located within the first data range corresponding to the sub-data block. The sub-data block includes a plurality of first data of a first type and a plurality of second data of a second type. The first data range corresponding to the sub-data block includes a first sub-data range and a second sub-data range. The plurality of first data are located within the first sub-data range, and the plurality of second data are located within the second sub-data range.

11. The apparatus as claimed in claim 10, characterized in that, The plurality of first data ranges include a plurality of first sub-data ranges and a plurality of second sub-data ranges, wherein the second data ranges include a third sub-data range of the first type and a fourth sub-data range of the second type; The index information of the data block indicates that there is an intersection between the first data range and the second data range among the plurality of first data ranges, including: The first sub-data range containing the target sub-data block in the plurality of first sub-data ranges intersects with the third sub-data range, and the second sub-data range containing the target sub-data block in the plurality of second sub-data ranges intersects with the fourth sub-data range, wherein the plurality of sub-data blocks include the target sub-data block.

12. The apparatus according to any one of claims 8-11, characterized in that, The index information includes a bitmap, which comprises multiple bits. These multiple bits correspond to multiple third data ranges. At least one of the multiple third data ranges intersects with the multiple first data ranges. The value of at least one bit corresponding to the at least one third data range is a first bit value, and the values ​​of the bits in the bitmap other than the at least one bit are second bit values; or... The index information includes the plurality of first data ranges.

13. The apparatus as claimed in claim 12, characterized in that, The index information includes a bitmap, and the device further includes a receiving unit; The receiving unit is configured to receive a creation request, the creation request including a first value, a second value, and the number of the plurality of bits, wherein the first value is less than the second value; The processing unit is further configured to obtain the bitmap based on the first value, the second value, and the number of the plurality of bits.

14. The apparatus according to any one of claims 8-13, characterized in that, The storage device is also used to store the index information, and the reading unit is also used to read the index information from the storage device.

15. A computing device cluster, characterized in that, It includes at least one computing device, each computing device including a processor and memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method as described in any one of claims 1-7.

16. A computer-readable storage medium, characterized in that, Includes computer program instructions, which, when executed by a cluster of computing devices, perform the method as described in any one of claims 1-7.

17. A computer program product containing instructions, characterized in that, When the instruction is executed by the computing device cluster, the computing device cluster performs the method as described in any one of claims 1-7.

Images