Address determination method and device

Abstract

The invention discloses an address determination method and device, and relates to the field of data storage. The address determination method comprises the following steps: acquiring a data identifier of to-be-processed data, and determining a region identifier of a first storage region corresponding to the data identifier of the to-be-processed data by utilizing an interval mapping relationship; and then, selecting a second storage address of the first storage address for storing the data to be processed from the address range of the first storage area. Wherein the interval mapping relation is used for indicating the data identification intervals corresponding to the area identifications of the multiple storage areas, and the second storage area is used for storing the first storage address of the data in the data identification interval corresponding to the second storage area in the disk. The data identifier interval comprises a plurality of continuous data identifiers, and the second storage area is one of the plurality of storage areas.

Description

Technical Field

[0001] The present application relates to the field of data storage, and in particular to a method and device for determining an address. Background Art

[0002] In data storage, not only the data itself is stored, but also the metadata of the data and the identifier of the data are stored, so that the metadata of the data can be located quickly according to the identifier of the data, and then the metadata is used to manage the data. The metadata and the identifier are often stored in a key-value (KV) storage method. After obtaining the data name and other identifiers of the data, the data stored in the KV storage method is queried to obtain the metadata corresponding to the identifier. A hash list is a common KV storage method. The hash list stores the metadata corresponding to multiple data belonging to the same data block or adjacent data blocks in different hash buckets. Furthermore, when managing the multiple data, it is necessary to access multiple hash buckets to obtain the corresponding metadata, thereby reducing the efficiency of managing the multiple data. Therefore, how to improve the management efficiency of multiple data has become a problem that needs to be solved urgently. Summary of the invention

[0003] The present invention provides an address determination method and device to improve the management efficiency of multiple data.

[0004] In a first aspect, an address determination method is provided, which may include: obtaining a data identifier of data to be processed, and using an interval mapping relationship to determine the region identifier of a first storage area corresponding to the data identifier of the data to be processed. Further, from the address range of the first storage area, a second storage address storing the first storage address of the data to be processed is selected. The interval mapping relationship is used to indicate the data identifier intervals corresponding to the region identifiers of multiple storage areas, respectively, and the second storage area is used to store the first storage address of the data in the data identifier interval corresponding to the second storage area in the disk. The data identifier interval contains multiple consecutive data identifiers, and the second storage area is one of the multiple storage areas.

[0005] In the present application, since the interval mapping relationship indicates the data identification interval corresponding to the area identifiers of multiple storage areas, and the data identification interval contains multiple continuous data identifiers, a storage area is used to store the first storage addresses corresponding to multiple data identifiers in its corresponding data identification interval. Therefore, when managing the data corresponding to multiple data identifiers belonging to the same data identification interval, only one storage area can be accessed to obtain the first storage addresses corresponding to the multiple data identifiers, thereby managing the data corresponding to the multiple data identifiers. Compared with the above-mentioned hash table, when managing multiple data identifiers belonging to the same identification interval, it is necessary to access multiple storage areas to obtain the storage addresses corresponding to the multiple data identifiers, which is a low efficiency problem. The efficiency of obtaining the first storage address of the stored data is improved, thereby improving the efficiency of data management.

[0006] The address determination method can be applied to a data storage system, or to a computing device that supports a data storage system to implement the address determination method. Here, a computing device executing the address determination method provided in this embodiment is used as an example for description, and the computing device includes at least one hard disk.

[0007] In a possible implementation, selecting a second storage address of the first storage address storing the data to be processed from the address range of the first storage area includes: substituting the data identifier of the data to be processed into a mapping model to obtain the second storage address. The mapping model is used to indicate the mapping relationship between the data identifier and the address in the storage area. The storage space indicated by the address in the storage area is used to store the data indicated by the data identifier that has a mapping relationship with the address in the storage area, at the first storage address in the disk.

[0008] In the present application, the mapping relationship between the data identifier and the address in the storage area is reflected by a mapping model, so the data identifier may not be stored in the storage area, and only the first storage address of the data in the disk is stored, avoiding the problem of storing the data identifier and the first storage address in the KV form, thereby reducing the storage space occupied. In addition, the second storage address of the first storage address storing the data to be processed can be directly located by using the mapping model, thereby improving the efficiency of determining the first storage address of the data to be processed, thereby improving the management efficiency of the data to be processed.

[0009] In a possible implementation manner, the mapping model is a mapping model corresponding to the first storage area.

[0010] In a possible scenario, multiple storage areas have corresponding mapping models respectively.

[0011] In one possible implementation, before selecting the second storage address of the first storage address storing the data to be processed from the address range of the first storage area, the address determination method also includes: fitting the address of the storage space in the first storage area where the first storage address has been stored and the data identifier corresponding to the stored first storage address to obtain a mapping model.

[0012] In the present application, the mapping model obtained by fitting the address of the storage space storing the first storage address and the data identifier corresponding to the stored first storage address can accurately represent the correspondence between the address of the storage space storing the first storage address and the data identifier corresponding to the first storage address. Furthermore, the second storage address corresponding to the data identifier of the data to be processed can be accurately and directly determined using the mapping model, which improves the efficiency of determining the first storage address of the data to be processed stored in the second storage address, thereby improving management efficiency. Furthermore, the mapping relationship between the data identifier and the address in the storage area is reflected through the mapping model, so the data identifier may not be stored in the storage area, and only the first storage address of the data in the disk is stored, reducing the storage space occupied in the storage area.

[0013] In a possible implementation, the storage area includes a first sub-area and a second sub-area, and the addresses of the first sub-area and the second sub-area are continuous. From the address range of the first storage area, the second storage address of the first storage address storing the data to be processed is selected, including: in the address range of the first target sub-area, the second storage address of the first storage address storing the data to be processed is selected. Among them, the first target sub-area is a sub-area corresponding to the data identification range where the data identification of the data to be processed is located in the first sub-area and the second sub-area. The data identification range of a sub-area is a range composed of data identifications of data stored in the sub-area at the first storage address; or, the data to be processed is newly written data, and the first target sub-area is a sub-area of ​​the first storage address storing the newly written data.

[0014] In the present application, the first target sub-region is determined according to the data identification range where the data identification of the data to be processed is located, thereby avoiding the need to determine the second storage address of the first storage address storing the data to be processed from the entire storage area, that is, the determined address range is narrowed, thereby reducing the time consumption of determining the second storage address, and improving the efficiency of determining the second storage address. And when the data to be processed is newly written data, the first storage address of the newly written data is directly written into a specific storage space, and the storage space belongs to the first target sub-region, so that in the scenario of newly written data, the first storage address of the newly written data can still be stored, thereby improving the management efficiency of specific types of data (newly written data).

[0015] In a possible implementation, selecting a second storage address of the first storage address storing the to-be-processed data in the address range of the first target sub-region includes: substituting the data identifier of the to-be-processed data into a mapping model of the first target sub-region to obtain the second storage address. The mapping model of the first target sub-region is used for the mapping relationship between the data identifier in the data identifier range of the first target sub-region and the address in the first target sub-region.

[0016] In the present application, if there is a mapping model corresponding to the first target sub-region, the mapping model of the first target sub-region can be used to determine the second storage address according to the data identifier of the data to be processed, so that the mapping model of the first target sub-region can be used to accurately and quickly locate the second storage address, thereby improving the efficiency of determining the second storage address.

[0017] Exemplarily, the first mapping model corresponding to the first sub-region is used for the mapping relationship between the data identifier in the data identifier range of the first sub-region and the address in the first sub-region, and the second mapping model corresponding to the second sub-region is used for the mapping relationship between the data identifier in the data identifier range of the second sub-region and the address in the second sub-region. The mapping model of the first target sub-region is any one of the first mapping model and the second mapping model.

[0018] In a possible implementation, if there is no mapping model corresponding to the first target sub-region, selecting a second storage address of the first storage address storing the data to be processed in the address range of the first target sub-region includes: using the address of the first storage space in the first target sub-region as the second storage address. The address determination method also includes: fitting the second storage address and the data identifier of the data to be processed to obtain the mapping model corresponding to the first target sub-region.

[0019] In the present application, by using the address of the first storage space in the first target sub-region as the second storage address to store the first storage address of the data to be processed, when there is no mapping model corresponding to the first target sub-region, the first storage address of the data to be processed is written into the storage region, thereby improving the writing capability of the first storage address. In addition, the data identifier stored in the first target sub-region and the second storage address storing the first storage address corresponding to the data identifier are fitted to obtain the mapping model of the first target sub-region. This is beneficial for determining the address of the storage space storing the first storage address of the newly written data when there is new data to be written later, thereby improving the writing efficiency of the data and reducing the read-write amplification.

[0020] In a possible implementation, the first sub-region is a sub-region of a first storage address storing newly written data. The address determination method further includes: when the first data is partially deleted, adjusting the address boundary of the second sub-region close to the first sub-region to the first address. The first address is the address of the storage space of the first storage address storing the first data in the first storage region, and the first data is any data stored in the first sub-region at the first storage address.

[0021] In the present application, when the first data is partially deleted, the address boundary of the first sub-region and the second sub-region is adjusted to facilitate determination of the data identification range of the first sub-region and the second sub-region, and update of the corresponding mapping models. The problem of serious read / write amplification caused by the overlap of the data identification range of the first sub-region and the second sub-region is avoided, and the efficiency of reading and writing addresses is improved. The updated mapping model can accurately locate the address of the first storage address corresponding to the storage data identification in the storage space, improving the efficiency of determining the address.

[0022] In a possible scenario, the storage area further includes a management sub-area and a reserved sub-area, and the management sub-area is used to store a mapping model or a data identification range and / or an address range of each sub-area, etc. The reserved sub-area can be used to occupy when a new first storage address is added in the first sub-area (a sub-area storing a first storage address of newly written data).

[0023] In a possible implementation, the address determination method further includes: if there is free storage space in the second target sub-region, moving the first storage address stored after the free storage space in the second target sub-region along the storage direction, so that the addresses of the first storage addresses stored in the second target sub-region are continuous. And the address of the storage space in the second target sub-region where the first storage address is stored and the data identifier corresponding to the stored first storage address are fitted, and the mapping model corresponding to the second target sub-region is updated. The second target sub-region is any sub-region;

[0024] In the present application, when storing the free storage space in the second target sub-region, the first storage address previously stored in the free storage space in the second target sub-region is moved to eliminate the free storage space in the second target sub-region, thereby reducing the read-write amplification. In addition, since the storage space in the storage region is recycled, after eliminating the free storage space, the utilization rate of the storage space in the storage region can be improved.

[0025] In a possible implementation, the address determination method further includes: acquiring a first storage address of the to-be-processed data stored at the second storage address, and then updating, deleting or reading the to-be-processed data stored in the storage space indicated by the first storage address of the to-be-processed data.

[0026] In the present application, data processing is performed on the data to be processed stored in the storage space indicated by the first storage address of the data to be processed, and while the efficiency of determining the second storage address is improved, the efficiency of the overall process of processing the data to be processed is improved.

[0027] In a possible implementation, the address determination method further includes: writing the first storage address of the to-be-processed data in the disk into the storage space indicated by the second storage address.

[0028] In the present application, data processing is performed on the data to be processed stored in the storage space indicated by the first storage address of the data to be processed, and while the efficiency of determining the second storage address is improved, the efficiency of the overall process of writing the data to be processed is improved.

[0029] In one possible scenario, if the second storage address corresponding to the data identifier of the data to be processed is not determined according to the mapping model, information is sent to a host connected to the computing device so that the host displays a control component of the processing flow and receives a trigger operation of the control component by the user. In response to the trigger operation, the processing flow selected by the user is determined, and the computing device performs corresponding processing according to the processing flow selected by the user.

[0030] In a second aspect, an address determination device is provided, which is applied to a computing device, and includes various modules for executing the address determination method in the first aspect or any optional design of the first aspect. For example, the address determination device includes: an acquisition module, a determination module, and a selection module.

[0031] The acquisition module is used to acquire the data identifier of the data to be processed.

[0032] The determination module is used to determine the region identification of the first storage area corresponding to the data identification of the data to be processed by using the interval mapping relationship; wherein the interval mapping relationship is used to indicate the data identification intervals corresponding to the region identifications of multiple storage areas, and the second storage area is used to store the first storage address of the data in the data identification interval corresponding to the second storage area in the disk. The data identification interval contains multiple continuous data identifications, and the second storage area is one of the multiple storage areas.

[0033] The selection module is used to select a second storage address of the first storage address storing the data to be processed from the address range of the first storage area.

[0034] In a possible implementation, the selection module is specifically used to substitute the data identifier of the data to be processed into the mapping model to obtain the second storage address. The mapping model is used to indicate the mapping relationship between the data identifier and the address in the storage area. The storage space indicated by the address in the storage area is used to store the data indicated by the data identifier that has a mapping relationship with the address in the storage area, at the first storage address in the disk.

[0035] In a possible implementation manner, the mapping model is a mapping model corresponding to the first storage area.

[0036] In a possible implementation, the address determination device further includes a fitting module. The fitting module is used to fit the address of the storage space in the first storage area where the first storage address is stored and the data identifier corresponding to the stored first storage address to obtain a mapping model.

[0037] In a possible implementation, the storage area includes a first sub-area and a second sub-area, and the addresses of the first sub-area and the second sub-area are continuous. The selection module is specifically used to select the second storage address of the first storage address storing the data to be processed in the address range of the first target sub-area. Among them, the first target sub-area is the sub-area corresponding to the data identification range where the data identification of the data to be processed is located in the first sub-area and the second sub-area. The data identification range of a sub-area is the range composed of the data identification of the data stored in the sub-area at the first storage address, or, the data to be processed is newly written data, and the first target sub-area is the sub-area of ​​the first storage address storing the newly written data.

[0038] In a possible implementation, the selection module is specifically used to substitute the data identifier of the data to be processed into a mapping model of the first target sub-region to obtain the second storage address. The mapping model of the first target sub-region is used for the mapping relationship between the data identifier in the data identifier range of the first target sub-region and the address in the first target sub-region.

[0039] In a possible implementation, if there is no mapping model corresponding to the first target sub-region, the selection module is specifically used to use the address of the first storage space in the first target sub-region as the second storage address. The above-mentioned fitting module is used to fit the second storage address and the data identifier of the data to be processed to obtain the mapping model corresponding to the first target sub-region.

[0040] In a possible implementation, the first sub-region is a sub-region storing a first storage address of newly written data. The address determination device further includes a first adjustment module. The first adjustment module is used to adjust the address boundary of the second sub-region close to the first sub-region to the first address after the first data is partially deleted. The first address is the address of the storage space of the first storage address storing the first data in the first storage region, and the first data is any data stored in the first sub-region at the first storage address.

[0041] In a possible implementation, the address determination device further includes a second adjustment module. The second adjustment module is used to move the first storage address previously stored in the free storage space in the second target sub-region if there is free storage space in the second target sub-region, so that the address of the first storage address stored in the second target sub-region is continuous. The second target sub-region is any sub-region. The above-mentioned fitting module is used to fit the address of the storage space in the second target sub-region where the first storage address has been stored and the data identifier corresponding to the stored first storage address, and update the mapping model corresponding to the second target sub-region.

[0042] In a possible implementation, the address determination device further includes an address acquisition module and a data processing module. The address acquisition module is used to acquire the first storage address of the to-be-processed data stored at the second storage address. The data processing module is used to update, delete or read the to-be-processed data stored in the storage space indicated by the first storage address of the to-be-processed data.

[0043] In a possible implementation, the address determination device further includes a writing module, which is used to write the first storage address of the processed data in the disk into the storage space indicated by the second storage address.

[0044] In a third aspect, the present application provides a computing device, which includes a memory and a processor, wherein the memory is used to store instructions; and the processor executes the instructions to implement the method in the first aspect or any possible implementation of the first aspect.

[0045] In a fourth aspect, the present application provides a computer-readable storage medium. The storage medium stores a computer program or instruction, and when the computer program or instruction is executed by a computing device, the method in the first aspect or any optional implementation of the first aspect is implemented.

[0046] In a fifth aspect, the present application provides a computer program product. The computer program product includes a computer program or instructions, and when the computer program or instructions are executed by a computing device, the method in the first aspect or any optional implementation of the first aspect is implemented.

[0047] The beneficial effects of the second to fifth aspects above can refer to the description of the first aspect or any implementation of the first aspect, and will not be repeated here. Based on the implementations provided by the above aspects, this application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a schematic diagram of a chained hash storage KV scenario;

[0049] Figure 2 It is a schematic diagram for skip table storage;

[0050] Figure 3 A structural diagram of a data storage system provided by this application;

[0051] Figure 4 A flowchart of an address determination method provided in this application;

[0052] Figure 5 A schematic diagram of the fitting mapping model provided for this application;

[0053] Figure 6 Schematic diagram of the storage area provided for this application Figure 1 ;

[0054] Figure 7 Schematic diagram of the storage area provided for this application Figure 2 ;

[0055] Figure 8 A schematic diagram of a process for determining a second storage address provided by the present application;

[0056] Fig. 9 A schematic diagram of the flow of the data writing method provided by this application;

[0057] Fig.10 A flowchart of the data processing method provided in this application;

[0058] Fig.11 A flowchart of the data updating method provided in this application;

[0059] Fig.12 Regional variations provided for this application Figure 1 ;

[0060] Fig.13 Regional variations provided for this application Figure 2 ;

[0061] Fig.14 Regional variations provided for this application Figure 3 ;

[0062] Fig.15Regional variations provided for this application Figure 4 ;

[0063] Fig.16 A schematic diagram of the structure of an address determination device provided in this application Figure 1 ;

[0064] Fig.17 A schematic diagram of the structure of an address determination device provided in this application Figure 2 . DETAILED DESCRIPTION

[0065] To facilitate understanding, the technical terms involved in this application are first introduced.

[0066] KV is a commonly used data structure, which consists of a unique key and a value. The key is a unique identifier used to identify the value, and the value is the data associated with the key.

[0067] Block: A storage service based on persistent media for storing data.

[0068] Write amplification is the amount of physical data that is actually written being a multiple of the amount of data written. Since flash memory must be erased before data can be rewritten, and the granularity of erase operations is much lower than write operations, performing these operations moves (or overwrites) user data and metadata multiple times. Therefore, to overwrite data, it is necessary to read some used parts of the flash memory, update them, and write to the new location, along with erasing it if the new location has been used before.

[0069] In the field of big data storage such as video data, as the amount of stored data continues to increase, the storage requirements for storage devices are also getting higher and higher. For example, it is required to store extremely large amounts of data, release the storage space occupied by expired data in a timely manner, and have high performance requirements for accessing stored data. Taking video data as an example, the stored KV has the following characteristics: batch writing, the newly generated key is greater than the historically generated key, and the KV is periodically recycled. The key can correspond to the data name, blockid+offset, etc., and the value can correspond to the storage address of the video data, etc.

[0070] To meet the above storage requirements, two possible implementation solutions are provided below.

[0071] Solution 1: Construct a KV index table. In the KV index table, the key is blockid+offset, the blockid corresponds to a unique block, the offset is the offset within the block, such as 0-5 megabytes (MByte), and the value is the address (first address) of the block. Table 1 below shows a possible example of a KV index table.

[0072] Table 1

[0073]

[0074] It is worth noting that the above Table 1 is only an example, and the KV index table may also include more key-value pairs.

[0075] In a possible example, a method of using the KV index table is shown below.

[0076] When searching for the address corresponding to block20-60M, it is necessary to traverse the above KV index table to obtain the KV corresponding to block20-60M, such as block20-60M-0x08000001.

[0077] If the address corresponding to block50-60M is searched, the above KV index table is traversed in the same way. If the KV corresponding to block50-60M is not obtained, it is returned that it does not exist or is saved.

[0078] When there is newly written data in the storage device, the data block id of the newly written data storage and the offset of the data in the aforementioned data block will be obtained to obtain the identification of the newly written data, such as block30-50M. Then, the address (0x0C000001) of block30-50M and block3 is maintained to the above-mentioned KV index table, and the storage space with the vacancy mark is found by traversing the above-mentioned KV index table, and then the block30-50M-0x0C000001 is written to the storage space. And the vacancy mark is deleted. After writing block30-50M-0x0C000001 in the above Table 1, the following Table 2 is obtained.

[0079] Table 2

[0080]

[0081] When the data corresponding to block05-63M is deleted, the KV corresponding to block05-63M in the KV index table also needs to be deleted. Use block05-63M to traverse the KV index table, determine the KV corresponding to block05-63M, such as block05-63M-0x00000001, and delete block05-63M-0x00000001. In addition, add a vacancy mark at the location where block05-63M-0x00000001 is stored.

[0082] Because the above-mentioned KV index table stores the identifier and address of the data in the storage space indicated by the empty mark after traversing to the empty mark, the KVs stored in adjacent positions in the KV index table are disordered and have large differences. Therefore, the KV index table cannot be compressed. As a result, the KV index table occupies a large amount of memory space. In addition, when reading data, it is necessary to traverse the KV index table to determine the KV corresponding to the blockid+offset specified by the user, and the reading efficiency is low. Similarly, when recycling KV, it is necessary to traverse the KV index table to find the KV to be recycled, and the recycling efficiency is low. When writing KV, it is necessary to traverse the KV index table to find the empty mark, and then insert the KV into the storage space indicated by the empty mark, and the writing efficiency is low.

[0083] Solution 2: Use Redis to store KV. Redis not only supports the storage of KV structured data, but also provides storage for list, set, zset, hash and other data structures. Table 3 below shows the index types and underlying implementations supported by Redis.

[0084] Table 3

[0085]

[0086] According to the above Table 3, the above KV is often stored through an ordered set or hash table provided by Redis. The underlying implementation of the ordered set index is a skip table, and the underlying implementation of the hash table index is a dictionary.

[0087] The following is a possible example, using a hash table to store KV. Figure 1 As shown, Figure 1 The following is a schematic diagram of a chained hash storage KV scenario. The process of storing KV may include:

[0088] ①: Use the hash function to calculate the hash bucket corresponding to each key in the hash table space.

[0089] For example, when the key is key1, through hash(key1)%8=1, the KV corresponding to key1 is calculated and stored in hash bucket 1. When the key is key9, through hash(key9)%8=6, the KV corresponding to key9 is calculated and stored in hash bucket 6.

[0090] ②: Store the KV corresponding to each key in the corresponding hash bucket.

[0091] Similarly, when reading a value, the hash function is used to calculate the hash bucket corresponding to the key to be read, and then the value corresponding to the key is obtained from the hash bucket.

[0092] Mapping keys to hash buckets through hash functions and using hash buckets to store discrete values ​​results in low similarity between discrete values ​​and weak compression capabilities, which in turn results in large storage space usage. In addition, the data identifier and the corresponding first storage address are stored in the form of KV in the hash bucket, which also results in large storage space usage. In addition, the values ​​of data stored in blocks are stored in different buckets. Therefore, when reading data in a block, multiple buckets need to be accessed, resulting in low reading efficiency. Since the memory of the hash structure itself is discrete, that is, KV is stored in the form of a linked list in the hash bucket, when a new KV is written to the hash bucket, there is a problem of migrating other KVs, resulting in the actual amount of data written being greater than the amount of data newly written to the KV. Therefore, direct persistence will cause serious write amplification and low write efficiency.

[0093] The following provides a possible example of using a skip table to store KV. Figure 2 As shown, Figure 2 This is a schematic diagram of skip table storage.

[0094] Each node in the jump list stores KV, such as Figure 2 The nodes 1, 2, 3, 4, and 5 are arranged in sequence according to the key sequence number. Figure 2 A two-level index is set in the figure. In the first-level index L1, there are nodes 2 and 5, in the second-level index L2, there is node 3, and the bottom layer L0 is 1, 2, 3, 4, and 5. Multi-level index is set to improve the retrieval efficiency.

[0095] Taking reading the value of node 5 as an example, we can first traverse the second-level index to determine node 3, then traverse the first-level index to determine node 5, and then read the value of node 5.

[0096] Since each node in the skip list directly stores KV, it occupies a large amount of memory space. In addition, the value of the data stored in the block is stored in different non-contiguous nodes. Therefore, when reading a block of data, multiple nodes need to be accessed, resulting in a low cache hit rate and poor read performance. The memory between each node in the skip list is discrete, so when a new KV is written to the skip list, the actual amount of data written is greater than the amount of data of the newly added node, resulting in serious persistent write amplification and low write efficiency.

[0097] In summary, both KV index tables and Redis storage of KV have poor efficiency in managing KV, such as read efficiency and write efficiency.

[0098] In order to avoid the poor management efficiency of the above-mentioned management of KV using KV index table and Redis, the present application provides an address determination method, which can be applied to a computing device, and the computing device can be deployed on a storage device, or the computing device is used to manage data in a storage device. The address determination method includes: obtaining the data identifier of the data to be processed, and using the interval mapping relationship to determine the area identifier of the first storage area corresponding to the data identifier of the data to be processed. Then, from the address range of the first storage area, a second storage address storing the first storage address of the data to be processed is selected. Among them, the interval mapping relationship is used to indicate the data identification interval corresponding to the area identifiers of multiple storage areas, and the second storage area is used to store the first storage address of the data in the data identification interval corresponding to the second storage area in the disk. The data identification interval contains multiple continuous data identifiers, and the second storage area is one of the multiple storage areas.

[0099] In the present application, since the interval mapping relationship indicates the data identification interval corresponding to the area identifiers of multiple storage areas, and the data identification interval contains multiple continuous data identifiers, a storage area is used to store the first storage addresses corresponding to multiple data identifiers in its corresponding data identification interval. Therefore, when managing the data corresponding to multiple data identifiers belonging to the same data identification interval, only one storage area can be accessed to obtain the first storage addresses corresponding to the multiple data identifiers, thereby managing the data corresponding to the multiple data identifiers. Compared with the above-mentioned hash table, when managing multiple data identifiers belonging to the same identification interval, it is necessary to access multiple storage areas to obtain the storage addresses corresponding to the multiple data identifiers, which is a low efficiency problem. The efficiency of obtaining the first storage address of the stored data is improved, thereby improving the efficiency of data management.

[0100] Next, the address determination method provided in the present application is described in detail with reference to the accompanying drawings.

[0101] like Figure 3 As shown, Figure 3 Schematic diagram of a data storage system provided in this application. The data storage system includes a host 310 and a storage device 320. Figure 3 In the application scenario shown, users can access data through applications on the host 310. The computer running these applications can be called a "host". The host 310 can be a physical machine or a virtual machine. Physical machines include but are not limited to desktop computers, servers, laptops, and mobile devices.

[0102] In a possible example, the host 310 accesses the storage device 320 through a network to access data. For example, the network may include a switch 330 .

[0103] In another possible example, the host 310 may also communicate with the storage device 320 through a wired connection, such as a universal serial bus (USB) or a peripheral component interconnect express (PCIe) bus.

[0104] Figure 3 The storage device 320 shown can be a centralized storage system. The centralized storage system is characterized by having a unified entrance, and all data from external devices must pass through this entrance, which is the engine 321 of the centralized storage system. The engine 321 is the core component of the centralized storage system, and many advanced functions of the storage system are implemented in it.

[0105] like Figure 3 As shown, the engine 321 may have one or more controllers. Figure 3 The example of engine 321 including one controller is used for explanation. In a possible example, if engine 321 has multiple controllers, any two controllers may have a mirror channel to achieve the function of any two controllers backing up each other, thereby avoiding hardware failures that cause the entire storage device 320 to be unavailable. It should be understood that if engine 321 includes multiple controllers, engine 321 may also be referred to as an array controller of storage device 320.

[0106] The engine 321 also includes a front-end interface 3211 and a back-end interface 3214, wherein the front-end interface 3211 is used to communicate with the host 310, thereby providing data access services for the host 310. The back-end interface 3214 is used to communicate with the hard disk to expand the capacity of the storage device 320. Through the back-end interface 3214, the engine 321 can connect more hard disks to form a very large storage resource pool.

[0107] In hardware, such as Figure 3As shown, the controller includes at least a processor 3212 and a memory 3213. The processor 3212 is a central processing unit (CPU) for processing data access requests from outside the storage device 320 (server or other storage system), and is also used to process requests generated inside the storage device 320. Exemplarily, when the processor 3212 receives a write data request sent by the host 310 through the front-end interface 3211, the data in the write data request will be temporarily stored in the memory 3213. When the total amount of data in the memory 3213 reaches a certain threshold, the processor 3212 sends the data stored in the memory 3213 to at least one of the mechanical hard disk 3221, the mechanical hard disk 3222, the solid state drive (SSD) 3223 or other hard disks 3224 through the back-end port for persistent storage.

[0108] Memory 3213 refers to an internal memory that directly exchanges data with the processor. It can read and write data at any time and at a very fast speed. It serves as a temporary data storage for the operating system or other running programs. The memory includes at least two types of memories. For example, the memory can be either a random access memory or a read-only memory (ROM). For example, the random access memory is DRAM or SCM. DRAM is a semiconductor memory, and like most random access memories (RAM), it is a volatile memory device. However, DRAM and SCM are only exemplary descriptions in this embodiment, and the memory can also include other random access memories, such as static random access memory (SRAM), etc. As for the read-only memory, for example, it can be a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), etc.

[0109] In addition, the memory 3213 can also be a dual in-line memory module or a dual in-line memory module (DIMM), that is, a module composed of a dynamic random access memory (DRAM), or an SSD. In practical applications, multiple memories 3213 and different types of memories 3213 can be configured in the controller. This embodiment does not limit the number and type of memory 3213. In addition, the memory 3213 can be configured to have a power-saving function. The power-saving function means that when the system loses power and then powers on again, the data stored in the memory 3213 will not be lost. A memory with a power-saving function is called a non-volatile memory.

[0110] The memory 3213 stores a software program, and the processor 3212 runs the software program in the memory 3213 to manage the data and the metadata of the data. For example, according to the first identifier of the data, the interval mapping table is queried to obtain the storage area matching the first identifier, and then the storage address matching the first identifier is determined according to the mapping relationship, so as to manage the data in the storage address matching the first identifier.

[0111] like Figure 3 As shown, in this system, the engine 321 may not have a hard disk slot, and the hard disk needs to be placed in the hard disk frame 322, and the back-end interface 3214 communicates with the hard disk frame 322. The back-end interface 3214 exists in the form of an adapter card in the engine 321, and two or more back-end interfaces 3214 can be used simultaneously on one engine 321 to connect multiple hard disk frames. Alternatively, the adapter card can also be integrated on the motherboard, and the adapter card can communicate with the processor 3212 via the PCIe bus.

[0112] It should be noted that Figure 3 Only one engine 321 is shown in the figure, however, in actual applications, the data storage system may include two or more engines 321, and redundancy or load balancing is performed between the multiple engines 321.

[0113] The hard disk enclosure 322 includes a control unit 3225 and a plurality of hard disks. The control unit 3225 may have various forms. In one case, the hard disk enclosure 322 is an intelligent disk enclosure, such as Figure 3As shown, the control unit 3225 includes a CPU and a memory. The CPU is used to perform operations such as address conversion and reading and writing data. The memory is used to temporarily store data to be written to the hard disk, or read from the hard disk to send data to the controller. In another case, the control unit 3225 is a programmable electronic component, such as a data processing unit (DPU). The DPU has the versatility and programmability of the CPU, but is more specialized and can run efficiently on network data packets, storage requests or analysis requests. The DPU is distinguished from the CPU by a large degree of parallelism (needing to process a large number of requests). Optionally, the DPU here can also be replaced by a processing chip such as a graphics processing unit (GPU) and an embedded neural network processor (NPU). Generally, the number of control units 3225 can be one, or two or more. The functions of the control unit 3225 can be offloaded to the network card 3226. In other words, in this embodiment, the hard disk frame 322 does not have a control unit 3225 inside, but the network card 3226 completes data reading and writing, address conversion and other computing functions. At this time, the network card 3226 is an intelligent network card. It can include a CPU and a memory. The CPU is used to perform operations such as address conversion and reading and writing data. The memory is used to temporarily store data to be written to the hard disk, or data read from the hard disk to be sent to the controller. It can also be a programmable electronic component, such as a DPU. There is no ownership relationship between the network card 3226 and the hard disk in the hard disk frame 322. The network card 3226 can access any hard disk in the hard disk frame 322 (such as Figure 3 The mechanical hard disk 3221, mechanical hard disk 3222, solid state disk 3223 and other hard disk 3224 are shown, so it is more convenient to expand the hard disk when the storage space is insufficient.

[0114] According to the type of communication protocol between the engine 321 and the hard disk frame 322, the hard disk frame 322 may be a serial attached small computer system interface (SAS) hard disk frame, or it may be an NVMe (Non-Volatile Memory express) hard disk frame and other types of hard disk frames. The SAS hard disk frame adopts the SAS3.0 protocol, and each frame supports 25 SAS hard disks. The engine 321 is connected to the hard disk frame 322 through the onboard SAS interface or the SAS interface module. The NVMe hard disk frame is more like a complete computer system, and the NVMe hard disk is inserted in the NVMe hard disk frame. The NVMe hard disk frame is then connected to the engine 321 through the RDMA port.

[0115] In an optional implementation, the storage device 320 is a centralized storage system with integrated disk and controller, and the storage device 320 does not have the above-mentioned hard disk frame 322, and the engine 321 is used to manage multiple hard disks connected through the hard disk slots. The function of the hard disk slots can be implemented by the back-end interface 3214.

[0116] In another optional implementation, Figure 3 The storage device 320 shown is a distributed storage system, which includes a computing device cluster and a storage device cluster. The computing device cluster includes one or more computing devices, and each computing device can communicate with each other. The computing device can be a physical computing device, such as a server, a desktop computer, or a controller of a storage array. In terms of hardware, the computing device can include a processor, a memory, and a network card. Among them, the processor is a CPU, which is used to process data access requests from outside the computing device, or requests generated inside the computing device. Exemplarily, when the processor receives a write data request sent by a user, the data in these write data requests will be temporarily stored in the memory. When the total amount of data in the memory reaches a certain threshold, the processor sends the data stored in the memory to the storage device for persistent storage. In addition, the processor is also used to calculate or process data, such as metadata management, deduplication, data compression, virtualized storage space, and address conversion. In an example, any computing device can access any storage device in the storage device cluster through a network. The storage device cluster includes multiple storage devices. A storage device includes one or more controllers, a network card, and multiple hard disks, and the network card is used to communicate with the computing device.

[0117] For example, Figure 3 The data storage system shown can be applied to scenarios where data needs to be stored and cleared regularly, such as in the field of security, where the captured video data is stored in the storage device 320 and deleted after a set time. Figure 3 In the data storage system, the user determines the data to be processed on the host 310, obtains the data identifier of the data to be processed, and then uses the interval mapping relationship to determine the region identifier of the first storage region corresponding to the data identifier of the data to be processed. From the address range of the first storage region, a second storage address of the first storage address storing the data to be processed is selected.

[0118] It should be noted that Figure 3 The illustrated architecture of the data storage system is only an example, and the device type or number of devices within the system can be configured according to actual needs, which is not limited in the embodiments of the present application.

[0119] On the one hand, the present application provides an address determination method, which can be applied to Figure 3 The data storage system shown. Figure 4 As shown, Figure 4 The following is a flow chart of an address determination method provided in the present application. The address determination method can be executed by a computing device. The computing device can be deployed at Figure 3 The storage device 320 shown in FIG. 1 may be deployed on the host 310 or the computing device may be connected to the host 310. Figure 3 The storage device 320 shown in FIG. Figure 4 As shown, the address determination method may include the following steps S410 to S430.

[0120] S410: The computing device obtains a data identifier of the data to be processed in the processing request.

[0121] In a possible example, the computing device obtains the data identifier of the data to be processed from an acquired processing request, where the processing request carries the data identifier of the data to be processed.

[0122] In one possible scenario, the processing request is a processing request sent by the host 310 to the computing device, such as a processing request triggered by operations such as downloading of the host 310, a processing request for storing captured video data in the security field, or a processing request for querying, updating, deleting, etc. the aforementioned video data.

[0123] In another possible scenario, the processing request is generated by the computing device according to a service executed by the computing device.

[0124] The data identifier is an identifier of the storage location of the data to be processed in the disk.

[0125] Exemplarily, the data identifier is the id of the block of the data to be processed in the hard disk enclosure 322 and the offset of the storage address of the data to be processed in the block, that is, blockid+offset.

[0126] For a description of how to generate the data identifier of the data to be processed in a processing request, see the following Fig. 9 The contents of S920 will not be elaborated here.

[0127] S420: The computing device determines the region identifier of the first storage region corresponding to the data identifier of the data to be processed by using the interval mapping relationship.

[0128] The interval mapping relationship is used to indicate the data identification intervals corresponding to the area identifications of the multiple storage areas. The second storage area is used to store the first storage address of the data in the data identification interval corresponding to the second storage area in the disk. The data identification interval contains multiple continuous data identifications, and the second storage area is one of the multiple storage areas.

[0129] Exemplarily, the interval mapping relationship is generated by the computing device before executing the address determination method of the present application. The user sets the number of data identifiers included in the data identifier interval in the interval mapping relationship and the size of the storage area according to the amount of data to be stored. The computing device receives the interval mapping relationship input by the user.

[0130] In a possible example, assuming that the storage area is a storage space in the hard disk frame 322 or the memory 3213, the area identifier of the storage area can be the first address or data identifier of the storage area, and the first address is such as 0x08000001. Since the size of each storage area is fixed, the first address of each storage area is also fixed. The digital identifier is such as storage area 0, storage area 1, etc., and the digital identifier indicates the first address of storage area 0, storage area 1, etc.

[0131] With respect to the above interval mapping relationship, two possible examples are provided below.

[0132] Example 1: The interval mapping relationship is an interval mapping table. Figure 4 As shown, the interval mapping shows the data identification intervals corresponding to the area identifications of multiple storage areas, that is, storage area 0 corresponds to the data identification interval block0-999, storage area 1 corresponds to the data identification interval block1000-1999, storage area 2 corresponds to the data identification interval block2000-2999, storage area 3 corresponds to the data identification interval block3000-3999, and storage area 9 corresponds to the data identification interval block9000-9999, etc. The data identifications in the aforementioned data identification intervals are all continuous. The data identification interval and the corresponding storage area can be used cyclically.

[0133] The computing device may query the interval mapping table using the data identifier of the data to be processed, determine the data identifier interval to which the data identifier of the data to be processed belongs, and then obtain the region identifier of the first storage region corresponding to the data identifier interval to which the data identifier of the data to be processed belongs.

[0134] For example, the data identifier of the data to be processed is block10-25M, and the computing device determines that block10-25M belongs to block0-999, and then obtains the first address of the storage area 0 corresponding to block0-999.

[0135] In the present application, since the interval mapping relationship indicates the data identification interval corresponding to the area identifiers of multiple storage areas, and the data identification interval contains multiple continuous data identifiers, one storage area is used to store the first storage addresses corresponding to multiple data identifiers in its corresponding data identification interval. Therefore, when the computing device manages the data corresponding to multiple data identifiers belonging to the same data identification interval, it can access only one storage area to obtain the first storage addresses corresponding to the multiple data identifiers, and then manage the data corresponding to the multiple data identifiers. Compared with the above-mentioned hash table, when managing multiple data identifiers belonging to the same identification interval, it is necessary to access multiple storage areas to obtain the storage addresses corresponding to the multiple data identifiers, which is a low efficiency problem. The efficiency of obtaining the first storage address of the stored data is improved, thereby improving the efficiency of data management.

[0136] Example 2: The addresses of the multiple storage areas are continuous and the data identification intervals corresponding to the storage areas contain the same number of data identifications. The interval mapping relationship only shows the area identification (such as storage area 0) of the storage area corresponding to the smallest data identification interval (such as block0-999).

[0137] For example, the data identifier of the data to be processed is block21000-25M, the number of data identifiers included in the data identifier interval is 1000, and the size of the storage area is 64M. The computing device can determine that the data identifier of the data to be processed belongs to block2000-2999 and is the third data identifier interval. Therefore, the computing device obtains the first address of storage area 2 based on the first address of storage area 0 plus the size of 128M (64M x 2).

[0138] S430: The computing device selects, from the address range of the first storage area, a second storage address of the first storage address storing the data to be processed.

[0139] The computing device obtains the address range of the first storage area from the area identifier of the first storage area determined above, and then selects the second storage address of the first storage address storing the data to be processed from the address range of the first storage area according to the data identifier of the data to be processed.

[0140] Exemplarily, the computing device may obtain the first address of the first storage area according to the area identifier of the first storage area, and further determine the address range of the first storage area according to a set storage area size, such as 64M.

[0141] Two possible implementations are provided below to illustrate the data stored in the storage area.

[0142] In a possible implementation, the first storage address and the data identifier of the data corresponding to the first storage address can be stored in the storage area in KV format. For the content of storing the first storage address and the data identifier in KV format, please refer to the contents of Table 1 and Table 2 above, which will not be repeated here.

[0143] When the storage area stores the first storage address and data identifier in KV format, the computing device uses the data identifier of the data to be processed to traverse the address range of the first storage area, finds the KV corresponding to the data identifier of the data to be processed, and thereby obtains the second storage address in the KV corresponding to the data identifier of the data to be processed.

[0144] In another possible implementation, the mapping relationship between the data identifier and the address in the storage area is reflected by a mapping model, so the data identifier may not be stored in the storage area, and only the first storage address of the data in the disk is stored.

[0145] Then, the computing device substitutes the data identifier of the data to be processed into the mapping model to obtain the second storage address. The mapping model is used to indicate the mapping relationship between the data identifier and the address in the storage area. The storage space indicated by the address in the storage area is used to store the data indicated by the data identifier that has a mapping relationship with the address in the storage area, at the first storage address in the disk.

[0146] In the present application, the computing device reflects the mapping relationship between the data identifier and the address in the storage area through a mapping model, so the data identifier may not be stored in the storage area, and only the first storage address of the data in the disk is stored, avoiding the problem of storing the data identifier and the first storage address in the KV form, thereby reducing the storage space occupied. In addition, the computing device can directly locate the second storage address of the first storage address storing the data to be processed by using the mapping model, thereby improving the efficiency of determining the first storage address of the data to be processed, thereby improving the management efficiency of the data to be processed.

[0147] In one possible scenario, a corresponding mapping model is maintained for each storage area in the computing device. When selecting the second storage address from the address range of the first storage area, the computing device substitutes the data identifier of the data to be processed into the mapping model corresponding to the first storage area to obtain the second storage address.

[0148] In a possible example, the mapping model corresponding to the first storage area is obtained by fitting the address of the storage space in the first storage area where the first storage address has been stored, and the data identifier corresponding to the stored first storage address.

[0149] like Figure 5 As shown, Figure 5A schematic diagram of the fitting mapping model provided for the present application, if the address of the storage space in which the first storage address has been stored in the first storage area, and the data identifier corresponding to the stored first storage address are data identifier 1: block00-63M and the corresponding address is 0x00000001, data identifier 2: block10-63M and the corresponding address is 0x00000401, data identifier 3: block20-63M and the corresponding address is 0x00000801, by fitting the aforementioned data identifiers and the addresses of the storage space, a corresponding mapping model can be obtained.

[0150] In one possible scenario, Figure 5 The mapping model is obtained by fitting a machine learning model, such as a linear regression model. In other embodiments of the present application, the mapping model can also be obtained by fitting a deep learning model, a table or a tree index, such as a B-tree index.

[0151] In the present application, the computing device uses the address of the storage space storing the first storage address and the data identifier corresponding to the stored first storage address to fit the mapping model, which can accurately represent the correspondence between the address of the storage space storing the first storage address and the data identifier corresponding to the first storage address. Furthermore, the computing device can accurately and directly determine the second storage address corresponding to the data identifier of the data to be processed using the mapping model, thereby improving the efficiency of determining the first storage address of the data to be processed stored in the second storage address, thereby improving management efficiency. Furthermore, the computing device uses the mapping model to reflect the mapping relationship between the data identifier and the address in the storage area, so the data identifier may not be stored in the storage area, and only the first storage address of the data in the disk is stored, thereby reducing the storage space occupied in the storage area.

[0152] In a possible embodiment, the storage area includes a first sub-area and a second sub-area, and addresses of the first sub-area and the second sub-area are continuous.

[0153] Since the first storage addresses stored in the first sub-region and the second sub-region have different characteristics, such as the first storage addresses in the first sub-region are relatively continuous, and the data identifiers corresponding to the first storage addresses are also relatively continuous. The first storage addresses in the second sub-region are relatively discrete, and the data identifiers corresponding to the first storage addresses are also relatively discrete. Therefore, different mapping models will be used to represent the relationship between the address of the storage space in which the first storage address has been stored and the data identifier corresponding to the stored first storage address in the first sub-region or the second sub-region, respectively. Thereby, the management efficiency of the first storage addresses in each sub-region is improved.

[0154] Two possible examples are shown below to illustrate the representation of the mapping model.

[0155] In the first possible example, since the data identifiers of the first storage address stored in the first sub-region and the data corresponding to the stored first storage address are continuous and close, a linear regression model may be used to fit the mapping relationship between the data identifier in the data identifier range of the first sub-region and the address of the storage space in the first sub-region;

[0156] In the second possible example, since the data identifiers of the first storage address stored in the second sub-region and the data corresponding to the stored first storage address are relatively discrete, a linear regression model can be used to fit the mapping relationship between the data identifiers in the data identifier range of the second sub-region and the addresses in the second sub-region, or a table or B-tree index can be used to represent the mapping relationship between the data identifiers in the data identifier range of the second sub-region and the addresses of the storage space in the second sub-region.

[0157] It is worth noting that the above examples are for illustration only and should not be construed as limitations on the present application. In the first possible example, the mapping model may also be represented by a table or a B-tree index.

[0158] In the present application, the computing device uses different basic models (linear regression model, B-tree index, etc.) to fit corresponding mapping models according to the characteristics of the first storage addresses stored in the first sub-region and the second sub-region, so as to adopt different mapping models for different sub-regions, thereby improving the accuracy and efficiency of determining the first storage addresses stored in different sub-regions.

[0159] In a possible embodiment, the above storage area is described. Figure 6 As shown, Figure 6 Schematic diagram of the storage area provided for this application Figure 1 .

[0160] The storage area is provided with a management sub-area 610 and a data sub-area 620. The management sub-area 610 stores the above-mentioned mapping model, the data identification range or address range of each sub-area, etc. The data sub-area 620 only stores the first storage address corresponding to the plurality of data. The data sub-area 620 is divided into a first sub-area and a second sub-area. The first sub-area is a sub-area for storing the first storage address of the newly written data.

[0161] In a possible example, the storage area further includes a reserved sub-area, and the reserved sub-area is used for occupation when a new first storage address is added in the first sub-area.

[0162] For example, Figure 7 As shown, Figure 7 Schematic diagram of the storage area provided for this application Figure 2The reserved sub-area is located on both sides of the first sub-area and the second sub-area. When the storage area needs to store the first storage address of the newly written data, the computing device will occupy part of the position in the reserved sub-area after the first sub-area to store the first storage address of the newly written data. The space of the two reserved sub-areas can be recycled, that is, the entire data sub-area 620 can be recycled.

[0163] In the present application, the computing device sets a reserved sub-area in the storage area to avoid the problem that the first storage address is written into other storage areas when the first sub-area or the second sub-area needs to occupy more space. This ensures that the first storage addresses corresponding to multiple data identifiers in a data identification interval can be stored in one storage area, and then the computing device only needs to read one storage area when managing the first storage addresses of the aforementioned multiple data identifiers, and the reading efficiency is optimal.

[0164] In a possible example, the management sub-area 610 is also used to store other management fields, such as data storage duration, etc.

[0165] Based on the configuration of the first sub-area and the second sub-area in the storage area, the above S430 may be specifically implemented as follows:

[0166] The computing device selects a second storage address of the first storage address storing the data to be processed in the address range of the first target sub-area.

[0167] The first target sub-region is a sub-region corresponding to the data identification range of the data identification of the data to be processed in the first sub-region and the second sub-region. The data identification range of a sub-region is a range composed of the data identification of the data stored in the sub-region at the first storage address.

[0168] Exemplarily, if the data to be processed is not newly written data, when the data identifier of the data to be processed is within the data identifier range of the first sub-region, the first target sub-region is the first sub-region; when the data identifier of the data to be processed is within the data identifier range of the second sub-region, the first target sub-region is the second sub-region.

[0169] For example, the data identification range of the first sub-region is (block50, block110), and the data identification range of the second sub-region is (block10, block40). If the data identification of the data to be processed is block560-63M, the computing device determines that the first sub-region is the first target sub-region. If the data identification of the data to be processed is block300-63M, the computing device determines that the first sub-region is the first target sub-region.

[0170] In the present application, when the first target sub-region is the first sub-region or the second sub-region, the computing device determines the first target sub-region according to the data identifier range of the data identifier of the to-be-processed data, thereby avoiding determining the second storage address of the first storage address storing the to-be-processed data from the entire storage region, that is, narrowing the determined address range. This reduces the time consumption of determining the second storage address and improves the efficiency of determining the second storage address.

[0171] If the data to be processed is newly written data, the first target sub-region is a sub-region of a first storage address storing the newly written data.

[0172] For example, if the data identifier of the data to be processed is block 1110-63M, the data to be processed is newly written data, and the first target sub-region is a sub-region of the first storage address storing the newly written data, such as the next storage space after the first sub-region, the next storage space after the second sub-region, or when the first sub-region or the second sub-region does not exist, the first target sub-region is a free storage space in the data sub-region 620.

[0173] In the present application, when the data to be processed is newly written data, the computing device directly writes the first storage address of the newly written data into a specific storage space, which belongs to the first target sub-area, so that in the newly written data scenario, the first storage address of the newly written data can still be stored, thereby improving the management efficiency of specific types of data (newly written data).

[0174] In a possible implementation manner, the computing device substitutes the data identifier of the data to be processed into the mapping model of the first target sub-region to obtain the second storage address.

[0175] The mapping model of the first target sub-region is used for the mapping relationship between the data identifiers in the data identifier range of the first target sub-region and the addresses in the first target sub-region.

[0176] Exemplarily, the mapping model includes a first mapping model and a second mapping model, the first mapping model is used for a mapping relationship between a data identifier in a data identifier range of a first sub-region and an address in the first sub-region, and the second mapping model is used for a mapping relationship between a data identifier in a data identifier range of a second sub-region and an address in the second sub-region. The mapping model of the first target sub-region is any one of the first mapping model and the second mapping model.

[0177] In the present application, if there is a mapping model corresponding to the first target sub-region, the computing device determines the first target sub-region to which the data to be processed belongs based on the data identifier of the data to be processed, and then uses the mapping model of the first target sub-region to determine the second storage address, so that the mapping model of the first target sub-region can accurately and quickly locate the second storage address, thereby improving the efficiency of determining the second storage address.

[0178] For example, if the data identifier of the data to be processed is within the data identifier range of the first sub-region, the first target sub-region is the first sub-region, and the computing device substitutes the data identifier of the data to be processed into the first mapping model to obtain the second storage address. The first mapping model and the second mapping model are stored in the management sub-region 610.

[0179] like Figure 8 As shown, Figure 8 A flow chart of determining a second storage address provided for the present application. Assume that the data identification range of the first sub-area is (block36, block60), that is, the first storage addresses corresponding to the data identifications in the range of block36-block60 are all stored in the first sub-area, and the data identification range of the second sub-area is (block10, block35), that is, the first storage addresses corresponding to the data identifications in the range of block10-block35 are all stored in the second sub-area.

[0180] Figure 8 It is shown that: ①: If the data identifier of the data to be processed is block50, the computing device determines that the storage address corresponding to the data identifier of the data to be processed is stored in the first sub-region, and determines that the first mapping model (model 1) corresponding to the first sub-region is the mapping model of the first target sub-region. ②: Using the first mapping model, determine the second storage address corresponding to the data identifier of the data to be processed.

[0181] For another example, if the data identifier of the data to be processed is within the data identifier range of the second sub-region, the first target sub-region is the second sub-region, and the computing device substitutes the data identifier of the data to be processed into the second mapping model (model 2) to obtain the second storage address.

[0182] For another example, if the data to be processed is newly written data, the first target sub-region is the sub-region of the first storage address storing the newly written data. Exemplarily, the computing device substitutes the data identifier of the data to be processed into the first mapping model to obtain the second storage address. Alternatively, the computing device uses the address of a free storage space in the storage area as the second storage address. If the first sub-region exists, the free storage space is the next storage space after the first sub-region. If the first sub-region does not exist and the second sub-region exists, the free storage space is the next storage space after the second sub-region. If the first sub-region and the second sub-region do not exist, the free storage space is any storage space in the data sub-region.

[0183] Two implementations of determining that the data to be processed is newly written data are shown below.

[0184] Implementation method 1: If the data identifier of the data to be processed is not within the data identifier range of the first sub-region, then the data to be processed is newly written data.

[0185] In implementation mode 2, the computing device further receives an attribute identifier of the data to be processed, the attribute identifier indicating whether the data to be processed is newly written data. If the attribute identifier indicates that the data to be processed is newly written data, then the data to be processed is newly written data.

[0186] After the computing device writes the first storage address storing the data to be processed into the second storage address, the data identification range of the first sub-region is adjusted to include the data identification of the data to be processed, that is, the first sub-region includes the storage space indicated by the second storage address.

[0187] In one possible scenario, if the computing device does not determine the second storage address corresponding to the data identifier of the data to be processed according to the mapping model, a message is sent to the host 310 to enable the host 110 to display the control component of the processing flow and receive the user's trigger operation on the control component. In response to the trigger operation, the computing device determines the processing flow selected by the user, and the computing device performs corresponding processing according to the processing flow selected by the user. The trigger operation may include a click operation, a slide operation, etc.

[0188] The processing flow may include an end process or a data writing process.

[0189] If the processing flow selected by the user is to end the processing, the computing device directly suspends the processing of the processing request.

[0190] If the processing flow selected by the user is data writing processing, the computing device executes the following according to the data carried in the processing request and the corresponding data identifier: Fig. 9 The contents of S930-S950 are not repeated here.

[0191] Exemplarily, the computing device reports an error to the host 310 while sending the control component of the processing flow to the host 310 .

[0192] Further, after the second storage address is determined in the above S430, different operations may be performed in different scenarios. It may be configured according to actual needs and is not limited in the embodiment of the present application.

[0193] For example, the data writing scenario is taken as an example to illustrate the above-mentioned address determination method. Fig. 9 As shown, Fig. 9 The data writing method provided in this application is a flow chart. The data writing method can be executed by a computing device. The computing device can be deployed at Figure 3 The computing device shown in FIG. 1 may be deployed on a host 310 or the computing device may be connected to Figure 3 For use with the computing equipment shown in the figure. Fig. 9 As shown, taking the data to be processed as data a as an example, the data writing method may include the following steps S910 to S950.

[0194] S910: The computing device obtains a write request.

[0195] In one possible scenario, the write request is a write request sent by the host 310 to the computing device, such as a processing request triggered by downloading operations of the host 310 or a processing request for storing captured video data in the security field, and the write request carries data a.

[0196] In another possible scenario, the write request is generated by the computing device according to a service executed by the computing device.

[0197] S920: The computing device writes data a and obtains a first storage address of the data a and a data identifier of the data a.

[0198] The computing device writes data a into the hard disk frame 322 .

[0199] In a possible implementation, the computing device allocates a block in the hard disk frame 322 to data a according to the data volume of data a, and then writes the data a into the allocated block to obtain the blockid of the block and the offset of the first data in the block.

[0200] Exemplarily, the data identifier of the data a may be blockid+offset, and the first storage address of the data a is used to indicate the address of the data a in the hard disk enclosure 322, such as the first address of the block to which the data a belongs.

[0201] For example, a block size of 64M and a data identifier of data a of identifier 1 are used for explanation. Two possible examples are provided below.

[0202] Example 1, assuming that the size of the existing data is 69M, since the existing data is larger than 64M, the computing device allocates two blocks for storage of the existing data, such as block1 and block2. The computing device stores 64M of the existing data in block1 and 5M of the existing data in block2, so that the first storage address of the existing data includes the first address of block1 and the first address of block2, and the corresponding data identifiers are block10-63M and block20-5M.

[0203] Example 2: Based on the above example 1, the computing device will also write data a, which is 20M. The computing device allocates data a to block2 for storage. Since part of data a has been stored in block2 from 0 to 5M, data b will be stored starting from the first address of block2 with an offset of 5M, so that the first storage address of data a is the first address of block2, and identifier 1 is block25-25M.

[0204] S930: The computing device determines the region identifier of the first storage region corresponding to the data identifier of the data a by using the interval mapping relationship.

[0205] For a detailed description of S930, see the above Figure 4 The content shown in S420 is not repeated here.

[0206] S940: The computing device selects a second storage address of the first storage address storing data a from the address range of the first storage area.

[0207] In this embodiment, S940 has the following differences compared to S430:

[0208] If there is no mapping model corresponding to the first target sub-region, the computing device uses the address of the first storage space in the first target sub-region as the second storage address. Then, the computing device fits the second storage address and the data identifier of the data to be processed to obtain the mapping model corresponding to the first target sub-region.

[0209] In a possible example, if the first mapping model does not exist in the management sub-area 610, the computing device determines a free storage space from the storage area as the storage space indicated by the second storage address. The storage space is the first storage space of the first sub-area.

[0210] For example, if the second mapping model exists, the computing device uses the address of a storage space after the second sub-region as the second storage address. If the second mapping model does not exist, the computing device uses the address of any storage space in the data sub-region 620 as the second storage address.

[0211] After writing the first storage address storing the data a into the second storage address of the first sub-region, the computing device will fit the second storage address and the data identifier of the data a to obtain a mapping model corresponding to the first sub-region.

[0212] In the present application, the computing device uses the address of the first storage space in the first target sub-region as the second storage address to store the first storage address of the data to be processed, so as to write the first storage address of the data to be processed into the storage region when there is no mapping model corresponding to the first target sub-region, thereby improving the writing capability of the first storage address. In addition, the computing device fits the data identifier stored in the first target sub-region and the second storage address storing the first storage address corresponding to the data identifier to obtain the mapping model of the first target sub-region. This is beneficial for determining the address of the storage space storing the first storage address of the newly written data when there is new written data in the future, thereby improving the data writing efficiency and reducing the write amplification.

[0213] In a possible scenario, the computing device also writes data b, and then writes the address of the first storage storing data b into the next storage space after the second storage address. The computing device performs fitting according to the second storage address and the data identifier of the data to be processed, and the address of the next storage space after the second storage address and the data identifier of data b, to obtain a mapping model of the first sub-area.

[0214] For the description of the fitted mapping model, please refer to the above Figure 5 The contents shown will not be repeated here.

[0215] S950: The computing device writes the first storage address of the data a into the second storage address.

[0216] Taking the storage of the first storage address in the storage area as an example, if the computing device does not compress the storage address, the complete first storage address of the data is written into the second storage address.

[0217] Alternatively, the computing device compresses the first storage address of data a, and writes the compressed first storage address of data a into the second storage address.

[0218] Exemplarily, the computing device performs prefix compression on the first storage address, saves the prefix in the management sub-area 610 , and stores the remaining suffix in the storage space in the corresponding data sub-area 620 .

[0219] For example, the three first storage addresses are: 0xFFFF0080, 0xFFFF0090, 0xFFFF0100, and the prefix obtained by prefix compression is 0xFFFF0, and then the addresses stored in the first sub-area are 080, 090, and 100. The prefix can be maintained by the mapping model in the management sub-area 610.

[0220] It is worth noting that the above prefix compression method is only an example. In other embodiments of the present application, Huffman coding or dictionary compression may also be used.

[0221] In the present application, by compressing the first storage address of data a and then writing it into the storage space indicated by the second storage address, the amount of data written is reduced, the occupancy of storage space in the storage area is reduced, and the management efficiency of the first storage address is improved.

[0222] In a possible scenario, the mapping model not only has a mapping relationship between a second storage address storing a first storage address and a data identifier of data corresponding to the first storage address, but also maintains a prefix common to multiple first storage addresses corresponding to multiple second storage addresses. The computing device determines the suffix of the first storage address stored in the storage area according to the data identifier of the data to be processed, and obtains the prefix of the storage address corresponding to the first identifier from the management sub-area 610, and after splicing the prefix and the suffix, a complete storage address can be obtained.

[0223] For example, the following uses a data processing (query, update or delete, etc.) scenario as an example to illustrate the above-mentioned address determination method. Fig.10 As shown, Fig.10 The data processing method provided in this application is a flowchart. The data processing method can be executed by a computing device. The computing device can be deployed on Figure 3 The storage device 320 shown in FIG. 1 may be deployed on the host 310 or the computing device may be connected to the host 310. Figure 3 The storage device 320 shown in FIG. Fig.10 As shown, taking the data to be processed as data a as an example, the data processing method may include the following steps S1010 to S1050.

[0224] S1010. The computing device obtains the data identifier of data a.

[0225] In a possible example, the data identifier of the data a is data carried in a management request sent by the host 310 to the computing device.

[0226] If the management request carries a data name, the computing device converts the data name into a corresponding blockid+offset.

[0227] In another possible example, the management request is generated by the computing device according to a service executed by the computing device.

[0228] Exemplarily, the management request may be a query request, a read request, an update request, a delete request, or the like.

[0229] S1020: The computing device determines the region identifier of the first storage region corresponding to the data identifier of the data a by using the interval mapping relationship.

[0230] For a detailed description of S1020, see the above Figure 4 The content shown in S420 is not repeated here.

[0231] S1030: The computing device selects a second storage address of the first storage address storing data a from the address range of the first storage area.

[0232] In this embodiment, the content of S1030 is different from that of S940 in the following ways:

[0233] If there is no mapping model corresponding to the second sub-region, the computing device uses the address of the first storage space in the second sub-region as the second storage address.

[0234] For example, when only one first storage address is stored in the second sub-region, the second mapping model is not fitted in the second sub-region. The first storage address is stored in the previous storage space before the address boundary between the first sub-region and the second sub-region, that is, the first storage space in the second sub-region. Therefore, the address of the first storage space in the second sub-region is the second storage address storing the first storage address. In actual situations, when the sub-region of the first storage address is changed, the second storage address storing the first storage address may not change. For the foregoing content, please refer to the following Fig.12 or Fig.13 Description of the.

[0235] In another possible embodiment, when only one first storage address is stored in the first sub-area or the second sub-area, the management sub-area 610 stores the correspondence between the data identifier of the data stored at the first storage address and the second storage address storing the first storage address. Therefore, the computing device can directly determine the second storage address of the first storage address storing data in the first sub-area or the second sub-area based on the correspondence.

[0236] S1040: The computing device obtains the first storage address of the data a stored at the second storage address.

[0237] The computing device reads the first storage address of the data a stored at the second storage address.

[0238] S1050: The computing device updates, deletes or reads the data a stored in the storage space indicated by the first storage address of the data a.

[0239] Since the first storage address is only the first address corresponding to the storage data a, the computing device can determine the data a according to the first address and the offset in the data identifier of the data a.

[0240] With respect to a computing device updating, deleting or reading data a stored in a storage space indicated by a first storage address of the data a, three possible embodiments are provided below.

[0241] In a first possible embodiment, the computing device reads or queries the data a stored in the storage space indicated by the first storage address of the data a.

[0242] In a second possible embodiment, the computing device updates the data a stored in the storage space indicated by the first storage address of the data a according to the data b in the update request.

[0243] The data a stored in the storage space indicated by the first storage address of the computing device updating the data a can be specifically implemented as follows: Fig.11 content, Fig.11 This is a flow chart of the data updating method provided in the present application. The data updating method may include steps S1110 to S1130.

[0244] S1110. The computing device obtains data b.

[0245] The computing device may obtain the data b carried by the update request from the update request, where the size of the data b is consistent with the size of the data a.

[0246] S1120. The computing device deletes the data a stored in the storage space indicated by the first storage address of the data a.

[0247] The computing device erases the data a stored in the storage space indicated by the first storage address of the data a, so that no data is stored in the storage address.

[0248] S1130. The computing device writes the data b into the storage space indicated by the first storage address of the data a.

[0249] Since the size of data b is consistent with that of data a, the computing device can directly write data b into the storage space indicated by the first storage address of data a.

[0250] In the present application, the computing device directly determines the storage space indicated by the first storage address of the data a according to the mapping model, and updates the data a in the storage space indicated by the first storage address of the data a. This enables the data in the storage space indicated by the first storage address of the data a to be directly located according to the number identifier of the data a, thereby improving the efficiency of data management.

[0251] In a third possible embodiment, the computing device deletes the data a stored in the storage space according to the storage space indicated by the first storage address of the data a. Two possible scenarios are provided below.

[0252] In a first possible scenario, the computing device partially deletes the first data in the storage space indicated by the first storage address of the first data. Two possible implementation methods are provided below.

[0253] In a first possible implementation, after the first data (data c) is partially deleted, the address boundary of the second sub-region close to the first sub-region is adjusted to the first address. The first address is the address of the storage space in the first storage region that stores the first storage address of data c. The data c is any data stored in the first sub-region at the first storage address.

[0254] Exemplarily, the data identifier of the data c is within the data identifier range of the first sub-region, and the data c occupies at least an entire block, or the first data is complete data in a block. The data c is partially deleted, which means that the data to be deleted is part of the data in the entire block corresponding to the data c, or the data to be deleted is part of the data in a complete block.

[0255] Take data c as data 1 and data 2 as an example. The size of a block is 64M, and data 1 is 65M. Therefore, the stored data 1 will be stored in two blocks. Correspondingly, the first storage addresses corresponding to the two blocks will be stored in the above storage area, such as block10-63M and block20-1M, including a whole block. Therefore, if the data identifier of the data to be deleted is block10-20M, data 1 is partially deleted. If the data identifier of the data to be deleted is block20-1M, it means that the complete data in a block is completely deleted.

[0256] The size of data 2 is 20M, and the data 2 will be stored in a block, such as block 2 above, and the offset is 1-21M. If the data identifier of the data to be deleted is block 21-10M, data 2 is partially deleted.

[0257] The following describes how the computing device adjusts the address boundary of the second sub-region close to the first sub-region to the first address.

[0258] The computing device adjusts the boundary address between the first sub-region and the second sub-region to the tail address of the storage space storing the first storage address of the first data, so as to move the first storage address storing the first data from the first sub-region to the second sub-region.

[0259] In one possible example, Fig.12 As shown, Fig.12 Regional variations provided for this application Figure 1 Before partially deleting the first data, the first address of the storage space storing the first storage address of the first data is the address boundary between the first sub-region and the second sub-region. After partially deleting the first data, the computing device adjusts the address boundary from the first address of the storage space storing the first storage address of the first data to the last address of the storage space storing the first storage address of the first data.

[0260] In another possible example, Fig.13 As shown, Fig.13 Regional variations provided for this application Figure 2 Before partially deleting the first data, the first address of the storage space storing the first storage address of the first data is not the address boundary between the first sub-region and the second sub-region. After partially deleting the first data, the computing device adjusts the existing address boundary to the tail address of the storage space storing the first storage address of the first data.

[0261] In a second possible implementation, after the data d is partially deleted, the address boundary between the first sub-region and the second sub-region does not change, wherein the data d is any data stored in the second sub-region at the first storage address.

[0262] In the present application, when the first data is partially deleted, the computing device adjusts the address boundary of the first sub-region and the second sub-region to facilitate determining the data identification range of the first sub-region and the second sub-region, and updating the corresponding mapping models. This avoids the problem of serious read / write amplification of the computing device caused by the overlap of the data identification range of the first sub-region and the second sub-region, and improves the efficiency of reading and writing addresses. The updated mapping model can accurately locate the address of the first storage address corresponding to the storage data identification in the storage space, improving the efficiency of determining the address.

[0263] In the above two possible implementations, after the address boundary of the first sub-region and the second sub-region is adjusted or the data d is partially deleted, the computing device updates the respective data identification ranges according to the data identification corresponding to the first storage address stored in the first sub-region and the second sub-region respectively. The computing device updates the respective address ranges according to the addresses of the storage spaces storing the first storage address in the first sub-region and the second sub-region respectively. And, the computing device re-fits the data identification corresponding to the stored first storage address according to the address of the storage space storing the first storage address in the second sub-region to update the second mapping model. Alternatively, the computing device re-fits the data identification corresponding to the stored first storage address according to the address of the storage space storing the first storage address in the first sub-region to update the first mapping model.

[0264] In a second possible scenario, if there is free storage space in the second target sub-region, the computing device moves the first storage address stored after the free storage space in the second target sub-region along the storage direction, so that the addresses of the first storage addresses stored in the second target sub-region are continuous. The second target sub-region is any sub-region of the first sub-region and the second sub-region. The computing device fits the address of the storage space in the second target sub-region where the first storage address is stored and the data identifier corresponding to the stored first storage address, and updates the mapping model corresponding to the second target sub-region.

[0265] For example, the computing device stores the first storage address along the low address of the second target sub-area to the high address (such as 0x00000001→0x00000801). If the storage space (0x00000002, 0x00000010) is free, it is necessary to move the first storage address stored after the free storage space (0x00000001, 0x00000002) along the storage direction, that is, move the first storage address stored in the storage space before the free storage space, so that the address of the first storage address stored in the second target sub-area is continuous.

[0266] In a possible example, if all data e are deleted, the computing device will delete the first storage address storing data e in the storage area, so that there is free storage space in the second target sub-area. The data e is any data stored in the second target sub-area at the first storage address.

[0267] The description of the data e can be found in the description of the data c above, which will not be repeated here. Deleting all the data e means that the data to be deleted is the data of the entire block corresponding to the data e, or the data to be deleted is the complete data in a block. For example, if the identifier of the data to be deleted is block10-63M, block20-1M or block21-21M in the above example, it means that the data e is completely deleted.

[0268] When the second target sub-region is the second sub-region, if the address of the free storage space is the left boundary of the second sub-region, the computing device only needs to adjust the lower address boundary of the second sub-region to the tail address of the free storage space.

[0269] If the address of the free storage space is not the left boundary of the second sub-region, the computing device will move the first storage address stored before the free storage space in the second sub-region to make the addresses of the first storage addresses stored in the second sub-region continuous.

[0270] When the second target sub-region is the first sub-region, Fig.14 As shown, Fig.14 Regional variations provided for this application Figure 3 If the address of the free storage space is the left boundary of the first sub-region, the computing device only needs to adjust the left boundary of the second sub-region to the tail address of the free storage space.

[0271] like Fig.15 As shown, Fig.15 Regional variations provided for this application Figure 4 If the address of the free storage space is not the left boundary of the first sub-region, the computing device will move the first storage address stored before the free storage space in the first sub-region to make the addresses of the first storage addresses stored in the second sub-region continuous.

[0272] In a possible scenario, in order to ensure that the first sub-region is continuous with the second sub-region, when the computing device moves the first storage address in the first sub-region, it also needs to move the first storage address in the second sub-region.

[0273] In the present application, when storing the free storage space in the second target sub-region, the first storage address previously stored in the free storage space in the second target sub-region is moved to eliminate the free storage space in the second target sub-region, thereby reducing the read-write amplification of the computing device. In addition, since the storage space in the data sub-region 620 is recycled, after eliminating the free storage space, the utilization rate of the storage space in the data sub-region 620 can be improved.

[0274] In this scenario, after the computing device completes moving the first storage address in the first sub-region or the second sub-region, the computing device updates the address boundary of the first sub-region or the second sub-region according to the address of the storage space storing the first storage address in the first sub-region or the second sub-region, and updates the data identification range of the first sub-region or the second sub-region according to the data identification of the data corresponding to the first storage address stored in the first sub-region or the second sub-region. The computing device performs fitting based on the address of the storage space storing the first storage address in the first sub-region or the second sub-region and the data identification corresponding to the stored first storage address to update the second mapping model corresponding to the first sub-region or the second sub-region. For a description of fitting, please refer to the above Figure 5 The content is not repeated here.

[0275] It is understandable that in order to implement the functions in the above embodiments, the computing device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and method steps of each example described in the embodiments disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

[0276] Combined with the above Figures 4 to 15 , describes in detail the address determination method provided by this embodiment, and will now be combined with Fig.16 , describing the address determination device provided according to this embodiment.

[0277] Fig.16 A schematic diagram of the structure of an address determination device provided in this application Figure 1 The address determination device can be used to implement the functions of the computing device in the above method embodiment, and thus can also achieve the beneficial effects of the above method embodiment. In this embodiment, the address determination device can be a module (such as a chip) of the computing device.

[0278] like Fig.16 As shown, the address determination device 1600 includes an acquisition module 1610, a determination module 1620 and a selection module 1630. The address determination device 1600 is used to implement the above Figures 4 to 15 The functions in the method embodiments shown in .

[0279] The acquisition module 1610 is used to acquire the data identifier of the data to be processed.

[0280] The determination module 1620 is used to determine the region identifier of the first storage area corresponding to the data identifier of the data to be processed by using the interval mapping relationship; wherein the interval mapping relationship is used to indicate the data identifier intervals corresponding to the region identifiers of multiple storage areas, and the second storage area is used to store the first storage address of the data in the data identifier interval corresponding to the second storage area in the disk. The data identifier interval includes multiple continuous data identifiers, and the second storage area is one of the multiple storage areas.

[0281] The selection module 1630 is used to select a second storage address of the first storage address storing the data to be processed from the address range of the first storage area.

[0282] To further achieve the above Figures 4 to 15 The present application also provides an address determination device, such as Fig.17 As shown, Fig.17 A schematic diagram of the structure of an address determination device provided in this application Figure 2 The address determination device 1600 also includes a fitting module 1640 , a first adjustment module 1650 , a second adjustment module 1660 , an address acquisition module 1670 , a data processing module 1680 and a writing module 1690 .

[0283] The fitting module 1640 is used to fit the address of the storage space in the first storage area where the first storage address is stored and the data identifier corresponding to the stored first storage address to obtain a mapping model. Alternatively, the second storage address and the data identifier of the data to be processed are fitted to obtain a mapping model corresponding to the first target sub-area. Alternatively, the address of the storage space in the second target sub-area where the first storage address is stored and the data identifier corresponding to the stored first storage address are fitted to update the mapping model corresponding to the second target sub-area. The second target sub-area is any sub-area of ​​the first sub-area and the second sub-area.

[0284] A first adjustment module 1650 is used to adjust the address boundary of the second sub-region close to the first sub-region to the first address after the first data is partially deleted. The first address is the address of the storage space of the first storage address storing the first data in the first storage region. The first data is any data stored in the first sub-region at the first storage address, and the first sub-region is the sub-region of the first storage address storing the newly written data;

[0285] The second adjustment module 1660 is configured to move the first storage address previously stored in the free storage space in the second target sub-region if there is free storage space in the second target sub-region, so that the addresses of the first storage addresses stored in the second target sub-region are continuous. The second target sub-region is any one of the first sub-region and the second sub-region.

[0286] The address acquisition module 1670 is used to acquire the first storage address of the to-be-processed data stored at the second storage address.

[0287] The data processing module 1680 is used to update, delete or read the data to be processed stored in the storage space indicated by the first storage address of the data to be processed.

[0288] The writing module 1690 is used to write the to-be-processed data at the first storage address in the disk into the storage space indicated by the second storage address.

[0289] It should be understood that the computing device according to the embodiment of the present application may correspond to the address determination device 1600 in the embodiment of the application, and may correspond to executing the method according to the embodiment of the present application. Figures 4 to 15 The corresponding corresponding subjects, and the operations and / or functions of each module in the address determination device 1600 are respectively to achieve Figures 4 to 15 For the sake of brevity, the corresponding processes of each method in the corresponding embodiments are not repeated here.

[0290] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instruction is loaded and executed on a computer, the process or function described in the embodiment of the present application is executed in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable device. The computer program or instruction may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer program or instruction may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium, for example, a floppy disk, a hard disk, a tape; it may also be an optical medium, for example, a digital video disc (DVD); it may also be a semiconductor medium, for example, a solid state drive (SSD).

[0291] In each embodiment of the present application, if there is no special explanation and logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced to each other, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships. The various digital numbers involved in the embodiments of the present application are only used to distinguish for the convenience of description and are not used to limit the scope of the embodiments of the present application. The size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and inherent logic.

Claims

1. A method for determining an address, characterized in that: The method comprises: Obtaining the data identifier of the data to be processed; Determine the region identifier of the first storage region corresponding to the data identifier of the data to be processed by using the interval mapping relationship; The interval mapping relationship is used to indicate the data identification intervals corresponding to the area identifications of the plurality of storage areas respectively, and the second storage area is used to store the first storage address of the data in the data identification interval corresponding to the second storage area in the disk; the data identification interval includes a plurality of continuous data identifications; and the second storage area is one of the plurality of storage areas; A second storage address of the first storage address storing the data to be processed is selected from the address range of the first storage area.

2. The method according to claim 1, characterized in that The selecting, from the address range of the first storage area, a second storage address of the first storage address storing the data to be processed comprises: Substituting the data identifier of the data to be processed into the mapping model to obtain the second storage address; Among them, the mapping model is used to indicate the mapping relationship between the data identifier and the address in the storage area; the storage space indicated by the address in the storage area is used to store the data indicated by the data identifier that has the mapping relationship with the address in the storage area, at the first storage address in the disk.

3. The method according to claim 2, characterized in that The mapping model is a mapping model corresponding to the first storage area.

4. The method according to claim 2 or 3, characterized in that: Before selecting the second storage address of the first storage address storing the data to be processed from the address range of the first storage area, the method further includes: The mapping model is obtained by fitting the address of the storage space in the first storage area where the first storage address has been stored and the data identifier corresponding to the stored first storage address.

5. The method according to any one of claims 2 to 4, characterized in that The storage area includes a first sub-area and a second sub-area, and the addresses of the first sub-area and the second sub-area are continuous; The selecting, from the address range of the first storage area, a second storage address of the first storage address storing the data to be processed comprises: In the address range of the first target sub-region, a second storage address of the first storage address storing the data to be processed is selected; Among them, the first target sub-region is a sub-region corresponding to the data identification range where the data identification of the data to be processed is located in the first sub-region and the second sub-region, and the data identification range of a sub-region is a range composed of data identifications of data stored in the sub-region at a first storage address; or, the data to be processed is newly written data, and the first target sub-region is a sub-region of the first storage address storing the newly written data.

6. The method according to claim 5, characterized in that The selecting, in the address range of the first target sub-region, a second storage address of the first storage address storing the data to be processed comprises: Substitute the data identifier of the data to be processed into the mapping model of the first target sub-region to obtain the second storage address; the mapping model of the first target sub-region is used for the mapping relationship between the data identifier in the data identifier range of the first target sub-region and the address in the first target sub-region.

7. The method according to claim 5, characterized in that If there is no mapping model corresponding to the first target sub-region, selecting a second storage address of the first storage address storing the data to be processed from the address range of the first target sub-region includes: Using the address of the first storage space in the first target sub-area as the second storage address; The method further includes: fitting the second storage address and the data identifier of the data to be processed to obtain a mapping model corresponding to the first target sub-region.

8. The method according to any one of claims 5 to 7, characterized in that The first sub-region is a sub-region of a first storage address storing newly written data; The method also includes: when the first data is partially deleted, adjusting the address boundary of the second sub-region close to the first sub-region to the first address; the first address is the address of the storage space in the first storage region that stores the first storage address of the first data; the first data is any data stored in the first sub-region at the first storage address.

9. The method according to any one of claims 5 to 7, characterized in that The method further comprises: If there is free storage space in the second target sub-region, then move the first storage address previously stored in the free storage space in the second target sub-region so that the addresses of the first storage addresses stored in the second target sub-region are continuous; the second target sub-region is any sub-region; The address of the storage space in the second target sub-region where the first storage address has been stored is fitted with the data identifier corresponding to the stored first storage address, and the mapping model corresponding to the second target sub-region is updated.

10. The method according to any one of claims 1 to 9, characterized in that The method further comprises: Obtaining a first storage address of the to-be-processed data stored at the second storage address; The data to be processed stored in the storage space indicated by the first storage address of the data to be processed is updated, deleted or read.

11. The method according to any one of claims 1 to 9, characterized in that The method further comprises: The to-be-processed data is written into the storage space indicated by the second storage address at the first storage address in the disk.

12. An address determination device, characterized in that: The device comprises: An acquisition module, used to acquire data identifiers of data to be processed; A determination module, used to determine the region identifier of the first storage area corresponding to the data identifier of the data to be processed by using the interval mapping relationship; wherein the interval mapping relationship is used to indicate the data identifier intervals corresponding to the region identifiers of multiple storage areas, and the second storage area is used to store the first storage address of the data in the data identifier interval corresponding to the second storage area in the disk; the data identifier interval includes multiple continuous data identifiers; and the second storage area is one of the multiple storage areas; A selection module is used to select a second storage address of the first storage address storing the data to be processed from the address range of the first storage area.

13. The device according to claim 12, characterized in that The selection module is specifically used to substitute the data identifier of the data to be processed into the mapping model to obtain the second storage address; Among them, the mapping model is used to indicate the mapping relationship between the data identifier and the address in the storage area; the storage space indicated by the address in the storage area is used to store the data indicated by the data identifier that has the mapping relationship with the address in the storage area, at the first storage address in the disk.

14. The device according to claim 13, characterized in that The mapping model is a mapping model corresponding to the first storage area.

15. The device according to claim 13 or 14, characterized in that The device also includes a fitting module; The fitting module is used to fit the address of the storage space in the first storage area where the first storage address has been stored and the data identifier corresponding to the stored first storage address to obtain the mapping model.

16. The device according to any one of claims 13 to 15, characterized in that The storage area includes a first sub-area and a second sub-area, and the addresses of the first sub-area and the second sub-area are continuous; the selection module is specifically used to select the second storage address of the first storage address storing the data to be processed in the address range of the first target sub-area; Among them, the first target sub-region is a sub-region corresponding to the data identification range where the data identification of the data to be processed is located in the first sub-region and the second sub-region, and the data identification range of a sub-region is a range composed of data identifications of data stored in the sub-region at a first storage address; or, the data to be processed is newly written data, and the first target sub-region is a sub-region of the first storage address storing the newly written data.

17. The device according to claim 16, characterized in that The selection module is specifically used to substitute the data identifier of the data to be processed into the mapping model of the first target sub-region to obtain the second storage address; the mapping model of the first target sub-region is used for the mapping relationship between the data identifier in the data identifier range of the first target sub-region and the address in the first target sub-region.

18. The device according to claim 16, characterized in that If there is no mapping model corresponding to the first target sub-region, the device further includes a fitting module; The selection module is specifically configured to use the address of the first storage space in the first target sub-area as the second storage address; The fitting module is used to fit the second storage address and the data identifier of the data to be processed to obtain a mapping model corresponding to the first target sub-region.

19. The device according to any one of claims 16 to 18, characterized in that The first sub-region is a sub-region of a first storage address storing newly written data; the device also includes a first adjustment module; The first adjustment module is used to adjust the address boundary of the second sub-region close to the first sub-region to the first address after the first data is partially deleted; the first address is the address of the storage space in the first storage region that stores the first storage address of the first data; the first data is any data stored in the first sub-region at the first storage address.

20. The device according to any one of claims 16 to 18, characterized in that The device also includes a second adjustment module and a fitting module; The second adjustment module is used for moving the first storage address previously stored in the free storage space in the second target sub-region if there is free storage space in the second target sub-region, so that the addresses of the first storage addresses stored in the second target sub-region are continuous; the second target sub-region is any sub-region; The fitting module is used to fit the address of the storage space storing the first storage address in the second target sub-region and the data identifier corresponding to the stored first storage address, and update the mapping model corresponding to the second target sub-region.

21. The device according to any one of claims 12 to 20, characterized in that The device also includes an address acquisition module and a data processing module; The address acquisition module is used to acquire the first storage address of the to-be-processed data stored at the second storage address; The data processing module is used to update, delete or read the data to be processed stored in the storage space indicated by the first storage address of the data to be processed.

22. The device according to any one of claims 12 to 20, characterized in that The device also includes a writing module; The writing module is used to write the to-be-processed data at the first storage address in the disk into the storage space indicated by the second storage address.

23. A computing device, characterized in that The method comprises a memory and a processor, wherein the memory is used to store computer instructions; when the processor executes the computer instructions, the method according to any one of claims 1 to 11 is implemented.

24. A computer-readable storage medium, characterized in that: The method comprises computer program instructions, which, when executed by a computing device, perform the method according to any one of claims 1 to 11.

Images