Object data processing method and device, storage medium and electronic equipment

Abstract

The invention discloses an object data processing method and device, a storage medium and electronic equipment, and relates to the technical field of computers. Under the condition that write-in operation is completed based on the object data, a data storage address and a transition layer address distributed for the object data are obtained, the data storage address is used for indicating the write-in position of the object data, and the transition layer address determines the mapping relation between the logic unit address of the object data and the transition layer address; adding a first mapping relation between the logic unit address and the transition layer address in the first mapping relation set; and in the second mapping relation set, a second mapping relation between the transition layer address and the data storage address is added, so that the mapping relation between the data storage address and the logic unit address is effectively dissociated based on the transition layer address, and the problem of low efficiency of existing data operation is solved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to a method and apparatus for processing object data, a storage medium, and an electronic device. Background Art

[0002] A centralized storage device provides block storage space for a server. To improve storage efficiency, many new functional features have been added to existing centralized storage devices, such as garbage collection functions, etc. In addition, to reduce the wear of flash media and extend the life of the storage medium, most centralized storage devices also support the deduplication function.

[0003] During the process of a centralized storage device performing data operations according to data operation instructions of a server, it usually depends on the mapping relationship between the logical unit address provided by the server and the data storage address on the storage device to perform related operations. In a centralized storage device, based on its provided related storage optimization functions, duplicate data blocks in the storage will be identified and eliminated, resulting in a very complex mapping relationship between the logical unit address and the data storage address, and thus significantly low efficiency in the process of performing related data operations based on the above mapping relationship in the centralized storage device.

[0004] In view of the above problems, no effective solution has been proposed yet. Summary of the Invention

[0005] Embodiments of this application provide a method and apparatus for processing object data, a storage medium, and an electronic device, so as to at least solve the technical problem of significantly low efficiency in the process of related data operations.

[0006] This application provides a method for processing object data, including:

[0007] Receiving a data write request, where the data write request carries a logical unit address and object data to be written;

[0008] When the write operation based on the object data is completed, obtaining the data storage address and the transition layer address allocated for the object data, where the data storage address is used to indicate the write position of the object data, and the transition layer address determines the mapping relationship between the logical unit address of the object data and the transition layer address;

[0009] Adding a first mapping relationship between the logical unit address and the transition layer address to a first mapping relationship set;

[0010] Adding a second mapping relationship between the transition layer address and the data storage address to a second mapping relationship set.

[0011] The present application also provides a processing device for object data, including:

[0012] A receiving unit, configured to receive a data writing request, where the data writing request carries a logical unit address and object data to be written;

[0013] An obtaining unit, configured to obtain a data storage address and a transition layer address allocated for the object data when a writing operation is completed based on the object data, where the data storage address is used to indicate the writing position of the object data, and the transition layer address determines the mapping relationship between the logical unit address and the transition layer address of the object data;

[0014] A first adding unit, configured to add a first mapping relationship between the logical unit address and the transition layer address in a first mapping relationship set;

[0015] A second adding unit, configured to add a second mapping relationship between the transition layer address and the data storage address in a second mapping relationship set.

[0016] The present application also provides an electronic device, including: a memory, configured to store a computer program; a processor, configured to implement the steps of any of the above object data processing methods when executing the computer program.

[0017] The present application also provides a computer-readable storage medium, in which a computer program is stored, where the computer program implements the steps of any of the above object data processing methods when executed by a processor.

[0018] The present application also provides a computer program product, including a computer program, where the computer program implements the steps of any of the above object data processing methods when executed by a processor.

[0019] In the technical solution of the present application, by receiving a data writing request and obtaining the data storage address and the transition layer address allocated for the object data after the writing operation based on the object data is completed, an efficient data management method is implemented. Among them, the data storage address is used to indicate the writing position of the object data, and the transition layer address determines the mapping relationship between the logical unit address and the transition layer address of the object data. This method realizes the effective dissociation of the mapping relationship between the data storage address and the logical unit address based on the transition layer address by adding the first mapping relationship between the logical unit address and the transition layer address in the first mapping relationship set and adding the second mapping relationship between the transition layer address and the data storage address in the second mapping relationship set.

[0020] This strategy of dissociation mapping relationship significantly improves the flexibility and efficiency of data operations compared to the method of directly mapping the logical unit address to the data storage address in the prior art. By introducing the intermediate layer address, the system can flexibly manage and adjust the mapping relationship among different sets of mapping relationships, thereby optimizing the data access path, reducing unnecessary data movement and copying operations, and thus solving the technical problem of significantly low efficiency in the process of related data operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] To more clearly illustrate the embodiments of the present application, the accompanying drawings required for use in the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0022] Figure 1 is a hardware structure block diagram of a server device for a method of processing object data according to an embodiment of the present application;

[0023] Figure 2 is a flowchart of a method of processing object data according to an embodiment of the present application;

[0024] Figure 3 is a schematic diagram of a method of processing object data according to an embodiment of the present application;

[0025] Figure 4 is a flowchart of another method of processing object data according to an embodiment of the present application;

[0026] Figure 5 is a schematic structural diagram of a device for processing object data according to an embodiment of the present application;

[0027] Figure 6 is a schematic structural diagram of an electronic device for processing object data according to an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some, rather than all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0029] It should be noted that in the description of this application, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. The terms "first", "second", etc. in this application are used to distinguish similar objects and not to describe a specific order or sequence.

[0030] To enable those skilled in the art of this technology to better understand the solution of this application, the following further details this application in conjunction with the accompanying drawings and specific embodiments.

[0031] The method embodiments provided in the embodiments of this application can be executed in a server device or a similar computing device. Taking the operation on a server device as an example, Figure 1 is a hardware structure block diagram of a server device for a method of processing object data in an embodiment of this application. As Figure 1 shown, the server device may include one or more ( Figure 1 only one is shown in the figure) processors 102 (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Among them, the above server device may further include a transmission device 106 for communication functions and an input / output device 108. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only schematic and does not limit the structure of the above server device. For example, the server device may further include more or fewer components than those Figure 1 shown in the figure, or have a different configuration from that Figure 1 shown.

[0032] The memory 104 can be used to store computer programs. For example, software programs and modules of application software, such as the computer program corresponding to the data processing method of the memory in the embodiments of this application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implements the above method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely disposed relative to the processor 102, and these remote memories can be connected to the server device through a network. Examples of the above network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0033] The transmission device 106 is used to receive or send data via a network. Specific examples of the above-mentioned network may include a wireless network provided by a communication provider of a server device. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, abbreviated as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a Radio Frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0034] It should be noted that with the rapid development of flash memory technology, many new functional features have been added to existing storage devices, such as garbage collection functions, etc. In addition, in order to reduce the wear on the flash memory medium and extend the life of the storage medium, most storage devices also support the deduplication function (Deduplication, duplicate data deletion).

[0035] In these storage devices, the deduplication function and the garbage collection function run inside the storage device and are transparent to the server using the storage device. From the perspective of the server, the storage device provides a storage volume of a specified size. From the perspective of the storage device, when the server uses the volume storage space, it will use data blocks of a specific size as the unit and specify the starting address of the expected write or read to make corresponding access requests.

[0036] For storage devices that support the deduplication function, the address requested by the server cannot be directly mapped to the address where the actual data is stored. This is because when storing data, if there are multiple data blocks with the same content, there is no need to repeatedly store the same data on multiple media, but only one data block needs to be saved, thus saving a large amount of storage space.

[0037] Since there is a one-to-many relationship between the logical unit address presented by the storage device that supports the deduplication function and the actual data storage address, it is necessary to maintain a mapping relationship table from the logical unit address to the data storage address in the storage device. At the same time, since the garbage collection function will periodically clean up invalid data blocks in the background to free up flash memory space, this may cause the data storage address to change, and it is necessary to modify the corresponding relationship between the logical unit address and the data storage address and the corresponding relationship between the fingerprint value and the data storage address.

[0038] Therefore, in a storage device with deduplication and garbage collection functions, the conventional metadata management method requires multiple corresponding relationship tables, making metadata management complex and occupying a relatively large amount of storage space. After the garbage collection function moves data and changes the data storage address, it involves modifying multiple records in multiple corresponding relationship tables, and it is necessary to ensure that all these corresponding relationship tables are correctly modified before other related requests can be made, which has a greater impact on performance. In addition, there is a one-to-many relationship between the corresponding relationship of the data storage address to the logical unit address, and this corresponding relationship makes the data structure and the modification of the value of the key-value very complex in the software design of key-value. In other words, there are technical problems of low efficiency in the existing data operation methods.

[0039] To solve the above technical problems, the present application provides an alternative implementation manner, as Figure 2 shown, the above method for processing object data can be applied to a storage device deployed at a storage service site, including:

[0040] S202, receiving a data write request, where the data write request carries a logical unit address and object data to be written;

[0041] S204, in the case where the write operation is completed based on the object data, obtaining the data storage address and the transition layer address allocated for the object data, where the data storage address is used to indicate the write position of the object data, and the transition layer address determines the mapping relationship between the logical unit address and the transition layer address of the object data;

[0042] S206, adding a first mapping relationship between the logical unit address and the transition layer address in the first mapping relationship set;

[0043] S208, adding a second mapping relationship between the transition layer address and the data storage address in the second mapping relationship set.

[0044] In the above implementation manner of the present application, the data write request may be an operation request sent by a client or a server, and is used to request to write object data into the current storage device.

[0045] It can be understood that the data write request in the above step S202 may carry a logical unit address and object data to be written. Among them, the above logical unit address is a logical address used to identify the storage device in the system of the above client or server, so as to distinguish different storage volumes or partitions. The above object data may be a data entity requested by the client or the server to be written into the storage system, and may be a file, a part of a file, or other types of data blocks.

[0046] Further, in the above step S204, when the object data completes the write operation on the current storage device, the data storage address and the transition layer address corresponding to the object data can be obtained. The data storage address is used to indicate the actual position of the written object data on the current storage device; the transition layer address is a configured intermediate address used to establish a mapping relationship between the logical unit address and the data storage address. Further, the above transition layer address can also be used to manage the storage and retrieval of data, as well as the deduplication function or the garbage collection function.

[0047] It can be understood that in the embodiment of the present application, based on the above logical unit address, transition layer address, and data storage address, the relationship can be established through the first mapping relationship set and the second mapping relationship set. The first mapping relationship set includes the mapping relationships between multiple logical unit addresses and multiple transition layer addresses. The mapping relationship between the logical unit address and the transition layer address can be used to convert the high-level logical address into the intermediate transition layer address; the second mapping relationship set includes the mapping relationships between multiple transition layer addresses and multiple data storage addresses. The mapping relationship between the transition layer address and the data storage address can be used to convert the transition layer address into the actual data storage address.

[0048] It should be noted that the above mapping relationship can be managed in the form of key-value pairs. The specific data structure management form can be one or more of a skip list, a hash table, a B+ tree, etc.

[0049] After the above data write operation is completed, according to the write situation of the current object data, the corresponding mapping relationships can be added to the first mapping relationship set and the second mapping relationship set respectively, so as to realize the management of the relevant address information involved in the current write operation. Furthermore, the complex mapping relationship between the logical unit address and the data storage address is dissociated through the transition layer address, improving the management efficiency of the mapping relationship, and further improving the operation efficiency of performing relevant data operations based on the mapping relationship.

[0050] It should be noted that in the process of providing the deduplication function by a conventional storage device, it is necessary to read the existing data from the storage medium of the storage device and compare each data block with the data block to be written in this request one by one. As the scale of the data storage volume in the storage device increases, the time-consuming of this comparison and search method is very high, and there are significant technical problems of low efficiency.

[0051] To solve the above technical problems, the storage device in the embodiment of the present application can also solve the above technical problems according to the data fingerprint information and the mapping relationship related to the data fingerprint information.

[0052] In an alternative embodiment, after receiving the data write request in step S202 above, it further includes:

[0053] S1. Determine the data fingerprint information that matches the object data;

[0054] S2. When the object data is successfully written, add a third mapping relationship between the data fingerprint information and the transition layer address in the third mapping relationship set.

[0055] In the above embodiment, the data fingerprint information can be a representation of a data feature, usually generated by a hash function, and is used to uniquely identify the data content. It can be understood that in the embodiments of the present application, the data fingerprint information can be used to quickly compare whether the data is the same, thereby improving the efficiency of the deduplication operation.

[0056] The above third mapping relationship set can be used to store the mapping relationships between multiple data fingerprint information and multiple transition layer addresses. In this embodiment, when the object data write operation is completed, the above third mapping relationship can be added to the above third mapping relationship set, and then used for subsequent data management and data retrieval operations.

[0057] Through the above embodiment of the present application, when the object data is successfully written into the storage system, the system will add a new third mapping relationship to the third mapping relationship set. The third mapping relationship associates the data fingerprint information with the transition layer address, thereby establishing an index in the storage system, so that the corresponding transition layer address can be quickly found through the data fingerprint, and then the actually stored data can be accessed.

[0058] The following further describes the data write method based on the above embodiment.

[0059] In an alternative embodiment, before obtaining the data storage address and the transition layer address allocated for the object data when the object data write operation is completed, it further includes:

[0060] S1. Determine multiple sub-object data according to the object data, where the data volume of the sub-object data is a preset size;

[0061] S2. Determine the data fingerprint values corresponding to the multiple sub-object data respectively, where the data fingerprint information that matches the object data includes the data fingerprint values corresponding to the multiple sub-object data respectively;

[0062] S3. Write the data fingerprint values corresponding to the sub-object data respectively in multiple consecutive sub-units in the first storage unit;

[0063] S4. Write multiple sub-object data respectively in multiple consecutive storage units associated with the first storage unit.

[0064] In the implementation of the present application, the minimum operation granularity requested by the host can be pre-set, and the storage device will follow this setting when calculating the fingerprint value. Specifically, each data unit of the minimum operation granularity will generate a corresponding fingerprint value. These fingerprint values will be combined to determine the overall fingerprint information of the object data to be written. When performing the deduplication function, that is, when determining whether the contents of two data blocks are the same, this granularity will also be used for comparison. Common minimum operation granularities include 8KB, 16KB, 32KB, etc.

[0065] During the data writing process, these fingerprint values can be stored together with the object data. In a preferred embodiment, to achieve data block alignment, a minimum data width for disk writes can be defined at the underlying storage device. Space for storing fingerprint values is reserved within this width, while the remaining portion of the width is used to store data at multiple minimum operation granularities.

[0066] In another alternative embodiment, if the current storage device is configured with a cache function, when there are few I / O requests and the minimum width of the data to be flushed to disk is not reached, a waiting period of, for example, 0.1 milliseconds, can be first maintained in the cache. If the minimum width of the data to be flushed to disk is still not reached within this period, zero padding can be used to fill the gap to ensure that the underlying data is saved.

[0067] In the above embodiment, a metadata management method is provided, and a three-layer address management scheme is designed to avoid the one-to-many management structure of the correspondence between data storage addresses and logical unit addresses, and reduce the types of relationship mapping tables. When the garbage collection function changes the data storage address, the operation steps also become simpler, thereby improving the processing performance of the storage device.

[0068] The following combination Figure 3 The above data writing operation and the above mapping relationship set are described in detail. Figure 3 FIG. 4 is a schematic diagram of metadata management within a current storage device.

[0069] like Figure 3 , the mapping relationship between the logical unit address, the transition layer address and the data storage address is shown.

[0070] in, Figure 3"Logical Unit Address 1", "Logical Unit Address 2", and "Logical Unit Address 3" shown in [figure] represent the logical addresses in the storage system, which are used by the server when sending data write requests. The intermediate layer addresses X, Y, and Z are the intermediate addresses between the logical unit addresses and the data storage addresses, and are used to manage and locate data in the storage system. The data storage addresses X2 and Y2 are the actual positions of the data on the physical storage medium. These addresses indicate the specific locations where the data is written.

[0071] Furthermore, Figure 3 The "First Mapping Relationship Set" on the left in [figure] describes the correspondence between the logical unit addresses and the intermediate layer addresses. Each logical unit address maps to a specific intermediate layer address (e.g., Logical Unit Address 1 maps to Intermediate Layer Address X, Logical Unit Address 2 maps to Intermediate Layer Address Y, and Logical Unit Address 3 maps to Intermediate Layer Address Z).

[0072] Figure 3 The "Second Mapping Relationship Set" in [figure] describes the correspondence between the intermediate layer addresses and the data storage addresses. For example, Intermediate Layer Address X maps to Data Storage Address X2, and Intermediate Layer Address Y maps to Data Storage Address Y2.

[0073] Figure 3 The "Third Mapping Relationship Set" on the right in [figure] describes the correspondence between the fingerprint values and the intermediate layer addresses. The fingerprint values are used to uniquely identify the data content and help achieve data deduplication.

[0074] During the write operation, the data fingerprint information can be written together with the object data. Figure 3 The fingerprint values Hx, He, and Hj shown in [figure] have a corresponding relationship with data blocks X, E, and J, that is, the fingerprint value of data block X can be fingerprint value Hx.

[0075] In Figure 3 In [figure], in order to facilitate data block alignment, the minimum disk write data width at the bottom layer of the storage device can be defined, and space for storing fingerprint values is reserved in the disk write data width. If the I / O requests are few and do not meet the minimum disk write data width, it can wait in the cache for a certain period of time, or fill it with zeros to reach the minimum disk write data width to complete the bottom layer data storage operation.

[0076] Through Figure 3 the mapping relationships and data management strategies shown in [figure], the storage system can effectively manage the writing and storage of data. Especially when using advanced storage functions (such as deduplication and compression), through Figure 3 the mapping relationship management method in [figure], the storage space can be optimized and the data access efficiency can be improved.

[0077] It should be noted that in the embodiment of the present application, in order to further improve the data operation efficiency, a reference count is further configured for the mapping relationship between the transition layer address and the data storage address, which is used to indicate the number of logical unit addresses mapped to the same transition layer address.

[0078] In an alternative embodiment, after receiving the data write request, the following steps are further included:

[0079] S1. Obtain data fingerprint information that matches the object data;

[0080] S2-1. When the third mapping relationship set includes a fourth mapping relationship that matches the data fingerprint information, obtain a reference transition layer address according to the fourth mapping relationship; add a fifth mapping relationship between the logical unit address and the reference transition layer address in the first mapping relationship set; according to the reference transition layer address, search for a sixth mapping relationship in the second mapping relationship set, where the sixth mapping relationship is the mapping relationship between the reference transition layer address and the reference storage address; increase the reference count that matches the sixth mapping relationship from a first value to a second value, where the reference count is used to indicate the number of logical mapping relationships associated with the reference transition layer address;

[0081] S2-2. When the third mapping relationship set does not include a mapping relationship that matches the data fingerprint information, determine to perform a write operation based on the object data.

[0082] The following combines Figure 4 to illustrate a complete data write process.

[0083] As Figure 4 In, first execute S402, and the server requests new data to be written.

[0084] Then execute S406 to calculate the fingerprint value of the newly written data. The method of calculating the fingerprint value in the current step can be implemented in the same way as the above embodiment.

[0085] Then execute S408: Query the transition layer address based on the fingerprint value. It can be understood that querying the transition layer address based on the fingerprint value can specifically be performed in the above third mapping relationship set.

[0086] When a record is found, execute S410-1 to S410-3, and use the transition layer address in the record (i.e., the fourth mapping relationship); S410-2: Add a record of the logical unit address to the transition layer address (i.e., the fifth mapping relationship), and update the reference count (i.e., the reference count corresponding to the sixth mapping relationship); complete the write request operation.

[0087] When no record is found, perform S412-1 to S412-8, save the data and fingerprint information; wait for the return of the data storage address; allocate an idle transition layer address; add a record from the transition layer address to the data storage address (i.e., the second mapping relationship), and set the reference count to 1; add a record from the fingerprint value to the transition layer address (i.e., the third mapping relationship); add a record from the logical unit address to the transition layer address (i.e., the first mapping relationship); and complete the write request operation.

[0088] Through the above embodiments of the present application, the optimization of the data write operation in the storage system is achieved. For example, the storage efficiency can be improved and data redundancy can be reduced by using data fingerprint values. During the data write process, by calculating the fingerprint value of the newly written data and querying whether there is a record of the same fingerprint value in the "corresponding relationship between the fingerprint value and the transition layer address", data deduplication can be achieved. If the same fingerprint value is found, it means that the data already exists in the storage system and there is no need to write it again, thus saving storage space. For duplicate data, the system does not repeatedly store the same data block on the physical storage medium, but refers to the existing data block through the mapping relationship. This method reduces the wear of the storage medium and improves the efficiency of the storage system. In addition, when duplicate written data is queried, the system only needs to add a record in the "corresponding relationship between the logical unit address and the transition layer address" to complete the write request operation. This operation is much faster than actually writing data to the storage medium, thus improving the performance of the write operation.

[0089] Furthermore, in the above embodiments of the present application, through the reference count, when adding a record in the "corresponding relationship between the transition layer address and the data storage address", the reference count is set to 1, which helps to track the usage of data blocks and provides a basis for subsequent data management (such as garbage collection).

[0090] In an alternative embodiment, when the server data operation request received is a data read request, such as Figure 3 in, the logical unit address specified by the server can be obtained from the data read request. The current storage device first queries the transition layer address from the first mapping relationship set, that is, the "corresponding relationship between the logical unit address and the transition layer address", using the logical unit address as the key, and then queries the data storage address from the second mapping relationship set, "the corresponding relationship between the transition layer address and the data storage address", using the transition layer address as the key. Finally, the data is obtained from the data storage address to complete the read request operation.

[0091] The following describes the garbage collection operation performed based on the above three-layer address structure. In an alternative embodiment, after adding the second mapping relationship between the transition layer address and the data storage address in the second mapping relationship set, it further includes:

[0092] S1. In response to a storage space recycling instruction, determine a first storage address of the current object data to be operated on, and a second storage address after movement.

[0093] S2. According to the first storage address, search for a seventh mapping relationship in the second mapping relationship set, where the seventh mapping relationship is the mapping relationship between the first storage address and the first intermediate layer address.

[0094] S3. According to the second storage address, update the seventh mapping relationship to an eighth mapping relationship, where the eighth mapping relationship is the mapping relationship between the second storage address and the first intermediate layer address.

[0095] In an embodiment of the present application, a garbage collection method based on a three-layer address scheme is provided. By adding an intermediate layer address, metadata management is optimized, thereby improving the efficiency of garbage collection and the overall performance of the storage system.

[0096] First, when a storage space recycling instruction is received, garbage collection is triggered. The garbage collection function in the current storage device can periodically clean invalid data blocks in the background to release flash memory space. Due to the characteristic of flash memory that the minimum granularity of a flash memory erasable block cleared and recycled at one time is relatively large, much larger than the minimum granularity that can be operated on during a single read / write operation, garbage collection is required to reduce the idleness of flash memory space and quickly recycle flash memory space. To reduce the idleness of flash memory space and quickly recycle flash memory space, usually, part of the valid data in multiple erasable block granularities is integrated, merged and written into a new block, and then these multiple erasable blocks are emptied and recycled. After such an operation, the original data storage address has changed.

[0097] In the above embodiment of the present application, by adding an intermediate layer address, when the background garbage collection function moves data, only the mapping relationship between the intermediate layer address and the data storage address needs to be updated and adjusted, and the logical unit address mapping relationship (i.e., the mapping relationship between the logical unit address and the intermediate layer address) and the fingerprint value mapping relationship (i.e., the mapping relationship between the fingerprint value and the intermediate layer address) do not need to be modified anymore. Thus, the use of the reverse one-to-many relationship mapping table from the data storage address to the logical unit address is avoided, thereby simplifying the metadata management in the garbage collection process.

[0098] In an optional embodiment, the data deletion operation performed on the above three-layer address management scheme can also significantly improve the operation efficiency. Specifically, after the reference count matching the sixth mapping relationship is increased from a first value to a second value, it further includes:

[0099] S1. In response to a data deletion request, according to the first logical unit address carried in the data deletion request, search for a ninth mapping relationship matching the data deletion request in the first mapping set.

[0100] S2. When the second transition layer address indicated by the ninth mapping relationship matches the reference transition layer address, search for the sixth mapping relationship in the second mapping set according to the reference transition layer address;

[0101] S3. Reduce the reference count that matches the sixth mapping relationship from the second value to the third value;

[0102] S4. When the third value is less than or equal to the target threshold, label the data object corresponding to the reference storage address as a deletable data object according to the sixth mapping relationship.

[0103] Further, after the above-mentioned step of labeling the data object corresponding to the reference storage address as a deletable data object when the third value is less than or equal to the target threshold, it further includes:

[0104] S1. Obtain the data fingerprint information that matches the deletable data object according to the reference storage address;

[0105] S2. Search for the fourth mapping relationship in the third mapping relationship set according to the data fingerprint information;

[0106] S3. Search for the fifth mapping relationship in the first mapping relationship set according to the reference transition layer address included in the fourth mapping relationship;

[0107] S4. Delete the fourth mapping relationship in the third mapping relationship set and delete the fifth mapping relationship in the first mapping relationship set.

[0108] It can be understood that in the above implementation manner, when the server specifies the logical unit address to initiate a block data deletion request to the storage device, the storage device can use the logical unit address as the key to search for the "corresponding relationship between the logical unit address and the transition layer address" and delete one record found, and use the transition layer address in this record as the key to search for the "corresponding relationship between the transition layer address and the data storage address". After finding it, first reduce the reference count of this record by 1; if the reference count is still greater than zero, the current data deletion request operation is completed; if the reference count is zero, this record is deleted, and the data block associated with the data storage address is marked as invalid data waiting for the garbage collection function to process. Read the fingerprint value corresponding to the data from the disk data width where the data is located as the key to search for the "corresponding relationship between the fingerprint value and the transition layer address" and delete one record found, and notify the garbage collection function module to complete the current data deletion request operation.

[0109] Through the above-described embodiments of the present application, the use of fingerprint values and data together with the storage reduces the use of mapping tables such as data storage addresses to fingerprint values. By reducing the use of relationship mapping tables, the consumption of metadata storage space is reduced. By adding transition layer addresses, the background garbage collection function is prevented from excessive reverse lookups and modifications to mapping relationships, thereby reducing conflicts caused by concurrent operations of background programs on foreground I / O requests.

[0110] In one optional implementation, when a server initiates a modification write request to a storage device, the storage device first deletes the old data before executing the write request for the new data. This process appears as a single request to the server externally, but is internally split into two separate requests. Specifically, the old data deletion operation may include the following steps:

[0111] First, the storage device finds the corresponding old data storage address according to the logical unit address, and finds the corresponding transition layer address in the "correspondence relationship from the logical unit address to the transition layer address".

[0112] Next, the transition layer address is used to find the old data storage address in the "correspondence between transition layer address and data storage address", and a deletion operation is performed.

[0113] The new data writing operation may include the following steps:

[0114] The storage device calculates the fingerprint value of the newly written data and uses the fingerprint value as the key to search in the "correspondence between fingerprint value and transition layer address".

[0115] If a record is found, it means the data already exists. Directly use the transition layer address in the record to add a new record in the "Correspondence between Logical Unit Address and Transition Layer Address".

[0116] If no record is found, the new data and fingerprint value are handed over to the storage device for storage, and a new data storage address is returned.

[0117] Then, an idle transition layer address is allocated in the transition layer address management module, and a new record is added to the "correspondence between the transition layer address and the data storage address", and the reference count is set to one.

[0118] Finally, a new record is added to the "correspondence between fingerprint value and transition layer address" and the "correspondence between logical unit address and transition layer address" to complete the write request operation.

[0119] With this three - layer address scheme, the use of the mapping table from the data storage address to the fingerprint value can be reduced, thereby reducing the consumption of metadata for storage space. At the same time, by adding the transitional layer address, it is possible to avoid frequent modification of the logical unit address mapping table and the fingerprint value mapping table during data movement by the background garbage collection function, reduce the concurrent operation conflicts of the background program with the foreground I / O requests, and improve the stability and efficiency of the system.

[0120] Through the above - mentioned implementation manners of the present application, the implementation manners of the present application utilize a three - layer address scheme, reduce the types of mapping relation tables, save the storage space occupied by metadata management, and solve the problems of complex software design and difficult operation caused by the one - to - many relationship in the "corresponding relationship between the data storage address and the logical unit address" in traditional metadata management. Compared with the traditional metadata management, when the background garbage collection function moves data, fewer mapping relation tables need to be modified, thereby reducing the complexity of system processing.

[0121] Through the description of the above implementation manners, those skilled in the art can clearly understand that the method according to the above - mentioned embodiments can be implemented by means of software plus a necessary general - purpose hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation manner.

[0122] As Figure 5 shown, the embodiments of the present application further provide a processing device for object data, including:

[0123] A receiving unit 502, configured to receive a data writing request, where the data writing request carries a logical unit address and the object data to be written;

[0124] An obtaining unit 504, configured to obtain the data storage address and the transitional layer address allocated for the object data when the write operation based on the object data is completed, where the data storage address is used to indicate the write position of the object data, and the transitional layer address determines the mapping relationship between the logical unit address and the transitional layer address of the object data;

[0125] A first adding unit 506, configured to add a first mapping relationship between the logical unit address and the transitional layer address in the first mapping relationship set;

[0126] A second adding unit 508, configured to add a second mapping relationship between the transitional layer address and the data storage address in the second mapping relationship set.

[0127] Optionally, the above - mentioned processing device for object data further includes: a third adding unit, configured to determine the data fingerprint information matching the above - mentioned object data; when the above - mentioned object data is written, add a third mapping relationship between the above - mentioned data fingerprint information and the above - mentioned transitional layer address in the third mapping relationship set.

[0128] Optionally, the processing device for the object data further includes: a first writing unit, configured to obtain the data fingerprint information matching the object data; when the third mapping relationship set includes a fourth mapping relationship matching the data fingerprint information, obtain a reference transition layer address according to the fourth mapping relationship; in the first mapping relationship set, add a fifth mapping relationship between the logical unit address and the reference transition layer address; according to the reference transition layer address, look up a sixth mapping relationship in the second mapping relationship set, where the sixth mapping relationship is a mapping relationship between the reference transition layer address and a reference storage address; increase a reference count matching the sixth mapping relationship from a first value to a second value, where the reference count is used to indicate the number of logical mapping relationships associated with the reference transition layer address; when the third mapping relationship set does not include a mapping relationship matching the data fingerprint information, determine to perform the writing operation based on the object data.

[0129] Optionally, the processing device for the object data further includes: a second writing unit, configured to determine a plurality of sub-object data according to the object data, where the data volume of the sub-object data is a preset size; determine data fingerprint values respectively corresponding to the plurality of sub-object data, where the data fingerprint information matching the object data includes the data fingerprint values respectively corresponding to the plurality of sub-object data; write the data fingerprint values respectively corresponding to the sub-object data into a plurality of consecutive sub-units in the first storage unit; write the plurality of sub-object data into a plurality of consecutive storage units associated with the first storage unit.

[0130] Optionally, the second writing unit is further configured to: in response to a storage space recovery instruction, determine a first storage address of the current object data to be operated and a second storage address after movement; according to the first storage address, look up a seventh mapping relationship in the second mapping relationship set, where the seventh mapping relationship is a mapping relationship between the first storage address and a first transition layer address; according to the second storage address, update the seventh mapping relationship to an eighth mapping relationship, where the eighth mapping relationship is a mapping relationship between the second storage address and the first transition layer address.

[0131] Optionally, the first writing unit is further configured to: in response to a data deletion request, look up, in the first mapping set, a ninth mapping relationship that matches the data deletion request according to the first logical unit address carried in the data deletion request; in the case where the second transition layer address indicated by the ninth mapping relationship matches the reference transition layer address, look up, in the second mapping set, the sixth mapping relationship according to the reference transition layer address; reduce the reference count that matches the sixth mapping relationship from the second value to a third value; in the case where the third value is less than or equal to a target threshold, label the data object corresponding to the reference storage address as a deletable data object according to the sixth mapping relationship.

[0132] Optionally, the first writing unit is further configured to: obtain the data fingerprint information that matches the deletable data object according to the reference storage address; look up the fourth mapping relationship in the third mapping relationship set according to the data fingerprint information; look up the fifth mapping relationship in the first mapping relationship set according to the reference transition layer address included in the fourth mapping relationship; delete the fourth mapping relationship in the third mapping relationship set and delete the fifth mapping relationship in the first mapping relationship set.

[0133] For the description of the features in the corresponding embodiments of the object data processing device, reference may be made to the relevant descriptions in the corresponding embodiments of the object data processing method, which will not be elaborated here one by one.

[0134] An embodiment of the present application further provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the embodiments of the above object data processing method. The electronic device may be Figure 1 the terminal device or server shown. In this embodiment, the electronic device is taken as a mobile phone or a computer as an example. As Figure 6 shown, the electronic device includes a memory 602 and a processor 604. A computer program is stored in the memory 602, and the processor 604 is configured to execute the steps in any one of the above method embodiments through the computer program.

[0135] Optionally, in this embodiment, the electronic device may be at least one network device among multiple network devices in a computer network.

[0136] Optionally, in this embodiment, the processor may be configured to execute the following steps through a computer program:

[0137] S1, receive a data writing request, where the data writing request carries a logical unit address and object data to be written;

[0138] S2. When the write operation is completed based on the object data, obtain the data storage address and the transition layer address assigned to the object data, where the data storage address is used to indicate the write position of the object data, and the transition layer address determines the mapping relationship between the logical unit address of the object data and the transition layer address;

[0139] S3. In the first mapping relationship set, add the first mapping relationship between the logical unit address and the transition layer address;

[0140] S4. In the second mapping relationship set, add the second mapping relationship between the transition layer address and the data storage address.

[0141] Optionally, those of ordinary skill in the art can understand that Figure 6 The structure shown is only schematic, and the electronic device can also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a handheld computer, and terminal devices such as Mobile Internet Devices (MID), PAD, etc. Figure 6 It does not limit the structure of the above-mentioned electronic device. For example, the electronic device may further include more or fewer components (such as a network interface, etc.) than those shown Figure 6 in the figure, or have a different configuration from that shown Figure 6 in the figure.

[0142] Among them, the memory 602 can be used to store software programs and modules, such as the program instructions / modules corresponding to the processing method and device of the object data in the embodiments of the present application. The processor 604 executes various functional applications and data processing by running the software programs and modules stored in the memory 602, that is, implements the above-mentioned processing method of the object data. The memory 602 includes the receiving unit 502, the obtaining unit 504, the first adding unit 506, and the second adding unit 508 in the above-mentioned processing device of the object data. The memory 602 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 602 may further include a memory remotely set relative to the processor 604, and these remote memories can be connected to the terminal through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof. This example will not be elaborated here.

[0143] Optionally, the above-mentioned transmission device 606 is used to receive or send data via a network. Specific examples of the above-mentioned network may include a wired network and a wireless network. In one example, the transmission device 606 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers through a network cable, so as to communicate with the Internet or a local area network. In one example, the transmission device 606 is a Radio Frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0144] In addition, the above-mentioned electronic device further includes: a display 608, which is used to display the above-mentioned target page; and a connection bus 610, which is used to connect each module component in the above-mentioned electronic device.

[0145] An embodiment of the present application also provides a computer-readable storage medium, in which a computer program is stored. Wherein, the computer program is configured to execute the steps in any one of the above-mentioned method embodiments for processing object data when running.

[0146] In an exemplary embodiment, the above-mentioned computer-readable storage medium may include, but is not limited to: various media that can store computer programs such as USB flash drives, read-only memories (Read-Only Memory, abbreviated as ROM), random access memories (Random Access Memory, abbreviated as RAM), mobile hard disks, magnetic disks, or optical discs.

[0147] An embodiment of the present application also provides a computer program product. The above-mentioned computer program product includes a computer program, and when the computer program is executed by a processor, it implements the steps in any one of the above-mentioned method embodiments for processing object data.

[0148] An embodiment of the present application also provides another computer program product, including a non-volatile computer-readable storage medium. The non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the steps in any one of the above-mentioned method embodiments for processing object data.

[0149] Those skilled in the art may further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0150] The above has introduced in detail a method for processing object data provided by this application. Specific examples have been used herein to elaborate on the principle and implementation manner of this application. The description of the above embodiments is only used to help understand the method and its core idea of this application. It should be noted that for those of ordinary skill in the art in this technical field, without departing from the principle of this application, several improvements and modifications can be made to this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A method for processing object data, characterized in that, including: Receiving a data writing request, where the data writing request carries a logical unit address and object data to be written; When the writing operation is completed based on the object data, obtaining a data storage address and a transition layer address allocated for the object data, where the data storage address is used to indicate the writing position of the object data, and the transition layer address determines the mapping relationship between the logical unit address of the object data and the transition layer address; Adding a first mapping relationship between the logical unit address and the transition layer address to a first mapping relationship set; Adding a second mapping relationship between the transition layer address and the data storage address to a second mapping relationship set.

2. The method according to claim 1, wherein: After receiving the data writing request, it further includes: Determining data fingerprint information matching the object data; When the writing of the object data is completed, adding a third mapping relationship between the data fingerprint information and the transition layer address to a third mapping relationship set.

3. The method according to claim 2, wherein: After receiving the data writing request, it further includes: Obtaining the data fingerprint information matching the object data; When a fourth mapping relationship matching the data fingerprint information is included in the third mapping relationship set, obtaining a reference transition layer address according to the fourth mapping relationship; Adding a fifth mapping relationship between the logical unit address and the reference transition layer address to the first mapping relationship set; According to the reference transition layer address, searching for a sixth mapping relationship in the second mapping relationship set, where the sixth mapping relationship is the mapping relationship between the reference transition layer address and a reference storage address; Increasing a reference count matching the sixth mapping relationship from a first value to a second value, where the reference count is used to indicate the number of logical mapping relationships associated with the reference transition layer address; When a mapping relationship matching the data fingerprint information is not included in the third mapping relationship set, determining to perform the writing operation based on the object data.

4. The method according to claim 2, wherein: Before obtaining the data storage address and the transition layer address allocated for the object data when the writing operation is completed based on the object data, it further includes: Determining a plurality of sub-object data according to the object data, where the data volume of the sub-object data is a preset size; Determining data fingerprint values respectively corresponding to the plurality of sub-object data, where the data fingerprint information matching the object data includes the data fingerprint values respectively corresponding to the plurality of sub-object data; Writing the data fingerprint values respectively corresponding to the sub-object data into a plurality of consecutive sub-units in a first storage unit; Writing the plurality of sub-object data into a plurality of consecutive storage units associated with the first storage unit.

5. The method according to claim 4, wherein: After adding the second mapping relationship between the transition layer address and the data storage address in the second mapping relationship set, the method further includes: In response to a storage space recovery instruction, determining a first storage address of the current object data to be operated and a second storage address after movement; According to the first storage address, searching for a seventh mapping relationship in the second mapping relationship set, where the seventh mapping relationship is a mapping relationship between the first storage address and a first transition layer address; According to the second storage address, updating the seventh mapping relationship to an eighth mapping relationship, where the eighth mapping relationship is a mapping relationship between the second storage address and the first transition layer address.

6. The method according to claim 3, wherein: After increasing the reference count matching the sixth mapping relationship from a first value to a second value, the method further includes: In response to a data deletion request, according to the first logical unit address carried in the data deletion request, searching for a ninth mapping relationship matching the data deletion request in the first mapping set; When the second transition layer address indicated by the ninth mapping relationship matches the reference transition layer address, searching for the sixth mapping relationship in the second mapping set according to the reference transition layer address; Reducing the reference count matching the sixth mapping relationship from the second value to a third value; When the third value is less than or equal to a target threshold, marking the data object corresponding to the reference storage address as a deletable data object according to the sixth mapping relationship.

7. The method according to claim 6, wherein: After marking the data object corresponding to the reference storage address as a deletable data object according to the fifth mapping relationship when the third value is less than or equal to a target threshold, the method further includes: Obtaining the data fingerprint information matching the deletable data object according to the reference storage address; Searching for the fourth mapping relationship in the third mapping relationship set according to the data fingerprint information; Searching for the fifth mapping relationship in the first mapping relationship set according to the reference transition layer address included in the fourth mapping relationship; Deleting the fourth mapping relationship in the third mapping relationship set and deleting the fifth mapping relationship in the first mapping relationship set.

8. An apparatus for processing object data, characterized in that The method includes: A receiving unit, configured to receive a data writing request, where the data writing request carries a logical unit address and object data to be written; An obtaining unit, configured to obtain a data storage address and a transition layer address allocated for the object data when a writing operation is completed based on the object data, where the data storage address is used to indicate a writing position of the object data, and the transition layer address determines a mapping relationship between the logical unit address and the transition layer address of the object data; A first adding unit, configured to add a first mapping relationship between the logical unit address and the transition layer address in a first mapping relationship set; A second addition unit, configured to add a second mapping relationship between the transition layer address and the data storage address in the second mapping relationship set.

9. An electronic device, characterized in that, Comprising: A memory, configured to store a computer program; A processor, configured to implement the steps of the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium, wherein the computer program implements the steps of the method according to any one of claims 1 to 7 when executed by a processor.

Images