Storage unit reference state maintenance method and device, product, equipment and medium

Abstract

The invention discloses a storage unit reference state maintenance method and device, a product, equipment and a medium, relates to the technical field of computers, and is applied to a distributed storage system, the system comprises a logic storage unit and a physical storage unit, and the method comprises the following steps: initiating a reference operation request to the physical storage unit through the logic storage unit carrying a reference key value, processing the reference operation request based on the reference key value through a physical storage unit; the idempotency of the reference count is ensured by the reference key value; if the logic storage unit receives an operation success response returned by the physical storage unit, executing a reference operation corresponding to the reference operation request in the logic storage unit; and sending an update notification to the physical storage unit through the logic storage unit, so that the physical storage unit deletes the reference key value and updates the reference count after receiving the update notification, thereby realizing maintenance of the reference state of the physical storage unit. In conclusion, the application reduces the occupation of the metadata resource space, and improves the system performance and benefit.

Description

Technical Field

[0001] The present invention relates to the field of computer technologies, and particularly relates to a method, apparatus, product, device, and medium for maintaining the reference status of storage units. Background Art

[0002] With the explosive growth of data volume in the digital age, duplicate data consumes a large amount of transmission bandwidth and storage space. As a data reduction means, duplicate data deletion technology has emerged as the times require. The core of the deduplication technology is that only one physical data block instance (Physical Data Block, PDB) is stored for the same data block, and other identical data blocks (logical data block instances, Logical Data Block, LDB) realize data sharing by storing references to the PDB. The reference count of the PDB instance reflects the number of times it is referenced by the LDB.

[0003] In a distributed storage system, LDBs and PDBs are scattered in different service processes and storage nodes, and the update of the reference count between the two depends on network transmission. Due to situations such as service failures, node anomalies, or network fluctuations, the update operation of the reference count is prone to the phenomenon of repeated sending. Therefore, it is crucial to ensure the idempotency of the reference count update operation. The idempotency of the update operation means that no matter whether the operation is executed once or multiple times, the impact on the system remains the same, so as to avoid data errors caused by repeated calculations.

[0004] Figure 1 Shows the interaction process of the reference operation between LDB and PDB in different scenarios in the traditional technology. It can be seen that the common way to achieve the idempotency of the reference count update operation in the traditional technology is that when the LDB requests to update the reference count, it carries a reference key value that uniquely identifies itself. After receiving the request, the PDB confirms the reference relationship with the LDB through the reference key value. However, the reference key value usually occupies at least dozens of bytes. As the number of references increases, the reference key values stored persistently will occupy a large amount of metadata space, making the already limited and precious metadata space resources even more tense, affecting the system performance and efficiency. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a method, apparatus, product, device, and medium for maintaining the reference status of storage units, which can reduce the occupation of metadata resource space and improve the system performance and efficiency. The specific solutions are as follows:

[0006] In a first aspect, the present application discloses a method for maintaining the reference status of storage units, which is applied to a distributed storage system. The distributed storage system includes logical storage units and physical storage units. The method includes:

[0007] The reference operation request is initiated to the physical storage unit by the logical storage unit carrying the reference key value, and the physical storage unit processes the reference operation request based on the reference key value; wherein, the reference key value is used to ensure the idempotency of the reference count, and the reference count is used to record the cumulative number of times the physical storage unit is referenced by different logical storage units;

[0008] If the logical storage unit receives the operation success response returned by the physical storage unit, the reference operation corresponding to the reference operation request is executed in the logical storage unit;

[0009] The update notification is sent to the physical storage unit by the logical storage unit, so that after receiving the update notification, the physical storage unit deletes the reference key value and updates the reference count to maintain the reference status of the physical storage unit.

[0010] Optionally, the reference operation request includes a reference creation request and a reference deletion request.

[0011] Optionally, if the reference operation request is a reference creation request, the physical storage unit processes the reference operation request based on the reference key value, including:

[0012] The physical storage unit queries whether there is a temporary reference corresponding to the reference key value; wherein, the temporary reference is a temporary record created to identify the reference relationship between the logical storage unit and the physical storage unit, and the temporary reference includes the reference key value;

[0013] If there is no temporary reference, a temporary reference is created and a creation success response is returned to the logical storage unit;

[0014] If there is a temporary reference, a creation success response is directly returned to the logical storage unit.

[0015] Optionally, deleting the reference key value and updating the reference count includes:

[0016] The physical storage unit queries whether there is a temporary reference;

[0017] If there is a temporary reference, deleting the reference key value is achieved by deleting the temporary reference, and the reference count is incremented by 1.

[0018] Optionally, the method for maintaining the reference status of the storage unit further includes:

[0019] Determine whether the temporary reference is deleted within the first verification time;

[0020] If the temporary reference is not deleted within the first verification time, a verification request is initiated to the logical storage unit by the physical storage unit;

[0021] If the verification result returned by the logical storage unit indicates that the logical storage unit still has a reference requirement for the physical storage unit, delete the temporary reference and increment the reference count by 1.

[0022] Optionally, if the reference operation request is a reference deletion request, the physical storage unit processes the reference operation request based on the reference key value pair, including:

[0023] The physical storage unit queries whether there is a temporary reference corresponding to the reference key value; wherein, the temporary reference is a temporary record created to identify the reference relationship between the logical storage unit and the physical storage unit, and the temporary reference includes the reference key value;

[0024] If there is a temporary reference, delete the temporary reference and return a deletion success response to the logical storage unit;

[0025] If there is no temporary reference, directly return a deletion success response to the logical storage unit.

[0026] Optionally, deleting the reference key value and updating the reference count includes:

[0027] The physical storage unit queries whether there is a temporary reference;

[0028] If there is a temporary reference, delete the reference key value by deleting the temporary reference and decrement the reference count by 1.

[0029] Optionally, the method for maintaining the reference status of the storage unit further includes:

[0030] Determine whether the temporary reference has been deleted within the second verification time;

[0031] If the temporary reference has not been deleted within the second verification time, the physical storage unit initiates a verification request to the logical storage unit;

[0032] If the verification result returned by the logical storage unit indicates that the logical storage unit still has a reference requirement for the physical storage unit, delete the temporary reference and decrement the reference count by 1.

[0033] Optionally, the method for maintaining the reference status of the storage unit further includes:

[0034] Determine whether the reference count has been decremented to 0;

[0035] If the reference count has been decremented to 0, delete the stored content in the physical storage unit and release the relevant resources occupied by the stored content.

[0036] Optionally, the method for maintaining the reference status of the storage unit further includes:

[0037] When a data writing request initiated by a user is received through a logical storage unit, it is determined whether the data to be written in the data writing request duplicates the stored content in the physical storage unit;

[0038] Based on the determination result, it is determined whether to write the data to be written in the physical storage unit.

[0039] Optionally, determining whether to write the data to be written in the physical storage unit based on the determination result includes:

[0040] If the data to be written does not duplicate the stored content in the physical storage unit, the data to be written is written in the physical storage unit.

[0041] Optionally, determining whether to write the data to be written in the physical storage unit based on the determination result includes:

[0042] If the data to be written duplicates the stored content in the physical storage unit, there is no need to write the data to be written in the physical storage unit, and it jumps to the step of initiating a reference creation request to the physical storage unit by carrying a reference key value through the logical storage unit.

[0043] Optionally, the update notification is sent using an asynchronous communication mechanism.

[0044] Optionally, the numerical type of the reference count is an integer type.

[0045] Optionally, during the process of deleting the reference key value and updating the reference count, it further includes:

[0046] Encapsulating the operation of deleting the reference key value and the operation of updating the reference count into the same database transaction to ensure that the operation of deleting the reference key value and the operation of updating the reference count take effect simultaneously.

[0047] Optionally, the logical storage unit includes a file in a file storage system, a logical data block in a block storage system, an object in an object storage system, or a virtual volume in a virtualized storage system; the physical storage unit includes a physical disk block, a solid-state drive storage page, or a persistent memory unit.

[0048] In a second aspect, the present application discloses a storage unit reference status maintenance device applied to a distributed storage system. The distributed storage system includes a logical storage unit and a physical storage unit. The device includes:

[0049] A request initiation and processing module, configured to initiate a reference operation request to the physical storage unit by carrying a reference key value through the logical storage unit, and process the reference operation request based on the reference key value through the physical storage unit; wherein, the reference key value is used to ensure the idempotency of the reference count, and the reference count is used to record the cumulative number of times the physical storage unit is referenced by different logical storage units;

[0050] A local operation execution module, configured to, if the logical storage unit receives an operation success response returned by the physical storage unit, execute a reference operation corresponding to the reference operation request in the logical storage unit;

[0051] A count update trigger module, configured to send an update notification to the physical storage unit through the logical storage unit, so that after receiving the update notification, the physical storage unit deletes the reference key value and updates the reference count, so as to maintain the reference status of the physical storage unit.

[0052] In a third aspect, the present application discloses a computer program product, including computer programs / instructions, which, when executed by a processor, implement the steps of the foregoing method for maintaining the reference status of a storage unit.

[0053] In a fourth aspect, the present application discloses an electronic device, including:

[0054] A memory, configured to store computer programs;

[0055] A processor, configured to execute the computer programs to implement the foregoing method for maintaining the reference status of a storage unit.

[0056] In a fifth aspect, the present application discloses a computer-readable storage medium, configured to store computer programs; wherein, when the computer programs are executed by a processor, the foregoing method for maintaining the reference status of a storage unit is implemented.

[0057] It can be seen that the present application proposes a method for maintaining the reference status of a storage unit, which is applied to a distributed storage system. The distributed storage system includes a logical storage unit and a physical storage unit. The method includes: initiating a reference operation request to the physical storage unit by carrying a reference key value through the logical storage unit, and processing the reference operation request by the physical storage unit based on the reference key value; wherein, the reference key value is used to ensure the idempotency of the reference count, and the reference count is used to record the cumulative number of times the physical storage unit is referenced by different logical storage units; if the logical storage unit receives an operation success response returned by the physical storage unit, execute a reference operation corresponding to the reference operation request in the logical storage unit; send an update notification to the physical storage unit through the logical storage unit, so that after receiving the update notification, the physical storage unit deletes the reference key value and updates the reference count, so as to convert from persistent reference key values to maintaining the reference count.

[0058] Beneficial effects: In this application, the logical storage unit initiates a reference operation request to the physical storage unit by carrying a reference key value, and the physical storage unit processes the reference operation request based on the reference key value to ensure the idempotency of the reference count. After the logical storage unit receives the operation success response, it executes the reference operation corresponding to the reference operation request and sends an update notification to the physical storage unit, so that the physical storage unit deletes the temporary reference key value after receiving the notification, and maintains the cumulative number of times the physical storage unit is referenced through the reference count, realizing the conversion from persistently storing the reference key value to lightweight maintenance of the reference count. Compared with the traditional solution that needs to persistently store the reference key value of each logical storage unit, this application ensures the operation idempotency by temporarily using the reference key value, deletes the reference key value after completing the reference operation, and then maintains the referenced state of the physical storage unit through the reference count. In this way, this application gets rid of the influence of the number of references on the metadata space, avoids the problem of a sharp increase in metadata storage overhead caused by the expansion of the reference scale, and also reduces the resource consumption of operations such as metadata query, thereby greatly improving the metadata management efficiency, and further optimizing the performance and benefits of the distributed storage system. Brief Description of the Drawings

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to the provided drawings.

[0060] Figure 1 It is a schematic diagram of the basic principle for implementing the idempotency of the reference count based on the reference key value in the prior art;

[0061] Figure 2 It is a flowchart of a method for maintaining the reference state of a storage unit disclosed in this application;

[0062] Figure 3 It is a flowchart of a specific method for maintaining the reference state of a storage unit disclosed in this application;

[0063] Figure 4 It is a schematic diagram of the process of a logical and physical storage unit under normal reference operations disclosed in this application;

[0064] Figure 5 It is a schematic diagram of the process of a logical and physical storage unit under repeated reference operations disclosed in this application;

[0065] Figure 6 It is a flowchart of a method for handling reference creation exceptions and residual references disclosed in this application;

[0066] Figure 7Flowchart of a specific method for maintaining the reference status of storage units disclosed in this application;

[0067] Figure 8 Flowchart of the process for handling deletion reference exceptions and residual references disclosed in this application;

[0068] Figure 9 Schematic structural diagram of a device for maintaining the reference status of storage units disclosed in this application;

[0069] Figure 10 Structural diagram of an electronic device disclosed in this application. Detailed implementation manners

[0070] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0071] The common method for traditional implementation of the idempotency of reference count update operations is that when the LDB requests to update the reference count, it carries a reference key value that uniquely identifies itself. After receiving the request, the PDB confirms the reference relationship with the LDB through this reference key value. However, the reference key value usually occupies at least dozens of bytes. As the number of references continues to increase, the reference key values stored persistently will occupy a large amount of metadata space, making the already limited and precious metadata space resources even more tense, affecting the system performance and efficiency.

[0072] Therefore, the embodiments of this application propose a solution for maintaining the reference status of storage units, which can reduce the occupation of metadata resource space and improve the system performance and efficiency.

[0073] The embodiments of this application disclose a method for maintaining the reference status of storage units. Refer to Figure 2 As shown, the method includes:

[0074] Step S11: Initiate a reference operation request to the physical storage unit by carrying a reference key value through the logical storage unit, and the physical storage unit processes the reference operation request based on the reference key value; wherein, the reference key value is used to ensure the idempotency of the reference count, and the reference count is used to record the cumulative number of times the physical storage unit is referenced by different logical storage units.

[0075] In this embodiment, the logical storage unit includes a file in a file storage system, a logical data block in a block storage system, an object in an object storage system, or a virtual volume in a virtualized storage system; the physical storage unit includes a physical disk block, a solid-state drive storage page, or a persistent memory unit. If the logical storage unit is a file in a file storage system, the corresponding physical storage unit can be a physical disk block or a solid-state drive storage page; if the logical storage unit is a logical data block in a block storage system, the corresponding physical storage unit can be a physical disk block and a solid-state drive storage page; if the logical storage unit is an object in an object storage system, the corresponding physical storage unit can be a physical disk block, a solid-state drive storage page, and a persistent memory unit; if the logical storage unit is a virtual volume in a virtualized storage system, the corresponding physical storage unit can be a physical disk block and a solid-state drive storage page. In this way, this application is compatible with multiple storage systems such as files, blocks, objects, and virtualizations, and realizes efficient cross-system metadata processing.

[0076] In this embodiment, a reference operation request is initiated to the physical storage unit by the logical storage unit carrying a reference key value, and the physical storage unit processes the reference operation request based on the reference key value; the reference key value is used to ensure the idempotency of the reference count, and the reference count is used to record the cumulative number of times the physical storage unit is referenced by different logical storage units; the reference operation request includes a reference creation request and a reference deletion request.

[0077] In a specific implementation manner, when the reference operation request is a reference creation request, the physical storage unit is queried to determine whether there is a temporary reference corresponding to the reference key value; the temporary reference is a temporary record created to identify the reference relationship between the logical storage unit and the physical storage unit, and the temporary reference includes the reference key value; if there is no temporary reference, a temporary reference is created and a creation success response is returned to the logical storage unit; if there is a temporary reference, a creation success response is directly returned to the logical storage unit. That is, when the logical storage unit initiates a reference creation request, the physical storage unit first queries whether there is a temporary reference corresponding to the reference key value in the request. If not, a temporary reference is created and a creation success response is returned; if so, it means that the reference relationship has been established, and a success response is directly returned to avoid repeated operations. In this way, through the existence verification of the temporary reference, the idempotency of the creation operation is ensured.

[0078] In another specific implementation, when the reference operation request is a reference deletion request, it is queried through the physical storage unit whether there is a temporary reference corresponding to the reference key value; wherein, the temporary reference is a temporary record created to identify the reference relationship between the logical storage unit and the physical storage unit, and the temporary reference includes the reference key value; if there is a temporary reference, the temporary reference is deleted, and a deletion success response is returned to the logical storage unit; if there is no temporary reference, a deletion success response is directly returned to the logical storage unit. That is, when the logical storage unit initiates a reference deletion request, the physical storage unit first queries whether there is a temporary reference corresponding to the reference key value. If there is, the temporary reference is deleted and a deletion success response is returned; if not, it means the reference relationship has expired, and a success response is directly returned to avoid invalid operations. In this way, through the existence verification of the temporary reference, the idempotency of the deletion operation is ensured.

[0079] Step S12: If the logical storage unit receives an operation success response returned by the physical storage unit, a reference operation corresponding to the reference operation request is executed in the logical storage unit.

[0080] In a specific implementation, if the logical storage unit receives a creation success response returned by the physical storage unit, an operation corresponding to the reference creation request is executed in the logical storage unit. For example, in a file storage system, the logical storage unit is used to add a file entry to a directory. In another specific implementation, if the logical storage unit receives a deletion success response returned by the physical storage unit, an operation corresponding to the reference deletion request is executed in the logical storage unit. For another example, in a file storage system, the logical storage unit is used to remove a file entry from a directory or mark it as the "deleted" state.

[0081] Step S13: An update notification is sent from the logical storage unit to the physical storage unit, so that after receiving the update notification, the physical storage unit deletes the reference key value and updates the reference count to maintain the reference state of the physical storage unit.

[0082] In this embodiment, an update notification is sent from the logical storage unit to the physical storage unit, so that after receiving the update notification, the physical storage unit deletes the reference key value and updates the reference count to maintain the reference state of the physical storage unit. It should be noted that the update notification is sent using an asynchronous communication mechanism, that is, the logical storage unit does not need to wait for the physical storage unit to complete processing after sending the update notification, and can immediately process subsequent service requests, avoiding the blocking of the service IO (Input / Output) path caused by metadata update, and significantly improving the throughput of the distributed storage system.

[0083] In this embodiment, the reference count can be of an integer type. The integer type only occupies the storage space of several bytes, and its space occupancy does not increase with the increase in the number of references. Compared with the traditional solution where it is necessary to persistently store reference key values of dozens of bytes and the amount of metadata grows linearly with the reference scale, this application gets rid of the influence of the number of references on the metadata space, avoids the problem of a sharp increase in the metadata storage overhead caused by the expansion of the reference scale, and also reduces the resource consumption of operations such as metadata query, thereby greatly improving the metadata management efficiency and further optimizing the performance and long-term benefits of the distributed storage system.

[0084] In this embodiment, the operation of deleting the reference key value and the operation of updating the reference count are encapsulated into the same database transaction to ensure that the operation of deleting the reference key value and the operation of updating the reference count take effect simultaneously. That is, the atomicity of the transaction ensures that the two operations of deleting the reference key value and updating the reference count are executed as an inseparable whole, avoiding the problem that the reference key value is deleted but the reference count is not updated, or the reference count is updated but the reference key value is not deleted. The transaction ensures that the two take effect simultaneously and meets the idempotency requirement.

[0085] It can be seen that this application proposes a method for maintaining the reference state of storage units, which is applied to a distributed storage system. The distributed storage system includes a logical storage unit and a physical storage unit. The method includes: initiating a reference operation request to the physical storage unit through the logical storage unit carrying the reference key value, and processing the reference operation request by the physical storage unit based on the reference key value; wherein, the reference key value is used to ensure the idempotency of the reference count, and the reference count is used to record the cumulative number of times the physical storage unit is referenced by different logical storage units; if the logical storage unit receives an operation success response returned by the physical storage unit, then execute the reference operation corresponding to the reference operation request in the logical storage unit; send an update notification to the physical storage unit through the logical storage unit, so that after the physical storage unit receives the update notification, delete the reference key value and update the reference count, in order to convert from persistent reference key values to maintaining the reference count.

[0086] Beneficial effects: In this application, the logical storage unit initiates a reference operation request to the physical storage unit by carrying a reference key value, and the physical storage unit processes the reference operation request based on the reference key value to ensure the idempotency of the reference count. When the logical storage unit receives an operation success response, it executes the reference operation corresponding to the reference operation request and sends an update notification to the physical storage unit, so that the physical storage unit deletes the temporary reference key value after receiving the notification, and maintains the cumulative number of times the physical storage unit is referenced through the reference count, realizing the conversion from persistent storage of reference key values to lightweight maintenance of reference counts. Compared with the traditional solution that needs to persistently store the reference key values of each logical storage unit, this application ensures operation idempotency by temporarily using reference key values, deletes the reference key value after the reference operation is completed, and then maintains the referenced state of the physical storage unit through the reference count. In this way, this application gets rid of the influence of the number of references on the metadata space, avoids the problem of a sharp increase in metadata storage overhead caused by the expansion of the reference scale, and also reduces the resource consumption of operations such as metadata query, thus greatly improving the metadata management efficiency, and further optimizing the performance and benefits of the distributed storage system.

[0087] This embodiment of the application discloses a specific method for maintaining the reference state of a storage unit. Compared with the previous embodiment, this embodiment further explains and optimizes the technical solution to clarify the update process of the reference count when the reference operation request is a reference creation request. Refer to Figure 3 as shown, specifically including:

[0088] Step S13a1: Query whether there is a temporary reference through the physical storage unit.

[0089] Step S13a2: If there is a temporary reference, delete the temporary reference to delete the reference key value and increment the reference count.

[0090] In this embodiment, when the reference operation request is a reference creation request, query whether there is a temporary reference through the physical storage unit. If there is a temporary reference, delete the temporary reference to delete the reference key value and increment the reference count. Refer to Figure 4As shown in Case 1, Case 1 is the normal process of creating a reference. The logical storage unit first sends a request of "Create Reference (Key = key1)" to the physical storage unit. After receiving the request, the physical storage unit looks up the temporary reference (Key = key1), and since it is not found, it creates the temporary reference (Key = key1) and returns a response of "Creation Successful" to the logical storage unit. After receiving the response, the logical storage unit executes the steps of creating a reference in the data block and then notifies the physical storage unit that the reference (Key = key1) has been created. After receiving the notification, the physical storage unit looks up the temporary reference (Key = key1) again. At this time, the temporary reference already exists. The physical storage unit deletes the temporary reference (Key = key1) and increments the reference count by 1 to complete the normal process of creating a reference.

[0091] In this embodiment, Figure 5 Case 3 is the process of repeatedly creating a reference. The logical storage unit first sends a request of "Create Reference (Key = key1)" to the physical storage unit. The physical storage unit looks up the temporary reference (Key = key1), creates the temporary reference (Key = key1) when it is not found, and returns a response of "Creation Successful", but this response is lost. After the response times out, the logical storage unit resends the request of "Create Reference (Key = key1)". At this time, the physical storage unit looks up the temporary reference (Key = key1) again, and since it is already found, it still notifies the logical storage unit that the creation is successful. Then, the logical storage unit creates a reference in the data block and notifies the physical storage unit that the reference (Key = key1) has been created. After receiving the notification, the physical storage unit confirms again that the temporary reference (Key = key1) already exists, then deletes the temporary reference (Key = key1) and increments the reference count by 1 to complete the entire process of repeatedly creating a reference. In this way, it avoids repeatedly creating temporary references and reasonably increases or decreases the reference count.

[0092] Further, when a data writing request initiated by a user through the logical storage unit is received, it is determined whether the data to be written in the data writing request duplicates the stored content in the physical storage unit; whether to write the data to be written in the physical storage unit is determined according to the determination result. If the data to be written does not duplicate the stored content in the physical storage unit, the data to be written is written in the physical storage unit. If the data to be written duplicates the stored content in the physical storage unit, there is no need to write the data to be written in the physical storage unit, and the process jumps to the step of initiating a reference creation request to the physical storage unit through the logical storage unit carrying a reference key value. That is to say, when the logical storage unit receives a data writing request from a user, the data to be written is compared with the existing stored content in the physical storage unit. If the two do not duplicate, the data is normally written into the physical storage unit; if they duplicate, no writing operation is performed, but directly enter the step of initiating a reference creation request to the physical storage unit through the logical storage unit carrying a reference key value, so as to optimize the data storage process and effectively improve the utilization efficiency of storage resources.

[0093] In this embodiment, it is determined whether the temporary reference is deleted within the first verification time; if the temporary reference is not deleted within the first verification time, a verification request is initiated to the logical storage unit through the physical storage unit; if the verification result returned by the logical storage unit indicates that the logical storage unit still has a reference requirement for the physical storage unit, the temporary reference is deleted and the reference count is incremented by 1. Refer to Figure 6 As shown, the logical storage unit sends a "create reference (key = key1)" request to the physical storage unit. After the physical storage unit searches for the temporary reference (key = key1) and finds that it does not exist, it creates it and returns a create success response. The logical storage unit creates a local reference, but notifies that "reference has been created (key = key1)" fails. Then it enters the stage of processing residual references. After the deletion timeout (also after exceeding the first verification time), the physical storage unit sends a "check reference (key = key1)" request to the logical storage unit. If the logical storage unit replies that it is still referencing, the physical storage unit deletes the temporary reference (key = key1) and increments the reference count by 1. It can be seen that by judging the status of the temporary reference within the first verification time, unnormally deleted temporary references can be detected in time, avoiding their long-term residual occupation of storage resources, ensuring the accuracy of resource recovery, and enabling more reasonable utilization of storage resources.

[0094] The embodiment of the present application discloses a specific method for maintaining the reference status of a storage unit. Compared with the previous embodiment, this embodiment further explains and optimizes the technical solution to clarify the update process of the reference count when the reference operation request is a reference deletion request. Refer to Figure 7 As shown, it specifically includes:

[0095] Step S13b1: Query whether there is a temporary reference through the physical storage unit.

[0096] Step S13b2: If there is a temporary reference, delete the reference key value by deleting the temporary reference, and decrement the reference count by 1.

[0097] In this embodiment, when the reference operation request is a reference deletion request, query whether there is a temporary reference through the physical storage unit. If there is a temporary reference, delete the reference key value by deleting the temporary reference, and decrement the reference count by 1. See Figure 4 as shown in Case 2. Case 2 is the normal reference deletion process. The logical storage unit sends a request of "delete reference (key = key1)" to the physical storage unit. After receiving the request, the physical storage unit searches for the temporary reference (key = key1). If it does not exist, it creates a temporary reference (key = key1), and then returns a response of "deletion successful" to the logical storage unit. After receiving the response, the logical storage unit deletes the reference in the data block, and then notifies the physical storage unit that the reference (key = key1) has been deleted. After receiving the notification, the physical storage unit searches for the temporary reference (key = key1) again. At this time, the temporary reference already exists. The physical storage unit deletes the temporary reference (key = key1) and decrements the reference count by 1 to complete the normal reference deletion process.

[0098] In this embodiment, Figure 5 Case 4 is the repeated reference deletion process. The logical storage unit sends a request of "delete reference (key = key1)" to the physical storage unit. The physical storage unit searches for the temporary reference (key = key1). After finding that it does not exist, it creates a temporary reference (key = key1) and returns a response of "deletion successful", but this response is lost. After the response times out, the logical storage unit resends the request of "delete reference (key = key1)". The physical storage unit searches for the temporary reference (key = key1) again. After finding that it already exists, it notifies the logical storage unit that the deletion is successful. Then, the logical storage unit deletes the reference in the data block and notifies the physical storage unit that the reference (key = key1) has been deleted. After receiving the notification, the physical storage unit confirms that the temporary reference (key = key1) already exists, then deletes the temporary reference (key = key1) and decrements the reference count by 1, thus completing the entire process of repeated reference deletion. In this way, repeated deletion of temporary references is avoided, and the reference count is reasonably incremented and decremented.

[0099] In this embodiment, it is judged whether the temporary reference is deleted within the second verification time; if the temporary reference is not deleted within the second verification time, a verification request is initiated to the logical storage unit through the physical storage unit; if the verification result returned by the logical storage unit indicates that the logical storage unit still has a reference requirement for the physical storage unit, the temporary reference is deleted, and the reference count is decremented by 1. SeeFigure 8 As shown, the logical storage unit sends a request of "delete reference (key = key1)" to the physical storage unit. The physical storage unit looks up the temporary reference (key = key1), creates a temporary reference (key = key1) after finding it does not exist, and returns a response of "delete successful", but the response is lost (indicated by "X"). After the response times out, the logical storage unit resends the request of "delete reference (key = key1)". At this time, the physical storage unit finds that the temporary reference (key = key1) already exists and returns a response of "delete successful" again, but the response is lost again (indicated by "X"). Finally, the logical storage unit deletes the local reference. Then, it enters the stage of processing residual references (case 6: no reference): after the deletion times out (i.e., after the second verification time), the physical storage unit sends a request of "check reference (key = key1)" to the logical storage unit. The logical storage unit replies "no reference", and the physical storage unit deletes the temporary reference (key = key1) accordingly and decrements the reference count by 1. It can be seen that by judging the status of the temporary reference within the second verification time, the temporarily un-deleted references can be detected in time, avoiding their long-term residual occupation of storage resources, ensuring the accuracy of resource recovery, and enabling more reasonable utilization of storage resources.

[0100] In this embodiment, it is judged whether the reference count has been decremented to 0. If the reference count has been decremented to 0, the stored content in the physical storage unit is deleted, and the relevant resources occupied by the stored content are released. That is, once the system judges that the reference count has been decremented to 0, a series of resource cleaning operations will be triggered. First, the system deletes the stored content in the physical storage unit. In actual operation, according to different storage media and system architectures, corresponding efficient deletion algorithms are adopted. For example, for physical disk blocks, it involves the erasure of track and sector data; for solid-state drive storage pages, it is necessary to operate according to the write and erase mechanisms of flash memory to ensure that the stored content is completely erased and prevent storage chaos caused by data residue.

[0101] In this application, the definitions and partitioning methods and ways of the logical storage unit and the physical storage unit may be different according to the specific implementation technologies of different distributed storage systems, and the solutions of this application are not restricted by this. In a distributed storage system, taking the logical storage unit as the logical data block in the block storage system and the physical storage unit as the physical disk block as an example, the process of the storage unit reference status maintenance solution is as follows:

[0102] When a logical data block needs to perform a reference operation on a physical disk block, it will initiate a reference operation request to the physical disk block with a reference key value. The reference operation requests are divided into reference creation requests and reference deletion requests. The reference key value is used to ensure the idempotency of the reference count, and the reference count records the cumulative number of times a physical disk block is referenced by different logical data blocks. If a logical data block initiates a reference creation request, after receiving the reference creation request, the physical disk block will first check whether there is a temporary reference corresponding to the reference key value. If there is no temporary reference corresponding to the reference key value, it will create a temporary reference and return a creation success response to the logical data block; if there is a temporary reference corresponding to the reference key value, it will directly return a creation success response. For example, logical data block A initiates a "create reference (key = key1)" request to physical disk block B. Physical disk block B does not find the corresponding temporary reference, creates a temporary reference (key = key1) and then notifies logical data block A that the creation is successful. After receiving the response, logical data block A performs a reference creation operation in its own data block and then sends an update notification to physical disk block B. After receiving the notification, physical disk block B checks and finds that the temporary reference (key = key1) exists, deletes the temporary reference and increments the reference count, completing the reference creation process. If an exception such as a lost response occurs during this process, causing logical data block A to resend the request, as long as the temporary reference already exists in physical disk block B, it will still return a creation success response and the subsequent processing flow remains the same, thus avoiding the repeated creation of temporary references.

[0103] When a logical data block initiates a reference deletion request, the physical disk block also first checks whether there is a temporary reference corresponding to the reference key value. If there is a temporary reference corresponding to the reference key value, it deletes the temporary reference and returns a deletion success response; if there is no temporary reference corresponding to the reference key value, it directly returns a deletion success response. For example, logical data block C sends a "delete reference (key = key1)" request to physical disk block B. Physical disk block B finds the corresponding temporary reference, deletes it and returns a deletion success response to logical data block C. After receiving the response, logical data block C performs a reference deletion operation and then sends an update notification. After receiving the notification, physical disk block B confirms that the temporary reference has been deleted and decrements the reference count. If an exception such as a lost response occurs and the request is resent, the processing flow is similar, thus preventing the repeated deletion of temporary references.

[0104] Furthermore, in terms of data writing, when a logical data block receives a user's data writing request, it will compare the data to be written with the stored content in the physical disk block. If the data to be written does not match the stored content in the physical disk block, it will write the data in the physical disk block; if the data to be written matches the stored content in the physical disk block, it will not write the data in the physical disk block and will initiate a reference creation request to the physical disk block with the reference key value, thus optimizing the utilization efficiency of storage resources.

[0105] Furthermore, the system verifies temporary references. During the first verification period (for temporary references created when a reference is created) or the second verification period (for temporary references deleted when a reference is deleted), if the temporary reference has not been deleted, the physical disk block initiates a verification request to the logical data block. If the logical data block still requires a reference, the physical disk block deletes the temporary reference and increases or decreases the reference count accordingly, ensuring timely cleanup of residual references and ensuring efficient use of storage resources. It is understood that when the reference count reaches 0, the physical disk block deletes its stored content to further free up related resources.

[0106] In other words, the block storage system has transitioned from persistently storing reference key values to lightweight reference count maintenance. When a logical data block references a physical disk block, it initiates a request with the reference key value. These requests are categorized into two types: create reference requests and delete reference requests. When initiating a create reference request, the physical disk block first checks for a temporary reference. If none exists, it creates and responds; if one exists, it responds directly. After receiving the response, the logical data block executes the operation and sends an update notification. After receiving the notification, the physical disk block deletes the temporary reference and increments the reference count. When initiating a delete reference request, the physical disk block also first checks for a temporary reference. If one exists, it deletes and responds; if none exists, it responds directly. After receiving the response, the logical data block executes the operation and sends a notification. After the physical disk block confirms the response, it decrements the reference count. This shift in the block storage system from persistently storing reference key values to lightweight reference count maintenance reduces metadata resource space usage, reduces resource consumption for operations such as metadata queries, improves metadata management efficiency, and ultimately optimizes the performance and efficiency of the distributed storage system.

[0107] Furthermore, as the application scope of distributed storage systems continues to expand, storage nodes are widely distributed in different regions. The network conditions and storage resources of each region vary significantly. This application builds an intelligent task allocation system through in-depth analysis of regional characteristics. For areas with high network latency, local reference requests are prioritized to avoid delays caused by long-distance data transmission and ensure efficient local data access. For areas with tight storage resources, the reference storage structure is optimized, such as using a more compact data encoding method to reduce the space occupied by reference storage and improve storage utilization. Ultimately, a comprehensive improvement in the performance of cross-regional distributed storage systems is achieved.

[0108] Correspondingly, the embodiment of the present application also discloses a storage unit reference state maintenance device, see Figure 9 As shown, the device includes:

[0109] A request initiation and processing module 11 is configured to initiate a reference operation request to a physical storage unit by carrying a reference key value through a logical storage unit, and the physical storage unit processes the reference operation request based on the reference key value; wherein, the reference key value is used to ensure the idempotency of the reference count, and the reference count is used to record the cumulative number of times the physical storage unit is referenced by different logical storage units;

[0110] A local operation execution module 12 is configured to, if the logical storage unit receives an operation success response returned by the physical storage unit, execute a reference operation corresponding to the reference operation request in the logical storage unit;

[0111] A count update trigger module 13 is configured to send an update notification to the physical storage unit through the logical storage unit, so that after receiving the update notification, the physical storage unit deletes the reference key value and updates the reference count, so as to maintain the reference status of the physical storage unit.

[0112] Among them, for the more specific working processes of the above-mentioned various modules, reference can be made to the corresponding content disclosed in the foregoing embodiments, and details will not be elaborated herein.

[0113] It can be seen that the present application proposes a storage unit reference status maintenance device, which is applied to a distributed storage system. The distributed storage system includes a logical storage unit and a physical storage unit. The device includes: a request initiation and processing module 11, which is configured to initiate a reference operation request to the physical storage unit by carrying a reference key value through the logical storage unit, and the physical storage unit processes the reference operation request based on the reference key value; wherein, the reference key value is used to ensure the idempotency of the reference count, and the reference count is used to record the cumulative number of times the physical storage unit is referenced by different logical storage units; a local operation execution module 12, which is configured to, if the logical storage unit receives an operation success response returned by the physical storage unit, execute a reference operation corresponding to the reference operation request in the logical storage unit; a count update trigger module 13, which is configured to send an update notification to the physical storage unit through the logical storage unit, so that after receiving the update notification, the physical storage unit deletes the reference key value and updates the reference count, so as to convert from a persistent reference key value to maintaining a reference count.

[0114] Beneficial effects: In this application, the logical storage unit initiates a reference operation request to the physical storage unit by carrying a reference key value, and the physical storage unit processes the reference operation request based on the reference key value to ensure the idempotency of the reference count. When the logical storage unit receives an operation success response, it executes the reference operation corresponding to the reference operation request and sends an update notification to the physical storage unit, so that the physical storage unit deletes the temporary reference key value after receiving the notification, and maintains the cumulative number of times the physical storage unit is referenced through the reference count, realizing the conversion from persistently storing the reference key value to lightweight maintenance of the reference count. Compared with the traditional solution that needs to persistently store the reference key value of each logical storage unit, this application ensures the operation idempotency by temporarily using the reference key value, deletes the reference key value after completing the reference operation, and then maintains the referenced state of the physical storage unit through the reference count. In this way, this application gets rid of the influence of the number of references on the metadata space, avoids the problem of a sharp increase in metadata storage overhead caused by the expansion of the reference scale, and also reduces the resource consumption of operations such as metadata query, thus greatly improving the metadata management efficiency, and further optimizing the performance and benefits of the distributed storage system.

[0115] Furthermore, an embodiment of this application also provides a computer program product, including computer programs / instructions, which, when executed by a processor, implement the steps of the foregoing method for maintaining the reference state of the storage unit.

[0116] Furthermore, an embodiment of this application also provides an electronic device. Figure 10 It is a structural diagram of an electronic device 20 shown according to an exemplary embodiment, and the content in the figure should not be considered as any limitation to the scope of use of this application.

[0117] Figure 10 It is a schematic structural diagram of an electronic device 20 provided by an embodiment of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a display screen 23, an input / output interface 24, a communication interface 25, a power supply 26, and a communication bus 27. Among them, the memory 22 is used to store computer programs, and the computer programs are loaded and executed by the processor 21 to implement the relevant steps in the foregoing method for maintaining the reference state of the storage unit disclosed in any one of the embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0118] In this embodiment, the power supply 26 is used to provide operating voltages for various hardware devices on the electronic device 20; the communication interface 25 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and specific limitations thereof are not provided herein; the input / output interface 24 is used to obtain external input data or output data to the outside, and the specific interface type can be selected according to specific application requirements, and specific limitations are not provided herein.

[0119] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, a random access memory, a magnetic disk, an optical disc, etc. The resources stored thereon can include a computer program 221, and the storage method can be transient storage or permanent storage. Among them, in addition to the computer program that can be used to complete the storage unit reference status maintenance method executed by the electronic device 20 disclosed in any of the foregoing embodiments, the computer program 221 can further include a computer program that can be used to complete other specific tasks.

[0120] Furthermore, an embodiment of the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the storage unit reference status maintenance method disclosed above is implemented.

[0121] For the specific steps of this method, reference can be made to the corresponding content disclosed in the foregoing embodiments, and details are not repeated herein.

[0122] The various embodiments in this application are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts between the various embodiments, reference can be made to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and reference can be made to the description in the method part for related parts.

[0123] Those skilled in the art can further realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described according to their functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0124] The steps of the methods or algorithms described in connection with the embodiments disclosed in this specification may be implemented directly in hardware, in software modules executed by a processor, or in a combination thereof. The software modules may be located in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium well known in the art.

[0125] Finally, it should also be noted that in this specification, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variation thereof is intended to cover a non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the phrase "comprising a..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the above element.

[0126] The above has provided a detailed introduction to a method, apparatus, product, device and medium for maintaining the reference status of a storage unit provided in this application. Specific examples are used in this specification to elaborate on the principles and implementation manners of this application. The description of the above embodiments is only used to help understand the method and its core idea of this application; at the same time, for those of ordinary skill in the art, according to the idea of this application, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to this application.

Claims

1. A method for maintaining the reference state of a storage unit, characterized in that, Applied to a distributed storage system, the distributed storage system includes a logical storage unit and a physical storage unit, and the method includes: Initiating a reference operation request to the physical storage unit by the logical storage unit carrying a reference key value, and processing the reference operation request by the physical storage unit based on the reference key value; wherein, the reference key value is used to ensure the idempotency of the reference count, and the reference count is used to record the cumulative number of times the physical storage unit is referenced by different logical storage units; If the logical storage unit receives an operation success response returned by the physical storage unit, perform a reference operation corresponding to the reference operation request in the logical storage unit; Send an update notification to the physical storage unit through the logical storage unit, so that after receiving the update notification, the physical storage unit deletes the reference key value and updates the reference count to maintain the reference status of the physical storage unit.

2. The method for maintaining the reference status of a storage unit according to claim 1, wherein The reference operation request includes a reference creation request and a reference deletion request.

3. The method for maintaining the reference status of a storage unit according to claim 2, wherein If the reference operation request is the reference creation request, then the processing of the reference operation request by the physical storage unit based on the reference key value includes: Query by the physical storage unit whether there is a temporary reference corresponding to the reference key value; wherein, the temporary reference is a temporary record created to identify the reference relationship between the logical storage unit and the physical storage unit, and the temporary reference includes the reference key value; If there is no such temporary reference, create the temporary reference and return a creation success response to the logical storage unit; If there is such a temporary reference, directly return a creation success response to the logical storage unit.

4. The method for maintaining the reference state of a storage unit according to claim 3, wherein Deleting the reference key value and updating the reference count includes: Query by the physical storage unit whether there is such a temporary reference; If there is such a temporary reference, delete the reference key value by deleting the temporary reference and increment the reference count by 1.

5. The method for maintaining the reference state of a storage unit according to claim 4, wherein Further includes: Judge whether the temporary reference is deleted within a first verification time; If the temporary reference is not deleted within the first verification time, initiate a verification request to the logical storage unit by the physical storage unit; If the verification result returned by the logical storage unit indicates that the logical storage unit still has a reference requirement for the physical storage unit, delete the temporary reference and increment the reference count by 1.

6. The method for maintaining the reference status of a storage unit according to claim 2, wherein If the reference operation request is the reference deletion request, then the processing of the reference operation request by the physical storage unit based on the reference key value includes: Query by the physical storage unit whether there is a temporary reference corresponding to the reference key value; wherein, the temporary reference is a temporary record created to identify the reference relationship between the logical storage unit and the physical storage unit, and the temporary reference includes the reference key value; If there is such a temporary reference, delete the temporary reference and return a deletion success response to the logical storage unit; If the temporary reference does not exist, directly return a deletion success response to the logical storage unit.

7. The method for maintaining the storage unit reference status according to claim 6, wherein Deleting the reference key value and updating the reference count includes: Querying through the physical storage unit whether the temporary reference exists; If the temporary reference exists, deleting the reference key value is achieved by deleting the temporary reference, and the reference count is decremented by 1.

8. The method for maintaining the reference state of a storage unit according to claim 7, characterized in that, It further includes: Determining whether the temporary reference has been deleted within the second verification time; If the temporary reference has not been deleted within the second verification time, initiate a verification request to the logical storage unit through the physical storage unit; If the verification result returned by the logical storage unit indicates that the logical storage unit still has a reference requirement for the physical storage unit, delete the temporary reference and decrement the reference count by 1.

9. The method for maintaining the reference state of a storage unit according to claim 7, wherein It further includes: Determining whether the reference count has been decremented to 0; If the reference count has been decremented to 0, delete the stored content in the physical storage unit and release the relevant resources occupied by the stored content.

10. The method for maintaining the reference status of a storage unit according to claim 3, wherein It further includes: When receiving a data writing request initiated by a user through the logical storage unit, determine whether the data to be written in the data writing request is the same as the stored content in the physical storage unit; Determine whether to write the data to be written in the physical storage unit according to the judgment result.

11. The method for maintaining the reference status of a storage unit according to claim 10, characterized in that, The determining whether to write the data to be written in the physical storage unit according to the judgment result includes: If the data to be written is not the same as the stored content in the physical storage unit, write the data to be written in the physical storage unit.

12. The method for maintaining the storage unit reference status according to claim 10, wherein The determining whether to write the data to be written in the physical storage unit according to the judgment result includes: If the data to be written is the same as the stored content in the physical storage unit, there is no need to write the data to be written in the physical storage unit, and jump to the step of initiating the reference creation request to the physical storage unit by carrying the reference key value through the logical storage unit.

13. The method for maintaining the reference status of a storage unit according to claim 1, wherein The update notification is sent using an asynchronous communication mechanism.

14. The method for maintaining the reference state of a storage unit according to claim 1, wherein The numerical type of the reference count is an integer type.

15. The method for maintaining the reference status of a storage unit according to claim 1, wherein During the process of deleting the reference key value and updating the reference count, it further includes: Encapsulating the operation of deleting the reference key value and the operation of updating the reference count into the same database transaction to ensure that the operation of deleting the reference key value and the operation of updating the reference count take effect simultaneously.

16. The method for maintaining the reference status of a storage unit according to any one of claims 1 to 15, characterized in that, The logical storage unit includes a file in a file storage system, a logical data block in a block storage system, an object in an object storage system, or a virtual volume in a virtualized storage system; the physical storage unit includes a physical disk block, a solid-state drive storage page, or a persistent memory unit.

17. A storage unit reference status maintenance device, characterized in that, Applied to a distributed storage system, the distributed storage system includes a logical storage unit and a physical storage unit, and the device includes: A request initiation and processing module, configured to initiate a reference operation request to the physical storage unit by carrying a reference key value through the logical storage unit, and the physical storage unit processes the reference operation request based on the reference key value; wherein, the reference key value is used to ensure the idempotency of the reference count, and the reference count is used to record the cumulative number of times the physical storage unit is referenced by different logical storage units; A local operation execution module, configured to, if the logical storage unit receives an operation success response returned by the physical storage unit, perform a reference operation corresponding to the reference operation request in the logical storage unit; A count update trigger module, configured to send an update notification to the physical storage unit through the logical storage unit, so that after receiving the update notification, the physical storage unit deletes the reference key value and updates the reference count, so as to maintain the reference status of the physical storage unit.

18. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by a processor, the steps of the method for maintaining the reference status of a storage unit according to any one of claims 1 to 16 are implemented.

19. An electronic device, characterized in that, Comprising: A memory, configured to store a computer program; A processor, configured to execute the computer program to implement the method for maintaining the reference status of a storage unit according to any one of claims 1 to 16.

20. A computer-readable storage medium, characterized in that, For storing a computer program; wherein, when the computer program is executed by a processor, the method for maintaining the reference status of a storage unit according to any one of claims 1 to 16 is implemented.

Images