Distributed storage system client cache synchronization method, device and storage medium

By dividing files into basic data units and configuring cap permissions based on identity identification information, the problem of repeated data transmission during file synchronization in distributed storage systems is solved, improving efficiency and optimizing resource usage.

CN113672570BActive Publication Date: 2024-11-29JINAN INSPUR DATA TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202110779557.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-07-09
Publication Date
2024-11-29
Estimated Expiration
2041-07-09

AI Technical Summary

Technical Problem

In distributed storage systems, during the client cache synchronization process, a large number of duplicate data transmission results in file-level updates, reducing efficiency and wasting resources.

Method used

Split the file into several basic data units, and configure cap permissions based on the identity identification information of each basic data unit, and only synchronize when the content changes to reduce invalid data transmission.

Benefits of technology

Improve the efficiency of file synchronization, reduce the resource occupation of distributed storage system, and optimize the synchronization process.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113672570B_ABST
    Figure CN113672570B_ABST
Patent Text Reader

Abstract

The present application relates to a distributed storage system client cache synchronization method, device and storage medium. The distributed storage system client cache synchronization method includes: dividing a file into a number of basic data units according to a set length and storing them in a distributed storage system; the distributed storage system creates a CAP permission in response to a CAP permission request, and the distributed storage system controls the synchronization of the corresponding basic data units in the corresponding client cache through the CAP permission. Among them, the device and storage medium for implementing the distributed storage system client cache synchronization method are used to implement the distributed storage system client cache synchronization method. The present application implements the refinement of the CAP permission into the basic data unit level. During the file synchronization process, the synchronized file only needs to synchronize the basic data units in the file whose content has changed. There is no need to transmit a large amount of unchanged data between the distributed storage system and the client, which improves the efficiency of file synchronization and reduces the resource occupation of the distributed storage system.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of distributed storage systems, and in particular to a distributed storage system client cache synchronization method, device and storage medium. Background Art

[0002] In a distributed storage system, when a client accesses a file in the distributed storage system, the file data of the distributed storage system is first cached in the client's local cache. In order to ensure data consistency, the client will be configured with the cap permission for the file; the client that obtains the cap permission will access the data of the file in the distributed storage system in a synchronous read-write manner to ensure that the content of the file in each client's cache is consistent with the data of the file in the storage system.

[0003] In the prior art, since the CAP permission is configured for files, when one of the clients with the CAP permission for a file modifies a part of the data in the file, all the clients with the CAP permission for the file are required to update the files in their local caches to the latest file data stored in the distributed storage system. Since the update is in units of files, this results in a large amount of unmodified file content in the client's local cache also being replaced by the latest file data in the distributed file storage system. During the cache synchronization process, the transmission process of a large amount of duplicate data occupies a large amount of resources of the distributed storage system. On the one hand, the efficiency of client cache synchronization is reduced, and on the other hand, the resources of the distributed storage system are wasted. Summary of the invention

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the present application provides a distributed storage system client cache synchronization method, device and storage medium.

[0005] In a first aspect, the present application provides a distributed storage system client cache synchronization method, comprising:

[0006] Split the file into several basic data units according to the set length and store them in the distributed storage system;

[0007] The client sends a CAP permission request for the operated data to the distributed storage system; the CAP permission request includes a target basic data unit and client identification information;

[0008] The distributed storage system creates a cap permission according to the cap permission request;

[0009] The distributed storage system records the client identity information corresponding to each basic data unit according to the created cap authority;

[0010] When the content of the basic data unit changes, the target client is determined according to the client identification information corresponding to the basic data unit, and the target client is controlled so that the target client synchronizes the changed basic data unit to the client local cache.

[0011] Furthermore, dividing the file into a number of basic data units according to a set length and storing them in a distributed storage system includes:

[0012] Separating the file into several groups of valid data;

[0013] Each group of valid data is stored in data blocks with the same capacity respectively;

[0014] A corresponding block header is configured for the data block to form the basic data unit.

[0015] Furthermore, a mapping relationship between a first offset of the client cache file and a second offset of a corresponding basic data unit stored in the distributed storage system is established on the client, and the mapping relationship makes the operated data of the client cache file correspond to the basic data unit in the distributed storage system.

[0016] Furthermore, the first offset maps the physical address range of the cache file in the cache; the second offset maps the physical address range of the basic data unit in the distributed storage system, and the second offsets of the basic data units corresponding to the same file are set to be continuous.

[0017] Furthermore, the client sends a cap permission request for the operated data to the distributed storage system, including:

[0018] The client constructs a target basic data unit according to the operated data;

[0019] Obtain client identification information of the client;

[0020] The client creates a cap permission request using the target basic data unit and the client identity information and sends it to the distributed storage system.

[0021] Furthermore, the client uses the range of the first offset of the operated data to obtain the range of the second offset from the mapping relationship query, and then determines the target basic data unit according to the range of the second offset.

[0022] Furthermore, the distributed storage system creates the cap permission according to the cap permission request including:

[0023] The distributed storage system parses the cap permission request to obtain the target basic data unit and client identity identification information;

[0024] The distributed storage system configures cap permissions according to target basic data units and client identity identification information.

[0025] Furthermore, the distributed storage system records the client identity information corresponding to each basic data unit according to the created cap authority, including:

[0026] The distributed storage system parses the included basic data units according to the target basic data unit parameters in the cap authority;

[0027] The distributed storage system determines the client corresponding to each basic data unit according to the client identity identification information parameter in the cap authority;

[0028] Automatically record the client identification information corresponding to each basic data unit.

[0029] In a second aspect, the present application provides a device for implementing a distributed storage system client cache synchronization method, comprising: a distributed storage system,

[0030] and a client connected to the distributed storage system;

[0031] Wherein, the client includes a cache module, and the cache module is used to cache files;

[0032] The client comprises a target basic data unit construction module, the target basic data unit construction module is used to calculate the starting second offset of the starting basic data unit in the basic data unit corresponding to the operated data according to the first offset of the cache of the operated data, the target basic data unit construction module obtains the length of the basic data unit corresponding to the operated data; the target basic data unit construction module constructs the target basic data unit according to the starting second offset and the length of the basic data unit corresponding to the operated data;

[0033] The client comprises a cap permission request module, wherein the cap permission request module is used to generate a cap permission request including the target basic data unit and client identity identification information;

[0034] The client comprises a synchronization module, and the synchronization module synchronizes the content of the corresponding basic data unit to the local cache module in response to the synchronization instruction of the distributed storage system;

[0035] Wherein, the distributed storage system comprises a cap permission configuration module, and the cap permission configuration module creates cap permissions according to the cap permission request of the permission request module;

[0036] The distributed storage system includes a basic data unit monitoring module, and the basic data unit monitoring module monitors the basic data units with modified contents;

[0037] The distributed storage system includes a recording module, which is used to record each basic data unit and its corresponding client;

[0038] The distributed storage system includes a target client determination module, which determines the target client according to the recording module and the basic data unit monitoring unit;

[0039] The distributed storage system includes a synchronization notification module, which sends a synchronization instruction to a target client.

[0040] In a third aspect, the present application provides a storage medium for implementing a distributed storage system client cache synchronization method, wherein the storage medium for implementing a distributed storage system client cache synchronization method stores at least one instruction, and executing the instruction implements the distributed storage system client cache synchronization method.

[0041] The above technical solution provided by the embodiment of the present application has the following advantages compared with the prior art:

[0042] The client of the present application divides the file into several basic data units according to a certain length and stores them in the distributed storage system; the distributed storage system of the present application can configure the CAP permission for the basic data unit, so that when the client obtains the CAP permission of the corresponding basic data unit, the file synchronization can be achieved only by synchronizing the basic data units whose contents have changed in the file. Therefore, on the one hand, the present application can greatly reduce the amount of data communicated between the distributed storage system and the client during the file synchronization process, and improve the efficiency of file synchronization; on the other hand, the present application can also reduce the resource occupation of the distributed storage system by the communication process between the client and the distributed storage system during the file synchronization process, and ensure the performance of the distributed storage system. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0045] Figure 1A flowchart of a distributed storage system client cache synchronization method provided in an embodiment of the present application;

[0046] Figure 2 A flowchart of dividing a file into several basic data units according to a set length and storing them in a distributed storage system provided in an embodiment of the present application;

[0047] Figure 3 A flowchart of a client sending a cap permission request for the operated data to the distributed storage system provided in an embodiment of the present application;

[0048] Figure 4 A flowchart of the distributed storage system providing an embodiment of the present application creating a cap permission according to a cap permission request;

[0049] Figure 5 A flowchart of the distributed storage system provided in an embodiment of the present application recording the client identity information corresponding to each basic data unit according to the created cap authority;

[0050] Figure 6 A flowchart of controlling a target client to synchronize the changed basic data unit to a local cache of the client when the content of the basic data unit provided in an embodiment of the present application changes;

[0051] Figure 7 A diagram of the device architecture for implementing the distributed storage system client cache synchronization method provided in an embodiment of the present application;

[0052] Figure 8 An architectural diagram of a client provided in an embodiment of the present application;

[0053] Fig. 9 An architectural diagram of a distributed storage system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0055] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0056] First, the relevant concepts involved in this application are introduced. Data blocks are the smallest units for reading and writing data in a distributed storage system. When a client accesses data stored in a distributed storage system, it accesses data blocks as the basic unit; the distributed storage system configures a fixed size of data blocks. A fixed-capacity data block includes a block header, free space, and data space. The block header is used to store the physical address of the data block, the type of the data block, and the data block-related parameters, and the free space and data space are used to store valid data. When the valid data in the data block is modified to increase the length of the valid data, the capacity of the free space of the data block is reduced, and the capacity of the data space is increased; when the valid data in the data block is modified to reduce the length of the valid data, the free space capacity of the data block is increased, and the data space capacity is reduced.

[0057] Example 1

[0058] This embodiment provides a distributed storage system client cache synchronization method. Figure 1 As shown, the distributed storage system client cache synchronization method includes:

[0059] S100, dividing the file into a number of basic data units according to a set length and storing them in a distributed storage system.

[0060] During the specific implementation process, see Figure 2 As shown, dividing a file into several basic data units according to a set length and storing them in a distributed storage system includes:

[0061] S101, dividing the file into several groups of valid data;

[0062] S102, storing each group of valid data in a corresponding data space of a data block with the same capacity in the distributed storage system;

[0063] S103, configuring a corresponding block header for the data block to form the basic data unit. In the embodiment of the present application, the block header of the basic data unit includes the starting second offset and unit length of the basic data unit, and the unit length of the basic data unit is the capacity of the data block; when the client connected to the distributed storage system caches the basic data unit locally, it can obtain the starting second offset and unit length from the block header of the basic data unit.

[0064] S200, the client sends a cap permission request for the operated data to the distributed storage system;

[0065] During the specific implementation process, see Figure 3 As shown, the client sends a cap permission request for the operated data to the distributed storage system, including:

[0066] S201, the client constructs a target basic data unit according to the operated data;

[0067] Specifically, constructing the target basic data unit according to the operated data requires that the operated data correspond to the basic data unit. Therefore, in an embodiment of the present application, a second offset of the basic data unit is set, and the second offset maps the physical address range of the basic data unit in the distributed storage system; in an embodiment of the present application, a first offset of the client cache file is set, and the first offset maps the physical address range of the cache file in the client cache; and a mapping relationship between the first offset and the second offset is established on the client. In order to facilitate the representation of the target data unit during the specific implementation process, the second offsets of the basic data units corresponding to the same file are set to be continuous in this application.

[0068] The range of the second offset is obtained from the mapping relationship query using the range of the first offset of the operated data, and then the target basic data unit is determined according to the range of the second offset. Specifically, determining the target basic data unit according to the range of the second offset includes: using the range of the second offset to determine the starting second offset of the starting basic data unit in the basic data unit corresponding to the operated data, and in the embodiment of the present application, start_block_start_offset is used to represent the starting second offset of the starting basic data unit in the basic data unit corresponding to the operated data; using the range of the second offset to determine the length of the continuous basic data unit corresponding to the operated data, and in the embodiment of the present application, BLOCKS_FILE_SIZE is used to represent the length of the continuous basic data unit corresponding to the operated data; and using the starting second offset and length as parameters to construct the target basic data unit. It should be understood that when the second offsets of all the basic data units corresponding to the operands are discontinuous, multiple target basic data units need to be constructed.

[0069] S202, obtaining the client identification information of the client; the client identification information is a unique identification number used to distinguish the client, and in the embodiment of the present application, the client identification information is represented by clientID.

[0070] S203, the client creates a CAP permission request using the target basic data unit and the client identity information and sends it to the distributed storage system. The client adds the target basic data unit and the client identity information to a CAP permission request data frame according to a set format and sends it to the distributed storage system.

[0071] S300, the distributed storage system creates a cap permission according to the cap permission request;

[0072] During the specific implementation process, see Figure 4 As shown, the distributed storage system creates the cap permission according to the cap permission request, including:

[0073] S301, the distributed storage system parses the cap permission request to obtain the target basic data unit and client identity information;

[0074] S302, the distributed storage system configures cap permissions with the target basic data unit and client identification information as parameters. This application uses cap to represent cap permissions, cap = {start_block_start_offset, BLOCKS_FILE_SIZE, clientID}, which means: for a number of basic data units starting with the second offset of start_block_start_offset and with a total length of BLOCKS_FILE_SIZE, configure cap permissions to the client with clientID as the client identification information.

[0075] It should be understood that when the target basic data unit contains only one basic data unit, the starting second offset of the starting basic data unit is equivalent to the starting second offset block_start_offset of the basic data unit; the length BLOCKS_FILE_SIZE is equivalent to the unit length BLOCK_SIZE of the basic data unit. At this time, cap can be expressed as cap = {block_start_offset, BLOCK_SIZE, clientID}. The meaning is that for the basic data unit with block_start_offset as the starting second offset, the cap permission is configured to the client with clientID as the client identity identification information. It can be seen that the cap permission configuration of the present application can be refined to the basic data unit level.

[0076] S400, the distributed storage system records the client identity information corresponding to each basic data unit according to the created cap authority;

[0077] During the specific implementation process, see Figure 5 As shown, the distributed storage system records the client identity information corresponding to each basic data unit according to the created cap authority, including;

[0078] S401, the distributed storage system parses the basic data units contained therein according to the target basic data unit parameters in the cap authority;

[0079] The target basic data unit parameters include the starting second offset start_block_start_offset of the starting basic data unit in the basic data units corresponding to the operated data, and the length BLOCKS_FILE_SIZE of the continuous basic data units corresponding to the operated data.

[0080] In the embodiment of the present application, each basic data unit is represented by the starting second offset block_start_offset of the basic data unit and the unit length BLOCK_SIZE of the basic data unit; the basic data unit is represented by blockid, blockid={block_start_offset, BLOCK_SIZE}.

[0081] start_block_start_offset divides the target basic data unit into BLOCKS_FILE_SIZE divided by BLOCK_SIZE parts with BLOCK_SIZE as the step size, and obtains the second starting offset of each part as the corresponding block_start_offset, so as to parse BLOCKS_FILE_SIZE divided by BLOCK_SIZE basic data units from the target basic data unit parameters in the cap permission.

[0082] S402, the distributed storage system determines the client corresponding to each basic data unit according to the client identity identification information parameter in the cap authority; specifically, the client corresponding to the target basic data unit is consistent with the client corresponding to the corresponding parsed basic data unit.

[0083] S403, automatically record the client identity identification information corresponding to each basic data unit. In the embodiment of the present application, a map container is used to automatically record each basic data unit and its corresponding client identity identification information. The map container is represented by map = {blockid, [clientID1, clientID2, ...clientIDn]}, where [clientID1, clientID2, ...clientIDn] is a clientID sequence; the map container uses the clientID sequence to slice the client identity identification information clientID. When a new client is added to the cap permission of a basic data unit, the corresponding clientID in the map of the basic data unit is increased by adding a slice. When a client is deleted from the cap permission of a basic data unit, the corresponding clientID in the map of the basic data unit is deleted by deleting a slice.

[0084] S500, when the content of the basic data unit changes, determine the target client according to the client identification information corresponding to the basic data unit, and control the target client so that the target client synchronizes the changed basic data unit to the client local cache.

[0085] During the specific implementation process, see Figure 6 As shown, step S500 includes:

[0086] S501, the distributed storage system determines whether the content of the basic data unit has changed, and if so, executes step S502. In the specific implementation process, the file cached locally by the client is modified, and the client uses the cap authority to modify the corresponding basic data unit according to the modified content of the local cache file to synchronize the modified content to the corresponding basic data unit of the distributed storage system. The present application can monitor the modification behavior of the client on the basic data unit in the distributed storage system to determine whether the basic data content has changed.

[0087] S502, determining a target client according to the client identification information corresponding to the basic data unit, and controlling the target client so that the target client synchronizes the changed basic data unit to the client local cache.

[0088] During the synchronization process, once the content of a basic data unit is changed due to modification by the client, the client identity information of the client that modifies the basic data unit is obtained, the blockid of the basic data unit is obtained, the map of the basic data unit is found according to the blockid of the basic data unit, and then the target client is found according to map = {blockid, [clientID1, clientID2, ... clientIDn]}. The target client is all the clients in the map of the basic data unit except the client that modifies the basic data unit. The distributed storage system sends a synchronization instruction to the target client, and the target client synchronizes the modified basic data unit to the local cache after receiving the synchronization instruction.

[0089] It can be understood that when the contents of multiple basic data units are changed, step 502 is performed for each basic data unit respectively.

[0090] Example 2

[0091] The difference between Example 2 and Example 1 is that:

[0092] When the client sends a cap permission request for the operated data to the distributed storage system,

[0093] Different from the target basic data unit in comparative document 1; the target basic data unit of embodiment 2 is directly composed of several basic data units, and the target basic data unit in embodiment 2 can be expressed as {[block_start_offset1 (the starting second offset of the basic data unit), BLOCK_SIZE], ... [block_start_offsetN (the starting second offset of the basic data unit), BLOCK_SIZE]}. In the specific implementation process, embodiment 2 uses the first offset range to obtain the second offset range according to the mapping relationship to further determine the basic data unit corresponding to the operated data.

[0094] Based on the above changes, in Example 2, when the distributed storage system creates a cap permission according to the cap permission request,

[0095] The cap permission of Example 2 can be expressed as {[block_start_offset1 (the second starting offset of the basic data unit), BLOCK_SIZE], ... [block_start_offsetN (the second starting offset of the basic data unit), BLOCK_SIZE]; clientID}. The basic data unit to which the cap permission needs to be granted to the client is indicated before the semicolon, and the client to which the permission needs to be granted is indicated after the semicolon.

[0096] Directly configure the cap permissions for the corresponding basic data units to the corresponding clients according to each basic data unit in the target data unit.

[0097] Based on the above changes, in Example 2, when the distributed storage system records the client identity information corresponding to each basic data unit according to the created cap authority,

[0098] The Map container directly traverses each basic data unit parameter in the cap permission and records the corresponding client identity information.

[0099] Example 3

[0100] The present application embodiment provides a device for implementing a distributed storage system client cache synchronization method. Figure 7 As shown, the device for implementing the distributed storage system client cache synchronization method includes: a distributed storage system, and a client connected to the distributed storage system.

[0101] Wherein, the client includes a cache module, and the cache module is used to cache files;

[0102] The client includes a target basic data unit construction module, the target basic data unit construction module is used to calculate the starting second offset of the starting basic data unit in the basic data unit corresponding to the operated data according to the first offset range of the operated data cached by the client, and the target basic data unit construction module obtains the length of the basic data unit corresponding to the operated data; the target basic data unit construction module constructs the target basic data unit according to the starting second offset and the length of the basic data unit corresponding to the operated data;

[0103] The client comprises a cap permission request module, wherein the cap permission request module is used to generate a cap permission request including the target basic data unit and client identity identification information;

[0104] The client comprises a synchronization module, and the synchronization module synchronizes the content of the corresponding basic data unit to the local cache module in response to the synchronization instruction of the distributed storage system;

[0105] Wherein, the distributed storage system comprises a cap permission configuration module, and the cap permission configuration module creates cap permissions according to the cap permission request of the permission request module;

[0106] The distributed storage system includes a basic data unit monitoring module, and the basic data unit monitoring module monitors the basic data units with modified contents;

[0107] The distributed storage system includes a recording module, which is used to record each basic data unit and its corresponding client;

[0108] The distributed storage system includes a target client determination module, which determines the target client according to the recording module and the basic data unit monitoring unit;

[0109] The distributed storage system includes a synchronization notification module, which sends a synchronization instruction to a target client.

[0110] Example 4

[0111] An embodiment of the present application provides a storage medium for implementing a distributed storage system client cache synchronization method. The storage medium for implementing a distributed storage system client cache synchronization method stores at least one instruction, and executing the instruction implements the distributed storage system client cache synchronization method.

[0112] The client of the present application divides the file into several basic data units according to a certain length and stores them in the distributed storage system; the distributed storage system of the present application can configure the CAP permission for the basic data unit, so that when the client obtains the CAP permission of the corresponding basic data unit, the file synchronization can be achieved only by synchronizing the basic data units whose contents have changed in the file. Therefore, on the one hand, the present application can greatly reduce the amount of data communicated between the distributed storage system and the client during the file synchronization process, and improve the efficiency of file synchronization; on the other hand, the present application can also reduce the resource occupation of the distributed storage system by the communication process between the client and the distributed storage system during the file synchronization process, and ensure the performance of the distributed storage system.

[0113] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0114] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0115] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0116] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A distributed storage system client cache synchronization method, characterized in that: include: Establishing a mapping relationship between a first offset of a client cache file and a second offset of a corresponding basic data unit stored in a distributed storage system on a client, wherein the mapping relationship makes the operated data of the client cache file correspond to the basic data unit in the distributed storage system, wherein the first offset maps a physical address range of the cache file in the cache; the second offset maps a physical address range of the basic data unit in the distributed storage system, and the second offsets of the basic data units corresponding to the same file are set to be continuous; Split the file into several basic data units according to the set length and store them in the distributed storage system; The client sends a CAP permission request for the operated data to the distributed storage system; the CAP permission request includes a target basic data unit and client identification information; wherein the client uses the range of the first offset of the operated data to obtain the range of the second offset from the mapping relationship query, and then determines the target basic data unit according to the range of the second offset; The distributed storage system creates a cap permission according to the cap permission request; The distributed storage system records the client identity information corresponding to each basic data unit according to the created cap authority; When the content of the basic data unit changes, the target client is determined according to the client identification information corresponding to the basic data unit, and the target client is controlled so that the target client synchronizes the changed basic data unit to the client local cache.

2. The distributed storage system client cache synchronization method according to claim 1, characterized in that: Splitting a file into several basic data units according to a set length and storing them in a distributed storage system includes: Separating the file into several groups of valid data; Each group of valid data is stored in data blocks with the same capacity respectively; A corresponding block header is configured for the data block to form the basic data unit.

3. The distributed storage system client cache synchronization method according to claim 1, characterized in that: The client sends a cap permission request for the operated data to the distributed storage system, including: The client constructs a target basic data unit according to the operated data; Obtain client identification information of the client; The client creates a cap permission request using the target basic data unit and the client identity information and sends it to the distributed storage system.

4. The distributed storage system client cache synchronization method according to claim 1, characterized in that: The distributed storage system creates a cap permission according to the cap permission request, including: The distributed storage system parses the cap permission request to obtain the target basic data unit and client identity identification information; The distributed storage system configures cap permissions according to target basic data units and client identity identification information.

5. The distributed storage system client cache synchronization method according to claim 4, characterized in that: The distributed storage system records the client identity information corresponding to each basic data unit according to the created cap authority, including: The distributed storage system parses the included basic data units according to the target basic data unit parameters in the cap authority; The distributed storage system determines the client corresponding to each basic data unit according to the client identity identification information parameter in the cap authority; Automatically record the client identification information corresponding to each basic data unit.

6. A device for implementing a distributed storage system client cache synchronization method, characterized in that: include: Distributed storage systems, and a client connected to the distributed storage system; Establishing a mapping relationship between a first offset of a client cache file and a second offset of a corresponding basic data unit stored in a distributed storage system on a client, wherein the mapping relationship makes the operated data of the client cache file correspond to the basic data unit in the distributed storage system, wherein the first offset maps a physical address range of the cache file in the cache; the second offset maps a physical address range of the basic data unit in the distributed storage system, and the second offsets of the basic data units corresponding to the same file are set to be continuous; Wherein, the client includes a cache module, and the cache module is used to cache files; The client includes a target basic data unit construction module, the target basic data unit construction module is used to calculate the starting second offset of the starting basic data unit in the basic data unit corresponding to the operated data according to the first offset of the cache of the operated data, and the target basic data unit construction module obtains the length of the basic data unit corresponding to the operated data; the target basic data unit construction module constructs the target basic data unit according to the starting second offset and the length of the basic data unit corresponding to the operated data; The client comprises a cap permission request module, wherein the cap permission request module is used to generate a cap permission request including the target basic data unit and client identity identification information; The client comprises a synchronization module, and the synchronization module synchronizes the content of the corresponding basic data unit to the local cache module in response to the synchronization instruction of the distributed storage system; Wherein, the distributed storage system comprises a cap permission configuration module, and the cap permission configuration module creates cap permissions according to the cap permission request of the permission request module; The distributed storage system includes a basic data unit monitoring module, and the basic data unit monitoring module monitors the basic data units with modified contents; The distributed storage system includes a recording module, which is used to record each basic data unit and its corresponding client; The distributed storage system includes a target client determination module, which determines the target client according to the recording module and the basic data unit monitoring unit; The distributed storage system comprises a synchronization notification module, which sends a synchronization instruction to a target client and controls the target client to synchronize the changed basic data unit to the client local cache.

7. A storage medium for implementing a distributed storage system client cache synchronization method, characterized in that: The storage medium for implementing the distributed storage system client cache synchronization method stores at least one instruction, and executing the instruction implements the distributed storage system client cache synchronization method as described in any one of claims 1-5.

Citation Information

Patent Citations

  • Quota statistical system and method based on CAP (Capability) mechanism of CEPH cluster

    CN107172152A

  • Authority obtaining method, device and system

    CN112307486A