Method and device for improving IO read performance based on object affinity

By optimizing data layout and access strategies in a distributed storage system, ensuring that indexes, storage, and GC objects reside on the same node, the problems of read amplification and low efficiency in cross-network transmission are resolved, thereby improving IO read performance.

CN120687043BActive Publication Date: 2025-11-04CHINA ELECTRONICS CLOUD DIGITAL INTELLIGENCE TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202511185969.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-08-22
Publication Date
2025-11-04
Estimated Expiration
2045-08-22

AI Technical Summary

Technical Problem

The existing ROW+ append-write architecture suffers from read amplification and low cross-network transmission efficiency in distributed storage systems, leading to a decrease in read performance.

Method used

By pre-configuring the affinity relationship between garbage collection (GC) objects and cluster nodes, data layout and access strategies are optimized to ensure that indexes, storage, and GC objects always reside on the same node, reducing cross-network transmission and disk read operations.

Benefits of technology

It significantly improves the IO read performance of distributed storage systems, increases cache hit rate, and reduces performance loss from data transmission across networks and reading slow media.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120687043B_ABST
    Figure CN120687043B_ABST
Patent Text Reader

Abstract

The application relates to a method and device for improving IO read performance based on object affinity. The method comprises the following steps: a certain number of GC objects are pre-configured according to the cluster size, and the affinity of the GC objects and each node is calculated; when a storage object is applied through aggregated writing or GC moving writing, the associated GC object of the storage object is determined according to the identification information of the storage object, and whether the GC object is affinitive with the target node is judged; if not, the storage object is discarded and reselected, and if yes, data is written into the storage object; read service is performed according to the local cache of the storage object in the target node. Through the combination of the read-write cache mechanism, the affinity of the cluster node and the GC object is pre-calculated, and data is written into the storage object affinitive with the GC object; even if the GC moving operation is triggered due to continuous overlay writing, the front-end read IO can still obtain effective data from the cache of the original node, the performance loss of data cross-network transmission and reading of slow media is greatly reduced, and the read performance is significantly improved.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of hard disk read-write, in particular to a method and device for improving IO read performance based on object affinity and electronic equipment. BACKGROUND

[0002] In a distributed storage system, in order to optimize the hard disk write performance, the industry usually adopts the Redirect-On-Write (ROW) technology: the data written to the hard disk pool object is redirected and aggregated into large IO, and then sequentially written to the object allocated by the hard disk pool. Specifically, when data needs to be written to different storage units of the hard disk pool, the system distributes it to different storage disks (i.e. target) based on the data layout algorithm (such as consistent hashing) to achieve load balancing. In addition, the Append-Write mechanism is often used in the field of distributed storage to further improve the write performance, and the ROW technology also uses this feature. However, appending the same data will generate redundant old version data (i.e. garbage data), so garbage collection (GC) algorithm is needed to recycle such invalid space.

[0003] Although the above technology effectively improves the write performance, the read performance problem introduced by it still needs attention: on the one hand, the aggregation characteristics of the foreground data will cause read amplification phenomenon, for example, if the data aggregation granularity is 4K, and the user only reads 1K data, the system still needs to read the minimum 4K aggregation unit from the disk, resulting in a read amplification multiple of 4, and frequent such read disk operations will significantly reduce the read performance. On the other hand, the overwrite operation will invalidate the old data of the old object, triggering the GC recycling process, and the data needs to be migrated to a new object, and due to the hash consistency mechanism of the redundancy algorithm, the migrated data will be scattered and distributed to each node; if such data is directly read, it needs to be obtained from different nodes across the network, resulting in low transmission efficiency.

[0004] In summary, the existing ROW+append write architecture performs well in terms of write performance and space utilization, but has double bottlenecks of "read amplification" and "cross-node access" in the read path, which needs to be optimized. SUMMARY

[0005] In order to solve the problems of read amplification, low cross-network transmission efficiency and other problems existing in the existing ROW+append write architecture, the present application proposes a new method for improving IO read performance based on object affinity, which aims to optimize data layout and access strategy, and effectively improve the IO read performance of the distributed storage system.

[0006] The main technical strategy of the application comprises: pre-configuring garbage collection (GC) objects based on cluster size, and pre-calculating the affinity relationship of each GC object and each node in the cluster; when a storage object is applied for storage and data is stored through an aggregated write operation or a GC garbage collection moving write operation, the system will check the affinity of the GC object associated with the storage object to be applied for storage and the target node of the storage object; if an inaffinity condition is detected, the inaffinity storage object is discarded actively; through the above mechanism, it is ultimately ensured that the index object, the storage object and the corresponding GC object always maintain affinity, and the three are located on the same node.

[0007] To achieve the above object, the application provides the following technical scheme:

[0008] The first aspect of the application provides a method for improving IO read performance based on object affinity, which comprises:

[0009] S1. A certain number of garbage collection (GC) objects are pre-configured according to the size of the distributed storage cluster, and the affinity relationship of each GC object and each node in the cluster is pre-calculated;

[0010] S2. When a storage object is applied for storage through an aggregated write or a GC moving write, the GC object associated with the storage object is determined according to the identification information (such as object ID) of the storage object, and it is judged whether the GC object is in affinity with the target node of the storage object;

[0011] S3. If the judgment result is inaffinity, the storage object is discarded actively and reselected;

[0012] S4. If the judgment result is affinity, data is written into the storage object, and the storage object, the GC object associated with it and the corresponding index object are kept in the same node;

[0013] S5. The storage object is read in the local cache of the node.

[0014] Further, in the method of the application, the preconfigured number N of GC objects in step S1 satisfies: N is an integer power of 2, and N≥node number×2000.

[0015] Further, in the method of the application, the pre-calculated affinity relationship of each GC object and each node in the cluster in step S1 comprises:

[0016] The GC objects are uniformly mapped to each node according to consistent hashing, and the GC object identification set responsible for each node is saved.

[0017] Further, in the method of the application, the judgment of whether the GC object is in affinity with the target node of the storage object in step S2 comprises:

[0018] The GC object identifier is calculated by taking the storage object identifier modulo N, and it is detected whether the identifier falls into the set of GC object identifiers saved by the current node.

[0019] Further, the method of the present application further includes the following steps in the GC moving write process:

[0020] (1) When the garbage ratio of the storage object reaches a threshold, the GC object associated with the storage object triggers a GC moving operation at the current node;

[0021] (2) The valid data is moved (read first and then written) to a newly generated affinity storage object in the same node, and the cache is updated;

[0022] (3) After the GC moving is completed, the metadata is updated to point to the newly generated affinity storage object, and the original storage object is removed from the cache.

[0023] Further, in the method of the present application, the aggregation granularity of the aggregation write in step S2 is 1 MB, and the small IO is uniformly written to the disk after the aggregation in the node local cache is completed.

[0024] Further, in the method of the present application, step S4 further includes:

[0025] Caching data to the target node: caching the data of the aggregation write to the memory or cache medium of the node where the GC object associated with the storage object is located in units of storage objects;

[0026] Recording storage object metadata: persistently storing the metadata of the storage object (including the information of the associated GC object, the cache location, etc.);

[0027] Further, in the method of the present application, step S5 further includes:

[0028] When the user reads data, the target storage object is located through the index object, and it is checked whether the storage object exists in the cache of the current node; if it exists, the data is directly read from the local cache.

[0029] The second aspect of the present application provides a device for improving IO read performance based on object affinity, which comprises:

[0030] A pre-configuration module is configured to pre-configure a certain number of garbage collection (GC) objects according to the size of the distributed storage cluster, and to pre-calculate the affinity relationship between each GC object and each node in the cluster.

[0031] An affinity checking module is configured to determine the associated GC object according to the identification information (such as the object ID) of the storage object when the storage object is applied to be stored through the aggregated write or the GC moving write, and determine whether the GC object and the target node of the storage object are affinity.

[0032] A reselection module is configured to actively discard the storage object and reselect when the determination result is not affinity.

[0033] A data writing module is configured to write data into the storage object and keep the storage object, the associated GC object and the corresponding index object in the same node when the determination result is affinity.

[0034] A cache module is configured to cache the data of the affinity storage object in the local cache of the node.

[0035] A data reading module is configured to perform the reading service according to the local cache of the target node of the storage object.

[0036] The device realizes the steps of the method for improving the IO reading performance based on the object affinity in the foregoing description when running.

[0037] The third aspect of the present application provides an electronic device, comprising a memory and a processor.

[0038] The memory is configured to store a computer program.

[0039] The processor is configured to execute the computer program to realize the steps of the method for improving the IO reading performance based on the object affinity in the foregoing description.

[0040] The fourth aspect of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by the processor to realize the steps of the method for improving the IO reading performance based on the object affinity in the foregoing description.

[0041] In summary, the present application scheme combines the read-write cache mechanism, pre-calculates the affinity relationship between the cluster node (node) and the garbage collection (GC) object, and writes data into the storage object with the GC object affinity. Under this mechanism, even if the GC moving operation is triggered due to the continuous overlay write, the foreground read IO can still obtain effective data from the cache of the original node (node), thereby effectively reducing the performance loss of data cross-network transmission and reading slow medium, and significantly improving the reading performance.

[0042] Other features and advantages of the present application will be described in detail in the subsequent description, or can be understood by implementing the related technical solutions of the present application. The purpose and other advantages of the present application can be realized by the technical features and technical means explicitly indicated in the specification, claims and drawings, and obtained by the implementation process of these technical contents. Attached Figure Description

[0043] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings involved in the description of the embodiments will be briefly introduced below. It should be noted that the accompanying drawings only show some embodiments of this application. For those skilled in the art, other related drawings can be derived from these drawings without creative effort.

[0044] Figure 1 This is a flowchart illustrating the overall implementation of the method for improving IO read performance based on object affinity in this application.

[0045] Figure 2 This is a topology diagram of the distributed storage cluster involved in the embodiments of this application.

[0046] Figure 3 This is a schematic diagram of node affinity GC object filtering in an embodiment of this application.

[0047] Figure 4 This is a schematic diagram of the data read / write process in an embodiment of this application.

[0048] Figure 5 This is a schematic diagram of the GC relocation process in an embodiment of this application.

[0049] Figure 6 This is a structural diagram of the device for improving IO read performance based on object affinity according to this application.

[0050] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be noted that the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the protection scope of this application.

[0052] In this document, the terms "comprise" and "comprising" and any form of these terms are open-ended expressions that are used to refer to the objects or elements they precede. These terms are to be construed as specifying the presence of what follows, but do not preclude the presence of elements or additional objects that are not listed. The terms "based on" and "based upon" are to be construed as "based at least in part on," that is, the condition or parameter based upon which something is done can not be the sole factor, but rather there can be other related factors. The term "one embodiment" is to be understood as "at least one embodiment," that is, there is more than one embodiment. The terms "a" and "one" are to be understood as "at least one" or "one or more," that is, there can be more than one of the elements being described.

[0053] In this document, the terms "a" and "one" are used to refer to one or more than one (i.e. to "at least one") of the elements being described. The terms "a(n)" and "one" encompass the terms "plurality" and "multiple," unless the context clearly indicates otherwise. The term "based on" is to be construed as "based at least in part on," that is, the condition or parameter based upon which something is done can not be the sole factor, but rather there can be other related factors.

[0054] Figure 1 The overall implementation procedure of the method for improving IO read performance based on object affinity provided in the present application is shown, which includes the following steps:

[0055] S1. A certain number of garbage collection (GC) objects are pre-configured according to the size of the distributed storage cluster, and the affinity of each GC object to each node in the cluster is pre-calculated;

[0056] S2. When a storage object is applied through aggregated writing or GC moving writing, the associated GC object is determined according to the identification information (such as object ID) of the storage object, and it is determined whether the GC object and the target node of the storage object are affinitive;

[0057] S3. If the result of the determination is not affinitive, the storage object is discarded and reselected;

[0058] S4. If the result of the determination is affinitive, the data is written into the storage object, and the storage object, the associated GC object and the corresponding index object are kept in the same node;

[0059] S5. The storage object is read in the local cache of the node.

[0060] In order to more clearly illustrate the technical solutions of the present application, the following will be further described through specific embodiments of specific scenarios.

[0061] For example, Figure 2As shown, in the distributed storage cluster scenario involved in the embodiment, it is assumed that the cluster contains 3 nodes, and each node deploys 4 disks. The user data writing process is as follows: first, mapping to an index object, the master node of the index object is allocated according to a rule (for example, the master of index_obj0 is node0, the master of index_obj1 is node1, and the master of index_obj2 is node2); then, through aggregation writing, respective storage objects (assuming 3-copy redundancy is used) are applied, for example, storage objects obj0 (associated with tgt0, tgt4, tgt8), obj1 (associated with tgt1, tgt5, tgt9), and obj2 (associated with tgt2, tgt5, tgt9) are generated, and obj0, obj1, and obj2 are respectively associated with garbage collection (GC) objects gc_obj0, gc_obj1, and gc_obj2 through a hash algorithm.

[0062] In the prior art, due to differences in layout algorithms, the master node of the index object and the master node of the GC object may not be compatible (that is, the master node of the index object and the master node of the GC object are not on the same node, for example, the master of index_obj0 is on node0, and the master of gc_obj0 is on node1). When aggregating and writing the storage object obj0, the data is cached to the memory or cache disk of node0, at this time, directly reading the data from node0 can avoid reading the disk and network transmission, and significantly improves the read performance. However, as the continuous addition of the overwrite write, the data of the storage object obj0 is gradually overwritten to become garbage, triggering the gc_obj0 recovery process. Since the master node of gc_obj0 is on node1, the GC migration operation will be performed on node1: the valid data of obj0 needs to be read first, and then written to a new storage object obj00 (cached to the memory of node1), and at the same time, the metadata will be updated from the old storage object obj0 to point to obj00. At this time, when the foreground IO reads again, it will still try to read the new object obj00 from node0, but since obj00 is not generated on node0, its cache cannot be hit, resulting in a decrease in read performance. To solve the problem that the read cache cannot be hit due to the incompatibility of the object master node, the present application proposes a method for improving read performance based on object compatibility.

[0063] The specific steps of the method are as follows:

[0064] 1. Preparing GC objects: according to the size of the deployed cluster, a proper number of GC objects are prepared, for example, for a cluster with a size of 32 nodes or less, a total of 64K GC objects can be set for the cluster, and according to the hash consistency principle, each node can hash to 2K.

[0065] 2. Calculate the affinity of GC objects and nodes: calculate the GC object affinity of the current node according to the current cluster map information (the map records the node and the disk distribution under the node of the cluster, such as the node normal / failure, the disk normal / failure, and the like), that is, select the GC object affinity from the 64K GC objects and save it, as shown in Figure 3

[0066] 3. Data aggregation write: for example, 1M aggregation granularity is taken as an example, the customer issues an IO size of 4K, that is, 256 times of 4K small IO is aggregated into a 1M large IO and then written to the disk, the purpose is to improve the performance of writing to the disk. When the aggregation write application storage object obj is applied, the GC object affinity of the obj is hashed, that is, the 64K is taken modulo the storage object obj ID, as shown in Figure 4

[0067] 4. GC object affinity check: check whether the GC object obtained in step 3 is in the 2K range of the node, if not, it means that the storage object and the node are not affinity, repeat steps 3-4, until the selected storage object calculates the GC object in the 2K range of the node;

[0068] 5. Write cache: when the storage object obj is written, the data is cached in the corresponding node memory or ssd and the like high-speed medium in the unit of obj;

[0069] 6. Record the metadata of the storage object: store the metadata of the storage object;

[0070] 7. Read data: when the user reads data, first hash to the index master to read, at this time, the storage object obj is found through the metadata, and whether the storage object obj is in the cache is checked. If the obj is in the cache, the cache hit is read, the data can be directly read from the node memory or ssd and the like high-speed medium, without cross-network transmission and reading from the slow medium HDD disk, thereby improving the read performance;

[0071] 8. GC migration: when the garbage amount of the storage object obj1 reaches the recycling threshold, the GC garbage collection mechanism is triggered by the master of the GC object, and the valid data of obj1 is migrated to the storage object obj2 (the selection method of the storage object obj2 is the same as steps 3-4) by the way of reading first and then writing; At the same time, the data of obj2 is written to the cache, as shown in Figure 5

[0072] 9. Metadata modification and old cache elimination: after the GC migration is completed, the metadata is changed from obj1 to obj2, and the storage object obj1 is eliminated from the cache to save the cache space;

[0073] ​​​10. Read data: same as step 7, except that the metadata is adjusted from obj1 to obj2.

[0074] With the continuous overwriting of data, the method can solve the problem of read performance decline caused by the incompatibility of objects due to the continuous migration of GC, greatly improve the cache hit rate, and improve the read performance.

[0075] Figure 6 An apparatus for improving IO read performance based on object affinity is shown, the apparatus comprises:

[0076] A pre-configuration module is configured to pre-configure a certain number of garbage collection (GC) objects according to the size of the distributed storage cluster, and pre-calculate the affinity relationship between each GC object and each node in the cluster.

[0077] An affinity checking module is configured to determine the associated GC object according to the identification information (such as object ID) of the storage object when applying for a storage object through aggregated writing or GC migration writing, and determine whether the GC object and the target node of the storage object are compatible.

[0078] A re-selection module is configured to actively discard the storage object and reselect when the result is incompatible.

[0079] A data writing module is configured to write data into the storage object when the result is compatible, and keep the storage object, the associated GC object, and the corresponding index object in the same node.

[0080] A cache module is configured to cache the data of the compatible storage object in the local cache of the node.

[0081] A data reading module is configured to perform read services according to the local cache of the target node of the storage object.

[0082] The above apparatus realizes the steps of the method for improving IO read performance based on object affinity disclosed in the present application when running.

[0083] The flowcharts and block diagrams in the drawings show possible implementation manners of the apparatus, method, and computer program product according to various embodiments of the present application, including architecture, function, and operation. In these figures, each block can represent a module, a program segment, or a part of code, which contains one or more executable instructions for implementing the specified logical function. It should be noted that each block in the block diagram and / or flowchart, as well as the combination of these blocks, can use a dedicated hardware-based system to implement the specified function or operation, or a combination of dedicated hardware and computer instructions.

[0084] As Figure 7As shown, the embodiments of the present application also disclose an electronic device, comprising: a processor 310, a communication interface 320, a memory 330 for storing a processor-executable computer program, and a communication bus 340. Wherein the processor 310, the communication interface 320 and the memory 330 complete mutual communication through the communication bus 340. The processor 310 realizes the steps of the method for improving IO read performance based on object affinity by running the executable computer program.

[0085] It can be understood that, in addition to the memory and the processor, the electronic device can also include input devices (such as a keyboard), output devices (such as a display), and other communication modules. These input devices, output devices, and other communication modules all communicate with the processor through an I / O interface (i.e., an input / output interface).

[0086] The operations of the present application can be implemented by using one or more program design languages or combinations thereof to write computer program codes. The program design languages include but are not limited to the following types:

[0087] Object-oriented program design languages, such as Java, Smalltalk, C++, etc.

[0088] Conventional procedural program design languages, such as the "C" language or similar program design languages.

[0089] The execution modes of the program codes include but are not limited to:

[0090] Completely on a user computer;

[0091] Partially on a user computer and partially on a remote computer;

[0092] As a separate software package;

[0093] Completely on a remote computer or server.

[0094] In the scenario involving a remote computer, the remote computer can be connected to the user computer through any type of network, including but not limited to a local area network (LAN) or a wide area network (WAN). In addition, the remote computer can also be connected to an external computer through an Internet service provider, such as a connection through the Internet.

[0095] Further, the present application also discloses a computer-readable storage medium, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute each step of the method for improving IO read performance based on object affinity disclosed by the present application.

[0096] In the context of the present application, computer-readable storage media refers to tangible media capable of storing computer program code and related data. Specific examples include, but are not limited to, the following:

[0097] (1) Portable computer disks: removable magnetic storage media such as floppy disks.

[0098] (2) Hard disks: fixed storage devices including mechanical hard disks and solid-state hard disks.

[0099] (3) Random Access Memory (RAM): volatile storage media used for temporary storage of data and program code.

[0100] (4) Read-Only Memory (ROM): non-volatile storage media used for storing fixed programs and data.

[0101] (5) Erasable Programmable Read-Only Memory (EPROM) or Flash Memory: non-volatile storage media that supports multiple erasing and programming.

[0102] (6) Optical fiber storage device: storage media based on fiber optic technology.

[0103] (7) Portable Compact Disc Read-Only Memory (CD-ROM): read-only media that stores data in the form of optical discs.

[0104] (8) Optical storage device: storage media based on optical principles such as DVD, Blu-ray disc, etc.

[0105] (9) Magnetic storage device: storage media based on magnetic principles such as magnetic tape, magnetic disk, etc.

[0106] (10) Any suitable combination of the above: for example, using multiple storage media in combination to meet different storage needs.

[0107] These computer-readable storage media can be used to store the program code and related data described in the present application to support the running of programs and the persistent storage of data.

[0108] In particular, according to embodiments of the present application, the processes described in the flowcharts can be implemented as computer software programs. For example, embodiments of the present application relate to a computer program product comprising a computer program carried on a non-transitory computer-readable medium. The computer program contains program code for executing the object affinity-based IO read performance improvement method disclosed in the present application. When the computer program is executed by a processing device, the above-mentioned functions defined in the embodiments of the present application can be realized.

[0109] Although the above discussion includes a number of specific implementation details, these details should not be construed as limiting the scope of the application. The above description is merely of the preferred embodiments and the principles of the application. It is understood that those skilled in the art will be able to devise various arrangements which, although not explicitly described or shown herein, embody the principles of the application and, as such, within the spirit and scope of the application. Furthermore, there are many different ways to implement the above described functionality, and the present application is not limited to any particular implementation.

[0110] Those skilled in the art will further understand that they can make modifications or improvements to the above described embodiments without departing from the spirit and scope of the present application. Such modifications or improvements are intended to be included within the scope of the present application.

Claims

1. A method for improving I / O read performance based on object affinity, characterized in that, The method comprises: S1. Pre-configure a certain number of garbage collection (GC) objects according to the size of a distributed storage cluster, and pre-calculate the affinity of each GC object to each node in the cluster, including uniformly mapping the GC objects to each node according to consistent hashing, and saving the set of GC object identifiers that each node is responsible for; S2. When a storage object is applied through aggregated writing or GC moving writing, determine the associated GC object according to the identifier information of the storage object, and judge whether the GC object is affinitive to the target node of the storage object, including calculating the GC object identifier obtained by taking the modulus of the identifier of the storage object to the pre-configured number N of the GC object, and detecting whether the identifier falls into the set of GC object identifiers that the current node has saved; S3. If the judgment result is not affinitive, actively discard the storage object and reselect; S4. If the judgment result is affinitive, write data into the storage object, and keep the storage object, the associated GC object, and the corresponding index object in the same node; S5. Perform read service according to the local cache of the storage object in the node.

2. The method of claim 1, wherein, The N satisfies: N is an integer power of 2, and N≥node number×2000.

3. The method of claim 1, wherein, The method further comprises the following steps in the GC moving writing process: (1) When the garbage ratio of a storage object reaches a threshold, trigger a GC moving operation by the GC object associated with the storage object in the current node; (2) Move the valid data to a newly generated affinitive storage object in the same node, and update the cache; (3) After the GC moving is completed, update the metadata to point to the newly generated affinitive storage object, and remove the original storage object from the cache.

4. The method of claim 1, wherein, The aggregated granularity of the aggregated writing in step S2 is 1 MB, and the small IO is completed in the node local cache and then uniformly landed.

5. The method of claim 1, wherein, Step S4 further comprises: Cache data to the target node: cache the data of the aggregated writing to the memory or cache medium of the node where the GC object associated with the storage object is located in units of storage objects; Record the metadata of the storage object: persistently store the metadata of the storage object.

6. The method of claim 1, wherein, Step S5 further comprises: When a user reads data, locate the target storage object through the index object, check whether the storage object exists in the cache of the current node, and if so, directly read the data from the local cache.

7. A device for improving I / O read performance based on object affinity, characterized in that, The device comprises: A pre-configuration module configured to pre-configure a certain number of garbage collection (GC) objects according to the size of a distributed storage cluster, and pre-calculate the affinity of each GC object to each node in the cluster, including uniformly mapping the GC objects to each node according to consistent hashing, and saving the set of GC object identifiers that each node is responsible for; An affinity checking module configured to, when a storage object is applied through aggregated writing or GC moving writing, determine the associated GC object according to the identifier information of the storage object, and judge whether the GC object is affinitive to the target node of the storage object, including calculating the GC object identifier obtained by taking the modulus of the identifier of the storage object to the pre-configured number N of the GC object, and detecting whether the identifier falls into the set of GC object identifiers that the current node has saved; The reselecting module is configured to discard the storage object and reselect when the result of the judgment is incompatibility; The data writing module is configured to write data into the storage object and keep the storage object, the GC object associated with the storage object and the corresponding index object in the same node when the result of the judgment is compatibility; The cache module is configured to cache the data of the compatible storage object in the local cache of the node; The data reading module is configured to perform reading service according to the local cache of the target node.

8. An electronic device, comprising: The method comprises the following steps: a memory and a processor; the memory is configured to store a computer program; the processor is configured to execute the computer program to implement the steps of the method for improving IO reading performance based on object compatibility according to any one of claims 1-6.

Citation Information

Patent Citations

  • CRDT junk data recovery method and device, equipment and storage medium

    CN112559383A

  • Garbage collection method and equipment for partition storage equipment and storage medium

    CN118733482A