Data structure of hash mapping set supporting traversal and dynamic modification

Abstract

The invention discloses a data structure of a hash mapping set supporting traversal and dynamic modification, and belongs to the field, the data structure comprises N slots, and a continuous array is stored in each slot. N logical slots are added and used for storing data structure fragments, a continuous array is stored in each slot, each array element is a list and used for storing actual data, through a hash algorithm, all requests can be equally distributed on all the slots and then distributed on the bucket, the probability of lock conflicts is greatly reduced, and when all data are cyclically traversed for multiple times, by taking the bucket as a unit, the number of the slots is reduced, and the number of the slots is reduced. According to the data structure, only the bucket index needs to be recorded in two cycles, in the data traversing process, if data needs to be inserted into the scanned bucket, the bucket is directly used as a part of a scanning result, the scanning result is not influenced by deleting the data, and by adopting the data structure, multi-thread traversing and dynamic modification are supported, so that the adverse influence caused by the failure of an iterator can be avoided.

Description

Technical Field

[0001] The present invention relates to the field of data structures, and in particular to a data structure of a hash map set supporting traversal and dynamic modification. Background Art

[0002] In data structures, a hash map (or hash table) is an efficient way to store key-value pairs, which allows fast lookup, insertion, and deletion of values ​​by key.

[0003] In data structure applications, in multi-threaded scenarios, the performance bottleneck of accessing the same data structure comes from locking the data structure. Although the data structure can be sharded to increase concurrency and thus performance, the more commonly used STL, such as map, can be logically sharded, but when all data needs to be traversed through multiple accesses, there is a problem of iterator failure.

[0004] The present invention is used for a data storage module of a Sudis distributed high-speed cache database system 2.0 product. Summary of the invention

[0005] The purpose of the present invention is to provide a data structure of a hash map set that supports traversal and dynamic modification, so as to solve the problems raised in the above background technology.

[0006] To achieve the above object, the present invention provides the following technical solution: a data structure of a hash map set supporting traversal and dynamic modification, comprising N slots, each of which stores a continuous array, called buckets, and each array element (bucket) is a list; The slot is used to store data structure fragments. Each request is mapped to the corresponding slot by hashing the key word in the request, and the list is used to store the actual data.

[0007] Preferably, in the array element (bucket), the positioning of the bucket is achieved by hashing the key word in the request.

[0008] Preferably, locking is performed at the list level, which greatly reduces the granularity of the lock and can ensure the consistency of the data view of each visitor.

[0009] Preferably, the number of slots is fixed, and the number of buckets under each slot is fixed.

[0010] Compared with the prior art, the beneficial effects of the present invention are as follows: the data structure of the hash map set that supports traversal and dynamic modification adds logically N slots for storing data structure fragments, each slot stores a continuous array (buckets), and each array element (bucket) is a list for storing actual data. Through the hash algorithm, each request will be evenly distributed on each slot, and then distributed on the bucket, and the probability of lock conflict is greatly reduced. When performing multiple loops to traverse all the data, in units of buckets, only the bucket index needs to be recorded in two loops. During the data traversal process, if there is data that needs to be inserted into the bucket that has been scanned, it is directly used as part of the scan result, and deleting the data has no effect on the scan result. With this data structure, the hash structure that supports multi-threaded traversal and dynamic modification is not strongly related to whether the structure is modified when performing multiple rounds of traversal of data, giving full play to the advantages of multi-threading and avoiding iterator failure. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 Schematic diagram of the data structure of the present invention. DETAILED DESCRIPTION

[0012] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0013] See also Figure 1 ,The present invention provides a data structure of a hash map set supporting traversal and dynamic modification, including N slots, each of which stores a continuous array, called buckets, and each array element (bucket) is a list; Slots are used to store data structure fragments. Each request is mapped to the corresponding slot by hashing the key word in the request, and lists are used to store actual data.

[0014] Sharding data structures and dividing large data sets into small data blocks can improve concurrency and thus performance, increase the convenience of data access, improve the efficiency of data query and processing, and at the same time improve the scalability of the data system and enhance the reliability and practicality of the data. Using slots to store data structure shards can improve the system's performance, scalability, availability, and flexibility of data management.

[0015] Through the above embodiment, in the array element (bucket), the bucket is located by hashing the keyword in the request. Locking at the list level greatly reduces the granularity of the lock, which can ensure the consistency of the data view of each visitor. The number of slots is fixed, and the number of buckets under each slot is fixed.

[0016] By locating the bucket, data can be quickly mapped to the corresponding storage location, thereby improving data access efficiency. At the same time, locking at the list level not only ensures the consistency of each visitor's data view, but also reduces lock competition, improves the system's concurrent access capabilities, and ensures system performance.

[0017] When the present invention is used, the added N slots are used to store data structure fragments. In the continuous array (buckets) corresponding to each slot, each array element (bucket) is a list. The list is used to store actual data. When data access is required, after submitting the data request, the hash algorithm is first used to locate the slot and then correspond to the bucket through the key word. In this process, through the hash algorithm, each request will be evenly distributed on each slot and then distributed on the bucket, and the probability of lock conflict is greatly reduced. When multiple loops are performed to traverse all data, the bucket is used as a unit, and only the bucket search engine needs to be recorded in two loops, which simplifies the process of searching records. For traversing data, if there is data that needs to be inserted into the bucket that has been scanned during the data traversal process, it is directly used as part of the scan result. Deleting data has no effect on the scan result and is not strongly related to whether the structure is modified. The advantages of multi-threading are fully utilized to effectively avoid the adverse effects caused by iterator failure.

[0018] It should be noted that N logical slots are added to store data structure fragments. Each slot stores a continuous array (buckets), and each array element (bucket) is a list for storing actual data. Through the hash algorithm, each request will be evenly distributed on each slot and then distributed on the bucket, greatly reducing the probability of lock conflicts. When traversing all the data in multiple loops, in units of buckets, only the bucket index needs to be recorded in two loops. During the data traversal process, if there is data that needs to be inserted into a bucket that has been scanned, it will be directly used as part of the scan result, and deleting the data will not affect the scan result. With this data structure, the hash structure that supports multi-threaded traversal and dynamic modification is not strongly related to whether the structure is modified when traversing the data in multiple rounds, giving full play to the advantages of multi-threading and avoiding iterator failure.

[0019] The above content describes the use principle, features and beneficial effects of the present invention. Relevant personnel in the field can understand from the above content that the above content does not limit the present invention. The above embodiments and descriptions describe the basic principles and features of the present invention. Under the premise of conforming to the concept of the present invention, the present invention can also be subjected to various changes and improvements, and these improvements should fall within the scope of protection claimed by the present invention.

Claims

1. A data structure of a hash map set that supports traversal and dynamic modification, characterized in that: It includes N slots, each of which stores a continuous array, called buckets, and each array element (bucket) is a list; The slot is used to store data structure fragments. Each request is mapped to the corresponding slot by hashing the key word in the request, and the list is used to store the actual data.

2. A data structure of a hash map set supporting traversal and dynamic modification according to claim 1, characterized in that: In the array element (bucket), the bucket is located by hashing the key word in the request.

3. A data structure of a hash map set supporting traversal and dynamic modification according to claim 2, characterized in that: Locking at the list level greatly reduces the granularity of the lock, and can ensure the consistency of the data view of each visitor.

4. A data structure of a hash map set supporting traversal and dynamic modification according to claim 4, characterized in that: The number of slots remains fixed, and the number of buckets under each slot remains fixed.

Images