Grouping vector hash table implementation method and equipment applied to storage system and readable medium

Abstract

The invention provides an implementation method for optimizing a packet vector hash table by using single instruction multiple data (SIMD), which is used for a key-value pair and comprises the following steps of: splitting a hash value, splitting the hash value of a current key into a low-order part and a high-order part, and setting a control byte to contain low-order information of the hash value; a metadata array is designed, any key is grouped according to the initialization table capacity of the hash table, and information of each group is stored by using the control bytes; and during query or write-in or deletion operation, the control bytes in any group are compared in parallel by using an SIMD (Single Instruction Multiple Data) instruction, so that the search and write-in efficiency is optimized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a method for implementing a group vector hash table optimized by single instruction multiple data (SIMD), which is particularly suitable for a storage system that needs to consider both memory efficiency and persistent storage requirements. BACKGROUND

[0002] This section is intended to provide background information to facilitate a better understanding of embodiments of the present application as set forth in the claim section. It should be understood that the description herein does not constitute admission of prior art.

[0003] A hash table is a data structure used for fast storage and lookup of key-value pairs. In the prior art, hash tables optimized based on SIMD mainly target memory usage scenarios, while storage systems need to consider efficient access to both memory and disk.

[0004] Although the existing patent "Efficient hash table implementation method optimized by SIMD" improves the performance of hash tables in memory, its design does not consider the persistent storage requirements and cannot be directly applied to storage systems. Therefore, based on the above invention, the present application further optimizes the data storage structure on the basis of inheriting the SIMD optimization technology, and designs a hash table structure suitable for storage systems, making it suitable for storage systems. SUMMARY

[0005] Based on the above technical defects, the purpose of the present application is to provide a group vector hash table implementation method applied to a storage system, which further optimizes the data storage structure on the basis of inheriting the SIMD optimization technology, and designs a hash table structure suitable for storage systems, making it suitable for storage systems, and solving the problem of efficient access to both memory and disk in storage systems.

[0006] At the same time, the method of the present application is suitable for high-performance memory database systems, memory cache systems, distributed systems or real-time data processing systems, and disk storage systems.

[0007] Aspects of the present application provide a method for implementing a group vector hash table optimized by single instruction multiple data (SIMD) to improve lookup performance, memory access efficiency and space utilization.

[0008] In one aspect of the present application, an implementation method for optimizing a grouped vector hash table using single instruction multiple data (SIMD) is provided for key-value pairs, the method comprising: splitting a hash value, splitting a hash value of a current key into a low-bit part and a high-bit part, and setting the control byte to contain low-bit information of the hash value; designing a metadata array, grouping any key according to a hash table initialization table capacity, and storing information of each group using the control byte; during a query or write or delete operation, quickly locating a target slot by using SIMD instructions to compare the control byte in any group in parallel; serializing key-value data into a byte array and storing it in a memory space or a disk file, and during writing or querying, locating a specific storage location of the key-value data through an offset.

[0009] Further, the design of the metadata array further includes an initialization step: each group contains multiple slots, and the state information of the slots is set to occupied slots, empty slots, or deleted slots.

[0010] Further, the design of the metadata array further includes an initialization step: setting all control bytes to empty slots.

[0011] Further, the encoding rules of the control byte include: 0x00-0x7F represents occupied slots; 0x80 represents empty slots; and 0xFE represents deleted slots.

[0012] Further, the grouped vector hash table includes: a metadata array for storing control bytes; and a data array for storing key-value pairs; the metadata array and the data array are stored separately.

[0013] Further, a grouping storage strategy is adopted, and the grouping size of the metadata array is 16 bytes, which is aligned with the 128-bit SIMD instruction set.

[0014] Further, the data storage step includes: after serializing the key-value data into a byte array, recording the storage offset; and storing the key length, key data, value length, and value data compactly in the memory or disk file.

[0015] Further, the serialization format is: 2-byte key length; key data; 2-byte value length; and value data.

[0016] Further, a memory mapping file mechanism is further included to map the disk file to a continuous memory space.

[0017] Further, when the hash value is 32 bits, it is split into 25 high bits and 7 low bits, and the 7 low bits are used as control bytes. When the hash value is 64 bits, it is split into 57 high bits and 7 low bits, and the 7 low bits are used as control bytes.

[0018] Further, the write logic includes: using SIMD to quickly screen the same control byte positions and updating the value.

[0019] Further, compared with whether each same control byte position is the current key; if the current group of same control byte positions is the same key, update the value; if the current group of same control byte positions is not the same key, further screen whether the current group has a null slot, (1) if the current group has a null slot, the null slot control byte is written into the control byte of the current key, and the corresponding data storage position is written into the key value data; (2) if the current group has no null slot, continue to detect the next group.

[0020] Further, the query logic includes: the high bit part of the split hash value is used to locate the starting group of the current key storage, and the high bit hash value is used to locate the starting group; using SIMD instructions to compare the control bytes in parallel to generate a bit mask recording the matching position; traversing the keys of the matching position, if matching, return the value, otherwise continue to detect the next group or terminate the query.

[0021] Further, the deletion logic includes: if the current group has a null slot, set the deletion position to the null slot; if the current group has no null slot, set the deletion position to the tombstone marker.

[0022] Further, the method further includes an automatic expansion step: when the number of elements reaches the product of the number of groups and the load factor, trigger expansion.

[0023] Further, the threshold of the load factor is 14; the load factor is a preset proportion of the number of slots in each group, which is used to avoid performance degradation of detection.

[0024] Another aspect of the present application provides an electronic device, the device comprising: at least one processor; and a memory connected in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the implementation method of using single instruction multiple data (SIMD) optimized grouped vector hash table as described above.

[0025] Another aspect of the present application provides a computer readable storage medium having stored thereon computer program instructions executable by a processor to implement the implementation method of using single instruction multiple data (SIMD) optimized grouped vector hash table.

[0026] Another aspect of the present application provides a computer program product comprising a computer program, which, when executed by a processor, implements the implementation method of using single instruction multiple data (SIMD) optimized grouped vector hash table.

[0027] The purpose of the present application is to propose a grouping vector hash table implementation method applied to a storage system, so as to solve the problems of discontinuous memory access, low conflict processing efficiency, and significant performance decline under high load in the prior art. The present application has the following significant distinguishing features compared with the prior art: first, the innovative hash value splitting method (high 25 bits / low 7 bits); second, the special control byte coding design (0x00-0x7F / 0x80 / 0xFE); third, the grouping detection mechanism combined with SIMD optimization; fourth, the automatic expansion strategy of the load factor control. Fifth, the key value data storage structure is compact, which can be used in memory and also directly written into a hard disk, and can be used in a storage system. The specific implementation mode of the present application includes: adopting a grouping strategy, performing batch detection on the whole group, and providing the use premise of the SIMD instruction; using the SIMD instruction set to realize batch data comparison and improve the search efficiency; splitting the hash value of the key into two parts for query starting address positioning and metadata control byte, and optimizing the memory access mode. The present application realizes the technical effects: first, the search performance is significantly improved, especially under high load; second, the memory access mode is improved, and the cache efficiency is improved; third, the memory occupation is reduced, and the space utilization rate is improved; fourth, it can be applied to a storage system that takes into account the disk and the memory. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0029] Other features, objects and advantages of the present application will become more apparent through reading the following detailed description of the non-limiting embodiments with reference to the following drawings: Figure 1 The logical flow diagram of the implementation method provided by an embodiment of the present application is shown in the figure; Figure 2 The key value data storage logic diagram of the present application is shown in the figure; Figure 3 The 64-bit hash value splitting diagram of the present application is shown in the figure Figure 4 The control byte coding design diagram of the present application is shown in the figure; Figure 5 The SIMD optimization write logical flowchart of the present application is shown in the figure; Figure 6 The SIMD optimization query logical flowchart of the present application is shown in the figure; Figure 7A structural schematic diagram of an electronic device suitable for implementing the scheme in the embodiments of the present application. DETAILED DESCRIPTION

[0030] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0031] In a typical configuration of the present application, the devices of the terminal and the service network each include one or more processors, input / output interfaces, network interfaces and memories.

[0032] The memory can include non-permanent memory, random access memory and / or non-volatile memory in the form of computer readable medium, such as read-only memory or flash memory. The memory is an example of the computer readable medium.

[0033] The computer readable medium includes permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. The information can be computer program instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory, static random access memory, dynamic random access memory, other types of random access memory, read-only memory, electrically erasable programmable read-only memory, flash memory or other memory technology, read-only optical disc, digital versatile disc or other optical storage, magnetic cassette, magnetic tape disc storage or other magnetic storage device, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0034] In actual scenarios, the execution subject of the method can be a user device, or a device integrated by a user device and a network device through a network, or an application program running on the above device. The user device includes but is not limited to computers, mobile phones, tablet computers, smart watches, wristbands and various terminal devices, and the network device includes but is not limited to network hosts, single network servers, multiple network server sets or computer sets based on cloud computing, which can be used to implement part of the processing functions when setting an alarm. Here, the cloud is composed of a large number of hosts or network servers based on cloud computing, wherein the cloud computing is a kind of distributed computing, which is composed of a virtual computer set by a loose coupled computer group.

[0035] Embodiment one

[0036] The application provides an implementation method of optimizing a group vector hash table using single instruction multiple data (SIMD) for a key-value pair. The following will be described in combination with the specific implementation method of the application and the core components of the technical solution of the application: Step S100, split the hash value, split the hash value of the current key into a low part and a high part, and set the control byte to contain the low bit information of the hash value. One of the core components of the technical solution of the application corresponding thereto is a hash value splitting method.

[0037] Specifically, the hash value is split into two parts of high / low. If a 32-bit hash value is used, it is split into two parts of high 25 bits and low 7 bits. If a 64-bit hash value is used, it is split into two parts of 57 bits and 7 bits as shown in Figure 2 .

[0038] Further, since 7 bits only need 1 byte for storage, the byte corresponding to 7 bits is called a control byte. Then 25 bits are stored using uint32, which is used to locate the starting group of the current key storage.

[0039] For example, the hash value of a certain key (key) is 2324567033, the binary representation is 0b1000_1010_1011_1001_1101_0101_1111_1001, which is split into 0b1000_1010_1011_1001_1101_0101_1 and 0b111_1001, the front zero is filled to 0b0000_0001_0001_0101_0111_0011_1010_1011 and 0b0000_0000_0000_0000_0000_0000_0111_1001.

[0040] The specific algorithm is as follows: h1Mask32 uint32 = 0xffff_ff80 h2Mask32 uint32 = 0x0000_007f h1 = (hash & h1Mask32) >> 7 h2 = h & h2Mask32 Step S200, design a metadata array, group any key according to the table capacity initialized by the hash table, and store the information of each group using the control byte.

[0041] The second core component of the technical solution of the application corresponding thereto is grouping according to the table capacity initialized by the hash table.

[0042] Further, the grouping vector hash table comprises: a metadata array for storing control bytes; and a data array for storing key-value pairs; the metadata array and the data array are stored separately.

[0043] Further, a grouping storage strategy is adopted, and a grouping size of the metadata array is 16 bytes, which is aligned with a 128-bit SIMD instruction set.

[0044] Because 7-bit data is used as the control byte for comparison, and one sign bit is added, a total of 8 bits are used. In order to better utilize the 128-bit SIMD parallel operation, 16 control bytes are used as a group for batch comparison. Assuming that the initial capacity is 1000, and there are 16 elements in each group, a total of 1000 / 16 groups are obtained, and the number of groups is rounded up to 63.

[0045] Thirdly, a load factor.

[0046] Further, the threshold of the load factor is 14; and the load factor is a preset proportion of the number of slot positions in each group, which is used to avoid degradation of the detection performance.

[0047] In order to avoid that the control bytes are all full, resulting in that the detection is degraded to full-quantity comparison, the load needs to be reduced, and on average, only 14 elements are stored in each group of 16 slot positions (here, 14 can be 12, 13 or 15, and 14 is the best value obtained by balancing the space utilization rate and the detection degradation rate), and the maximum number of stored elements in each group is set as the load factor.

[0048] Grouping number adjustment. The number of groups is adjusted correspondingly due to the reduction of the load in each group (grouping according to the initial table capacity of the hash table). Grouping number = round up (initial capacity / load factor).

[0049] Fourthly, metadata array design.

[0050] Further, the design of the metadata array further comprises an initialization step: each group contains a plurality of slot positions, and the state information of the slot positions is set as occupied slot positions, empty slot positions or deleted slot positions.

[0051] As shown in Figure 3 , the design of the metadata array further comprises an initialization step: all the control bytes are set as empty slot positions. The encoding rule of the control bytes comprises: 0x00-0x7F represents occupied slot positions; 0x80 represents empty slot positions; and 0xFE represents deleted slot positions.

[0052] Specifically, the state information of each slot position is stored by using the control byte. The control byte contains low-bit information of the hash value.

[0053] Specifically, a grouping storage strategy is adopted, and each group has a fixed size.

[0054] Specifically, the control byte encoding design: 0x00 - 0x7F: occupied (specific control byte, store the low 7 bits of the hash value). 0x80 (0b1000_0000): empty slot position. 0xFE (0b1111_1110): deleted (tombstone), and the tombstone code here can be any number from 0x81-0xFF.

[0055] Specifically, the group size is fixed to 16 bytes, which is aligned with the SIMD instruction set.

[0056] Step S300, execute the operation.

[0057] In the query or write or delete operation, the target slot position is quickly located by using the SIMD instruction to compare the control bytes in the any group in parallel. Further, the optimized lookup and write efficiency is realized.

[0058] The write logic includes: using SIMD to quickly screen the same control byte position, and updating the value. Further, it is compared whether each same control byte position is the current key; if the current group same control byte position is the same key, the value is updated; if the current group same control byte position is not the same key, it is further screened whether the current group has an empty slot position, (1) if the current group has an empty slot position, the empty slot position control byte is written to the control byte of the current key, and the corresponding data storage position is written with the key value data; (2) if the current group has no empty slot position, the next group is continued to be detected.

[0059] Specifically, as shown in Figure 4 the same control byte position is screened by using SIMD. It is compared whether each same byte position is the current key, and if the same key is found, the value is updated. If the current group same control byte position does not find the same key, the empty position is quickly detected based on the metadata array and the empty slot position (0x80) by using SIMD, and if the empty position is found, the empty slot position control byte is written to the control byte of the current key, and the corresponding data storage position is written with the key value data.

[0060] The query logic includes: the high part of the split hash value is used to locate the starting group where the current key is stored, the high hash value is used to locate the starting group, the control bytes are compared in parallel by using the SIMD instruction to generate a bit mask recording the matching position, and the keys of the matching position are traversed, and if the matching is found, the value is returned, otherwise the next group is continued to be detected or the query is terminated.

[0061] Specifically, as shown in Figure 5The bit mask technology is used to quickly locate the matching position, the keys in the same position of the mask bit are traversed, and whether the keys are the same as the query keys is determined. If the keys are the same, the corresponding value is found and returned. If the same key is not found in the current group of the same control byte position, the empty slot bit (0x80) is quickly detected based on the metadata array and the empty slot bit. If the empty slot bit is found, the data corresponding to the current key does not exist. If the empty slot bit is not found, the next group is continuously detected, and the above steps are repeated until the same key or the empty slot bit is found.

[0062] The deletion logic includes: if the current group has an empty slot bit, the deletion position is set to the empty slot bit; if the current group does not have an empty slot bit, the deletion position is set to the tombstone marker.

[0063] Specifically, it is judged whether the current group has an empty slot bit. If yes, the deletion position is set to the empty slot bit for position reuse. If the current group does not have an empty slot bit, the deletion position is set to the tombstone marker (0xFE) instead of physical deletion, and the empty slot bit is not set, so as to ensure that the number of hash conflicts is located in the current group, but when the current group is not found, the next group is continuously searched. When rehash is triggered, the tombstone marker is cleaned.

[0064] The core component of the technical solution of the application corresponding to the above is that the automatic expansion is triggered.

[0065] Further, the method further includes an automatic expansion step: when the number of elements reaches the product of the number of groups and the load factor, expansion is triggered.

[0066] Specifically, when data is written, if the number of current elements reaches the product of the number of groups and the load factor, expansion is triggered. The load factor.

[0067] Step S400, data storage.

[0068] The key value data is serialized into a byte array and stored in a memory space or a disk file. When writing or querying, the specific storage position of the key value data is located through an offset.

[0069] The core component of the technical solution of the application corresponding to the above is that the key value data is stored.

[0070] Further, the data storage step includes: after the key value data is serialized into a byte array, the storage offset is recorded; the key length, key data, value length and value data are stored compactly in the memory or disk file.

[0071] Further, the serialization format is: 2-byte key length; key data; 2-byte value length; value data.

[0072] Further, the memory mapping file mechanism is further included, and the disk file is mapped into continuous memory space.

[0073] Unlike the memory data structure, it can be widely applied to various data types for storing system, the key value needs to be serialized into byte array for convenient storage.

[0074] In order to store the key value data compactly, the memory uses continuous space, and the disk storage uses file direct storage. In particular, the memory mapping file mechanism can also be used to map the disk file to the memory, thereby forming a continuous and scalable memory space. When writing key value data, first record the write offset of the current memory space or file, then write the length of the key using 2 bytes (which can be adjusted according to the length of the key), then write the key data, then write the length of the value of 2 bytes, then write the value data, and write the offset to the data slot corresponding to the control byte. When looking up the key value, according to the offset obtained from the data slot corresponding to the control byte, the length of the key of 2 bytes (which can be adjusted according to the length of the key) is obtained at the offset position of the memory space or file, and then the key length byte data is obtained by offsetting 2 bytes, that is, the key data can be obtained, and the value data can be obtained in the same way.

[0075] The present application has the following significant distinguishing features compared with the prior art: first, the innovative hash value splitting method (high 25 bits / low 7 bits); second, the special control byte coding design (0x00-0x7F / 0x80 / 0xFE); third, the grouping detection mechanism combined with SIMD optimization; fourth, the automatic expansion strategy of the load factor control. Fifth, the key value data storage structure is compact, which can be used in memory, and also can be directly written into hard disk, and can be used in storage system.

[0076] The specific implementation of the present application includes: adopting a grouping strategy, performing batch detection on the whole group, and providing a prerequisite for using SIMD instructions; using the SIMD instruction set to realize batch data comparison and improve the search efficiency; splitting the hash value of the key into two parts for query starting address positioning and metadata control byte, and optimizing the memory access mode. The technical effects of the present application are: first, the search performance is significantly improved, especially under high load; second, the memory access mode is improved, and the cache efficiency is improved; third, the memory occupation is reduced, and the space utilization is improved. Fourth, it can be applied to a storage system that takes into account the disk and the memory. For example: performance improvement. The search operation performance is improved by 30-50%. The performance decay is smaller under high load. The memory access efficiency is significantly improved. Resource utilization; the memory occupation is reduced by 20-30%; the cache hit rate is improved by 40-60%; the space utilization is improved by 15-25%.

[0077] Embodiment two

[0078] Based on the same inventive concept, the embodiments of the present application also provide an electronic device, the method corresponding to the electronic device can be the implementation method of the single instruction multiple data (SIMD) optimized grouping vector hash table in the foregoing embodiments, and the problem solving principle thereof is similar to that of the method. The electronic device provided by the embodiments of the present application comprises at least one processor, and a memory in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method and / or technical solution of the foregoing embodiments of the present application.

[0079] The electronic device can be a user device, or a device integrated by a user device and a network device through a network, or can also be an application program running on the above device, the user device includes but is not limited to computers, mobile phones, tablet computers, smart watches, wristbands and various terminal devices, and the network device includes but is not limited to network hosts, single network servers, multiple network server sets or computer sets based on cloud computing, etc., which can be used to realize part of the processing function when setting an alarm. Here, the cloud is composed of a large number of hosts or network servers based on cloud computing.

[0080] Figure 7 The structure of an electronic device suitable for implementing the method and / or technical solution in the embodiments of the present application is shown, the device 700 comprises a central processing unit 701, which can perform various appropriate actions and processes according to programs stored in a read-only memory 702 or programs loaded from a storage part 708 to a random access memory 703. In the random access memory 703, various programs and data required for system operation are also stored. The central processing unit 701, the read-only memory 702 and the random access memory 703 are connected to each other through a bus 704. An input / output (I / O, Input / Output) interface 705 is also connected to the bus 704.

[0081] The following components are connected to the I / O interface 705: an input part 706 including a keyboard, a mouse, a touch screen, a microphone, an infrared sensor, etc.; an output part 707 including a cathode ray tube, a liquid crystal display, an LED display, an OLED display, etc., and a speaker, etc.; a storage part 708 including one or more computer readable media such as a hard disk, an optical disk, a magnetic disk, a semiconductor memory, etc.; and a communication part 709 including a network interface card such as a local area network card, a modem, etc. The communication part 709 performs communication processing via a network such as the Internet.

[0082] In particular, the methods and / or embodiments of this application can be implemented as a computer program product. For example, embodiments disclosed herein include a computer program product that includes a computer program tangibly embodied on a computer readable medium. The computer program includes program code for executing the methods illustrated in the flowcharts. When the computer program is executed by the central processing unit 701, the above-mentioned functions defined in the methods of this application are performed.

[0083] Embodiment Three

[0084] Another embodiment of this application provides a computer readable storage medium having stored thereon computer program instructions which can be executed by a processor to implement the methods and / or techniques of any one or more embodiments of the application previously described.

[0085] In particular, this embodiment can take any combination of one or more computer readable media. The computer readable media can be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this document, the computer readable storage medium can be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0086] The program code contained on the computer readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wire line, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0087] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network ("LAN") or a wide area network ("WAN"), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0088] The flow diagrams and block diagrams in the drawings are representative of the architectural, functional, and operational aspects of possible implementations of apparatuses, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may

[0089] Those skilled in the art can clearly understand that, for the convenience and brevity, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0090] In several embodiments provided by the present application, it should be understood that the disclosed system, device and method can be implemented by other ways. For example, the device embodiments described above are merely schematic, for example, the division of the units is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of units or page components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0091] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0092] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of hardware plus software functional unit.

[0093] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of the present application.

[0094] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The plurality of units or devices stated in the device claim can also be realized by one unit or device through software or hardware. First, second, etc. The words are used to represent the name, and not to represent any specific order.

Claims

1. An implementation method of a group vector hash table optimized using single instruction multiple data (SIMD) for key-value pairs, the method comprising: splitting a hash value, splitting a hash value of a current key into a low part and a high part, setting the control byte to contain low information of the hash value; designing a metadata array, grouping any key according to a hash table initialization table capacity, and storing information of each group using the control byte; when querying or writing or deleting, quickly locating a target slot by using SIMD instructions to compare the control byte in any group in parallel; serializing key-value data into a byte array and storing it in memory space or a disk file, and when writing or querying, locating the specific storage location of the key-value data through an offset.

2. The method of claim 1, wherein the designing a metadata array further comprises an initialization step: each group contains a plurality of slots, the state information of the slots is set as occupied slots, empty slots or deleted slots; and the control byte is set as an empty slot.

3. The method of claim 2, the encoding rules of the control bytes comprising: 0x00-0x7F represents an occupied slot; 0x80 represents an empty slot; 0xFE represents a deleted slot.

4. The method of claim 1, wherein a group size of the metadata array is 16 bytes, which is aligned with a 128-bit SIMD instruction set.

5. The method of claim 1, further comprising a memory mapping file mechanism, which maps a disk file to a continuous memory space.

6. The method of claim 3, the write logic comprising: By using SIMD, the same position of the control byte is quickly screened, and the value is updated.

7. The method of claim 3, wherein the query logic comprises: the high part of the split hash value is used to locate the starting group where the current key is stored, and the high hash value is used to locate the starting group; by using SIMD instructions to compare the control byte in parallel, a bit mask is generated to record the matching position; the keys in the matching position are traversed, and if a match is found, the value is returned, otherwise the next group is detected or the query is terminated.

8. The method of claim 3, the deletion logic comprising: If the current group has an empty slot, the deleted position is set as an empty slot; If the current group has no empty slot, the deleted position is set as a tombstone marker.

9. The method of claim 1, further comprising an automatic expansion step: when the number of elements reaches the product of the number of groups and the load factor, expansion is triggered.

10. An electronic device, comprising: at least one processor; and a memory connected in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 9.

11. A computer readable medium having stored thereon computer program instructions executable by a processor to implement the method of any one of claims 1 to 9.

12. A computer program product comprising a computer program which, when executed by a processor, implements the method of any one of claims 1 to 9.

Images